good morning everyone welcome to Google
I O as an AI first company we are at an
exciting inflection point
let me start with few examples of how
generative AI is helping to evolve our
products let’s say you got this email
that your flight was canceled the
airline has sent a voucher but what you
really want is a full refund you could
reply and use help me write
just type in the prompt of what you want
an email to ask for a full refund hit
create and a full draft appears
as you can see it conveniently pulled in
flight details from the previous email
and it looks pretty close to what you
want to send
maybe you want to refine it further in
this case a more elaborate email might
increase the chances of getting the
refund
and there you go I think it’s ready to
send
help me write we’ll start rolling out as
part of our workspace updates
the next example is Maps imagine if you
could see your whole trip in advance
with immersive view for routes now you
can whether you’re walking cycling or
driving
let me show you what I mean say I’m in
New York City and I want to go on a bike
ride
Maps has given me a couple of options
close to where I am I like the one on
the waterfront so let’s go with that
looks Scenic I want to get a feel for it
first
click on immersive view for routes and
it’s an entirely new way to look at my
journey
I can zoom in to get an incredible
bird’s eye view of the ride
and and as we turn we get on to a great
bike path and if I want to check traffic
and weather and see how they might
change over the next few hours I can do
that looks like it’s going to pour later
so maybe I want to get going now
immersive view for routes will begin to
roll out over the summer and launch in
15 cities by the end of the year another
product made better by AI is Google
photos every month 1.7 billion images
are edited in Google photos
AI advancements give us more powerful
ways to do this let’s have a look
this is a great photo but as a parent
you always want your kid at the center
of it all and it looks like the balloons
got cut off in this one so you can go
ahead and reposition the birthday boy
magic editor
automatically recreates parts of the
bench and balloons that were not
captured in the original shot as a
Finishing Touch you can punch up the sky
it changes the lighting in the rest of
the photo so the edit feels consistent
it’s truly magical
we are excited to roll out magic editor
in Google photos later this year
our ability to make AI helpful for
everyone relies on continuously
advancing our foundation models
so I want to take a moment to share how
we are approaching them
today we are ready to announce our
latest Farm model and production Palm 2.
2 Bill zone are found fundamental
research and our latest infrastructure
it’s highly capable at a wide range of
tasks and easy to deploy
we are announcing over 25 products and
features powered by palm2 today
Palm 2 models deliver excellent
foundational capabilities across a wide
range of sizes we’ve affectionately
named them gecko order bison and unicorn
gecko is so lightweight that it can work
on mobile devices
fast enough for great interactive
applications on device even when offline
Palm to models are stronger in logic and
reasoning thanks to Broad training on
scientific and mathematical topics it’s
also trained on multilingual text so
spanning over 100 languages so it
understands and generates nuanced
results
while Palm 2 is highly capable it really
shines when fine-tuned on
domain-specific knowledge we recently
released Secom a version of pump to
fine-tune for security use cases it uses
AI to better detect malicious scripts
and can help Security Experts understand
and resolve threats
another example is metform2
in this case it’s fine-tuned on medical
knowledge this fine tuning achieved a 9x
reduction in inaccurate reasoning when
compared to the model approaching the
performance of clinician experts to
answer the same set of questions
in fact metform 2 was the first language
model to perform an expert level on
medical licensing exam style questions
and is currently the state of the art
we are also working to add capabilities
to metform 2 so that it can synthesize
information from Medical Imaging like
plane films and mammograms you can
imagine an AI collaborator that helps
Radiologists interpret images and
communicate the results
is the latest step in our decade-long
journey to bring AI in responsible ways
to billions of people
it builds on progress made by two
world-class teams the brain team and
deepmind
we recently brought these two teams
together into a single unit Google
deepmind using the computational
resources of Google they are focused on
building more capable systems safely and
responsibly
this includes our next Generation
Foundation model Gemini which is still
in training
Gemini was created from the ground up to
be multimodal highly efficient at tool
and API Integrations and built to enable
future Innovations like memory and
planning
while still early we are already seeing
impressive multimodal capabilities not
seen in Prior models once fine-tuned and
rigorously tested for safety Gemini will
be available at various sizes and
capabilities just like pom 2.
as we invest in more advanced models
we’re also deeply investing in AI
responsibility
this includes having the tools to
identify synthetically generated content
whenever you encounter it
if you look at a synthetic image it’s
impressive how real it looks so you can
imagine how important this is going to
be in the future
metadata allows content creators to
associate additional context with
original files giving you more
information whenever you encounter an
image we’ll ensure every one of our AI
generated images as that metadata as
models get better and more capable one
of the most exciting opportunities is
making them available for people to
engage with directly
that’s the opportunity we have at board
large language models have captured the
world’s imagination changing how we
think about the future of computing we
launched Bard as a limited access
experiment on a lightweight large
language model to get feedback and
iterate and since then the team has been
working hard to make rapid improvements
and launch them quickly
with palm 2 Bard’s Math logic and
reasoning skills made a huge leap
forward underpinning its ability to help
developers with programming Bard can now
collaborate on tasks like code
generation debugging and explaining code
Snippets
Bard has already learned more than 20
programming languages including C plus
plus go JavaScript Python kotlin and
even Google Sheets functions now Bart
can also help me understand the code
could you tell me what chess dot board
does in this code
now this is a super helpful explanation
of what it’s doing and makes things more
clear
all right let’s see if we can make this
code a little better
how would I improve this code
okay let’s see there’s a list
comprehension creating a function and
using a generator those are some great
suggestions now could you join them into
one single python code block okay now
Bart is rebuilding the code with these
improvements
okay great how easy was that we’ve also
heard that you want Dark theme so
starting today you can activate it right
in Bart or let it follow your OS
settings in the next few weeks Bard will
become more visual both in its responses
and your prompts
so if you ask
what are some must-see sites in New
Orleans
Bart’s going to use Google search and
the knowledge graph to find the most
relevant images the French Quarter the
Garden District these images are really
giving me a much better sense of what
I’m exploring we’ll also make it easy
for you to prompt Bart with images
giving you even more ways to explore and
create
imagine I’m 18 and I need to apply to
college
I won’t date myself with how long it’s
been but it’s still an overwhelming
process
so I’m thinking about colleges but I’m
not sure what I want to focus on I’m
into video games and what kinds of
programs might be interesting
okay this is a helpful Head Start
animation looks pretty interesting now I
could ask help me find colleges with
animation programs in Pennsylvania
okay great that’s a good list of schools
now to see where these are I might now
say show these on a map here Bart’s
going to use Google Maps to visualize
where the schools are
this is super helpful and it’s exciting
to see that there’s plenty of options
not too far from home now let’s start
organizing things a bit show these
options as a table
nice structured and organized but
there’s more I want to know add a column
showing whether they’re public or
private schools perfect this is a great
start to build on and now let’s move
this to Google Sheets so my family can
jump in later to help me with my search
you can see how easy it will be to get a
jump start embarred and quickly have
something useful to move over to apps
like docs or sheets to build on with
others okay now that’s a taste of what’s
possible when Bard meets some of
Google’s apps but that’s just the start
Bard will be able to tap into all kinds
of services from across the web with
extensions from incredible Partners like
instacart indeed Khan Academy and many
more we are removing the wait list and
opening up Bard to over 180 countries
and territories
Bart is also becoming available in more
languages Beyond English starting today
you’ll be able to talk to Bart in
Japanese and Korean and we’re pleased to
share that we’re on track to support 40
languages soon and now to hear more
about how large language models are
enabling Next Generation productivity
features right in workspace I’ll hand it
over to aparna
from the very beginning workspace was
built to allow you to collaborate in
real time with other people
now you can collaborate in real time
with AI
AI can act as a coach a thought partner
a source of inspiration as well as a
productivity booster across all of the
apps of workspace
our first steps with AI is a
collaborator review the help me write
feature in Gmail and docs which launched
to trusted testers in March
one of our most popular use cases is the
trustee job description
every business big or small needs to
hire people a good job description can
make all the difference
here’s how docs has been helping
say you run a fashion boutique and need
to hire a textile designer to get
started you enter just a few words as a
prompt
senior level job description for textile
designer docs will take that prompt send
it to our Palm 2 based model and let’s
see what I got back
not bad with just seven words the model
came back with a good starting point
written out really nicely for me now you
can take that and customize it for the
kind of experience education and skill
set that this role needs saving you a
ton of time and effort
next
let me show you how you can get more
organized with sheets
imagine you run a dog walking business
and need to keep track of things like
your clients Logistics about the dogs
like what time they need to be walked
for how long Etc
sheets can help you get organized in a
new sheet simply type something like
client and pet roster for a dog walking
business with rates and hit create
she sends this input to a fine-tuned
model that we’ve been training with all
sorts of sheet specific use cases
look at that the model
the model figured out what you might
need the generated table has things like
the dog’s name client info notes Etc
this is a good start for you to Tinker
with
sheets made it easy for you to get
started so you can go back to doing what
you love prompts are a powerful way of
collaborating with AI the right prompt
can unlock far more from these models
what if AI could proactively offer you
prompts
even better what if these prompts were
actually contextual and changed based on
what you’re working on
my niece Mira and I are working on a
spooky story together for summer camp
we’ve already written a few paragraphs
but now we’re stuck
let’s get some help
as you can see we launch a side panel
something the team fondly calls sidekick
sidekick instantly reads and processes
the document and offers them really neat
suggestions along with an open prompt
dialogue
if we look closely we can see some of
the suggestions like what happened to
the golden seashell
what are common mystery plot twists
let’s try the seashell option and see
what it comes back with
now what’s happening behind the scenes
is that we’ve provided the entire
document as context to the model along
with the suggested prompt and let’s see
what we got back
the golden seashell was eaten by a giant
squid that lives in the Cove this is a
good start let’s insert these as notes
so that we can continue our little
project and this is exactly what AI can
help with
I see a new suggestion on there for
generating images let’s see what this
does the story has a village a golden
seashell and other details and instead
of having to type all of that out the
model picks up these details from the
document and generates images
say you’re about to give an important
presentation and you’ve been so focused
on the content that you forgot to
prepare speaker notes
the presentation is in an hour oh
no need to panic
look at what one of the suggestions is
create speaker notes for each slide what
happened behind the scenes here is that
the presentation and other relevant
context was sent to the model to help
create these notes and once you’ve
reviewed them you can hit insert and
edit the notes to convey what you
intended
so you can now deliver the presentation
without worrying about the notes next up
we’re going to talk about search
to give you a sense of how we are
bringing generative AI in search
I’m going to invite Kathy onto the stage
let’s start with a search for what’s
better for a family with kids under
three and a dog Bryce Canyon or arches
now although this is the question that
you have you probably wouldn’t ask it in
this way today
you’d break it down into smaller ones
sift through the information and then
piece things together yourself
now search does the heavy lifting for
you
what you see here looks pretty different
so let me first give you a quick tour
you’ll notice a new integrated search
results page so you can get even more
out of a single search
there’s an AI powered snapshot that
quickly gives you the lay of the land on
a topic
and so here you can see that while both
parks are kid-friendly only Bryce Canyon
has more options for your furry friend
then if you want to dig deeper there are
links included in the snapshot
you can also click to expand your view
and you’ll see how the information is
corroborated so you can check out more
details and really explore the richness
of the topic
this new experience Builds on Google’s
ranking and safety systems that we’ve
been fine-tuning for decades
these new generative AI capabilities
will make search smarter and searching
simpler let’s say you’re searching for a
good bike for a five mile commute with
hills
this can be a big purchase so you want
to do your research
in the AI powered snapshot you’ll see
important considerations like motor and
battery for taking on those Hills and
suspension for a comfortable ride
below that you’ll see products that fit
the bill each with images reviews
helpful descriptions and current pricing
this is built on Google’s shopping graph
the world’s most comprehensive data set
of constantly changing products sellers
Brands reviews and inventory out there
with over 35 billion listings in fact
there are 1.8 billion live updates to
our shopping graph every hour
and for commercial queries like this we
also know that ads can be especially
helpful to connect people with useful
information and help businesses get
discovered online
they’re here clearly labeled and we’re
exploring different ways to integrate
them as we roll out new experiences in
search
and now that you’ve done some research
you might want to explore more
so right under the snapshot you’ll see
the option to ask a follow-up question
or select a suggested Next Step tapping
any of these options will bring you into
our brand new conversational mode
in this case maybe you want to ask a
follow-up about e-bikes so you look for
one in your favorite color red
and without having to go back to square
one Google search understands your full
intent and that you’re looking
specifically for e-bikes in red that
would be good for a five mile commute
with hills
and even when you’re in this
conversational mode it’s an integrated
experience so you can simply scroll to
see other search results
now maybe this e-bike seems to be a good
fit for your commute
with just a click you’re able to see a
variety of retailers that have it in
stock and some that offer free delivery
or returns
you’ll also see current prices including
deals and can seamlessly go to a
merchant site check out and turn your
attention to what really matters getting
ready to ride
these new generative AI capabilities
also unlock a whole new category of
experiences on search
it could help you create a clever name
for your Cycling Club craft the perfect
social post to show off your new wheels
or even test your knowledge on bicycle
hand signals
these are things you may never have
thought to ask search for before
this new search generative experience
also known as sge will be available in
Labs along with some other experiments
and they’ll be rolling out in the coming
weeks
AI is not only a powerful enabler it’s
also a big platform shift every business
and organization is thinking about how
to drive transformation
that’s why we are focused on making it
easy and scalable for others to innovate
with AI that means providing the most
advanced Computing infrastructure
including state-of-the-art tpus and gpus
and expanding access to Google’s latest
Foundation models that have been
rigorously tested in our own products to
tell you more about how we are doing
this with Google Cloud please welcome
Thomas
all of the Investments you’ve heard
about today are also coming to
businesses
there are three ways Google Cloud can
help you take advantage of the massive
opportunity in front of you
first you can build generative
applications using our AI platform
vertex AI
with vertex you can access Foundation
models for chat text and image you just
select the model you want to use
create prompts to tune the model
and you can even fine-tune the model’s
weights on your own dedicated compute
clusters
to help you retrieve fresh and factual
information from your company’s
databases your corporate internet your
website and Enterprise applications we
offer enterprise search
with both vertex and Enterprise search
you have sole control of your data and
the costs of using generative AI models
in other words your data is your data
and no one else’s
you can also choose the best model for
your specific needs across many sizes
that have been optimized for cost
latency and quality
the second way we’re helping you take
advantage of this opportunities by
introducing duet AI for Google Cloud
duet uses generative AI to provide
developers assistance
it can provide you contextual code
completion
offers suggestions tuned to your code
base and generate entire functions in
real time
you can even assist you with code
reviews and code inspection the third
way we’re helping you seize this moment
is by building all of these capabilities
on our AI optimized infrastructure
this infrastructure makes large-scale
training workloads up to eighty percent
faster and up to 50 percent cheaper
compared to any alternatives out there
look when you nearly double performance
for less than half the cost amazing
things happen
today we’re excited to announce a new
addition to this infrastructure family
the A3 virtual machines based on
nvidia’s latest h100 gpus
we provide the widest choice of compute
options for leading AI companies like
anthropic and mid-journey to build their
future on Google Cloud
and yes there’s so much more to come
next Josh is here to show you exactly
how we’re making it easy and scalable
for every developer to innovate with AI
and palm 2.
thanks Thomas now to show you just how
powerful the Palm API is I want to share
one concept that five engineers at
Google put together over the last few
weeks
the idea is called project tailwind and
we think of it as an AI first notebook
that helps you learn faster
like a real notebook your notes and your
sources power Tailwind
how it works is you can simply pick the
files from Google Drive and it
effectively creates a personalized and
private AI model that has expertise in
the information that you give it
now imagine that I’m a student taking a
computer science history class I’ll open
up tailwind and I can quickly see in
Google Drive all my different notes and
assignments and readings I can insert
them and what will happen when Tailwind
loads up as you can see my different
notes and articles on the side
here they are in the middle and it
instantly creates a study guide on the
right to give me bearings you can see
it’s pulling out key Concepts and
questions grounded in the materials that
I’ve given it
now I can come over here and quickly
change it to go across all the different
sources and type something like create
glossary for Hopper
and what’s going to happen behind the
scenes is it’ll automatically compile a
glossary associated with all the
different notes and articles relating to
Grace Hopper the computer science
history Pioneer look at this flomatic
Cobalt compiler all created based on my
notes
now project Tailwind is still in its
early days but we’ve had so much fun
making this prototype and we realized
it’s not just for students it’s helpful
for anyone synthesizing information from
many different sources that you choose
like writers researching an article or
analysts going through earnings calls or
even lawyers preparing for a case
imagine collaborating with an AI That’s
grounded in what you’ve read in all of
your notes
building bold AI requires a responsible
approach so let me hand it over to James
to share more thanks
hi everyone
I’m James in addition to research I lead
a new area at Google called technology
and Society generative AI makes it
easier than ever to create new content
but it also raises additional questions
about his trustworthiness
that’s why we’re developing and
providing people with tools to evaluate
online information in the coming months
we’re adding two new ways for people to
evaluate images
first without about this image Tool in
Google search you will be able to see
important information such as when and
where similar images may have first
appeared where else the image has been
seen online including news fact checking
and social sites as we begin to roll out
the generative image capabilities like
Sundar mentioned we will ensure that
every one of our AI generated images has
metadata a markup in the original file
to give you context if you come across
it outside of our platforms
not only that creators and Publishers
will be able to add similar metadata so
you’ll be able to see a label in images
in Google search marking them as AI
generated
as we apply our AI principles we also
start to see potential tensions when it
comes to being bold and responsible
here’s an example
Universal translate is an experimental
AI video dubbing service that helps
experts translator speakers voice while
also matching their lip movements let me
show you how it works with an online
college course created in partnership
with Arizona State University
what many college students don’t realize
is that knowing when to ask for help and
then following through on using helpful
resources is actually a Hallmark of
becoming a productive adult
universities
we use Next Generation translation
models to translate what the speaker’s
saying models to replicate the style and
the tone and then match the speaker’s
lip movements then we bring it all
together
this is an enormous step forward for
learning comprehension and we’re seeing
promising results of course completion
rates
but there’s an inherent tension here
you can see how this can be incredibly
beneficial but some of the same
underlying technology could be misused
by Bad actors to create deep fakes
so we’ve built the service of guard
rails to help prevent misuse and to make
it accessible only to authorized
partners
has to be wrap up I’ve been reflecting
on the big technology shifts that we’ve
all been a part of the shift with AI is
as big as they come and that’s why it’s
so important that we make AI helpful for
everyone
we are approaching it boldly with a
sense of excitement because as we look
ahead Google’s deep understanding of
information
combined with the capabilities of
generative AI can transform search and
all of our products yet again
we look forward to working together and
building together so on behalf of all of
us at Google thank you and enjoy the
rest of IO
[Music]
