How AI Is Changing Google Search and SEO

Hello, and welcome to a new episode of search
off the record, the podcast where
we take you a little bit behind the scenes of Google Search
and hopefully have some fun along the way.
Well, you probably have seen AI features in search,
and whenever I have to talk about AI features in search,
I'm really, really happy that I got
to see a presentation at Search Central live in Zurich
last year, and I think it's time to open this up to more people.
So I invited a guest today.
My guest today is Nikola Todorovic.
And would you like to introduce yourself nikola?
Yes thank you Martin.
So I have joined Google about 15 years ago over here
in the Zurich office.
And for all of that time, I've been
a part of the search organization, what used
to be called search quality.
Nowadays, it's search intelligence.
And I've been a part of the team that's called safesearch.
And for the last several years, I've been leading that team.
And also in the last couple of years
I was more involved in the ecosystem work,
working together with you with the folks from Search Console,
Google Trends, et cetera.
And so have some more experience on that front as well.
And we pushed you into the cold water of our stage in Zurich
as well.
And you had a really, really cool topic.
You talked a bit more about AI in search.
Would you like to tell us what led to that talk,
and what was the thinking behind it,
and what you want people to take away from that.
Yeah, well, clearly AI is the topic
that everybody is talking about right now.
A lot of people are wondering how is search evolving
and what will be the future of Search
the future of AI, et cetera.
And from that perspective, I think
it was valuable to bring that particular presentation.
Now, the presentation that you referred
to has showed a lot more things before the new wave of AI
came in.
I think that was the context that I
felt it was helpful to present to the audience over here.
Yeah, because I think everyone is talking about AI in search
as if it's a new thing.
But it has been there behind the scenes,
so to speak, before that.
So what makes these AI features that people are using now
and that are progressively enhancing the search
experience for them, so different from the features
we had before.
Would you consider these new features revolutionary
and completely different from what we've been doing so far,
or is it more like an evolution of what
we have been doing in the past, I think the way they are being
used, and I think it is a revolution that we are speaking
of right now, but clearly in the whole process
there was small steps.
But if you compare search now and search 10 years ago,
it's a very different product.
So I would say yes, it's like a big step change
and it is absolutely changing the way the users are searching.
So if you think about it, any feature is changing in some way.
For example, if you bring more images, videos, et cetera
then it is bringing this kind of experience.
So people are going more to image search.
For example, when we added what we
call the image universal blocks on the main page.
Now, this new wave is also changing
the way the users are searching, because they are uncovering
that search can actually answer to more complex questions.
And for that reason, we do see that user queries
or if you call them prompts now so they're getting longer,
they become more detailed and the average query length
is growing.
So we do see the new traffic.
And this new wave of traffic is a consequence
of users being able to see there is something new I can do over
here.
So that's from that perspective.
It is revolution, but it is obviously
a bunch of steps in between that happen and have been
improving search all the time.
Can you shed some light on the steps
in between that you think are outstanding,
and probably have paved the way for this kind of.
Before I jump into that, maybe it
would be interesting to tell you a little bit
about the process, how the changes happen in search.
Oh yeah.
And then I can add what are the particular changes
or that reflected this AI revolution.
So in principles Google Search is a huge product.
There's a lot of different components
and all like you, Gary illyes, John Mueller and others
have been talking about this it starts all starts with the web,
with the crawling, indexing, the ranking components and so
on, the new features on top, et cetera.
So we have thousands of changes in Google Search per year.
I'm not sure how many but it's certainly in thousands.
We know that because we're tracking all of them
and we are evaluating all of them.
We're measuring because the key point is, yes, we
have new technology.
We have things that are for example,
we know problems that happen very often, there
are changes that come to search are either
a consequence of the new technology that's coming up
as we see, oh, let's use this new technology because it
certainly will bring us something, some improvements.
Or alternatively we see how there's a problem.
I'm typing this query, but I'm getting this result is not
optimal to see this.
And when we do this, we as engineers on search,
we are making a kind of an experimental version
of Google Search that has something new, that
has something different compared to the production
version of Google Search.
And we need some way to tell what is better,
because we're not just launching this 5,000 changes,
because some engineer or some product manager has an intuition
that this probably will be better.
So let me add this thing there, this thing there.
No so we have to start and see how
I have to build the prototype of the new version.
Thankfully all the infrastructure at Google
is really amazing, so it helped us run this very quickly.
Once we have a good idea so we can build a new version,
run a comparison with the baseline, which
is the production system, and we run those things called side
by sides.
So you're getting random user queries
that will see a difference between the production
and your experiment.
And we have published the guidelines
that help human raters review those changes, those differences
between the baseline and experiment.
And out of these reviews, out of these human reviews,
we're getting statistics.
This statistic is telling us if the experiment is better
than the baseline, and if it is, then, well, you
would think, yeah, let's submit it and commit the changes
and go launch.
No, we will have something called launch review.
And that is a process where we are where the engineers are
talking to the leads who have the decision making power
in the end and make a call.
Yes this is better.
And sometimes it can be that your overall statistics look
improving but you have some really bad pattern of losses
in your experiment.
And, well, if there's a kind of reasonable way
how to fix those patterns, we're going
to bring the engineer back and let them fix those patterns
and make an improvement.
And so right now I'm just talking
about the standard good old process of the launch reviews
and the new experiments and everything that goes in search.
And this process has been going on and is still there.
So let me know if this what I was just explaining is clear.
Do you have any questions on that before I moving to the more
I territory, I'm just wondering if at some point,
we should break this out as a separate episode,
because I think we've mentioned both the search quality rater
guidelines and the experiments beforehand,
but I don't think we've ever gotten
such a nice explanation of how the process works
and how the different bits and pieces fit together.
So that was really, really cool.
But let's take it back to I now.
So I'm guessing the AI features underwent more or less
the same process.
Yeah, absolutely.
They do.
And I have to say yes, given that the world is obviously
changing, the competitive landscape has changed as well.
We also need to adapt to this new world.
However, a lot of AI inside of Google
has been developed for years before the generative AI came
to play.
As I mentioned in the beginning, I
am responsible for the saved search engineering team,
and we were one of the first places where Google
was able to comfortably apply artificial intelligence
machine learning models directly in search.
The reason why it was not so easy to just apply it
everywhere is because these models function
like a kind of a black box.
You don't always understand what's happening underneath.
It's a complex set of for example, neural networks or even
the older simpler even.
The linear models are the easiest ones
to understand and to debug, because it's not just
you can put your AI or ML system into search
and you'll reap the most benefit from your side
by side experiments that I just mentioned previously.
And now you will, get to something and launch it.
But then you will have problems with that
as well, because obviously the systems evolved,
the searches evolve and so on.
And then you will need to debug this and replace it.
And this kind of replacement and changes is complicated.
So the more you can understand how these things work,
what signals are you using.
What signals are important for the relevance, for the quality,
for the safety of the results.
So you do need to understand the system.
And the more complex the AI or the ML systems
are, then the more challenging it is.
But saved search has been one of the places
where could isolate outside of the main search ranking flow.
You can isolate the systems that just do process the images,
process the videos, process the text,
and they just give you of a signal on its own.
How explicit.
For example, our result can be.
And then the understanding of let's say that 10 years
ago or 15.
No, it's more like 12 years ago when
really the convolutional neural networks came in
to help us understand the images better.
And in many places, they were actually already doing things
better than humans and understanding images.
Then we could apply this as a kind of a standalone AI
system that runs on a topic.
And if we have problems, yes, the engineers
in the research team had the intuition
and could run an iteration and improve the neural network
itself.
But it's a kind of a very isolated space.
So you can more easily navigate.
And then the rest of the search stack
has still been on its own and running things along the way.
There have been various new technologies,
so starting with transformers, I think
that that's the biggest one like that in the end introduced all
the Gen AI world, but we were reaping
the benefits of transformers on search
long before all the stuff came in and we were open about it.
So we have announced publicly the systems like Bert, like mom,
and they have been able to transform the search and ranking
into a much better place.
And again, these systems were built in an isolation
as well, just like the search systems.
I think these systems were also built in isolation
as a new signals, and these new signals
were supporting the whole ranking infrastructure.
And it was one more thing on top of everything else.
Hopefully that makes sense.
That makes sense.
And I mean if you look at it, the new AI features
are kind of also they are integrated,
but they are also somewhat isolated, as in there's
an AI overview that lives in its own space,
and AI Mode is a completely different way of searching.
So they are kind of also independent of the rest
of the search, even though they use
the rest of the search infrastructure and search
stack and ranking systems.
Would you say that's the case as well,
or is that completely different from previous systems.
Yeah let's maybe start with the overviews,
because that's where I think this holds the most still,
because if you think of AI Overviews
like this is your normal search with perhaps a few fan outs.
I just introduced a new term.
I probably should please explain that.
I think the experts out there.
I don't think it's like probably many of them
have heard about it.
But anyway, a fan out is when you have your own search query.
But then we might identify some additional search query
that will yield the results that can
be relevant for your original search query as well.
And then we can fork and in parallel do the retrieval
for multiple search queries.
That can all come back into one original, more complex query
that you gave in.
And so as I initially said previously
that we do see longer queries.
This is also we can help and understand
more directions of what you were initially typing.
So we launch multiple queries.
Now we get all this retrieved back.
And then AI Overviews is combining
from an interesting selection of these results
and making a summary from what I can see in those results.
So in a sense, the whole retrieval system,
the whole ranking system is the old style, the old school,
and that one is the AI.
Overviews is a feature that stamps on top of this
and operates on its own in this AI.
This is the isolated space for the AI overview
where it combines, and it's really fascinating
what the language models have been able to do.
But yes, it can combine like text
that it sees on these sources, on the snippets
and titles, et cetera.
And an additional context it can get out of those pages and then
make a really nice summary in the end.
And I really like that.
And I think that also goes back to what you said earlier,
that the behavior changes and queries get
longer and more complicated, because I remember back
in the days when I don't the world was still monochrome
or something, when I searched, even on Google,
I searched kind of keyword like restaurant vegetarian zürich
and then over the years that became more conversational,
as in vegetarian restaurants in zürich,
which is already a change.
And nowadays, I ask questions or I type
in queries that are so much more vague
and I still get usable results, based on dietary restrictions.
Which restaurants would you recommend now
for a lunch in Zurich.
And then you get a bunch of stuff
and it works because of these fan out queries.
It asks a bunch of queries that I don't have to ask myself
anymore to get to the right result.
And what I find myself doing is I'm asking questions
where I don't even know what a good question is.
Beforehand, you would sit-in front of Google
and think, how do I even look for this.
There's an effect in, I don't let's
say there's a physical effect and I
are what was the name of that.
So you would try to find the name of the effect
first and then Google for the specific effect.
Once you had the name and now you're like,
what is the physical effect that makes water glow
when there's radiation there.
And then it kind of figures it out for you.
And I think that's one of the possibilities of features
like AI overview.
So from AI Overviews, what was the motivation and the idea
behind then going further towards AI Mode.
Yeah no I agree completely.
These are exactly the nice examples
of the way how search has evolved
with the AI Overviews and eventually also AI Mode,
but all the capability of understanding your intention
with some vagueness.
Or I mean, even if you're more detailed.
Yeah, I want to a vegetarian restaurant that serves falafels
and that has you should be able to get this
or that's open now near me all the kind of context
that you're getting it.
True I didn't think of that.
But yeah, even if you have more details, you.
Now get better results.
Yeah so either if you have vague like query or if you have.
Actually more details.
So both of these seems to work better.
And clearly.
This doesn't stop there yet because what we're seeing
with the large language models.
They're able to gather a lot of information on their own.
And so they're able to.
Things what is the capital of France.
You don't really need to do the.
Search for it.
So this is, one part of it's all in parametric memory
of the model.
And so AI Mode is able to communicate with you
in a obviously it's like.
Even longer queries or longer discussions because it
also enables you to do the multi-turn thing.
And I mean, you have different tools that do all that.
So with Gemini being the Google's version, but obviously
others like ChatGPT cetera have been there.
And we do see that users like that.
So the users like the conversational aspect, the user
like to communicate longer and so on.
So AI Mode is kind of search's answer to that.
And we have also seen obviously not every user in the world
is going to some of these chatbots.
And obviously a mode is kind of a part of search.
So the users of search might actually
want to use that and see how it's like.
And you do have also the option to transition from the AI
Overviews to AI Mode if you want to explore more and have
a longer conversation and more detail.
So I think it's overall really, really a nice addition.
And I get myself like many times entering query
on search or maybe directly into AI Mode
or going to the AI Overviews and say maybe we
should actually I want a longer conversation.
And then I'll go to AI Mode.
AI Mode is also still using the search.
So it does have its own fan outs.
It does have the linked results and citations as well.
So it is kind of in essence still based
on this kind of standard concept of how we do things on search.
But on its own, it has a kind of a bigger well,
like the infrastructure is new and all the it
has kind of bigger ownership or it's
no longer an isolation of it.
It's like the AI Mode is kind of it runs on search,
but it also has a bigger platform for its own.
I'm still processing the fact that yeah, of course,
it works in both directions.
It also works with if you have more details.
And I just like the ability to have multimodal search.
And I think AI Mode just adds to that really.
And that's pretty cool.
But one thing that we keep hearing from the ecosystem,
pretty much at every event we do, and it's everywhere,
is how do we make sure that with AI features
being part of Search, now, that the ecosystem continues
to thrive.
And I think that's an interesting challenge.
But also there are lots of opportunities thanks to AI
features these days.
And I know that we at Google try our best
to go on this journey together with the ecosystem,
but how do you see it from your perspective.
What is it that we do to make sure
the ecosystem thrives with these new features.
Yeah, the ecosystem impact.
And I think as you said, I've been on two or three Search
Central lives like twice in Zurich, once in Madrid.
This is clearly one the key question.
And you see them a lot in on the social media as well.
And I don't think there is a magic wand that
can clearly give the guidance.
OK, what do I do now.
What would the SEO experts do now in the new system.
My kind of guiding principle or the way I see here
is that the site owners, I think they do
need to continue making sure that their products, that
their websites, that their platforms are providing value
to the user.
Because ultimately, if you provide a particular value,
then the users will continue coming to you
and they will continue coming to you through Google as well.
So if for example, you're selling something
you have a product or platform that you have
some subscriptions, et cetera, you clearly
will if you are providing value to your clients
like they will continue coming to you.
We were talking about restaurants.
Obviously if you're like putting a menu, et cetera.
So yeah, the users will eventually
come as well to your restaurant so they will go over and see.
So in the AI centric or AI oriented system,
I think those kind of bringing the value still continues.
But just like in the previous evolutionary or revolutionary
steps, like on how the media has been disseminated,
thinking about the newspapers, the radio, the TV,
like the internet, all the stuff that all of these things
also remain to be in this world.
But people needed to continue providing value.
Because if you don't provide value,
nobody's going to buy your newspaper or book or nobody's
going to listen to the radio or to the podcast.
So I think everybody like including all of us,
there's a lot of question, is going
to take our jobs and so on.
I think we all need to continue thinking like,
how do we provide value on top of all of this.
And in many cases, this is about mastering the AI tools
and being able to use them in the best possible way.
So this is one of my recommendation to all the SEO
professionals and site owners and the whole ecosystem
that they continue providing value,
but then do not neglect the new technology
and make sure you use it in the best possible way for you.
Now, obviously I don't think we would over here recommend
like the best possible way is to just multiply all the content
and just generate because it's cheap and easy.
And now we're going to generate it's not going
to provide a ton of value.
But if you know you're using it to improve your grammar
to improve the style a little bit, make it more interesting
and so on, I don't think that's a wrong use of the technology,
but then there's plenty of ways, OK,
maybe AI can help me better understand your data.
Maybe I can help you understand the competition potentially
better as well.
And so on.
So clearly, this is something we can advise.
I find that really interesting because I'm
seeing a lot of excitement at the same time,
a lot of worrying in the community, in the ecosystem.
And I think it is that because on one hand,
it democratizes a lot of stuff that
has been traditionally difficult to do or just cumbersome to do.
At the same time, some people have misunderstood whatever
it was that they are trying to accomplish or to provide
to be these cumbersome bits and only these cumbersome bits.
So to give you an example, when it comes to let's say,
writing articles about I don't lifestyle or technical topics
because I'm more like a geek.
So I'm reading more technical things.
I really enjoyed when people were giving me interesting
details of technology from the days past, much older than I am,
so I wouldn't have any touching points with technology from
the 60s or the 70s.
And someone was like, hey, did you
know that the displays in old Hi Fi devices worked like this.
That was a really interesting article,
but obviously they also went and explained
what their experiences were with new technology as it came out.
And as they were provided with samples, sometimes even.
And that was interesting, but eventually that
turned into them effectively.
How do I put this nicely.
Putting words around spec Sheets from manufacturers.
And that wasn't really the value that I was looking for.
I'm not interested in knowing how many gigahertz
a certain new processor has, because I
can read that basically on the box, it says it on the box.
You don't have to tell me that this is now a 3 gigahertz
processor and it says it on the box.
Thank you.
And I had a key moment when I was buying a joystick back
in the days for a computer game.
And I didn't know what force feedback was.
And that's effectively like you have a different resistance
and it might move and vibrate the device
if there's any shaking happening in the surroundings.
And I didn't know what that was and it
said on the box it has force feedback.
And so I went to someone who worked at the shop,
and I anticipated them to be an expert on the topic.
So I'm like, so this says force feedback.
What does that mean.
And he literally said to me, oh, that
means that this joystick has force feedback.
And this is funny, but I'm seeing this a lot in articles
and on websites that they're effectively not giving me
any context.
They're just explaining what I can
glimpse and gather from the information that
is right in front of me.
And I think AI makes that easier.
Don't have to spend as much time to rattle off the spec Sheets
into a more readable human conversational form.
But chatbots do that, so you don't necessarily
have to do that on your website anymore.
But maybe you have tested it, and you
found it to be particularly good for your use case
or particularly unfit for your use case.
And then you can share this insight that AI doesn't have.
It doesn't know it hasn't used the technology,
it doesn't know this, but you do.
So you're the expert.
And I might be coming if you're using your electronics the way
that I use them, I might be interested in your opinion.
I might not be interested in this other person's opinion
because they are using their electronics differently.
But that's fine because there are
other people who are using their electronics the same way as they
do, just not me.
So I think there is still enough space
online for different outlets and people and opinions
and experiences, but I think we have
to increase the level of our content
to be useful and interesting testing for humans from humans
to humans.
And I don't think AI is going to take that away.
I think AI is going to bridge that.
Yeah, I absolutely agree.
I get often kind of nervous, when
I see the AI style reports like obviously internally we
want to use these tools like they helping us.
They help me understand the documentation more easily.
Can we ask questions like, notebook
has been a fascinating tool that can in a couple of minutes,
explain a complicated thing.
So yeah, I do believe there is still
a need for the human touch on top of.
On top of all of that, I do think
we need to understand the capabilities of the tools,
but in the end, US providing the value, US making sure that yes,
we're bringing something to the table.
And I think that's where we want to focus.
But yeah.
Are you using the coding tools.
Interestingly enough, yes.
And that's exactly where this stuff comes in, so handily.
So the code base in Google is huge
because it has a lot of stuff, and you've seen it yourself.
And just a couple of days ago, we
stumbled upon a specific piece of code,
and it was going through lots of layers
of indirection and abstraction to do something.
And we had a hypothesis where this is going in the end,
but we didn't know.
So we asked our internal tools like,
so we found this thing that does this thing,
but where does the information actually go.
So it was basically like we found this method that
tells us how big an image is.
But where does this information come from.
Does it have to download it or does it use
the image and index for this.
And we could have found that ourselves
by going 20, 30 minutes through abstraction layer
after abstraction layer to finally get
to where it's coming from.
Or we just ask the system and it's like, oh,
this is coming from here.
And that was the right spot.
And we're like, oh yeah, OK.
So it comes from where we expect it to come from.
Cool that's good to know.
So it is useful.
It does help.
It makes things faster.
Doesn't replace us making the effort of figuring out
if what we're doing makes sense in the first place,
and if it takes the right trade offs
and if it's the right choice.
Those things, I think, are not that automatable or AI able yet.
Yeah, maybe yet.
Yet that might change.
But I think these tools are useful.
But yeah, you're absolutely right.
It depends on how you use these tools.
But yeah.
And on top of that, I think there's always
a risk of introducing a bug that you don't understand and so on.
So I think the whole discussion of how
is the software of the future, is going
to be maintained by the AI and will remain
human maintainable or understandable right now,
you still have a bunch of people who
can understand what's going on.
We'll see how that will evolve and will take the system that
is fully I run and I automotive br
become at some point in the future,
more we will have an edge over the current system, architecture
or style of building systems.
We will see all that, but I think it's important for now,
at least for all of us in the engineering side,
to lean into the tools and make sure we continue using them
and be capable with them.
And I would love to hear from you all out there.
What do you think.
Are you using AI for something that you
wouldn't have expected before you tried it out,
or are you skeptical.
Have you made good experiences.
Have you made bad experiences with AI.
I'm just curious how you all out there
are experiencing this shift and this time of exploration
basically.
Anyway, thank you so much for being here.
I think that was really, really interesting.
We touched upon so many interesting things
from how we are running experiments
to how I evolved at Google into the thinking behind AI.
Overviews and AI Mode.
And thank you so much for your time and thanks
so much for being here.
Thank you Martin, it was a pleasure
joining you in the podcast and all of you out there.
If you'd like to hear more of this, please do subscribe.
We are on all your podcast platforms out there
and we're looking forward to hear from you.
So leave us a comment.
Leave us a leave us a subscription
and talk to you soon.
Bye bye bye everybody.
We've been having fun with these podcast episodes.
I hope you, the listener, have found
them both entertaining and insightful too.
Feel free to drop us a note on LinkedIn
or chat with us at one of our next events we go to.
If you have any thoughts, let us know.
And of course, do not forget to and subscribe.
Thank you so much for listening and goodbye.