Vibe Coding - yay or nay?

In this episode of search.
Off the record, I'm talking to John about vibe coding.
We talk a little bit about what it is
and what tools we have tried, what our experiences were,
and where we think these tools help us already,
and where they might not be as helpful as they want to be.
And we also want to hear from your experiences.
So let's dive in.
Hello, and welcome to another episode of search
off the record, your podcast coming from Google Search
relations right to you.
Wherever you're listening to podcasts,
we want to take you behind the scenes of search a little bit
and show and tell you a little bit of what's
inside of our heads.
With me today is John Mueller.
Hi, John.
Hi, Martin.
Same team, same place, same fun.
John, I want to talk to you about vibe coding websites.
OK, let's do it.
OK excellent.
I think there's a lot talk about AI becoming
a tool that kind of fire and forget, and you can do anything.
You can create videos, you can make music, you can do whatever,
including making programs and websites and apps and stuff.
And I tried it.
I finally tried it.
I tried AI Studio this time.
Did it work.
It was great, actually.
I wanted it to build me a website.
A static website didn't need.
Well, not really a static website.
I wanted it to build a client side tool with some JavaScript,
and it did that really well.
And the nice thing is, I was surprised that I could actually
read the code, and it looked more or less
like a standard Next.js application.
I wasn't too like thrown away with how
the weird setup sometimes looks like when
you have other people writing code for you.
So that was nice.
But then at some point, I got into a loop
where I asked her to not do something and it vibe coded.
It literally kind of vibe coded because I
was like and use this library.
And I was like, no, no, I built this myself.
And I'm like, no, no, no, no, don't do that.
Use this where I know it works.
And it's like, OK.
But it lied to me and got stuck.
And then I asked it for a half an hour.
I tried to make it not do what it wanted to do,
and want to do what I wanted to do.
And that was weird.
Did you try something and did you have other results
or did the same happen to you.
I've been making a lot of websites,
mostly for myself and to test things out using vibe coding.
I don't Martin.
Just an aside, was the UI in purple.
No, I heard that from many people.
It was an interestingly enough, not in purple.
OK, nice.
OK just checking.
So I think maybe taking a step back.
Vibe coding is this fancy new term
that someone came up with from the AI world
where basically you have an AI system, write the code for you,
and you basically talk to the AI system using English.
Or I'm pretty sure you can also do all of this in German
or any of the other languages where you basically just say
like, hey, I want a personal website
and it should look like this.
You describe what you want it to look like,
and it should have these pages on it.
And then the system goes off and basically creates those pages
for you, those files.
And basically from a technical point of view,
you have to do an additional step
after that, which is basically take these files
and put them on some server so that they actually
run as a normal website that you would do.
And I think the allure a little bit is well,
you speak to the system in English.
You don't have to know JavaScript
or any other programming language, basically.
And it goes off and does all of this technical stuff for you.
And then you take the output and you put it somewhere,
and then you have a normal website, which, practically
speaking, nobody can really recognize
as being like, this is a vibe coded website.
Of course, if you use some of the frameworks or platforms that
are used commonly by vibe coding,
people can recognize those patterns,
but otherwise, it's a normal website.
And I think one of the things that is perhaps tempting
is to take it to the next step and say like, oh, you should
also create my content for me.
It's like, now you made my website look nice,
but why don't you write an About Me page that
lists all of the things that I've done or is like,
why don't you write a marketing page that makes sure
that everyone clicks buy.
Which is like, sure can do that.
But it's not something where I'd say, where are you getting
the most value out of it.
Interesting yeah.
I mean, with content, with content,
it becomes tricky because then why would I
visit a website where party content has
been written when I can just talk to the eye directly, right.
But with the structure, I mean, if you're using frameworks,
then most of the things look roughly the same.
Interesting is when I apparently prefers purple.
So if I see a purple website, then I can ask myself,
is this really the preference of the editor
or the author of this website.
Or is this something that just happened because it
was Yeah, it's kind of weird.
I think also one of the things that is from my point of view,
kind of interesting is one of the selling
points is that you don't have to know how to code.
I was about to ask you that.
Yeah is it this fire and forget kind of thing
where you just describe it in English and plain English,
I want this with that and this button and this thing
and this, that.
And then it just generates the right thing
because it worked for me this one
time of 90% I think it depends.
It depends on the complexity that you're trying to do
and the long term setup that you want.
And also the almost like the cleanliness that you want,
because it can definitely create a generic website based
on the information that you give it,
but it will make a variety of assumptions along the way.
If you go and say like, I want a personal website
and these pages, it could go off and say,
well, I'm going to use a static site generator because those
exist, those are easy to use, and I will create these pages
for you.
Or it could go out and say, well, I
have all of these JavaScript frameworks
that have a lot of open source documentation and tooling.
I will make a JavaScript based website for you.
Or it could go out and say, well, actually you probably want
a CMS, and therefore I will create this complicated system
that has a database backend that lets you edit your homepage
and all of this stuff in a browser kind of thing.
All of these are reasonable assumptions
where if you talk to a developer they will also
make these assumptions.
But if you just tell the AI system like I want a website,
then it will pick one, like in ideal case,
maybe it asks you some questions like what do you actually
want to do with this.
Are you going to make changes or not.
Do you mind working in GitHub.
Do you want to do it in a UI.
Those would be reasonable questions to ask,
but often the system will just go off and say like, sure,
let me take care of this thing for you.
And then it goes off and does stuff.
So that's the kind of thing where
I think if you know what you're doing with regards how
to make a website and you have a concrete idea where you're like,
well, actually, I want a static site generator because I just
want to copy and paste the files to I don't Firebase hosting
or Vercel or anywhere.
Then you can tell it that ahead of time, when
you're setting this up and it's like,
hey, I want you to create a static site using maybe
my preferred framework and here's
the setup, the structure that it's supposed to have.
But for that need to have the technical understanding.
Yeah, exactly.
That's where I think the technical understanding comes
in.
And especially going to take a little bit past that
and you're thinking like, well, what about SEO.
And of course, in the end, you can always tell the AI system.
Now add some SEO to it.
But, how that works out is if you go to a developer
and add some SEO and it's like, what do you mean.
Sprinkle some meta tags and add some structured data.
It's like, what do you mean, add some SEO.
So those are the kind of things where
I think if you have the technical understanding.
You can guide it from the beginning and you can say,
actually it's like when you create my site,
I want you to do this and this.
Make sure that the pages have, I don't
canonicals here's the domain name I want to use.
Make sure to set up a sitemap file
or whatever that you're trying to do.
So on the one hand, knowing what technically you kind of want
helps to get there quite a bit.
What also helps a lot is if you understand things like how
a deploy script should look, or how a presubmit should work,
how a linter works, those kind of things,
then you can also include that in the configuration that you're
setting up, so that you can essentially say like,
this is the structure you should create
and you should make sure to before publishing,
make sure that these URLs all return content.
Make sure that I don't none of the JavaScript files
are blocked by robots.txt.
And it's create a robots.txt file
and make it useful of thing, all of these small details
that help a lot.
And I think especially the pre-submit stuff
is something where for a lot of the mainstream frameworks,
there's a lot of tooling out there that helps to make sure
that oh, I built this site and I published it
and it's actually I'm publishing the right thing.
I'm not publishing my source code and all of my API keys
or whatever you're doing that keeps happening to people.
I keep seeing that accidental.
Yeah, API key publishing.
But all right.
Assuming that it can build these things.
So it's not like a one click more or less kind of thing
where you just say some plain English
and then get whatever you want it back.
You need to still have some idea of where you want to go
and some of the technical lingo, because that
helps steer the output.
But when it's created, I also have to test it
and depending on what I have asked it to do,
testing manually might be a bit tricky.
And also when it fails in subtle ways, I mean,
if it fails with an error message,
you can just give it the error message.
You can say like, oh, I clicked this button
and then I got this error.
Can you fix that please.
I think please makes it more expensive.
I don't know.
So one of the weird things I heard,
or I read somewhere in one of the random leaks that happened
from the AI labs, is that they have dashboards specifically
for the coding tools about how often people
curse at the AI system.
Oh, OK.
Because it's like a sign.
It's like, oh, it's like our system did something wrong.
And like, it's sometimes just enough to count,
how many times people are cursing at it.
So I'm not saying that you should or I'm not
saying I'm also not judging.
If you're screaming at the machines, whatever.
If it's a toaster and it burns your toast,
like maybe it does good to scream at it,
but it's like you're using the toaster to make toast.
It's not the toaster's problem.
That toast burnt.
Oh, OK.
But is there a way for me to also use the tool
to have it test itself.
Or does that fall short because it doesn't actually
know what the expected.
Because I remember back in the days,
we used, I think it started with Selenium, and then at some point
used Puppeteer to remote control a browser from code.
But you would have to write that code so that it would actually
go and fill in the registration form,
put some products in the cart and so on and so forth.
Can I help you with that as well.
Sure like can you code this as well
and say like also test this thing in the browser.
I mean, at that point you really have
to have a strong technical understanding of what
you want to do.
I think maybe there are ways to go at it naively,
but basically for pretty much all of the mainstream
AI systems.
They have this mechanism for plugins, which is, I think,
like MCP, and there are other tools and skills
that you can set up.
I don't really understand all of them.
I don't have that much time to spend on,
not spending time with, I don't weird,
but they have this notion of plugins basically.
And there are tons of plugins out there,
including things that test pages from a static point of view
to understand, oh, is your HTML actually valid.
Is it reasonable HTML.
Is it semantically structured.
And also including things like browser use agents,
where basically it will spin up in the back end,
something like a Chromium browser,
and then basically use that as a way to I
don't access your pages.
So it could do that if you're hosting it locally,
if you have something like when you set up your site
and you have it hosted on a local port kind of thing
are like local server, basically local server.
Yeah So it can do that.
It can also access the live web page.
And a lot of course, depends on how you set things up.
So I basically just have a Unix Linux computer at home
that I can access where I have my stuff on it and on there.
I have this Chromium agent setup so
that it can use Chromium to access the pages.
And basically because it uses the AI system
to access your pages, it doesn't have
to know the exact IDs of the elements to click on.
It doesn't have to know the exact coordinates
to click those kind of things.
It can basically be controlled by text.
So one of the things, for example, that I did
is we like the apartment where we live.
They set up a weird rental portal website kind of thing
with I don't fancy React app, which apparently this
is how things are done now.
You don't get mail anymore.
You have to find your PDFs on a website somewhere,
and it's a really annoying thing.
And I don't know who coded it.
Maybe it's also vibe coded.
That would be weird.
But I basically used an AI system
to access that website using Chromium and to click through.
And if there is a cookie banner, just click Accept.
When it has a list of files, you should
check the files for new files and download any which are new.
And you don't have to go in and be
like, oh, Cookie, banner has this ID
and you have to click on this button with this ID
and you have to send it a click event or a post or whatever.
You basically just tell it if there's a cookie banner,
just click OK and get back to work kind of thing.
And that I think is really fascinating
where you can do these complex things that in the past
you could also do, but now you don't
have to spend as much time focusing on all the details.
For me, figuring out how to run Chromium and click
on a specific element like that would probably take a day.
I don't know.
It's like I haven't used it in a long time.
I don't have that much experience with what is it.
JavaScript, TypeScript.
I don't even know the language that's so awkward.
But I can go and tell it roughly what I want,
and it can create some code that does that.
Obviously, taking this code and creating a version
to sell to people is like completely different story.
But for me, for my personal use, it works fine.
Interesting but you would argue that you don't
have to know any technology.
You still have to know roughly what you want,
and you have to know enough to be inspecting what's going on.
I don't know.
I think maybe that's also just a bias
that I have which is hard for me to evaluate because I basically
I understand the command line tools,
I understand the basic infrastructure.
And if something goes wrong, I'm like, oh, you stupid machine,
you clicked instead of swiped kind of thing.
And you did this and did this in Chromium,
can't you try it in Firefox instead kind of thing
where I basically tell it specifically what I want,
but how much of that would just work if you told it vaguely?
And he was like, hey, this didn't work.
Try again.
I don't know.
I think if you're selling a service
and you have things like user data in a database,
then you probably want someone who
understands what they're doing.
But if you're just making something for yourself,
if you're making a static site for trying things out.
Or for a friend that just wants a website like that,
seems like the low risk kind of situation
where make something a little bit more fancy,
but it's not like I don't something
will break or burn down if you do it wrong.
Yeah for me, I wonder how much bias
is in there as well, because of course I'm a software engineer.
I to write my own stuff, but I tried specifically
to use AI for situations where I was bored enough to not like.
It's the thing I need to do, but I really
don't want to have to spend too much thought and coding on it.
So I try to counteract this bias by doing projects
where I was like, I need this thing,
but I don't want to spend time on it
and then want to offload the time.
And at first I was very impressed
how far I got very, very quickly.
What was a bit tricky for me was that I noticed at some point,
I got stuck, and I'm not sure if it's
because I used only one tool.
I might have actually tried other tools, probably.
And the other thing is, and to be fair,
that kind of tells me that my bias wasn't too big here.
I didn't really look at the code much.
So I don't know how maintainable it is, because that's one thing
that I was wondering.
Yeah I mean, I doesn't have a notion of cost of work.
It can just add code and code and patch
and put layers upon layers upon layers.
Whereas you, as someone with experience
might go, no, if I do this, if I patch it now, this quickly
and by adding this additional layer,
then that will come back later and bite me,
because I then have to deal with this additional layer.
I don't think AI has this notion.
I think that's something that definitely happens.
And you definitely see it if you understand code,
because you ask it to do something
and it's like, well, I have to do this common thing,
but I will just write my own code to do it,
rather than using your shared library that you
set up somewhere, or the existing
shared function that you already wrote last time.
It's like, oh, you need to these elements.
Let me write a custom sort function for you.
And it's like, well, you can, but this is not
what I would recommend if you wanted to work in a group
together with other people.
And you have to maintain your code for the long run.
But I find those are also the kind of things where
I can help you to refactor a little bit if you understand
what you're doing.
Oh, where I notice it, for example,
with some home automation things that I do.
I use home assistant, which uses primarily YAML files
for configuration.
It also has this whole web UI kind of thing,
but it uses YAML files.
And because of that, there are a whole bunch
of GitHub repositories with all of these YAML files.
It under, like the AI system has a lot of training material
to work on, and it basically can generate
these YAML files for you.
And I took a bunch of the things that I created over the years.
And basically told it to hey, does this still make sense.
Is this kind of modern use of the functions kind of thing.
And for some of them, it really came back with some good ideas
on how to do things differently.
So I found that kind of useful.
That is indeed nice.
So you set up your site or I guess your page or your tool
with AI Studio tool.
Yeah web app.
And did you end up publishing it or.
No that's when I hit the.
So I don't know why, but I was basically using I was like, OK,
it's working now ish.
I want to click on Publish.
And then it was like, OK, so now let's
integrate with Google Cloud.
And Google Cloud is a great platform,
except that in this case, I was like spinning up a project
and then dealing with the billing and all that stuff.
So Google Cloud is great for bigger projects,
but this was basically just HTML, JavaScript and CSS.
So I would have loved to see a range of options
where I could publish.
What I didn't think about at the time was
I could have exported all this thing as a zip file
and then just deploy it separately.
But then I have a lot of manual work again,
and then yeah, no manual work.
Yeah so the things I've started creating, I guess,
mostly test websites for me to try things out
to see how requests are made from Googlebot with
regards to all of the back and forth stuff there.
Most of them I created using Hugo as a static site generator.
Hugo I think Hugo, Hugo.
Oh, God.
Now, I don't know how to pronounce like trivial names.
Basically, I had it set up the whole setup for me
and I published it to Firebase.
Firebase hosting.
Wow OK nice.
Which has this free tier kind of setup thing.
And it also connects to Cloud functions
and has some basic database backend kind of things
that you can use.
And basically I had it create all of this website
and tried it out locally and then published it
to Firebase hosting.
And it basically told me what I need to do
is get the key here, put it in this file kind of thing
and trust that it doesn't steal your key
and hallucinate it for other people.
I don't it seemed pretty safe and I
had it set up to use GitHub as the storage place.
I think something like GitHub makes
a lot of sense for all of this stuff,
because you see all of the previous versions, which
is something that I'm super paranoid about with all
of the vibe.
Coding is like, I tell it to do something
and it's like, hey, I deleted everything and started over.
And then if you don't have a backup
of what the previous state was, it's like,
well, sucks to be you.
Yeah, maybe it works, maybe it doesn't.
So that basically worked out, I think fairly well for me.
And then it also set up the GitHub actions,
which basically mean that whenever
I make a change in the GitHub repository
where it stores all the files, it automatically publishes
it to Firebase hosting.
And this whole process kind of worked really smoothly
with regards to SEO.
The main things I looked at there
were, are these pages kind of reasonable HTML.
And they were and do they link out properly and they do.
So all of that I think mostly worked out.
Oh that's nice.
Yeah what tools did you try so far.
I initially started with something in VS Code.
I think it's Copilot.
They have some plugins for different AI systems.
But then recently, I switched to the command line tools
like Cloud code and Gemini CLI.
Oh, OK.
What I find interesting with those command
line tools is that you basically don't look at your code anymore.
You look at it on GitHub.
You see what happened there.
And if there are variables that you need to edit, setting files,
things like that do that manually,
but otherwise you interact with it primarily
by writing things out.
And I do that by writing things out in a Google Doc
and copy and paste them into the CLI system.
And then it does that.
And I find it almost like a more natural way of interacting.
And what I find interesting is both of these systems
have a notion of agents that go off and do things that
do things in parallel a little bit, that run things like tools
and other functionality in the backend.
So you can tell it like, hey, this is not working.
Can you check the logs to see what went wrong.
And in the back end it'll know.
Oh it's like you're using Firebase hosting.
The logs are hosted here.
The Project IDX you told me that already,
and then it goes off and uses the command line
tools to look at the server logs and figures
out what it needs to do.
So all of this kind of back and forth
functionality is available or is kind of happens for you
automatically.
It's a lot less of the old style use of AI
when I used it for coding, it's like you go to ChatGPT or Gemini
and it's like, hey, write me a JavaScript
snippet that does this.
And then you copy and paste it into your thing.
You basically tell the full thing
and it kind of takes care of the whole process.
Well that's quite nice.
I think with regards to maybe future looking things,
it feels like people are still deploying Drupal websites.
So yeah, it's not like these old like old school, oh my God.
It's like web is so fresh.
But it's like these old school jobs are going away.
People will continue to use Drupal and PHP and WordPress
and whatever, but it feels like a lot of the simpler things
are easily possible with this kind of vibe coded setup,
and especially if you know what you're doing,
or if you work with someone who knows what they're doing,
it's easier to make sure that it's not
like this hallucinated framework kind of thing that
looks like a website, but it's actually
just a giant GIF like could also happen or PDF kind of thing.
With all of the tooling and testing stuff around it,
I think it's possible to get reasonably far with this setup.
God, I feel like I could talk about this forever.
Good I'm so sorry for all of the listeners.
Keep going.
I learned things here.
What I found interesting is also because Gemini app CLI and Cloud
Code.
They basically use a terminal window for interactions.
You can just SSH into your website like from anywhere
and basically tell your system to update the website.
And that means you can run an SSH terminal on your phone
and you basically can use the voice to text mode
to enter stuff.
And you tell it what to do and it will update your website
like that.
I used this recently on a flight to the US.
There was Wi-Fi and I was like Wi-Fi is kind of spotty.
And doing something with a remote desktop
kind of environment was super annoying and slow and flaky.
But in a terminal window, doesn't use a lot of bandwidth.
You can do reasonable things and try things out.
That's true.
So I don't like I also saw some photos of this developer office
where people just have microphones and are whispering
to your computer kind of thing, which I don't know that feels
kind of weird, but it's definitely
an interesting way of coding when you don't have
to think so much about, what is the exact parameter
order for this function in JavaScript that I think exists.
Or maybe it's in Python.
I don't know where you basically tell it to add functionality,
and then you think about it a little bit more
as what do I want my website to do.
Rather than oh, it's like, which functions do I need to write.
True And the other thing that I'm seeing
is especially from the experiences that I've seen
and what you just said, whenever you are starting a project
and agencies tend to do that, a new customer comes or customer
comes with a new project, you have
this setup work even if you just set up Drupal or whatever,
and you might have some templates,
but then each project is oh yeah, that template fits.
But then we need to make these seven different changes.
And that just takes time and is not creative.
Interesting work.
It's tedious most of the time.
If I can do that and walk you through that process
quite quickly, then that would be really, really helpful.
I think you could probably do this
in minutes instead of hours.
And then when it becomes interesting
where the more subtle things happen,
I think that's where human time becomes valuable again.
I think that's yeah.
Yeah, I think for the Search Central audience,
being able to spin up websites quickly to try things out
is super helpful.
Making your own kind of web based tools is super easy to do.
If you want to see if your website is crawlable,
you just have it create a crawler and you try it out.
These things are super easy to do with regards
to agentic access, where it's like I don't maybe it goes off
and accesses Search Console using the API thing.
I don't know how that will evolve or any of it's like, oh,
look at Google Trends and then write me some articles
about the top trends.
That just feels like spam at that point.
But there's a lot of work to even just generate,
like a mock of a website or a mock of certain pages.
And a lot of that can be made it a lot easier
when you notice certain workflows that you keep
having to do where you need to take a data piece of data
in one tool, and then from that one tool to the next tool,
and then from that tool to the next.
You can probably pipeline this with AI as well.
Yeah which is what I sometimes do.
Yeah interesting.
Cool thank you so much, John.
I think that was really, really insightful
and especially hearing your experience with different tools
and how that related to the experience
that I just had was interesting.
I would like to hear from our audience in the comments
as well.
What tools are you using.
What have you found.
Where do they fall short of the promised land, which
is just whisper into a computer and the right thing falls out.
And where have they surprised you.
Positively I'm just curious to hear what people out there have
been experiencing.
Yeah all right.
Sounds cool.
Thank you so much for being here, John.
Thank you so much for talking with me.
And thanks to all of you out there for listening.
I'd like to say goodbye and wiedersehen.
Bye we've been having fun with these podcast episodes.
I hope you, the listener, have found
them both entertaining and insightful too.
Feel free to drop us a note on LinkedIn
or chat with us at one of our next events we go to.
If you have any thoughts, let us know.
And of course, do not forget to and subscribe.
Thank you so much for listening and goodbye.