Google’s AI search engine is batsh*t crazy

If you want to know how to get the island of cheese on your homemade pizza to stay afloat atop that sea of red sauce, mixing a little Elmer’s Glue into the sauce is not the answer. Yet that’s what Google’s new AI-powered search came up with when asked this extremely vital question.

That bit of cooking wisdom actually came from an 11-year-old post on a Reddit by someone with the online handle “f*cksmith.” 

Thus we learn three things:

1. Elmer’s Glue is a lot more versatile than you thought.

2. Anyone calling themselves “f*cksmith” is possibly not the most authoritative source of culinary advice.

3. Google’s AI Overview isn’t very good.

AI Overview (formerly known as Search Generation Experience, or SGE) is supposed to take the superpowers of generative AI — in this case, Google’s own foundational chatbot, Bard Gemini — and blend them with Google's mighty search prowess to answer all the world’s questions from a single screen. 

It is not going as planned. Despite being in public beta for more than a year, AI Overview is still suffering from a bad case of the whoopsies. 

For example: A semi-recent Washington post article notes that searching for a fictional restaurant in San Francisco — Danny’s Dan Dan Noodles [1] — produced an authoritative-sounding summary that suggested the restaurant “had long lines and crazy wait times,” including a two-year-long, 4,000-deep waitlist.

As you can see from the screen if you hold up a magnifying glass and squint, the AI Overview summary on top clearly draws bits and pieces from the sources listed immediately below it. The SF noodle joint with the 4,000-person waitlist is actually Laowai Noodles, cited in the SF Chronicle story in the middle. [2] The name of the fictional restaurant is actually the name of a dish (as you can see from the description at the bottom of the screen), and the “crazy wait times” line came from a Yelp review of another restaurant that serves Dan Dan Noodles. [3]

It’s a classic AI hallucination, something I kind of assumed we’d gotten past, but apparently not. This is how generative AI works — it takes a little information from Websites A, B, and C, then mashes them up into something that sounds plausible, if you don't look too hard at it (or try to make a reservation at a restaurant that only exists inside Google AI's tiny robot brain).

AI Overview is like an absent-minded kleptomaniac, constantly stuffing items into its pockets when it thinks no one is watching, and then can’t remember who it stole them from.

The angel of derp

Google’s own pitch for the product, which it officially introduced at its annual I/O Conference last week, is “Let Google do the Googling for you.” Seriously. It’s like the AI version of Here, Let Me Google That For You, a circa 2008 site designed as a “passive-aggressively fun way teach your friends [how to] use Google.” 

The good news is that Google’s AI search is much better than it was last August, when Gizmodo asked it to produce a summary of “the benefits of slavery:”  

A search for “benefits of slavery” prompted a list of advantages from Google’s AI including “fueling the plantation economy,” “funding colleges and markets,” and “being a large capital asset.” Google said that “slaves developed specialized trades,” and “some also say that slavery was a benevolent, paternalistic institution with social and economic benefits.”

You’d think Google AI was vying to be a certain candidate’s running mate, or possibly governor of Florida. 

Per Gizmodo, Google AI also talked about the positive effects of genocide, guns, and banning books, while offering up the best ways to prepare “Amanita ocreata, a poisonous mushroom known as the ‘angel of death’.” 

I suppose that’s better than telling people to inject bleach, but not by a lot.

Search me

Since then, after getting smacked around in the media by people like me (only younger and better looking), Google has gotten a lot more careful about where and when it shows the AI Overview. A search for "Are Jews the chosen ones?" produces the following AI snippet:

A search for "Are Jews taking over the world?" defaults to standard Google search results (minus the ads) about anti-semitic conspiracies. You can get an AI Overview for "How to build a business," but not one for "How to build a bomb," one for "Are puppies adorable?" but not for "Are Nazis deplorable?"

Still, the reputational damage has been done. Google’s AI search results are “ridiculous,” "misleading," and “horribly wrong,” not to mention offensive and possibly dangerous. The question is, why? How did people this smart screw up this badly?

The answer could be, maybe they’re not as smart as we thought they were. Or perhaps more accurate, generative AI is not as smart as we think it should be. As SEO maven Patrick Plestid [5] notes in an essay on why Google and Bing’s AI search efforts have failed:

Generative AI fundamentally operates like bad SEO [search engine optimization]. It takes widely available information, repackages that information into familiar, commonly used structure, and rephrases the wording so that it sounds original. 

As I noted in a recent post, bad SEO is bad for everybody except the soulless wraithes who make serious coin from it. It's turning the web into sludge. And now AI is doing that heavy lifting for them. 

AI Overview and things like it may one day replace traditional search engines for people who crave fast answers to simple questions. But I'm not sure that the rest of us want Google doing our Googling for us. And you definitely do not want to take cooking advice from a robot, lest it suggest adding spackle to thicken your Arrabbiata sauce.  

How do you keep the cheese from sliding off the top of your pies? Offer your suggestions in the comments below or email me: [email protected].

[1] I am unaffiliated with Dan’s Dan Dan Noodles, but I want to eat there. 

[2] I definitely want to eat at Laowai Noodles. If I put my name on the list now, maybe I’ll manage to get in before I am dead.

[3] Searching on that name now turns up zero AI Overview results, just recommendations for local Asian restaurants along with a couple of machine-generated answers to questions about AI Overview. Think maybe the Post story touched a nerve?

[4] Remember when Google knew everything? 

[5] His bio page contains the following: "Patrick Plestid is a Senior SEO Manager at Session Interactive. He's been called 'the Pablo Picasso of Google Sheets formatting,' 'the Marie Kondo of messy website architecture,' and 'the Don Draper of 'best SEO content strategist for business.' His boss regrets letting him write his own bio." No notes.