Lately, Google’s latest AI feature, AI Overviews, has been grabbing attention for all the wrong reasons. This tool, powered by large language models (LLMs), aims to offer concise summaries of search queries. However, it has come under fire for generating highly inaccurate responses. One alarming example was its recommendation for users to use glue on pizza to stop the cheese from sliding off—a culinary blunder waiting to unfold.
In a recent interview with Google CEO Sundar Pichai by The Verge, just before the criticism of AI Overviews gained traction, Pichai acknowledged these errors as “hallucinations” inherent in LLMs. He admitted that this issue remains an unsolved problem, despite efforts by Google engineers to rectify it. This admission suggests that users should brace themselves for more bizarre and erroneous outputs from AI Overviews in the future.
“There are still times it’s going to get it wrong, but I don’t think I would look at that and underestimate how useful it can be at the same time,” he said. “I think that would be the wrong way to think about it.”
“Are we making progress? Yes, we are,” he added. “We have definitely made progress when we look at metrics on factuality year on year. We are all making it better, but it’s not solved.”
Despite the glaring flaws, Pichai appears to downplay the significance of these errors. He emphasizes the utility of AI Overviews while acknowledging its shortcomings. Pichai asserts that although mistakes will happen, it’s crucial not to underestimate the tool’s overall usefulness. However, his optimism contrasts sharply with the public’s reaction, which has been marked by outrage and ridicule over the misinformation propagated by AI Overviews.
The fallout from these blunders is tarnishing Google’s flagship product, Search, which has already faced criticism for delivering subpar results. Observers online have been quick to highlight instances of AI Overviews generating blatantly false information. This backlash underscores the growing dissatisfaction with Google’s handling of search queries and raises questions about the company’s ability to compete in the realm of AI against formidable rivals like Perplexity and OpenAI.
“People expect AI to be leaps & bounds more accurate than traditional methods… but it’s often not!” writes AI consultant and SEO expert Britney Muller on X. “Google’s playing a risky game competing against Perplexity & OpenAI, when they could be developing AI for bigger, more valuable use cases beyond Search.”
Pichai’s remarks suggest that both Google and the wider online community are facing a tumultuous path ahead as they confront the complexities and constraints of AI technology. Although strides have been taken to enhance the precision of AI-generated content, significant hurdles lie ahead. The pursuit of genuinely dependable AI solutions for search functions is still ongoing, with numerous challenges to overcome.