Google’s AI Overviews: Insights and Challenges

Google has recently introduced AI Overviews, a new feature designed to generate written answers for specific search queries by summarizing online information using large language models (LLMs) similar to OpenAI’s ChatGPT. The aim of this feature is to enhance user experience by providing concise, informative answers directly on the search results page, minimizing the need for users to click through multiple links.

However, this innovation has not been without its challenges. The AI Overviews have faced criticism for giving incorrect and misleading information. Among the most notable errors were suggestions that people eat rocks and apply glue to pizza—missteps that can be traced back to the AI pulling information from satirical articles and jocular comments found on platforms like Reddit. These errors highlight the current limitations of relying heavily on user-generated content and the difficulties in distinguishing factual information from humor or satire.

In response to these issues, Google has acknowledged the problems and has taken proactive steps to improve the reliability of AI Overviews. Enhancements have been made in detecting nonsensical queries and reducing the dependency on potentially unreliable user-generated content. Concurrently, other players in the industry are dealing with similar challenges. Richard Socher, founder of You.com, emphasizes the importance of significant effort to ensure that LLMs do not disseminate harmful or inaccurate information. You.com, for instance, uses a custom-built web index and a citation mechanism to keep its LLMs in line.

Meanwhile, Google’s generative AI upgrade reflects a broader industry trend inspired by OpenAI’s release of ChatGPT in November 2022. Following this development, Microsoft has also enhanced its search engine, Bing, with comparable technology. This wave of AI adoption highlights the industry’s commitment to improving user experience yet underscores the shared challenges across these platforms in maintaining information accuracy and quality.

Experts in the field have voiced their concerns as well. Barry Schwartz has pointed out that Google should have foreseen the potential misuse and acted more cautiously, especially given that these AI-generated answers are displayed as default on one of the company’s most valuable products. Additionally, Lily Ray, a search engine optimization consultant, notes that AI search can provide summaries drawn from poor sources or defunct websites, which can lead to less useful information being presented to users.

As part of ongoing refinement, Google’s head of search, Liz Reid, announced that the company is scaling back on AI-generated overviews due to their unhelpful responses to queries. Efforts are being made to improve algorithms and to remove responses that do not adhere to Google’s policies. User feedback is also actively sought to continually refine the accuracy and usefulness of the AI Overviews. For those who prefer to disable this feature, there are multiple workarounds, like using the Web tab, switching browsers, or employing the Hide Google AI Overviews extension for Chrome.