Could AI Cause ‘Knowledge Collapse’? Study Finds A Troubling Pattern

Every AI Chatbot Tested Offered Less Variety Than a Google Search, Study Finds

Ask a chatbot the same question enough different ways, and the biggest AI models start doing something strange: they hand back less variety, not more. A new study covering 27 popular AI systems and 70 million individual statements found that larger language models tended to produce narrower, more repetitive information than smaller versions of the same models, even though bigger models are generally built to perform better.

That finding lands at an awkward moment. Search engines are increasingly steering people toward AI-written summaries instead of a list of links, and there is speculation that people will soon get most of their information through an AI middleman. Researchers from universities in Denmark and the United States wanted to know what happens to the diversity of information when that middleman does the talking. Their paper, posted to arXiv, an open site where researchers share papers, tests a concern known as “knowledge collapse,” where the range of ideas people can access shrinks into a narrow set of views.

Results came back mixed. Diversity in AI answers has improved over the past few years, which pushes back against the gloomier predictions. But every system tested, no matter how new or advanced, still produced less varied information than simply reading the top results of a Google search.

Researchers Measured AI Diversity Across 27 Models and 70 Million Claims

To measure how much variety shows up in AI answers, the research team built what amounts to a giant claim-sorting assembly line. They tested AI systems from four families, Llama, Gemma, Qwen, and OpenAI’s GPT models, ranging from small systems with about a billion internal settings to large ones with more than 70 billion. Each model answered questions on 155 topics, from broad subjects like democracy, free speech, and nuclear weapons to history and public figures tied to 12 countries, including the United States, India, Russia, Brazil, and Saudi Arabia. Country-specific picks ranged from the Berlin Wall to K-pop.

Every model answered each topic using 200 different human-written prompt styles, such as “write me an essay about” a given subject. That produced 1.7 million individual responses. An AI model then broke each response into individual factual statements, and human reviewers checked samples of that work. Statements making the same basic point were grouped together, so sentences all saying, for example, that democracy involves free elections counted as one core piece of information instead of many.

Diversity was scored with a formula ecologists use to count species in a habitat and how evenly the population is spread among them. Applied here, it rewards a model for covering many distinct points rather than repeating the same few. As a point of comparison, the researchers also pulled the top 40 Google search results for each topic.

Every AI Model Tested Trailed Web Search on Diversity

Not every model family improved, with Qwen the main exception. Google’s Gemma 3 and OpenAI’s GPT-5 posted the biggest jumps between generations, a sign that recent updates from those companies are paying off. On the study’s topics, GPT-5 offered more than five times the variety of an earlier OpenAI model, GPT-4o.

So how big is the gap? Search results were at least 18.7 percent more varied than the best-performing model, GPT-5. Because the researchers built the search comparison to be conservative, the real gap is probably wider.

One technique helped. Called retrieval-augmented generation, it lets an AI pull information from current web pages before answering instead of relying only on what it absorbed during training. Answers got noticeably more varied, but the boost was uneven. Topics tied to the United States, India, Russia, France, and China gained the most, while other countries gained less. The researchers think their U.S.-based search tool may have under-represented information specific to other countries.

Larger AI Models Produced Less Varied Answers Than Smaller Ones

Model size produced one of the odder results. After accounting for other factors, larger models were linked to less diversity, meaning smaller versions within the same model family tended to generate more varied claims than their bigger siblings. The researchers have a possible explanation: bigger models may memorize more of their training data, which could narrow what they say when asked about the same topic again and again.

AI Answers Leaned on English-Language Knowledge in Five of Eight Countries

A separate test asked whose knowledge shows up in AI answers. The researchers matched AI claims about country-specific history and public figures against English-language Wikipedia pages and against the same topics in each country’s own language, using eight countries. Prompts were written in English. For five of the eight countries, AI answers reflected the English-language pages clearly more than the local-language ones, and in no country did local-language material come out clearly ahead. Topics tied to the United States were represented more strongly than topics tied to any other nation.

Progress is real, but it has not reached everyone equally. As search increasingly hands people AI-written overviews, it matters that AI closes the gap with web search and gives local knowledge a fairer share of the answers. Otherwise, a tool meant to broaden what people know could narrow it without anyone noticing.

---