The Digital Depth Dilemma: Google Searching vs. ChatGPT for Learning
In the rapidly evolving landscape of digital information gathering, researchers have discovered a significant difference in how we learn when using traditional search engines versus AI chatbots. A groundbreaking study published in October’s PNAS Nexus reveals that despite the convenience of large language models (LLMs) like ChatGPT, people who use conventional search engines like Google develop deeper knowledge and understanding of subjects they research.
The comprehensive study, led by Shiri Melumad, a consumer psychology researcher at the University of Pennsylvania, involved more than 10,000 participants across seven carefully designed experiments. Participants were randomly assigned to research various topics—ranging from vegetable gardening to healthy lifestyle choices—using either Google or ChatGPT. After gathering information, they were asked to write advice for a friend based on what they had learned. The researchers then evaluated how much the participants had genuinely absorbed and how invested they were in the advice they provided. Melumad explains the significance of this research: “LLMs are fundamentally changing not just how we acquire information but how we develop knowledge. The more we learn about their effects—both their benefits and risks—the more effectively people can use them, and the better they can be designed.”
What makes this study particularly compelling is that the results remained consistent even when controlling for the information available. Even when participants were provided with identical sets of facts through simulated interfaces, those who accessed information through web searches demonstrated deeper knowledge than those who relied on chatbot summaries. The researchers used multiple methods to evaluate the depth of knowledge, including participant self-reporting, natural language processing tools, and assessments by independent human judges. The findings clearly showed that participants who used search engines developed a more thorough understanding of their topics compared to those who relied on AI chatbots for quick answers.
Beyond just measuring knowledge depth, the researchers discovered additional concerning patterns among LLM users. Participants who learned via ChatGPT showed less investment in the advice they provided, created less informative content, and were less likely to adopt the advice for themselves compared to those who conducted web searches. Perhaps most telling was an experiment featuring a version of ChatGPT that provided optional links to original sources—only about 25% of the approximately 800 participants in that experiment were motivated to click on even a single link. This suggests that the convenience of receiving pre-packaged information may discourage people from engaging more deeply with a subject.
Melumad attributes this difference to how we process information: “While LLMs can reduce the load of having to synthesize information for oneself, this ease comes at the cost of developing deeper knowledge on a topic.” This points to a fundamental cognitive trade-off—when we outsource the work of information synthesis to an AI, we miss out on the mental processing that leads to deeper understanding and retention. The researcher also suggests that more could be done to design search tools that actively encourage users to dig deeper rather than settling for surface-level summaries.
Not all experts interpret these findings in the same way, however. Daniel Oppenheimer, a psychologist from Carnegie Mellon University, while praising the research, suggests a different framing. Rather than concluding that people who synthesize information themselves gain deeper understanding, he believes it’s more accurate to say that “LLMs reduce motivation for people to do their own thinking.” Oppenheimer cautions against abandoning these useful tools entirely, emphasizing that “like all learning, the effectiveness of the tool depends on how you use it. What this finding is showing is that people don’t naturally use it as well as they might.” This perspective suggests that ChatGPT and similar tools can still be valuable for learning—if we approach them with the right mindset and supplement their use with additional engagement and critical thinking.
