CONNECT WITH US
Tech

Tech

"You’re right"—LLMs will get climate info wrong if you ask

Ars Technica - Tech logo

Published on

Add as a preferred source on Google
"You’re right"—LLMs will get climate info wrong if you ask

Many studies over the years have examined how people encounter climate science information. Some focused on the top results that Google surfaced (back when Google was still in the business of serving links), while others examined social media content or newspaper and TV news coverage. With large language models (LLMs) now forming a share of the public information diet, it's worth looking at how they handle climate change.

But there’s no direct way for researchers to look at the information people have seen. It is served ephemerally to individual users, confined to their chat session. A study led by researchers from University College London and the United Nations Educational, Scientific, and Cultural Organization (UNESCO) tried to tackle this question from multiple angles, testing LLMs with different prompts and evaluating their output. They started with benchmarks of simple factual questions (benchmarks that might even be in models’ training data at this point) but also analyzed more complex interactions—including when users think they’re right to push back on an answer.

LLM research often faces (no-true-Scotsman-tinged) criticism for focusing on a particular model rather than some other frontier model that may have been released after the experiment concluded. In this case, the study used 33 models, including multiple versions of the major frontier and open-weight models. The flagship frontier models used were Claude Fable 5, Claude Opus 5, Gemini 3.1 Pro Preview, Gemini 3.6 Flash, and GPT-5.6 Sol. (You can find the full list in the report.)



Source link

Disclaimer

We strive to uphold the highest ethical standards in all of our reporting and coverage. We TheMorningPulse.fyi want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It's possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.