With midterm elections just days away, you might be tempted to ask ChatGPT, Gemini, or Perplexity about a candidate’s record before heading to the polls. But a new study suggests you should think twice before trusting the answer, even when the chatbot provides sources to back it up.

A report from AI search analytics company Searchable found that roughly one in 12 answers about local political issues contradicted official government data. Of the 7,449 answers researchers evaluated, 610 were false, an 8.2% error rate.

The research examined four popular AI tools: ChatGPT, Gemini, Google AI Mode, and Perplexity. The questions covered issues voters regularly consider when evaluating politicians, including crime, housing costs, employment, education and healthcare.

But the most surprising finding wasn’t necessarily how often the chatbots got things wrong. It was that they sometimes produced incorrect answers despite consulting reliable government sources.

The researchers found significant differences in how the four AI tools performed.

Perplexity produced false answers in 22.6% of the tested responses, compared with 6.2% for Gemini and 4.4% for ChatGPT. Google AI Mode had the lowest error rate at 2.9%.

In other words, more than one in five Perplexity answers evaluated in the study contradicted official government data.

Errors also varied by subject. Questions about housing and the cost of living produced the most false answers overall, at 11.9%, followed by energy and environmental issues at 10.9% and schools at 10.1%.

Crime and public safety questions had the lowest aggregate error rate, at 4.7%.

Source link