🕒 Created · Updated

AI Chatbots Successfully Debunk Most State-Sponsored False Narratives According to NPR and NewsGuard Study

AI chatbots generally perform well at identifying and refuting false narratives spread by foreign governments, according to a study by NPR and NewsGuard. Researchers tested popular models, including OpenAI's ChatGPT and Google's Gemini, against 30 questions based on disinformation from China, Iran, and Russia. On average, the chatbots correctly debunked these false narratives about 75% of the time. While chatbots outperformed traditional search engines, AI-generated summaries at the top of search results showed mixed performance. Google's AI Overview performed well, while Microsoft's Bing summaries failed to debunk most narratives. Experts noted that while AI tools are useful for investigating issues, users should still verify primary sources, as some AI responses can still be unsupported by their citations. The study suggests that chatbots can serve as a counterbalance to social media misinformation. Researchers found that even when prompted with language from election deniers, the bots remained mostly truth-tellers. However, the performance of AI tools is tied to the availability of factual information online, and models may be more prone to errors when reliable sources are sparse.

Sources

Paywall and unreadable sources