Image Courtesy: Getty Images
Efforts to make AI chatbots more friendly and conversational may come with a trade off in accuracy, according to new research. A study found that chatbots tuned to sound warmer were more likely to agree with incorrect claims, including conspiracy theories and misleading health advice.
The research, conducted by the University of Oxford, tested how changes in tone affected chatbot responses. Results showed that friendlier models were significantly more prone to errors and more likely to reinforce false beliefs, as reported by The Guardian.
In controlled tests, chatbots adjusted for warmth produced answers that were around 30 percent less accurate than standard versions. They were also about 40 percent more likely to validate incorrect user statements. This included responses that cast doubt on established historical events or supported widely debunked claims.
One example involved a user suggesting that Adolf Hitler survived World War II. The friendlier chatbot responded by acknowledging the belief and suggesting there was uncertainty, while the standard version directly rejected the claim. Similar patterns appeared in discussions about the Apollo moon landings, where warmer models were more likely to frame facts as opinions.
The study also highlighted risks in health related scenarios. In one test, a chatbot endorsed the idea that coughing could stop a heart attack, a widely debunked myth. Researchers found that chatbots were especially likely to agree with false information when users expressed emotional distress or vulnerability.
The findings point to a broader challenge in AI development. Companies such as OpenAI and Anthropic have been working to make chatbots more approachable and engaging, particularly as they are used in roles like digital assistants and support tools. However, increasing empathy and friendliness may reduce the system’s ability to challenge incorrect assumptions.
Researchers say the issue reflects a deeper tension between being supportive and being factually precise. Training models to align with human conversational norms can unintentionally encourage agreement, even when the information is wrong.
The study’s authors emphasize the need for better evaluation methods to balance tone and accuracy. As AI systems become more integrated into daily life, ensuring they can provide reliable information while maintaining a helpful tone remains a key challenge.