Artificial intelligence tools like ChatGPT struggle to distinguish belief from fact, a new study has revealed.
A team from Stanford University in the US found that all major AI chatbots failed to consistently identify when a belief is false, making them more likely to hallucinate or spread misinformation.
The findings have worrying implications for the use of large language models (LLMs) in areas where determining between true and false information is critical.
“As language models (LMs) increasingly infiltrate into high-stakes domains such as law, medicine, journalism and science, their ability to distinguish belief from knowledge, and fact from fiction, becomes imperative,” the researchers noted.
“Failure to make such distinctions can mislead diagnoses, distort judicial judgments and amplify misinformation.”
The researchers evaluated 24 LLMs – including Claude, ChatGPT, DeepSeek and Gemini – using 13,000 questions to analyse their ability of distinguishing between beliefs, knowledge and facts.
All models tested failed at recognising false beliefs and statements, demonstrating a fundamental limitation in being able to link knowledge to truth.
“These findings expose a structural weakness in language models: their difficulties in robustly distinguishing between subjective conviction and objective truth depending on how a given assertion is formulated,” said Pablo Haya Coll, a researcher at the Computer Linguistics Laboratory of the Autonomous University of Madrid, who was not involved in the study.
“Such a shortcoming has critical implications in areas where this distinction is essential, such as law, medicine, or journalism, where confusing belief with knowledge can lead to serious errors in judgement.”
One possible solution to this shortcoming, according to Dr Coll, could be to train the model to be more cautious in its responses. While this may reduce the likelihood of hallucinations, it may also affect their usefulness.
The Stanford researchers called for tech companies developing AI tools to “urgently” improve the models before deploying them in high-stakes domains.
The findings were detailed in a study, titled ‘Language models cannot reliably distinguish belief from knowledge and fact’, which was published in the scientific journal Nature Machine Intelligence on Monday.
© Independent Digital News & Media Ltd