More and more people are asking AI models for medical advice, but two new studies show that they are unreliable, according to a report by the Wall Street Journal. In one study, five general-purpose AI models were asked to answer 250 medical questions. They failed 50% of the time, and one in five responses was judged to be “potentially dangerous.”
Another study found that general-purpose AI cannot follow the complex reasoning process doctors are trained to use. AI makes “diagnoses” confidently despite having insufficient information, and was wrong 80% of the time because it does not ask for more details. By contrast, doctors request additional information and order tests before reaching a conclusion.
In 2025, one in four Americans used AI for health information, while 14 million did not visit a doctor because they relied on advice from AI.
In 2025, about 25% of Americans used AI for health information, with the rate significantly higher among young adults aged 18–29, reaching 69%, according to a survey by the West Health-Gallup Center. At the same time, 14 million people (14%) said they did not go to a doctor because they relied on AI advice.
It is worth noting that several AI companies are working to improve their capabilities in the healthcare sector or launch specialized AI applications for medical issues. In addition, many of the models evaluated have been upgraded since the studies were conducted, which may improve their performance.
Source: Wall Street Journal
What is fyi.news?
