This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

Large language models (LLMs), such as ChatGPT and Claude, are increasingly being used to provide health information to the public. Although this is beneficial for improving access to treatment information, their limitations were revealed by a new study that found LLMs were biased in their treatment choices and that using these outputs in practice could drive up treatment costs.

A research team led by professor Daiju Ueda at Osaka Metropolitan University's Graduate School of Medicine, in collaboration with the University of Texas MD Anderson Cancer Center, asked five LLMs (GPT-5.4, Claude Sonnet 4.6, Gemini 3.1 Flash, Llama 4 and MedGemma) to recommend up to three FDA-approved drug products for a range of simulated scenarios.

Although an average of 5.5 FDA-approved products were available for each specified active ingredient, a single product accounted for an average of 97.5% of first-choice recommendations across the tested scenarios and models. The study is published in Nature Health.

"The recommendations were mostly for brand-name drugs, those highly visible in Google searches and treatments with dedicated Wikipedia pages," Ueda said.

The LLMs also tended not to recommend products with later FDA approval and those more commonly prescribed in actual physician practice. The study also identified recommendations that were inconsistent with regulatory information, including discontinued products.

S. data. They looked at a scenario in which physicians prescribed drugs in accordance with the LLMs' recommendations.

Source link