This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

As artificial intelligence (AI) becomes increasingly capable of assisting with health care tasks, a new study by researchers at the Icahn School of Medicine at Mount Sinai has found that adding a brief safety reminder reduced potentially harmful choices by AI models in clinical scenarios.

The study, published Sept. 26 in Communications Medicine, also found that AI models can be influenced by the context and instructions surrounding a clinical decision.

The findings, based on millions of outputs, suggest that how AI systems are prompted and guided may remain an important consideration in developing safe and reliable clinical applications, even as increasingly capable AI models and agents become better able to understand users' intent without lengthy or comprehensive prompts.

The reminder reduced potentially harmful choices across 19 of the 20 models tested, demonstrating a potential approach to strengthening safeguards.

The research team evaluated 20 large language models using 501 variations of 50 clinical scenarios, along with 100 cases adapted from deidentified hospital discharge records. Across more than 10 million responses, the models made approximately 1.18 million potentially harmful clinical choices. Without a safety reminder, potentially harmful choices accounted for 16.6% of model responses. Adding a brief safety reminder reduced that rate to 10.1%.

The findings, the researchers say, highlight the importance of evaluating not only whether an AI model can provide accurate clinical information but also how it responds when given an instruction that conflicts with patient safety.

Source link