This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

Artificial intelligence tools are being touted as a solution for so-called pajama time—the time doctors inevitably spend compiling patient notes after working hours. As hospitals increasingly deploy AI scribes to listen and take notes during patient encounters, the data the scribes produce becomes the scaffold on which insights about health care, from billing to research to clinical decision-making, are built.

This rapidly changing infrastructure is the basis of a recent article written by Taofeeq Oluwatosin Togunwa, M.D., and Jodyn Platt, Ph.D., of the Department of Learning Health Sciences at University of Michigan Medical School, published in the journal Learning Health Systems.

"It used to be that the doctor or clinician taking the notes was the filter for data. Now a machine is in the pipeline deciding what does and does not become data," said Platt.

Often, these systems are proprietary with opaque training data. And what happens if the AI is not trained on a patient's demographic, wondered Togunwa.

"Usually, these models are trained on English and American accents and then they perform well within that distribution. But you find concerning literature where these models actually don't perform as well with people that might have different types of accents," he noted.

The ensuing hallucinations—fabrications introduced by AI when inputs do not match training data—then end up as part of the official data set.

And while clinicians are supposed to review machine-scribed notes for accuracy, "there isn't a lot of incentive to go line by line if the note looks generally fine," he said.

Source link