AI program provided hallucinated notes to Ontario doctors, report says
· Toronto Sun
An auditor general report into the use of an artificial intelligence program by Ontario doctors in recent years found it hallucinated and made many mistakes in the notes it provided.
The report released Tuesday by auditor general Shelley Spence said AI Scribe, a program that purportedly relieves pressure on physicians to take notes while speaking with a patient, led to many mistakes including incorrect and incomplete information as well as “AI hallucinations.”
Visit saltysenoritaaz.com for more information.
The report said “hallucinations” occur when AI systems generate information that is made up, fabricated or not based on any data provided to it.
Since 2023, physicians in the province can use the technology only if patients consent.
Program creates SOAP note
Once authorized, the program listens to the conversation between a patient and doctor and compiles the information into a SOAP (subjective, objective, assessment and treatment plan) note.
However, Spence noted the types of inaccuracies found in the notes generated by AI found incorrect information in 12 of 20 vendors, hallucinations in nine of 20 and incomplete information in six of 20.
The inaccuracies “captured a different drug than what was prescribed by the doctor,” the report said.
Even more alarming were the AI hallucinations. The report found AI Scribe systems “fabricated information and made suggestions to patients’ treatment plans, such as referring the patient to therapy or ordering blood tests, even though these steps were not mentioned in the simulated recordings.”
Another example included statements that there were “no masses found” or that a patient had issues with anxiety, although it was never discussed.
RECOMMENDED VIDEO
Concerns about privacy risks
AI notes that generated missing or incomplete information occurred 85% of the time, which included key details that were omitted about the patients’ mental-health issues.
Another concern was the risk of loss of privacy from the potential exposure of patient information when the AI tool is used.
“When Ontarians see their doctor, they need to share intimate information about their health, their bodies and their personal lives to receive proper care,” the report said.
“Ontarians expect this extremely personal information to be kept private and confidential. Using AI to assist in providing health care must not come at the cost of compromising privacy.”