Using artificial intelligence to supplement radiologists’ evaluations of mammograms may improve breast cancer screening by reducing false positives without missing cases of cancer, according to a new study.
The researchers developed an algorithm that identified normal mammograms with very high sensitivity. They then ran a simulation on patient data to see what would have happened if all of the very low-risk mammograms had been taken off radiologists’ plates, freeing the doctors to concentrate on the more questionable scans.
The simulation revealed that fewer people would have been called back for additional testing but that the same number of cancer cases would have been detected.
“False positives are when you call a patient back for additional testing, and it turns out to be benign,” explains senior author Richard L. Wahl, a professor of radiology at Washington University’s Mallinckrodt Institute of Radiology (MIR) and a professor of radiation oncology.
“That causes a lot of unnecessary anxiety for patients and consumes medical resources. This simulation study showed that very low-risk mammograms can be reliably identified by AI to reduce false positives and improve workflows.”
Wahl previously collaborated with Whiterabbit.ai on an algorithm to help radiologists judge breast density on mammograms to identify people who could benefit from additional or alternative screening. That algorithm received clearance from the Food and Drug Administration (FDA) in 2020 and is now marketed by Whiterabbit.ai as WRDensity.
In the new study, published in the journal Radiology: Artificial Intelligence, Wahl and colleagues at Whiterabbit.ai worked together to develop a way to rule out cancer using AI to evaluate mammograms. They trained the AI model on 123,248 2D digital mammograms (containing 6,161 showing cancer) that were largely collected and read by Washington University radiologists. Then, they validated and tested the AI model on three independent sets of mammograms, two from institutions in the US and one in the UK.
First, the researchers figured out what the doctors did: how many patients were called back for secondary screening and biopsies; the results of those tests; and the final determination in each case. Then, they applied AI to the datasets to see what would have been different if AI had been used to remove negative mammograms in the initial assessments and physicians had followed standard diagnostic procedures to evaluate the rest.
For example, consider the largest dataset, which contained 11,592 mammograms. When scaled to 10,000 mammograms (to make the math simpler for the purposes of the simulation), AI identified 34.9% as negative. If those 3,485 negative mammograms had been removed from the workload, radiologists would have made 897 callbacks for diagnostic exams, a reduction of 23.7% from the 1,159 they made in reality. At the next step, 190 people would have been called in a second time for biopsies, a reduction of 6.9% from the 200 in reality.
At the end of the process, both the AI rule-out and real-world standard-of-care approaches identified the same 55 cancers. In other words, this study of AI suggests that out of 10,000 people who underwent initial mammograms, 262 could have avoided diagnostic exams, and 10 could have avoided biopsies, without any cancer cases being missed.
“At the end of the day, we believe in a world where the doctor is the superhero who finds cancer and helps patients navigate their journey ahead,” says coauthor Jason Su, cofounder and chief technology officer at Whiterabbit.ai.
“The way AI systems can help is by being in a supporting role. By accurately assessing the negatives, it can help remove the hay from the haystack so doctors can find the needle more easily.
“This study demonstrates that AI can potentially be highly accurate in identifying negative exams. More importantly, the results showed that automating the detection of negatives may also lead to a tremendous benefit in the reduction of false positives without changing the cancer detection rate.”
This work was supported by funding from Whiterabbit.ai Inc.
Washington University has equity interests in Whiterabbit.ai Inc. and may receive royalty income and milestone payments from a “collaboration and license agreement” with Whiterabbit.ai to develop a technology evaluated in this research. Wahl is the principal investigator on a research contract from Whiterabbit.ai. These agreements are managed by the Washington University Institutional Conflict of Interest Committee.
