Can Dr. Oz Really Replace Your Doctor With AI?
Let's see how well "virtual doctors" are doing so far.
As part of the ongoing firehose of drama coming out of the Trump administration, a new precious gem of an idea: New CMS administrator Dr. Oz is going to advocate for “AI Avatar” doctors.
How close are we, then, to autonomous AI functioning independently as a clinician? Well, one glimpse comes from one of the companies that could potentially be a candidate to create such virtual clinicians – and published a couple days ago in Annals of Internal Medicine.
Their proprietary workflow looks a bit like this, in which the AI gathers information up front, and then provides the physician with a summary plus recommendations:
Their little puff piece looked at a sample of these AI augmented virtual visits, all very bland low-risk encounters, overwhelmingly respiratory and genitourinary complaints. Then, independent adjudicators evaluated the suitability of recommendations made by both AI and physicians, focusing on cases in which the physician and AI were not concordant.
The contingency table comparing AI and physician scoring looks like this:
The key numbers are best summarised in the marginal score distributions – where you can see AI was more likely to provide “optimal” care, and less likely to provide “inadequate” or “potentially harmful” care. There’s a long way to go to graduate this sort of augmentation to a relatively independent agent, but it’s clear the vision of “AI Avatars” is not quite so farfetched – and, while flawed, no moreso than human physicians. There will be many vendors like this one building competing products, and we will certainly see innovation continue at the same head-spinning rate as Executive Orders.
Finally, say a prayer for those eight patients for whom both the AI and the physician were judged to make “potentially harmful” recommendations. Godspeed.