From George Clooney in ER to Noah Wyle in The Pitt, emergency department doctors have long been popular heroes. But will it soon be time to hang up the scrubs?

A groundbreaking Harvard study has found that AI systems outperformed human doctors in high-pressure emergency medicine triage, diagnosing more accurately in the potentially life and death moments when people are first rushed to hospital.

And here’s the proof:

Each study “participant” performed three different generations of five potential diagnoses, one at each stage of information availability. These stages were the clinical content available as generated by nurse triage, after an ED physician had documented the encounter, and then upon admission to the hospital. Additional physician members of the study team scored each list of possible diagnoses as the percentage having exact or near-exact correct diagnosis (“Bond Score”). Purple bar (o1-preview) wins!

Of course, 1) these are what internal medicine clinicians made out of emergency department clinical data, 2) the “exact diagnosis” is the not the intent of ED information gathering, it is exclusion of immediately sinister conditions and early treatment for prevention of ongoing morbidity/mortality, 3) the entire foundation of this demonstration is the information produced by the expert evaluation, management, and cognitive work of human emergency physicians, and 4) sure, neat trick, but another contrived experiment in silico, not in Real Life.

With predictable findings, they’ve also fed o1-preview NEJM Healer cases, Grey Matters cases, and Clinicopathological Conferences and compared them with previous 4o results. Another generation of models, another incremental improvement in performance:

It also ought to be noted – o1 is last year’s OpenAI flagship, whereas we’re well beyond that today. OpenAI o1 is also just a “general” LLM, rather than one specifically tuned or trained for optimal performance on clinical content.

As for the calls for sunsetting of clinical autonomy, we’re not quite yet there – but the transformation is well in progress. Even if there are ways to nitpick these results, the frog is already grasping at our heels.

Those days are assuredly over – and likely have been for some time.

This is a publication of the results of PANORAMIC and CanTreatCOVID trials, two trials conducted in the U.K. and Canada in the 2021 to 2024 timeframe. We’ve already seen Paxlovid has no value in the “standard risk” population – and this revisits its utility in a higher-risk population, but a more modern cohort than those enrolled in EPIC-HR back in 2021.

The research summary infographic from NEJM probably summarizes this the best:

The big difference seen here – in EPIC-HR, 7% of the placebo cohort suffered hospitalization or death within 28 days, as opposed to the ~1% in these trials.

Nirmatrelvir-ritonavir definitely reduces viral load faster than placebo, and about 1 in 10 reported symptom improvement sooner as a result. Adverse events were reported frequently with nirmatrelvir-ritonavir, but very few were classified as serious.

Toss it on the hill with the influenza antivirals: in the ambulatory population, no clear benefit in serious outcomes, probably minimal benefit for symptomatic recovery.

The preponderance of evidence now, in adults and children, is that plain amoxicillin ought to be the first choice:

Out of ~500,000 eligible encounters, ~400,000 received amoxicillin-clavulanic acid, and the results above show propensity score-matched cohorts. No reliable detectable differences in treatment failure, but amoxicillin-clavulanic acid gave rise to more downstream cases of yeast infections and Clostridium difficile.

But, not mentioned at all in this article: it’s likely most of these cases never needed anything more than time and conservative therapy. A clear reason it’s not possible to show a difference in outcomes between two therapies is when the natural course of the disease is to cure, regardless. Without stricter criteria for failure of conservative management and initiation of antibiotics in sinusitis, we can’t truly ask the question whether there is a difference in efficacy between these two agents.

At the least, however, if we can’t stop prescribing unnecessary antibiotics – prescribe the less harmful one.

This is a report generated secondhand from a “national technology sprint” at the Veterans Health Administration in which 11 scribe vendors were put through their paces on five simulated case recordings. In this report, we see the modified PDQI-9 scores for the pooled cohort of AI scribe notes, as compared with scoring for a set of human clinician notes generated off the same recordings.

Humans win:

As with every study, there are quirks reducing internal and external validity. These are recordings of “simulated” patient encounters using standardized patients off an illness script, and the Hawthorne effect will bias the humans involved to write higher-quality notes than usual. However, there are some other interesting observations: one of the encounters had a fair bit of background noise, while in another encounter both the patient and clinician were wearing masks. The ambient scribes performed the worst on these, further demonstrating the need for pristine audio to minimize transcription degradation.

AI ambient scribes are not as magical as the vendors would have you believe, but they’re also not as terrible as some evaluations might show. They’re simply tools requiring experience and adaptation in order to maximize their capability and value, and their incremental benefit is not evenly distributed.

These authors took a base audio file and used it to generate a “ground truth” clinical summary using a commercial ambient AI scribe. Then, they introduced several types of input degradation: 1) moving the microphone further from the audio source, 2) playing other types of audio such as rain, baby, construction, and toddler.

The impact of moving the microphone depended on the quality of the equipment, but basic computer built-in versions could start losing information by 2 meters. As for the other types of noise pollution, here are their results, stratified by sound level and impact on different types of error:

The typical impact is omission rather than hallucination – content simply no longer passes through the transcription engine. Heavy rain (“pink noise”) does an excellent job of obscuring transcription, while the nonsensical babbling of a toddler likewise sowed confusion.

The authors do note it is unlikely outdoor environmental volume would reach the thresholds seen in this experiment, but the baby and toddler could easily be as disruptive as the simulation in this study. The main lesson here is probably to have a higher index of suspicion for omissions depending on the consultation context.

The guidelines state the goal of treatment should be to raise serum sodium by >4-6 mEq with treatment – and, this little retrospective study observed it did a fine job at that:

And, no central access needed! (that’s probably also another enduring piece of dogmatic avoidance, as well)

The nature of their retrospective analysis precludes any patient-oriented outcomes being measured, and there’s no basis upon which to claim superiority of the bicarbonate solution – but it’s a little piece of reassuring data to reinforce ongoing uptake of this practice.

But, in a quite large, pragmatic, randomized, controlled trial – the PRONTO trial in the U.K. – it saved lives:

Patients either had “usual [emergency department] care” guided by standard, NEWS2-based sepsis protocols, or had combination NEWS2 + procalcitonin-guided assessment:

With 7,600 patients randomized, mortality was 13.6% in the procalcitonin-guided cohort and 16.6% with usual care.

The confusing bit – it’s not clear how. Antibiotics were provided to the same proportion of each cohort. Antibiotics within 3 hours were provided to the same proportion of each cohort. No individual subgroup screamed “procalcitonin made all the difference” – and the authors are unable to provide a conclusive explanation for their observed findings.

Even more interesting – only two-thirds of treating clinicians in the procalcitonin arm saw the result, and even fewer agreed with the recommendation arising from the risk score.

The only clue I can see:

There are slightly different clinician behaviors in the sickest patients, with fewer of them ultimately receiving antibiotics. The only theory I can imagine to explain the results of this trial is the procalcitonin result prevents clinicians from premature closure and anchoring on “sepsis” as a final diagnosis, ensuring alternative diagnoses are explored completely.

These observations are unlikely to reliably generalize outside the settings and models of care similar to those in NHS trusts, but these data still have fascinating implications.

In this vein HI-PEITHO is a trial deploying CDL for those “intermediate risk" PEs, as compared to a traditional strategy of systemic anticoagulation and observation. The primary outcome is a very impressive sounding composite of:

PE-related mortality, cardiorespiratory decompensation or collapse, or nonfatal symptomatic and objectively confirmed recurrence of PE, within seven days of randomization.

And, good news for the sponsors at Boston Scientific, the patients randomized to their fancy CDL did not suffer as many primary outcomes, and – emphasised sweetly in the text and the editorial – this was driven by reductions in “cardiorespiratory decompensation”. Cardiorespiratory decompensation sounds like a good thing to avoid!

This is what “cardiorespiratory decompensation” looked like in the two groups:

So, mostly small numbers and grossly “the same”, considering – except for the NEWS score. At the end of the day, the only Real Difference in the primary outcome was a few more patients with increased physiologic stress within 24 hours of randomisation.

What happened to these folks? Well, they went off and received a variety of rescue therapies – mostly a type of CDL, embolectomy, or systemic thrombolysis. And they did fine, regardless of whether they had the blanket early intervention, or watchful waiting with selective intervention:

So, what this trial is really telling us: there's no reason to do the special CDL on everyone because you can watch out for the ~5-10% of patients who get worse, and then provide them with an appropriate escalation in therapy.

Unfortunately, that doesn’t allow Boston Scientific to sell more catheters!

But, within narrow use cases, distinguishing between “real” and “fake” becomes much more challenging – and, it would seem, plain radiographs are one of those instances.

This study used GPT-4o and RoentGen to generate synthetic anatomic and chest radiographs, respectively, and then mixed these radiographs with similar matched authentic pairs. Seventeen radiologists were sampled to view and classify the radiographs as real or fake, as well as running these same images through four LLMs as a comparison.

So far, the deepfakes aren’t winning – but enough are slipping through:

When a radiograph appeared obviously synthetic, radiologists and LLMs alike were usually correct. However, a good number of synthetic ones – 60 to 80% – slipped through undetected, and GPT-5 and GPT-4o were the only ones with performance up to or exceeding human perceptions.

Qualitative surveying of the radiologists after the initial phases of the study indicated certain features were more suggestive of synthetic radiographs: absence of noise, overly-smooth cortex, and unnatural-appearing fracture patterns. A few examples:

There isn’t a dramatic earthshaking takeaway from these examples. The editors suggest issues of potentially fraudulent image generation and contamination of training sets with synthetic data. These are real issues to consider, but nothing that stands radiology on its head.

This is a study that’s as much about the behavioural intervention as it is the prevention of kidney stones – a long-term randomisation to “increased” vs. “standard” fluid intake after enrolment following renal colic. “Increased” intake was defined as a daily target of 2.5L, titrated down for those less than 75kg.

This is their fancy water bottle and tracker:

This is their measurement of daily fluid intake across the duration of the study:

And this is whether new stones showed up or not:

So, pretty much a wash – but, at least everyone got a cool water bottle!

The issues: 1) both arms increased their fluid intake from baseline, and 2) the null effect assumes the measure of adherence and fluid intake is reliable. These methods only measure fluid intake from the smart water bottle and do not take into account other sources. It is not truly possible to determine the extent to which the intervention cohort exposure differed from the control group, and therefore the relative utility of fluid intake for stone prevention.

There’s probably kernels of truth in the old dogma, based on increased risks observed when daily fluid intake was similar to this baseline of ~1-1.2L. However, there are likely diminishing returns at a certain level, making the 2.5L target precarious at best.

This study looks at patients in French ICUs with hypoxemic respiratory failure and paO2s below 200, and if you look closely, there are differences:

The primary patient-oriented outcome is not different. This is similar to the lack of difference identified by this same research group in their COVID-only cohort, SOHO-COVID – and not terribly surprising because about half of this cohort is also COVID-19-associated respiratory failure.

Then, SOHO-COVID saw fewer intubations in the HNFO cohort – and, consistently, again here.

It’s a bit challenging to interpret these data primarily due to the non-blinded nature of the intervention. That is to say, the presence of HNFO on a patient is likely to push clinicians to delay intubation – and it’s unclear the downstream effect on patient-oriented outcomes. There is probably some clue behind this in the subgroup analyses in the supplement:

I suspect HFNO is, indeed, slightly better – but the effect of the intervention on clinician behavior needs a closer look to ensure potential harms from delayed care (intubation) are not being passed on to the patient.

But, this trial involving prehospital transfusion following major trauma found no difference between whole blood and balanced blood components for their primary outcome:

“But what’s the breakdown of the massive transfusion component” you ask, and thus some secondary outcomes:

Prothrombin times were longer in the whole blood component group on arrival, but it is doubtful this translated into any meaningful differences in primary, secondary, or safety outcomes.

It’s possible limitations in the trial did not enroll the proper cohort from which to detect an advantage to whole blood – but it’s probably more likely there simply isn’t one.

It’s just a brief communication of a scenario provided to participants on an internet survey platform: if an AI identifies a hemorrhagic stroke, yet the radiologist persists in denying its existence, has the radiologist met their duty-of-care to the patient?

Interestingly, this survey found differing opinions depending on where AI was injected into the scenario. A “single read” scenario involved the AI finding being displayed to the radiologist prior to their initial read – and the jury of the public was quite unsympathetic. However, in a “double read” scenario, the radiologist viewed the study, then received the AI findings, and viewed the read again. In this second instance, they judged the radiologist more leniently.

This is just a small study and quite superficial in its question, but it’s another clue as to how the medicolegal environment might treat clinicians involved in AI recommendations.

Probably not the case!

In Canada, to treat patients with acute venous thromboembolism:

And no difference between arms as far as recurrent VTE during the treatment period.

Rivaroxaban is frequently chosen because of its simplified once-a-day dosing, and this is apparent in the adherence statistics gathered:

In spite of this limitation, apixaban retained its effectiveness in treating and preventing recurrent VTE.

These data include only patients <120kg, and only treatment of uncomplicated acute VTE – there are many other indications and permutations for DOACs, and these don’t automatically generalize to all of them. It will be interesting to see those follow-up studies as they materialize.

This is the overall setup:

Unfortunately, it’s not really a head-to-head comparison. Everyone who participated in the study was exposed to AIME – there’s no pre-visit human chat as a parallel group. The mechanism for evaluation, then, is a comparison of the pre-visit AI summary to the post-urgent care clinician human follow-up plan – and those urgent care visits were exposed to the AIME summary in the first place.

The chatbot was “fine”:

Independent clinical review indicated AIME had a generally reasonable set of diagnoses, as well as grossly acceptable management plans – rating higher than humans in some cases, though humans were still preferred more often. The primary domains in which AIME suffered were sort of under the umbrella of “logistics” or “care navigation”, in that their plans underperformed on being practical or cost-effective.

There are limitations, positives, and negatives to take away here – and the overall gist is that this chatbot probably needs a little more refinement to become clinically acceptable with a human-in-the-loop, and definitely not ready to shed any human oversight, at all. Some bits are harder to fix – the LLM attempting to build a differential diagnosis, for example – but the superficial logistics can likely be improved by injecting more local management context into the framework.

Importantly, it’s a nice baby step forward towards “real world” use – not just exam questions, simulations, or marketing promises.

This report illustrates the findings of a convenience sample of patients using an Ada Health product as a navigator for care by one of Ada Health’s clients, written up by – you guessed it – Ada Health. You can imagine these data have all the resilient integrity of a sandcastle.

But, here’s what they’re selling:

Patients came to the Ada Health platform, implemented for a private healthcare provider in Portugal, considering specialty care and were re-triaged to GPs. The prevailing narrative across the study is that their platform safely redirects patients to a “more appropriate” care setting. Bravo.

Whether these have either internal or external validity is highly suspect. The authors prompted 33,000 to take part in these surveys of pre-Ada Health intention and captured a mere self-selected 1,400 – half of which they lost to follow-up, leaving the 700-odd shown in the Sankey diagram above. Then, much depends on the healthcare environment wrapping around the entrants into a platform such as this – meaning the generalizability will be limited by the resemblance of your patient population to those belonging to this private healthcare organization, and the resources the organization has to offer. Finally, it’s Ada Health reporting on Ada Health, which always raises questions regarding things Left Unsaid (née Unreported).

Regrettably, this is likely just another piece of mostly unhelpful fluff. There are some funny bits – patients probably didn’t actually do what they said they would do when they had a post-encounter survey! – but the vast majority of this is fairly unrevealing.

And, now, this is a similar application – but in wastewater specific to an emergency department.

Depending on the local public wastewater infrastructure, a project such as in NZ might not be feasible or offer sufficient coverage – so why not monitor a site enriched with the latest and greatest? Here’s an example comparing positive tests in the Stanford ED to those viruses detected in the wastewater 2024-25:

Whilst there are those who are loathe to admit danger remains from SARS-CoV-2, and, likewise, the public nuisance of influenza – this is an interesting pilot offering another approach to monitoring for potential pandemic pathogens in the community.

Tested here is an AI radiology tool for verifying nasogastric tube placement, a common indication for hospital-based patients. Performed as part of a product evaluation to deploy in active use, this was a retrospective analysis of 1,605 chest x-rays performed in the NHS, and the study design looked like this:

And the results look like this:

In this scenario, the hope is the AI can be relied upon to ensure safe placement of NGT, obviating the need for radiologist review. Unfortunately, the AI tested here missed 13% of unsafe NGT – a situation in which use of a malpositioned NGT could lead to aspiration events and other patient-level risks.

Whether the outputs of this system can be incorporated into any sort of workflow effectively deburdening radiologists is not tested, only the system working in isolation. However, based on these results, the authors report they terminated plans for deployment of this specific model and use case.

Pretty simple sample of clinical vignettes being fed into the brand-new ChatGPT Health, asking the LLM to spit out a deterministic care destination. Over-triage is common to both humans and LLMs and is not terribly worrying, but the key concern is the under-triage: over half of patients with time-sensitive morbidity were not referred to immediate medical care.

It’s not quite as bad as folks are making it out:

Everyone who was “missed” was referred to urgent care within 24-48 hours, so it’s still better than throwing darts at the carnival.

However, there are a couple major issues:

• It’s an LLM. It’s not thinking, it’s not reasoning, it’s not rationalizing – it’s spitting out tokens based on probability distributions from patterns in its training set. If the input patterns match clinically appropriate outputs, it all works out. A general health LLM is unlikely to be enriched with those sorts of connections in this sort of a triage domain.

• Unsurprisingly, the under-triage got worse when provided with objective data – vital signs, lab results. Relatively normal conditions early in severe illness are falsely reassuring when matching patterns in the training set.

• These results are “best case scenario”, in many ways. Once humans get in the way with transactional chatting, as compared to organized clinical vignettes, the information exchange veers further away from the “textbook” patterns in the LLM training sets. It’s possible a back-and-forth could generate better results, but it’s far more likely human interaction will generate worse.

Generating better performance in this domain likely requires a combination of structured knowledge and bespoke small language model, and no one ought to suggest these very general platforms should be applied to narrow high-consequence use cases without extensive validation.

Aortic dissection does not have the same luxury of benign outcomes if missed, and thus the CT scanner goes brrrrrr. There has been some consideration into using the ADD-RS to reduce over-investigation, but this report from Victoria seems to indicate it is not clearly going to solve our problems with low overall yield:

The missing chunk on this data set – ADD-RS is designed to proceed to D-dimer, and these data does not indicate which of these patients incorporated D-dimer in a decision to proceed with angiography. It may the ADD-RS pathway is valid in reducing over-investigation, and there is a whole batch of other patients excluded from angiography based on ADD-RS and D-dimer testing, but these data don’t show us that bit. What it does show us is that it takes 50 CTs, most of which have some “high risk” feature, to find a single aortic dissection.

These data also don’t show us if any important alternative diagnoses were identified on these scans, so it may not be completely hopeless – but I think it goes without saying, whoever can invent A Better D-Dimer can have the next Nobel Prize for Medicine.