The "Stroke Miracle" at the NHS

I'm glad it's weirdly in the news again, because it gives me an opportunity to savage it.

Yesterday, the AMIA Informatics SmartBrief chose to highlight an AI stroke technology being deployed at the NHS – linking out, not to primary literature, but to a Guardian puff piece hailing its rollout: “Stroke centres in England given AI tool that will help 50% of patients recover”

If you run this headline by any clinician, you’ll get some wild whiplash as they screw their head around in confusion and shock. Stroke remains a challenge to treat, and only a handful of patients are eligible for acute therapy. Of those, the percentage that recover is dramatically less than 50%.

So, where do these numbers come from? And how does “AI” play into it?

This is one of Oxford’s incubator spinouts, Brainomix, and their e-Stroke software. The most commonly cited numbers you’ll find across various websites are these:

“An early-stage analysis of this AI technology has demonstrated an association with a reduction in the time between someone presenting with a stroke and receiving treatment by more than 60 minutes and it is correlated with a tripling in the number of stroke patients recovering with no or only slight disability – defined as achieving functional independence – from 16% to 48%.”

Magical numbers fulfilling the “too good to be true” idiom, but if the NHS has invested £30m into this product, there’s substantial political motivation to disseminate only the best data.

The actual report on this technology is here, and the numbers within are far more sobering. The e-Stroke evaluation consists of a retrospective review of stroke outcomes from a handful of comprehensive and acute stroke centers with a mix of before-and-after implementations between 2019-2023.

Then, the e-Stroke pathway actually has no effect on intravenous thrombolysis, the most common treatment for acute stroke, and its use generally delayed administration of IVT by a few minutes:

Where the e-Stroke software is supposedly useful – and this is where the “AI” technology is applicable – is in improving the workflow and diagnosis for triage to mechanical thrombectomy. There has been a faster uptake of MT in the intervention hospitals since implementation:

As with any retrospective evaluation over a long time period, the improvement seen is only an association with the use of e-Stroke, not causation, as there are potentially confounding process improvement initiatives occurring in the same timeframe.

But, the main selling point of this sort of software in accelerating diagnosis is to expedite door-in-door-out for transfers out of acute stroke centers to comprehensive stroke centers, and these data are all over the place:

Some of this is attributed to delays in ambulance dispatch, which confounds the reliability of these data.

The other issue was clinical acceptability, with the users having a reserved opinion:

These are consistent with other publications describing limitations in a real-world evaluation of the Brainomix software.

Lastly, we come to the 16% versus 48% number – and that comes from this predatory journal publication. These data are from a single acute stroke center evaluation in which 19 patients were treated prior to implementation of e-Stroke, as compared with 21 patients after implementation. The door-in-door-out times were better with e-Stroke, and this is a potentially valid observation. However, the patient cohorts are small and heterogenous (and favoring the post-implementation in NIHSS and ASPECTS):

Based on these tiny numbers – and, not even broken down into the patients who underwent MT, just those evaluated by the e-Stroke software – there’s no way to rely upon the mRS 0-2 numbers reported.

So, the entire narrative is a flimsy edifice of cards being pushed (as you can see in the authorship contributions) by Oxford and employees of Brainomix. There is absolutely potential value in augmenting radiology interpretation of CTA and CT perfusion scans, and anything that helps reduce time to transfer is fantastic. However, these results are cherry-picked and overstated, and it undermines the credibility of those promoting this innovation.