A Radiology Swing and Miss

Miss rate too high for even this simple clinical question.

Across overburdened public healthcare systems around the world, a common challenge remains ever-increasing utilization of advanced and plain radiology. The promise of AI – at least, in this current generation – is to offload or augment as much straightforward plain radiography as feasible.

Tested here is an AI radiology tool for verifying nasogastric tube placement, a common indication for hospital-based patients. Performed as part of a product evaluation to deploy in active use, this was a retrospective analysis of 1,605 chest x-rays performed in the NHS, and the study design looked like this:

And the results look like this:

In this scenario, the hope is the AI can be relied upon to ensure safe placement of NGT, obviating the need for radiologist review. Unfortunately, the AI tested here missed 13% of unsafe NGT – a situation in which use of a malpositioned NGT could lead to aspiration events and other patient-level risks.

Whether the outputs of this system can be incorporated into any sort of workflow effectively deburdening radiologists is not tested, only the system working in isolation. However, based on these results, the authors report they terminated plans for deployment of this specific model and use case.