What *Is* the Effect of Triage Decision-Support?

Evaluate the model, or evaluate the model's effects?

There’s a mildly interesting little article in NEJM AI regarding a decision-support system deployed in the Emergency Department to augment assignment of the Emergency Severity Index.

The decision-support system and associated report is not necessarily interesting – like much of NEJM AI, it’s a puff piece authored by Beckman-Coulter and the folks with financial interests associated with the project. What is interesting is this Figure:

You can see the black lines with the AI recommended triage category – and then its various effects on human assignment following its rollout. The AI certainly affects the humans – but not in such a fashion where you can see, say, the humans and AI converging as the system engenders trust. The human assignments, in this half-year report, have changed from their prior baselines, but stay persistently diverged from the AI expertise.

It’s a nice little example of the difference between model performance and real-world clinical impact.