Hype Vs. Reality In Cosmos AI

Just how does Epic's new prediction engine shake out?

Considering the oversized influence Epic exerts on acute care medicine in the U.S. – and, to a lesser extent, around the world – a goodly amount of attention is being paid to their announced foray into AI. One development, likely to reshape the valuation of ambient scribe companies, is the forthcoming integration of ambient technology into Epic, initially based off Microsoft’s Dragon Copilot engine.

The second, and arguably more important, is how they’re going to leverage Epic Cosmos for decision-support and forecasting. With 300 million patient records over 16.3 billion patient encounters, Cosmos is a virtually unmatched proprietary asset. On top of this, they’ve utilized the transformer architecture to create a predictive engine – a set of models they describe as CoMET.

The paper they’ve published is only obliquely about the clinical plan for Cosmos AI, and focused moreso on comparing the performance of the CoMET transformer models of varying size. However, the overall gist of the vision can be gleaned from this glossy information graphic:

Most of the news coverage will likely discuss the clinical implications of these prediction engines, but the article is also here to tell us: bigger is better, as far as transformer goes. The comparison features CoMET-S with 62M parameters vs. CoMET-M with 119M parameters vs. CoMET-L with 1B parameters. Larger is better for most, although the relevance of the differences is variable and task-specific:

Further interesting is the bit under the hood where they look at comparison between these “general” prediction models and task-specific, supervised, machine-learning models. The authors tested both a 119M parameter supervised CoMET and an XGBoost decision tree classifier, and ultimately compared their CoMET to the XGBoost versions. The predictive ability of CoMET was mostly better than the XGBoost models, although it is not yet well-described where, or whether, the performance is sufficient to be clinically useful:

Regardless of the present limitations, this is just scratching the surface of how care might be impacted by the seamless integration of the Cosmos data set into everyday workflow.