LLMs Attempting Data Merge
Harmonization is Real Hard.
It’s 2025! I can stream esoteric music from 50 years ago through my phone, almost anywhere in the world. Access to the accumulated accounts of nearly all of human history is just clicks away.
But, it’s still frequently impossible to migrate health data from one institution to another.
The project overview from this study encapsulates the mapping problem and describes their work quite nicely:
Some terms are obvious exact matches. Others can be inferred easily from minor differences in data structures. Others represent bespoke variances in flowsheets and clinical practice, and may or may not manifest the same type of data.
How did these authors do in using their “LLM Augmented Concept Mapping” for data pairs generated in acute and intensive care units?
Not great, Bob! But, it’s probably reasonable to use a process like this to reduce the amount of manual review required – though these authors do not audit the results of their semantic match alignment. A small amount of error might be tolerable when generating a data set for research purposes, but less ideal if attempting to create a definitive care record.