Decomposing Error and Style in Automated Clinical Coding

2026-09-21

AI Summary

研究解决自动化临床编码中模型误差评估问题，通过引入编码风格ψ来区分系统性差异和真正错误，提高ICD F1评分。

Abstract

In automated clinical coding, where the label space spans tens of thousands of diagnosis and procedure codes, models are currently evaluated against a single gold annotation, treating any deviation as error. But we find when two teams code the same 110 ACI-Bench encounters, they agree on only 73% of codes (Jaccard similarity) for the same note; even after an independent clinical audit removes erroneous codes, agreement rises only to 77%. Is that gap error or something systematic? We model the systematic component as coding style $ψ$, a coder- or site-specific policy over what to code and how much to document, and recast coding as $p(\mathrm{code}\mid\mathrm{note},ψ)$, estimating $ψ$ with a 10-dimension rubric. If style were noise, conditioning on it would do nothing. Instead, across five datasets a model conditioned with a data-matching style raises ICD F1 by up to 26 points and an extreme mismatched one lowers it by up to 21. Four prompt based coding methods spanning 39-49 F1 converge to 52-56 once style is supplied (All p<0.05). Much of what single-gold evaluation charges to model error is recoverable, unmodeled style.

Problem

• automated clinical coding
• coding style
• systematic component
• gold annotation
• Jaccard similarity

Innovation

• coding style
• systematic component
• style-conditioned model
• ICD F1 improvement