ClinicalEncoder26AM: A Multlilingual Diagnosable ColBERT Model; Evidences from the MultiClinNER Shared Task

2026-05-27

AI Summary

This work addresses the low accuracy of symptom, disease, and procedure entity recognition in multilingual clinical and biomedical texts by proposing a diagnosable multilingual ColBERT architecture. Built upon BGE-M3, the model undergoes clinical post-training that integrates synthetic medical records, doctor–patient dialogues, and annotated corpora. It jointly optimizes named entity–level and sentence-level representations through multi-adapter distillation and incorporates a lightweight CNN boundary detection head alongside BIO sequence labeling to enable multi-granular supervision. While preserving the ability to process long texts within a single window, the approach substantially improves data efficiency and cross-lingual generalization. Evaluated on the MultiClinNER shared task, the model achieves state-of-the-art multilingual entity recall and ranks among the top five across all entity types and languages in character-weighted F1 scores.

Abstract

ClinicalEncoder26AM is a multilingual Diagnosable ColBERT for clinical and biomedical texts, which aligns at multiple levels its token-level semantic with ClinicalMap25, a clinical latent space inspired by BioLORD-2023 and enriched with synthetic and annotated supervision. The post-training recipe builds upon BGE-M3, and combines synthetic clinical notes, patient--doctor conversations, and annotated resources such as MedMentions, while considering both named-entity-level and sentence-level representations in a multi-adapter distillation, along with a ColBERT-style retrieval objective. In this system demonstration paper, we evaluate the model in the MultiClinNER shared task by finetuning it as a BIO tagger for patient symptoms, disorders, and procedure spans, using a lightweight two-layer CNN head to improve local boundary detection. The resulting system remains simple, processes most documents in a single 8192-token window, and achieves state-of-the-art multilingual entity recall, while achieving Top 5 overall across all entity types and languages in Character-weighted F1 scores. Training curves further show that ClinicalEncoder26AM is markedly more data-efficient than the base M3 model, supporting the usefulness of its clinical post-training for downstream information extraction. The model can be downloaded on https://huggingface.co/Parallia/ClinicalEncoder26AM-Diagnosable-Colbert-L2-for-multilingual-medical-texts

Problem

• multilingual clinical NER
• named entity recognition
• clinical information extraction
• biomedical text processing
• data-efficient learning

Innovation

• Multilingual Clinical NER
• Diagnosable ColBERT
• Clinical Semantic Space
• Multi-Adapter Distillation
• Data-Efficient Post-Training