Open-Linguistic Concept Unified Learning for Cross-Site Interpretable Dermatology Image Diagnosis

2026-08-04

AI Summary

Current concept-based dermatological diagnostic models are constrained by dataset-specific concept taxonomies, limiting their generalizability across healthcare institutions and imaging modalities—such as dermoscopic and clinical photographs—and suffer from rigid intervention mechanisms that hinder the deployment of interpretable computer-aided diagnosis (CAD) systems. To address these challenges, this work proposes UniCon, a novel framework that constructs a shared semantic space via a unified concept prototype codebook, integrates open-language-driven multidimensional semantic descriptions to alleviate label sparsity, and introduces a reliability-gated intervention interface enabling clinicians to flexibly and confidently correct model predictions across sites. UniCon achieves state-of-the-art diagnostic accuracy while, for the first time, enabling cross-terminology semantic alignment and reliable human-in-the-loop intervention, substantially enhancing both model generalizability and clinical utility.

Abstract

Human-interpretable computer-aided diagnosis is crucial for clinical decision making. Concept-based models excel by providing transparent reasoning and enabling post-hoc, clinician-in-the-loop interventions. However, their rigid dataset-specific adaptation inherently restricts cross-site generalization. Applying them across diverse modalities, such as dermoscopic and clinical photographs, is challenging due to heterogeneous concept taxonomies varying in availability, granularity, and semantics across cohorts. Consequently, adapting Foundation Vision-Language Models (FVLMs) demands costly label engineering and repeated post-training. Existing intervention mechanisms remain rigidly tied to predefined concepts, lacking adaptability and hindering scalable dermatology CAD deployment. To address these bottlenecks, we propose UniCon, an open-linguistic unified concept learning framework for multimodal interpretable vision-language diagnosis. UniCon resolves these challenges through three contributions: (1) A shared semantic representation space via a unified concept prototype codebook, seamlessly coordinating heterogeneous concept systems across modalities without dataset-specific retraining. (2) Open-linguistic based multi-faceted semantic specifications to overcome sparse textual label limitations, improving boundary sensitivity in uncertain clinical contexts. (3) A robust, cross-site adjustable intervention interface powered by reliability-gated bottleneck aggregation, enabling consistent reasoning and transferable clinician corrections. Extensive experiments demonstrate that beyond securing top-tier diagnostic accuracy, UniCon successfully bridges disparate clinical taxonomies, unlocking unprecedented cross-site intervention capabilities. Code is available at https://github.com/wuchengyu123/UniCon.

Problem

• cross-site generalization
• heterogeneous concept taxonomies
• interpretable dermatology diagnosis
• vision-language models
• clinician-in-the-loop intervention

Innovation

• open-linguistic
• unified concept learning
• cross-site generalization
• interpretable diagnosis
• vision-language model