SecondOpinion: Anatomy-Aware Gated Reasoning for Efficient Medical Image Analysis

2026-08-03

AI Summary

This work addresses the inefficiency of conventional medical image analysis models that uniformly process all cases regardless of complexity. The authors propose SecondOpinion, a dual-stream framework wherein a primary stream rapidly handles all inputs, while an anatomically guided auxiliary stream is activated only when a learnable gating mechanism—GateKeeper—deems the initial prediction unreliable. Crucially, the gate is explicitly trained as a binary correctness classifier, enabling on-demand invocation of anatomical reasoning. Results on chest X-ray and pelvic fracture datasets demonstrate that the model matches or exceeds state-of-the-art performance, with auxiliary stream activation rates ranging from 9.23% to 45.71%, closely aligned with task difficulty. This approach significantly enhances computational efficiency and task adaptability.

Abstract

Deep learning models for medical image analysis typically apply a fixed amount of computation to every input, regardless of case difficulty. Anatomy-guided dual-stream architectures have been shown to improve diagnostic performance, but they evaluate both streams unconditionally, even on cases a single stream could already resolve confidently. We propose SecondOpinion, a framework in which a fast primary stream processes every case, while a second, anatomy-guided stream is invoked only when GateKeeper, a gating mechanism trained explicitly as a binary correctness classifier, judges that the primary stream's prediction needs additional scrutiny, much as a clinician might seek a second opinion on a difficult case. When activated, the two streams are combined through a lightweight cross-attention fusion module. We evaluate SecondOpinion on a unified five-class chest X-ray dataset and a pelvic fracture dataset, the latter including a held-out, harder subset of fractures that are invisible on X-ray but confirmed via CT. SecondOpinion matches or exceeds prior state-of-the-art performance on both tasks, while activating its anatomy-guided stream on only 9.23% of chest X-ray cases, rising to 24.12% on visible fractures and 45.71% on invisible fractures, an activation rate that tracks task difficulty directly. These results suggest that supervising a gating signal toward correctness, rather than relying on unsupervised confidence, allows a model to allocate anatomical reasoning where it is actually needed.

Problem

• medical image analysis
• computational efficiency
• case difficulty
• anatomy-guided reasoning
• adaptive inference

Innovation

• gated reasoning
• anatomy-aware
• medical image analysis
• adaptive computation
• second opinion