A Dual-Feature Extractor Framework for Accurate Back Depth and Spine Morphology Estimation from Monocular RGB Images

2025-07-30

AI Summary

To address the ionizing radiation risk in X-ray–based scoliosis assessment and the poor robustness and generalizability of monocular RGB-based depth estimation—caused by illumination sensitivity—we propose the Grid-Aware Multiscale Adaptive Network (GAMA-Net). GAMA-Net employs a dual-encoder architecture integrated with a Patch-Based Hybrid Attention (PBHA) module to jointly exploit surface texture and geometric priors, and introduces an Adaptive Multi-Scale Feature Fusion (AMFF) mechanism to enhance sensitivity to subtle dorsal depth variations. Evaluated on public benchmarks, GAMA-Net achieves depth estimation accuracies of 78.2%, 93.6%, and 97.5% under the standard δ₁, δ₂, and δ₃ metrics, respectively, and attains a spinal morphology classification accuracy of 97.0%. These results significantly outperform existing RGB-only methods, establishing a novel, radiation-free, and portable paradigm for clinical scoliosis assessment.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements

Abstract

Scoliosis is a prevalent condition that impacts both physical health and appearance, with adolescent idiopathic scoliosis (AIS) being the most common form. Currently, the main AIS assessment tool, X-rays, poses significant limitations, including radiation exposure and limited accessibility in poor and remote areas. To address this problem, the current solutions are using RGB images to analyze spine morphology. However, RGB images are highly susceptible to environmental factors, such as lighting conditions, compromising model stability and generalizability. Therefore, in this study, we propose a novel pipeline to accurately estimate the depth information of the unclothed back, compensating for the limitations of 2D information, and then estimate spine morphology by integrating both depth and surface information. To capture the subtle depth variations of the back surface with precision, we design an adaptive multiscale feature learning network named Grid-Aware Multiscale Adaptive Network (GAMA-Net). This model uses dual encoders to extract both patch-level and global features, which are then interacted by the Patch-Based Hybrid Attention (PBHA) module. The Adaptive Multiscale Feature Fusion (AMFF) module is used to dynamically fuse information in the decoder. As a result, our depth estimation model achieves remarkable accuracy across three different evaluation metrics, with scores of nearly 78.2%, 93.6%, and 97.5%, respectively. To further validate the effectiveness of the predicted depth, we integrate both surface and depth information for spine morphology estimation. This integrated approach enhances the accuracy of spine curve generation, achieving an impressive performance of up to 97%.

Problem

• Estimating back depth and spine morphology from RGB images
• Overcoming environmental sensitivity in RGB-based spine analysis
• Enhancing accuracy with dual-feature extraction and fusion

Innovation

• Dual encoders extract patch and global features
• Patch-Based Hybrid Attention module interacts features
• Adaptive Multiscale Feature Fusion dynamically combines information