Multimodal RGB-HSI Feature Fusion with Patient-Aware Incremental Heuristic Meta-Learning for Oral Lesion Classification

2025-11-15

AI Summary

Early screening of oral cancer and precancerous lesions in low-resource settings is hindered by severe scarcity of annotated patient data. Method: We propose a patient-aware incremental heuristic meta-learning framework that jointly leverages RGB images, reconstructed 31-band hyperspectral (HSI) data (generated via a ConvNeXt-v2 encoder for RGB-to-HSI translation), hemoglobin-sensitive indices, handcrafted texture and spectral shape features, and demographic/clinical metadata. A probabilistic stacking module coupled with posterior smoothing enables dynamic multimodal fusion and explicit uncertainty modeling. Contribution/Results: The framework significantly enhances generalization to unseen patients. On a cross-patient test set, it achieves a macro-F1 score of 66.23% and accuracy of 64.56%, demonstrating the practical efficacy of hyperspectral reconstruction and uncertainty-aware meta-learning for real-world oral cancer screening.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Early detection of oral cancer and potentially malignant disorders is challenging in low-resource settings due to limited annotated data. We present a unified four-class oral lesion classifier that integrates deep RGB embeddings, hyperspectral reconstruction, handcrafted spectral-textural descriptors, and demographic metadata. A pathologist-verified subset of oral cavity images was curated and processed using a fine-tuned ConvNeXt-v2 encoder, followed by RGB-to-HSI reconstruction into 31-band hyperspectral cubes. Haemoglobin-sensitive indices, texture features, and spectral-shape measures were extracted and fused with deep and clinical features. Multiple machine-learning models were assessed with patient-wise validation. We further introduce an incremental heuristic meta-learner (IHML) that combines calibrated base classifiers through probabilistic stacking and patient-level posterior smoothing. On an unseen patient split, the proposed framework achieved a macro F1 of 66.23% and an accuracy of 64.56%. Results demonstrate that hyperspectral reconstruction and uncertainty-aware meta-learning substantially improve robustness for real-world oral lesion screening.

Problem

• Classifying oral lesions using multimodal RGB-HSI fusion
• Addressing limited annotated data in low-resource cancer detection
• Improving robustness through hyperspectral reconstruction and meta-learning

Innovation

• RGB-to-HSI reconstruction for hyperspectral feature extraction
• Fusion of deep, spectral, and clinical metadata features
• Incremental heuristic meta-learner with patient-level smoothing