BriMA: Bridged Modality Adaptation for Multi-Modal Continual Action Quality Assessment

2026-02-22

AI Summary

This work addresses the challenges of modality imbalance and missing data in multimodal continual action quality assessment, often caused by sensor failures or incomplete annotations. To tackle these issues, the authors propose BriMA, a novel approach that introduces a bridging modality adaptation mechanism to reconstruct missing modalities by fusing task-agnostic and task-specific representations. BriMA further incorporates a memory-guided bridging completion module and a modality-aware replay strategy that prioritizes informative samples less affected by modality distortion and distribution shift. Evaluated on the RG, Fis-V, and FS1000 datasets, BriMA consistently improves correlation by 6–8% and reduces error by 12–15%, demonstrating significantly enhanced robustness and continual learning capability under non-stationary modality-missing conditions.

Technology Category

Machine Learning: Multimodal LearningComputer Vision: Multi-modal VisionIntelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAGEconomics, Online Markets and Human Computation: Data quality aspects of human-annotated datasetsUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Action Quality Assessment (AQA) aims to score how well an action is performed and is widely used in sports analysis, rehabilitation assessment, and human skill evaluation. Multi-modal AQA has recently achieved strong progress by leveraging complementary visual and kinematic cues, yet real-world deployments often suffer from non-stationary modality imbalance, where certain modalities become missing or intermittently available due to sensor failures or annotation gaps. Existing continual AQA methods overlook this issue and assume that all modalities remain complete and stable throughout training, which restricts their practicality. To address this challenge, we introduce Bridged Modality Adaptation (BriMA), an innovative approach to multi-modal continual AQA under modality-missing conditions. BriMA consists of a memory-guided bridging imputation module that reconstructs missing modalities using both task-agnostic and task-specific representations, and a modality-aware replay mechanism that prioritizes informative samples based on modality distortion and distribution drift. Experiments on three representative multi-modal AQA datasets (RG, Fis-V, and FS1000) show that BriMA consistently improves performance under different modality-missing conditions, achieving 6--8\% higher correlation and 12--15\% lower error on average. These results demonstrate a step toward robust multi-modal AQA systems under real-world deployment constraints.

Problem

• Action Quality Assessment
• Multi-Modal Learning
• Modality Missing
• Continual Learning
• Non-stationary Modality Imbalance

Innovation

• modality adaptation
• continual learning
• action quality assessment
• missing modality
• multi-modal learning