GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs

2026-07-15

AI Summary

This work addresses the challenge of recognizing micro-gestures, where subtle local motions are easily confounded by static appearance and background noise, and existing multimodal large language models struggle to effectively model such sparse dynamics. To this end, we propose a Gated Motion-aware Tokenization (GMoT) module that explicitly extracts motion energy prior to temporal modeling through spatially weighted pooling and inter-frame differencing, followed by semantic gating to produce compact, motion-aware representations. Fine-grained and interpretable reasoning is achieved via semi-supervised anatomy-focused captioning and reward-guided optimization. We further introduce the Body-Region Grounding Recall metric and a cross-domain overlapping-label transfer protocol. On the iMiGUE and SMG benchmarks, our method achieves Top-1 accuracies of 67.32% and 73.11%, respectively—improving upon the Qwen3-VL-8B baseline by 6.80 and 3.11 percentage points—and demonstrates markedly enhanced robustness to interference, cross-domain transferability, and anatomical localization precision.

Abstract

Micro-gesture recognition demands the detection of fleeting, spatially localized movements that are frequently overwhelmed by dominant static appearances and background noise. While Multimodal Large Language Models (MLLMs) excel at general video understanding, they inherently struggle with subtle kinematics and often rely on static posture priors. To this end, we propose GMoT, a Gated Motion-Aware Tokenization module that explicitly distills sparse kinematic evidence into a compact sequence prior to temporal modeling. GMoT dynamically spotlights action-relevant regions via spatially weighted pooling, extracts adjacent-frame temporal differencing to capture precise motion energy, and adaptively fuses these cues into the visual stream using a conservatively initialized semantic gate. To transition from simple classification to evidence-grounded reasoning, we further introduce a progressive reward-guided policy refinement paradigm, supported by a semi-supervised annotation pipeline that generates anatomically focused captions. Beyond achieving the best Top-1 accuracy among the compared methods on iMiGUE (67.32\%) and SMG (73.11\%), improving the Qwen3-VL-8B baseline by +6.80 and +3.11 points, our framework introduces Body-Region Grounding (BRG) Recall as an anatomical-grounding proxy conditioned on correct predictions, together with an overlapping-label cross-domain transfer protocol between iMiGUE and SMG. Extensive evaluations demonstrate that our GMoT-augmented model improves in-domain accuracy, retains clear gains under label-preserving corruptions, and improves accuracy-oriented cross-domain transfer under explicit small-split caveats while maintaining high anatomical grounding in its generated rationales.

Problem

• micro-gesture recognition
• subtle kinematics
• motion-aware tokenization
• multimodal LLMs
• fine-grained video reasoning

Innovation

• Gated Motion-Aware Tokenization
• Micro-gesture Recognition
• Multimodal LLMs
• Temporal Differencing
• Anatomical Grounding