CASTLE2026 Team WDL Technical Report

2026-05-30

AI Summary

This work addresses the challenge of integrating multi-source heterogeneous evidence—including video, ASR transcripts, auxiliary images, and spatiotemporal metadata—in long-form, multi-view first-person video question answering spanning over 600 hours. To tackle this, the authors propose an evidence-aware multimodal reasoning framework built upon the Qwen large language model. The framework employs a question-type-adaptive routing mechanism to dynamically invoke specialized prompting modules and performs iterative reasoning through keyframe sampling, ASR-based retrieval, and multimodal prompt engineering, culminating in a confidence-weighted voting ensemble. Evaluated on the CASTLE Challenge @ EgoVis 2026, the method achieves top-ranking performance. Ablation studies demonstrate that LoRA fine-tuning boosts accuracy from 0.21 to 0.50, with further improvement to 0.58 upon incorporating frame sampling, underscoring the approach’s effectiveness and novelty.

Abstract

The CASTLE Challenge @ EgoVis 2026 evaluates long-form egocentric video question answering over 600+ hours of multi-perspective recordings. Each four-choice question requires evidence from videos, transcripts, auxiliary photos, people, days, rooms, and temporal context. We propose an evidence-aware multimodal reasoning pipeline based on Qwen. Our system parses question hints, retrieves ASR chunks, attaches auxiliary images, samples candidate video frames, and routes questions into static visual, speech/text, temporal, and mixed types with specialized prompts. Multiple inference passes are aggregated by confidence-weighted voting and converted into the official Codabench format. In ablation, LoRA improves the score from 0.21 to 0.50, and more sampled frames further raise it to 0.58. Our final system ranks first in the CASTLE Challenge @ EgoVis 2026.

Problem

• egocentric video
• question answering
• multimodal reasoning
• temporal context
• evidence integration

Innovation

• evidence-aware reasoning
• multimodal question answering
• LoRA fine-tuning
• egocentric video understanding
• confidence-weighted voting