Multi-Agent Debate and Visual Information Extraction for SeePhys Pro: A 1st-Place Technical Report from ICML 2026 AI4Math Track 3 Challenge

2026-07-23

AI Summary

This work addresses the challenge that large language models struggle to accurately solve university-level physics problems when critical information is embedded in accompanying images. To bridge this modality gap, the authors propose a two-stage framework: first, visual information extraction converts image content into structured textual descriptions; second, a multi-agent debate mechanism orchestrates collaborative reasoning among three heterogeneous solvers. The approach innovatively integrates modality alignment with multi-agent collaboration, revealing that answer selection reliability is pivotal to performance gains and that the utility of visual augmentation increases significantly with the proportion of image-dependent information in the problem. Evaluated on the ICML 2026 AI4Math Track 3 challenge, the method improves accuracy on the public test set from 0.643 to 0.802 and achieves first place on both public and private leaderboards (0.743 on the private set).

Abstract

This technical report presents our approach to Challenge Track~3: SeePhys Pro at the 3rd AI for Math Workshop, where the task is to answer college-level physics questions whose statement and figure may be given partly or entirely as an image. Visual physics problems become substantially harder for large language models when the decisive information resides in a figure rather than in the text, and this modality gap widens as more of the problem migrates into the image. We address the task with a two-stage framework: a visual information extraction stage that re-expresses figure content as solver-readable text to close the modality gap, and a reasoning stage that orchestrates three heterogeneous solvers through multi-agent debate. Our analysis yields two findings: the gain from orchestration comes from reliable answer selection rather than from additional debate, and the value of a figure aid scales with how much of the problem is locked inside the image. The resulting pipeline improves overall accuracy over a single-agent baseline from 0.643 to 0.802 on the public split, and won 1st place on both the public and the private leaderboard (private overall 0.743).

Problem

• visual physics problems
• modality gap
• image-based question answering
• information extraction
• college-level physics

Innovation

• multi-agent debate
• visual information extraction
• heterogeneous solvers
• modality gap
• physics reasoning