Boosting Multimodal Reasoning with MCTS-Automated Structured Thinking

2025-02-04

AI Summary

Multimodal large language models (MLLMs) struggle to balance performance and efficiency in complex visual reasoning, as existing approaches rely on massive datasets and brute-force search, resulting in coarse-grained cognitive modeling and inefficient data utilization. Method: We propose AStar—a novel automated structured thinking paradigm grounded in Monte Carlo Tree Search (MCTS). AStar introduces a hierarchical MCTS-driven cognitive modeling mechanism that dynamically integrates internal reasoning capabilities with external structured prompt orchestration, augmented by lightweight reasoning distillation to reduce computational overhead. Contribution/Results: On the MathVerse benchmark, AStar achieves 54.0% accuracy using only a 7B-parameter model—surpassing GPT-4o (50.2%)—while substantially reducing both training data requirements and inference cost. The framework establishes a new paradigm for efficient, interpretable visual reasoning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models
• Computer Vision: Large Vision Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Multimodal large language models (MLLMs) exhibit impressive capabilities but still face challenges in complex visual reasoning. While recent efforts attempt to enhance MLLMs' reasoning by incorporating OpenAI o1-like structured thinking through explicit search structures or teacher-guided distillation, they often struggle to balance performance and efficiency. A critical limitation is their heavy reliance on extensive data and search spaces, resulting in low-efficiency implicit insight extraction and data utilization. To address this, we propose AStar, an Automated Structured thinking paradigm for multimodal reasoning via Monte Carlo Tree Search (MCTS). AStar automatically derives high-level cognitive reasoning patterns from limited data using MCTS-powered hierarchical structures. Building on these explicit patterns, we design a unified reasoning framework that seamlessly integrates models' internal reasoning capabilities and external reasoning guidelines, enabling efficient inference with minimal tree iterations. This novel paradigm strikes a compelling balance between performance and efficiency. Extensive experiments demonstrate AStar's effectiveness, achieving superior accuracy (54.0$%$) on the MathVerse benchmark with a 7B backbone, surpassing GPT-4o (50.2$%$) while maintaining substantial data and computational efficiency.

Problem

• Enhances multimodal reasoning efficiency
• Reduces reliance on extensive data
• Balances performance and computational efficiency

Innovation

• Monte Carlo Tree Search
• hierarchical structures
• unified reasoning framework