🤖 AI Summary
Large language models (LLMs) excel at textual mathematical reasoning but exhibit substantial performance degradation on visual geometric reasoning tasks, primarily due to the challenges of image-based geometric understanding, multi-step spatial reasoning, and the scarcity of large-scale, diverse, and reasoning-annotated geometric datasets.
Method: We introduce GeoThought—the first large-scale, diverse geometric reasoning dataset featuring explicit chain-of-thought and reflective reasoning steps, systematically covering hierarchical geometric reasoning processes. Our approach integrates vision-language description generation, multimodal large language model (MLLM) architecture, and error-correcting chain-of-thought training.
Contribution/Results: The resulting GeoThought-MLLM achieves state-of-the-art performance on both in-domain and cross-domain geometric reasoning benchmarks. Error analysis demonstrates that explicit reflection mechanisms effectively mitigate conceptual misclassifications and spatial relationship misunderstandings, significantly enhancing geometric semantic comprehension.
📝 Abstract
Large language models (LLMs) have demonstrated strong reasoning capabilities in text-based mathematical problem solving; however, when adapted to visual reasoning tasks, particularly geometric problem solving, their performance substantially declines because geometric problems present unique challenges. Specifically, these challenges stem from two key factors: first, the intrinsic complexity of geometry requiring detailed image comprehension and multi-step reasoning, and second, the limitations of existing datasets which lack sufficient scale, diversity, and explicit reasoning traces, consequently hindering effective model training. To address these challenges, we developed the GeoThoughts dataset, a comprehensive geometric reasoning corpus with two subsets: Geo-Thought-6K with 6,243 samples and its augmented version Geo-Thought-Augmented-10K containing 10,834 samples. Each entry includes visual descriptions, step-by-step solutions, explicit reasoning chains, reflection steps, and final answers. Using this dataset, we developed GeoThought-MLLM, a mathematical reasoning multimodal model that generates detailed thinking processes during problem-solving. Our model outperforms existing benchmarks in geometric tasks, demonstrating that training with our Chain-of-Thought dataset improves geometric reasoning capabilities across both in-domain and out-of-domain settings. Finally, we analyze failure cases and observe that errors primarily arise from incorrect interpretation of mathematical concepts or spatial misjudgment. By invoking CoT to correct these mistakes, the model produces correct answers.