Hard Vision, Easy Vision: What GPT-6 Astra Reveals Across Computer Vision

2026-09-28

AI Summary

This study investigates the capability boundaries of general-purpose AI in computer vision. Through comprehensive multi-benchmark evaluation and cross-system comparison, we systematically assess generalist models such as GPT-6 Astra across 34 capabilities and 55 benchmarks, using specialized models and human performance as references. Our analysis delineates a new landscape of visual capabilities, revealing that while semantic reasoning approaches human-level proficiency, significant gaps persist in geometric precision and 3D reconstruction. Furthermore, we demonstrate that Astra substantially outperforms baselines in spatial reasoning, yet high-precision perception remains a critical frontier challenge. This work quantifies the extent to which generalist interfaces can substitute for traditional computer vision pipelines, offering empirical guidance for the future development of visual intelligence.

Abstract

Frontier general-purpose systems are rapidly expanding beyond visual understanding into capabilities traditionally handled by dedicated computer-vision models. As these capabilities expand, a central question for the computer-vision community is how far this reach extends, and what remains hard. We evaluate GPT-6 Astra alongside five frontier general-purpose AI systems across 34 capabilities and 55 benchmarks spanning nine areas of computer vision. We compare their performance with dedicated models and humans where suitable references are available. Astra demonstrates broad visual capability, with substantial gains over other frontier systems in visual and spatial reasoning and several forms of structured prediction. Across the state-of-the-art systems, a consistent pattern emerges. Capabilities involving semantic interpretation, reasoning, and object-centric prediction increasingly approach or reach available reference levels. In contrast, larger gaps remain when tasks require metric geometric accuracy, faithful reconstruction, temporally consistent dense prediction, or specialized fine-grained visual knowledge. Additional reasoning and specialist tools close selected gaps, but their benefits vary across capabilities. These results map a changing landscape of computer vision in which increasingly sophisticated visual tasks are accessible through a general-purpose interface, while precise and fidelity-sensitive perception remains an important frontier.

Problem

• Computer Vision
• General-purpose AI systems
• Visual understanding
• Frontier models
• Benchmark evaluation

Innovation

• General-purpose AI systems
• Computer vision benchmarks
• Visual and spatial reasoning
• Structured prediction
• Dense prediction