Adapting Like Humans: A Metacognitive Agent with Test-time Reasoning

2025-11-28

AI Summary

Current vision-language models (VLMs) lack online adaptation capability during test-time inference for novel tasks, whereas humans dynamically optimize strategies via metacognitive mechanisms. To bridge this gap, we propose the Metacognitive Test-time Reasoning framework (MCTR), a dual-module VLM architecture comprising metalevel and object-level components. MCTR integrates a hierarchical memory system, natural-language memory storage, context-aware reasoning, dynamic knowledge retrieval, and metacognitive reinforcement learning—enabling real-time rule discovery and autonomous policy updating at test time. Evaluated across 45 Atari games, MCTR achieves state-of-the-art (SOTA) performance on 9 of 12 unseen games, significantly outperforming existing baselines. This work represents the first systematic incorporation of human-inspired metacognition into VLM test-time adaptation, establishing a new paradigm for robust generalization and continual strategy optimization in open-ended visual reasoning.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Computer Vision: Large Vision Models
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent Vision-Language Models (VLMs) exhibit strong perceptual reasoning abilities, yet they often struggle to adapt efficiently when encountering novel tasks at test time. In contrast, humans leverage the metacognitive model with memory, enabling continuous strategy refinement through metacognitive control when faced with new challenges. To bridge this gap, we propose metacognitive test-time reasoning (MCTR), a framework that equips models with the ability to learn, adapt, and improve during test time through metacognitive self-updating. Inspired by the dual structure of human metacognition, MCTR comprises meta-level and object-level VLM reasoning modules, each equipped with dedicated memory systems for hierarchical adaptive reasoning. Specifically, MCTR consists of (1) a meta-reasoning module which incrementally builds a structured memory by discovering and storing task-relevant rules, environmental patterns, and action-outcome relationships from test-time observations as natural language descriptions; and (2) an action-reasoning module that determines optimal actions through context-aware perception and strategic reasoning by dynamically retrieving and integrating knowledge from memory. The action-reasoning module continuously updates its policy through proposed metacognitive test-time reinforcement learning, adapting as knowledge memory evolves. We evaluate MCTR on 45 Atari games (33 seen, 12 unseen). MCTR demonstrates robust test-time adaptation, achieving 9/12 top-1 results on unseen games compared with baselines. Analyses through ablations, learning dynamics, and case studies reveal the complementary contributions of both components and show meta-reasoning evolving toward human-like adaptation strategies.

Problem

• Enhancing Vision-Language Models' adaptation to novel tasks during test time
• Bridging human metacognitive reasoning with AI through memory-augmented modules
• Enabling continuous strategy refinement via hierarchical adaptive reasoning systems

Innovation

• Metacognitive framework with dual reasoning modules
• Hierarchical memory systems for adaptive test-time learning
• Self-updating policy via metacognitive reinforcement learning