Toward a Metrology for Artificial Intelligence: Hidden-Rule Environments and Reinforcement Learning

2025-09-07

AI Summary

This work addresses the challenge of simultaneously inferring latent control rules and learning optimal policies via reinforcement learning in partially observable Hidden Rule Games (GOHR), specifically for a 6×6 board-clearing task. To this end, we introduce a standardized GOHR benchmark platform and systematically compare feature-centric versus object-centric state representations in terms of rule inference efficiency and policy generalization. We propose a novel Transformer-based advantage Actor-Critic (A2C) architecture that jointly optimizes rule induction and action selection. Experimental results demonstrate that object-centric representations substantially improve cross-rule transferability. Our method outperforms baselines on both multi-episode rule-switching and zero-shot rule generalization tasks, validating its effectiveness in implicit rule reasoning and co-learning of representation and policy.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Reinforcement Learning
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We investigate reinforcement learning in the Game Of Hidden Rules (GOHR) environment, a complex puzzle in which an agent must infer and execute hidden rules to clear a 6$ imes$6 board by placing game pieces into buckets. We explore two state representation strategies, namely Feature-Centric (FC) and Object-Centric (OC), and employ a Transformer-based Advantage Actor-Critic (A2C) algorithm for training. The agent has access only to partial observations and must simultaneously infer the governing rule and learn the optimal policy through experience. We evaluate our models across multiple rule-based and trial-list-based experimental setups, analyzing transfer effects and the impact of representation on learning efficiency.

Problem

• Investigating reinforcement learning in hidden-rule puzzle environments
• Developing strategies for inferring rules from partial observations
• Evaluating representation impact on learning efficiency and transfer

Innovation

• Transformer-based A2C algorithm for training
• Feature-Centric and Object-Centric state representations
• Partial observation hidden rule inference