iARCS: Iterative Agentic RL for Controllable 3D Scene Generation

2026-08-06

AI Summary

Existing 3D scene generation methods struggle to reliably satisfy task-critical functional constraints such as navigability and reachability, limiting the practical utility of synthetic data. This work proposes an iterative agent-based reinforcement learning framework that first enhances physical plausibility and layout quality through pretraining with generic rewards, then leverages a large language model (LLM) to generate executable, task-specific reward programs. These LLM-generated rewards are integrated into a feedback-driven reinforcement learning loop for iterative refinement. By uniquely combining LLM-synthesized reward functions with iterative reinforcement learning, the approach significantly improves adherence to functional constraints while preserving scene diversity, thereby enhancing downstream task performance.

Abstract

Synthetic 3D scene generation is increasingly used as a data source for computer vision and embodied AI, but existing generators often optimize perceptual realism without reliably satisfying task-critical functional constraints. This mismatch limits the usefulness of synthetic data for downstream training, where accessibility, traversability, and spatial rule compliance are often essential. We present iARCS, an iterative agentic reinforcement learning framework that adapts a pretrained scene generator to natural-language task requirements. iARCS uses a two-stage strategy: universal-reward pretraining to improve physical plausibility and layout quality, followed by task-specific fine-tuning with LLM-generated reward programs that are iteratively refined from training feedback. Experiments show improved constraint fidelity on walkability, reachability, and clearance-focused tasks, effective task-specific constraint optimization, and competitive scene diversity. We further show that data generated by iARCS improves a base generator, supporting its value as a practical synthetic data generation tool rather than only a controllable scene editing method.

Problem

• 3D scene generation
• functional constraints
• synthetic data
• task-critical requirements
• controllable generation

Innovation

• iterative agentic reinforcement learning
• controllable 3D scene generation
• LLM-generated reward programs
• task-specific constraint optimization
• synthetic data generation