GitSwarm: Decentralized Compounding Inference

2026-10-04

AI Summary

This study addresses the challenge that reasoning computations in long-horizon tasks are difficult to accumulate and reuse across attempts, and intermediate results lack persistence. To this end, we propose the "compound interest reasoning" paradigm. This method constructs an asynchronous multi-agent system based on a shared Git repository, introducing the first decentralized framework that supports branched collaboration for exploration, verification, and synthesis. Through atomic commits and explicit semantic dependency recording, it enables homogeneous agent collaboration with structured memory. Experiments demonstrate that the proposed framework achieves full solutions on IMOProofBench, attains 79.4% accuracy on ProgramBench, and significantly enhances neural architecture search performance, with a high proportion of intermediate results being effectively reused in subsequent attempts.

Abstract

Long-horizon problem solving and scientific research require computation to accumulate across successive attempts. Partial solutions, experimental findings, and unsuccessful approaches can inform later work, yet most inference-time computation is organized around individual trajectories or candidates rather than a persistent body of reusable work. We call this paradigm compounding inference: organizing inference-time computation so that intermediate work persists and can be inspected, extended, combined, or challenged by subsequent computation. We instantiate compounding inference in GitSwarm, an asynchronous system where homogeneous agents independently decide how to advance a task while collaborating through structured persistent memory. Agents explore, experiment, verify, refine, and synthesize previous work in a shared, branch-able Git repository. Atomic commits preserve intermediate artifacts, while explicit semantic dependencies record how later contributions build on work across branches. We evaluate GitSwarm on long-horizon problem solving and sustained GPU-backed experimental research. On IMOProofBench-Advanced, GitSwarm solves all 30 problems in one run using GPT-5.5. On ProgramBench, it achieves a $79.4\%$ mean score, versus $65.1\%$ for the strongest reported baseline under the stated budget. On three neural architecture research tasks (Residual Matrix Transformer, Looped Transformer, NanoChat), GitSwarm improves upon the starting architectures through successive experimentation. Beyond final performance, we measure whether computation accumulates: on ProgramBench, $94.7\%$ of contributions are subsequently built upon, while the selected solution's ancestry covers $82-93\%$ of the contribution graph. These results show that inference-time computation can accumulate across otherwise independent episodes, forming an evolving body of work that subsequent inference can reuse.

Problem

• compounding inference
• long-horizon problem solving
• inference-time computation
• decentralized collaboration
• knowledge accumulation

Innovation

• Compounding Inference
• Multi-Agent Collaboration
• Persistent Memory
• Asynchronous System
• Git-based Repository