AlgoPilot: Fully Autonomous Program Synthesis Without Human-Written Programs

2025-01-11

AI Summary

This work addresses the challenge of autonomously generating correct, human-readable algorithms without human prior knowledge. We propose an unsupervised trajectory language model (TLM)-guided reinforcement learning framework: the TLM is pretrained on execution traces of random Python functions, implicitly capturing syntactic and semantic constraints; an RL agent then searches directly over executable, interpretable algorithmic trajectories within this learned constraint space. To our knowledge, this is the first approach to synthesize classic algorithms—such as bubble sort—from scratch in a zero-shot, template-free, label-free, and example-free setting, yielding both functional correctness and human interpretability. Experiments on sorting tasks demonstrate efficacy, eliminating reliance on handcrafted templates or supervised signals common in prior algorithm discovery methods. Our framework establishes a new paradigm for autonomous computational logic discovery.

Technology Category

Natural Language Processing: Code Generation / Program Synthesis from Natural LanguageMachine Learning: Unsupervised & Self-Supervised LearningSearch and Optimization: Learning to Search

Application Category

Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Program synthesis has traditionally relied on human-provided specifications, examples, or prior knowledge to generate functional algorithms. Existing methods either emulate human-written algorithms or solve specific tasks without generating reusable programmatic logic, limiting their ability to create novel algorithms. We introduce AlgoPilot, a groundbreaking approach for fully automated program synthesis without human-written programs or trajectories. AlgoPilot leverages reinforcement learning (RL) guided by a Trajectory Language Model (TLM) to synthesize algorithms from scratch. The TLM, trained on trajectories generated by random Python functions, serves as a soft constraint during the RL process, aligning generated sequences with patterns likely to represent valid algorithms. Using sorting as a test case, AlgoPilot demonstrates its ability to generate trajectories that are interpretable as classical algorithms, such as Bubble Sort, while operating without prior algorithmic knowledge. This work establishes a new paradigm for algorithm discovery and lays the groundwork for future advancements in autonomous program synthesis.

Problem

• Automatic Algorithm Generation
• Machine Learning
• Codeless Programming

Innovation

• Reinforcement Learning
• Pre-trained Model
• Algorithm Generation