Vibe-Coding: Feedback-Based Automated Verification with no Human Code Inspection, a Feasibility Study

2026-04-16

AI Summary

This work addresses the challenge of reliably verifying large language model (LLM)-generated adaptive managers for collective adaptive systems (CAS) and ensuring their runtime correctness without manual code inspection. The authors propose an approach that integrates adaptive cycles with a feedback-driven Vibe-Coding mechanism, introducing Functional Constraints Logic (FCL)—a novel first-order temporal logic—to formally specify functional constraints. By synergistically combining architectural constraints, FCL-based verification, and runtime monitoring, the method generates fine-grained error feedback to iteratively guide the LLM toward correct implementations. Evaluated on the Dragon Hunt CAS case study, the approach produces effective adaptive managers within only a few iterations, substantially outperforming baseline methods that rely on coarse-grained metrics, thereby demonstrating the feasibility of fully automated verification and repair.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Adversarial Agents

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Vibe coding inherently assumes iterative refinement of LLM-generated code through feedback loops. While effective for conventional software tasks, its reliability in runtime-adaptive systems is unclear -- especially when generated code is not manually inspected. This paper studies feedback-based automated verification of LLM-generated adaptation managers in Collective Adaptive Systems (CAS). We focus on the key challenges of verification in the loop: how to detect failures of generated code at runtime and how to report them precisely enough for an LLM to fix them. We combine the adaptation loop with a vibe-coding feedback loop where correctness is checked against (i) generic architectural constraints and (ii) functional constraints formalized in Functional Constraints Logic (FCL), a novel first-order temporal logic over potentially finite traces. Conducting the Dragon Hunt CAS case study, we show that fine-grained constraint violations provide actionable feedback that typically yields a valid adaptation manager within a few iterations, while simple coarse metric-based feedback often stalls. Our findings suggest that feedback precision is the dominant factor for reliable vibe coding in systems designed by domain experts with no programming skills, thereby obviating the need for human code inspection.

Problem

• automated verification
• LLM-generated code
• Collective Adaptive Systems
• runtime feedback
• code correctness

Innovation

• Vibe-Coding
• automated verification
• Functional Constraints Logic (FCL)
• Collective Adaptive Systems
• feedback loop