Spaghetti Architect: A Contamination-Resistant, By-Construction-Labelled, Multi-Language Code Dataset Generator

2026-07-20

AI Summary

Existing code corpora commonly suffer from uncontrolled semantics, structural disorder, ambiguous difficulty levels, lack of standardized references, and training data contamination. This work proposes a language-agnostic, on-demand generation framework based on a JSON intermediate representation, employing a deoptimizing transpiler to produce deliberately redundant and fully flattened programs in five languages (Python, JavaScript, Go, Java, and C++), with correctness guaranteed via oracle validation. The resulting dataset is intrinsically correct, contamination-resistant, and annotated with self-contained difficulty labels capturing both intrinsic and incidental complexity, enabling fine-grained control over code readability and complexity. Experiments demonstrate that the generated data effectively reflects a difficulty gradient—model accuracy drops to zero as intrinsic difficulty increases—and exhibits minimal performance discrepancy (<0.012) between development and novel test sets, with self-annotations providing significantly greater gains for weaker models than for stronger ones.

Abstract

Mined code corpora are abundant but uncontrolled: a snippet's semantics, surface "messiness," and difficulty are whatever the wild contained; there is no known-optimal reference to grade against; and any public sample may already sit in a model's training set. We present Spaghetti Architect, a tool that mints code datasets with the control such corpora lack. An anti-optimization transpiler maps a clean, language-agnostic JSON intermediate representation to deliberately redundant, fully-flattened programs in five languages (Python, JavaScript, Go, Java, C++); every program is compiled, run, and checked against a reference oracle, so each instance is correct by construction. The clean IR is a known-optimal reference, messiness is dialed by strictly-nested anti-pattern profiles, each instance is labelled along two orthogonal difficulty axes, intrinsic (problem size) and incidental (presentation at fixed semantics), and contamination is resisted by minting fresh variants from a private held-out seed. We give construct-validity evidence that the quality order moves established complexity and readability metrics, and report baselines on a four-model open ladder: exact match rises with scale, and the intrinsic knob collapses arithmetic-aggregation accuracy of even the strongest model to zero. Further, development-set scores equal freshly re-minted held-out counterparts within $|Δ|\le 0.012$ (comprehension) and $\le 0.011$ (refactoring); on identical programs, refactoring equivalence ($0.73 \rightarrow 0.99$) is scale-invariant while output prediction collapses; and ablating the generator's self-annotations shows they inflate the weakest model an order of magnitude more than the strongest ($-0.173$ vs $-0.017$): the annotated ladder resolves one of three adjacent pairs where the unannotated resolves all three. Open source (MIT), dependency-free, archived under a persistent DOI.

Problem

• code dataset
• contamination
• controlled generation
• difficulty labeling
• reference oracle

Innovation

• contamination-resistant
• by-construction correctness
• anti-optimization transpilation
• difficulty-controlled code generation
• self-annotated dataset