The Best Programming Language for Tokenmaxxing: An Investigation of Coding Agent Behavior Across Programming Languages

2026-07-24

AI Summary

This study systematically investigates the impact of programming language choice on token consumption during code generation by reasoning agents. Through controlled experiments, it evaluates token efficiency across five state-of-the-art models solving problems of equivalent difficulty in Python, Java, Rust, and OCaml. The work introduces a multidimensional trajectory analysis framework integrating trace re-execution, test-result vector abstraction, intermediate solution annotation, and natural language analysis. It presents the first quantitative assessment of cross-language token usage disparities among agents, revealing that language familiarity significantly influences generation behavior: in less familiar languages, agents are more prone to producing uncompilable code, redundantly modifying already correct solutions, and resorting to Python prototyping to circumvent direct implementation in the target language.

Abstract

Although coding agents are now very effective in a variety of programming languages, this paper first shows that the cost (in tokens) can very significantly by programming language. We evaluate five recent models on programming problems in Python, Java, Rust, and OCaml. We carefully control for problem difficulty, and show that there can be stark variation in token consumption that is consistent across models. To understand why, we analyze both the structure and content of agent trajectories. First, we re-execute every intermediate solution and abstract each trajectory as a sequence of test-outcome vectors, then label the work between successive solutions. This reveals agents repeatedly producing noncompiling solutions in unfamiliar languages and revising solutions that already pass. Second, we analyze trajectory text, finding that agents plan solutions in code comments, distrust the provided tests in favor of inputs they invent, and sidestep unfamiliar target languages by prototyping in Python. Our results show that by-language token efficiency is a metric that should be considered when benchmarking and developing multilingual agents, and, for the tokenmaxxer, a guide to the most expensive language to work in.

Problem

• token efficiency
• coding agents
• programming languages
• multilingual agents
• token consumption

Innovation

• token efficiency
• coding agents
• multilingual programming
• agent trajectory analysis
• language-dependent behavior