DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving

2026-02-28

AI Summary

Large language models (LLMs) struggle in autonomous driving tasks due to the discretization of numerical values into textual tokens, which fails to preserve the semantic ordering of digits and consequently undermines numerical reasoning accuracy and control command precision. To address this limitation, this work proposes DriveCode—a domain-specific numerical encoding method tailored for autonomous driving. DriveCode introduces dedicated numerical embeddings and a numerical projector that directly map continuous numerical values into the LLM’s latent space, enabling seamless fusion with multimodal features. Evaluated on the OmniDrive, DriveGPT4, and DriveGPT4-V2 datasets, the proposed approach significantly improves trajectory prediction and control command generation, demonstrating its effectiveness and superiority in LLM-driven autonomous driving systems.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language models (LLMs) have shown great promise for autonomous driving. However, discretizing numbers into tokens limits precise numerical reasoning, fails to reflect the positional significance of digits in the training objective, and makes it difficult to achieve both decoding efficiency and numerical precision. These limitations affect both the processing of sensor measurements and the generation of precise control commands, creating a fundamental barrier for deploying LLM-based autonomous driving systems. In this paper, we introduce DriveCode, a novel numerical encoding method that represents numbers as dedicated embeddings rather than discrete text tokens. DriveCode employs a number projector to map numbers into the language model's hidden space, enabling seamless integration with visual and textual features in a unified multimodal sequence. Evaluated on OmniDrive, DriveGPT4, and DriveGPT4-V2 datasets, DriveCode demonstrates superior performance in trajectory prediction and control signal generation, confirming its effectiveness for LLM-based autonomous driving systems.

Problem

• numerical encoding
• large language models
• autonomous driving
• tokenization
• numerical reasoning

Innovation

• numerical encoding
• large language models
• autonomous driving
• multimodal integration
• number projector