Technical Challenges in Maintaining Tax Prep Software with Large Language Models

2025-04-25

AI Summary

Frequent tax code revisions impose high maintenance costs and error-prone updates on tax compliance software. Method: This paper proposes an LLM-based automated code diff generation and secure integration framework. It employs rule-guided prompt engineering and structured parsing of IRS official publications to systematically model and address six domain-specific challenges in tax-law-to-code translation—including legal ambiguity, implicit logical dependencies, and cross-year provision coupling. Contribution/Results: We introduce the first LLM-driven code diff paradigm tailored for tax software. Evaluated on ChatGPT and Llama, our approach demonstrates feasibility in accurately parsing statutory text and generating executable, semantically faithful code changes. Experiments show significant improvements in maintenance efficiency and correctness, establishing a reusable technical foundation for sustainable evolution of tax compliance systems.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

As the US tax law evolves to adapt to ever-changing politico-economic realities, tax preparation software plays a significant role in helping taxpayers navigate these complexities. The dynamic nature of tax regulations poses a significant challenge to accurately and timely maintaining tax software artifacts. The state-of-the-art in maintaining tax prep software is time-consuming and error-prone as it involves manual code analysis combined with an expert interpretation of tax law amendments. We posit that the rigor and formality of tax amendment language, as expressed in IRS publications, makes it amenable to automatic translation to executable specifications (code). Our research efforts focus on identifying, understanding, and tackling technical challenges in leveraging Large Language Models (LLMs), such as ChatGPT and Llama, to faithfully extract code differentials from IRS publications and automatically integrate them with the prior version of the code to automate tax prep software maintenance.

Problem

• Automating tax software updates from IRS publications using LLMs
• Reducing manual errors in tax law amendment interpretations
• Translating tax regulation changes to executable code efficiently

Innovation

• Automate tax code updates using Large Language Models
• Translate IRS publications to executable specifications
• Integrate code differentials automatically for maintenance