How Researchers Use and Verify AI Coding Assistants: Tasks and Validation Practices in Scientific Programming

2026-09-18

AI Summary

研究调查了527名研究人员如何使用AI编码助手及验证其代码正确性，主要任务包括数据处理、可视化等，验证方式多为非正式且个人化。

Abstract

Generative AI has entered research programming, yet there is little evidence about which tasks researchers hand to it or how they decide whether its code is correct. We draw on 527 free-text responses to a 2025 survey of researchers who write code, most of them at U.S. universities. In each response, a researcher recounts a single task from their own work, the way they used an AI tool for it, and what they did to assess the result. We coded the task and the evaluation strategies reported, and related both to programming experience, research area, and confidence ratings. Use was concentrated in five tasks: data handling, visualization, debugging, mathematical/scientific computing, and statistical analysis. Evaluation was informal and individual. Over half of accounts described running the generated code, while automated tests and review by another person were rare. Use cases and evaluation strategies varied little with programming experience, but confidence did: Less experienced programmers trusted the AI more than themselves, and experienced programmers the reverse. Evaluation confidence was not associated with the strategies reported. Its strongest correlates were confidence in the tool and in oneself. Validating AI contributions to scientific code rested largely on individual judgment, outside shared infrastructure for testing or review. Interfaces could support task-appropriate evaluation rather than leave it to the user.

Problem

• AI Coding Assistants
• Validation Practices
• Scientific Programming
• Task Delegation
• Code Verification

Innovation

• AI Coding Assistants
• Validation Practices
• Scientific Programming
• Task-Specific Use
• Individual Judgment