Model-Assisted and Human-Guided: Perceptions and Practices of Software Professionals Using LLMs for Coding

2025-10-10

AI Summary

This study investigates software practitioners’ real-world usage practices, perceptions, and adoption challenges regarding large language models (LLMs). Employing a mixed-methods survey—comprising quantitative questionnaires and qualitative interviews with 131 global practitioners—the work systematically characterizes LLM application patterns and human-AI collaboration dynamics across coding, debugging, and documentation generation. Results indicate that LLMs significantly enhance development efficiency, reduce cognitive load, and accelerate skill acquisition; however, critical limitations persist, including hallucination, constrained contextual understanding, and risks to code security and ethical integrity. The primary contribution is the proposal of a “prudent collaboration” framework—a first-of-its-kind empirically grounded, cross-regional foundation for designing, evaluating, and engineering AI-augmented programming tools.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Applications

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models have quickly become a central component of modern software development workflows, and software practitioners are increasingly integrating LLMs into various stages of the software development lifecycle. Despite the growing presence of LLMs, there is still a limited understanding of how these tools are actually used in practice and how professionals perceive their benefits and limitations. This paper presents preliminary findings from a global survey of 131 software practitioners. Our results reveal how LLMs are utilized for various coding-specific tasks. Software professionals report benefits such as increased productivity, reduced cognitive load, and faster learning, but also raise concerns about LLMs' inaccurate outputs, limited context awareness, and associated ethical risks. Most developers treat LLMs as assistive tools rather than standalone solutions, reflecting a cautious yet practical approach to their integration. Our findings provide an early, practitioner-focused perspective on LLM adoption, highlighting key considerations for future research and responsible use in software engineering.

Problem

• Understanding how software professionals practically use LLMs in coding workflows
• Investigating practitioners' perceptions of LLM benefits and limitations for development
• Exploring the assistive role of LLMs rather than standalone solutions in coding

Innovation

• Model-assisted human-guided coding with LLMs
• LLMs utilized for various coding-specific tasks
• Treating LLMs as assistive tools not standalone solutions