Exploring Individual Factors in the Adoption of LLMs for Specific Software Engineering Tasks

2025-04-03

AI Summary

This study investigates how individual cognitive and behavioral factors differentially influence software engineers’ adoption of large language models (LLMs) across five specific tasks—including code generation and information retrieval. Drawing on a survey of 188 practicing engineers, it extends the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) to a task-level granularity for the first time and empirically validates the model using structural equation modeling (SEM). Results reveal task-dependent heterogeneity in the effects of individual factors: several factors exhibit paradoxical inhibitory effects when considered in isolation, and distinct drivers and barriers emerge for each task. Based on these findings, the study proposes two actionable strategies—seamless environmental integration and peer-mediated knowledge sharing—to guide LLM system design and team-level adoption management, thereby contributing both theoretical refinement of technology acceptance frameworks and practical, evidence-based recommendations for AI tool deployment in software engineering contexts.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The advent of Large Language Models (LLMs) is transforming software development, significantly enhancing software engineering processes. Research has explored their role within development teams, focusing on specific tasks such as artifact generation, decision-making support, and information retrieval. Despite the growing body of work on LLMs in software engineering, most studies have centered on broad adoption trends, neglecting the nuanced relationship between individual cognitive and behavioral factors and their impact on task-specific adoption. While factors such as perceived effort and performance expectancy have been explored at a general level, their influence on distinct software engineering tasks remains underexamined. This gap hinders the development of tailored LLM-based systems (e.g., Generative AI Agents) that align with engineers' specific needs and limits the ability of team leaders to devise effective strategies for fostering LLM adoption in targeted workflows. This study bridges this gap by surveying N=188 software engineers to test the relationship between individual attributes related to technology adoption and LLM adoption across five key tasks, using structural equation modeling (SEM). The Unified Theory of Acceptance and Use of Technology (UTAUT2) was applied to characterize individual adoption behaviors. The findings reveal that task-specific adoption is influenced by distinct factors, some of which negatively impact adoption when considered in isolation, underscoring the complexity of LLM integration in software engineering. To support effective adoption, this article provides actionable recommendations, such as seamlessly integrating LLMs into existing development environments and encouraging peer-driven knowledge sharing to enhance information retrieval.

Problem

• Examining individual factors affecting LLM adoption in software engineering tasks
• Investigating task-specific impacts of cognitive and behavioral attributes on LLM use
• Addressing gaps in tailored LLM system design for engineers' needs

Innovation

• Surveyed engineers using structural equation modeling
• Applied UTAUT2 to analyze adoption behaviors
• Integrated LLMs into development environments