ai tooling

This work addresses the limited accessibility of large language model (LLM) and agent workflow development for engineers without machine learning expertise, primarily due to the absence of integrated testing, debugging, and reproducibility capabilities. To bridge this gap, the authors propose a novel IDE-native AI observability workflow, implemented as the AI Toolkit plugin for JetBrains IDEs. This approach seamlessly embeds trace capture and evaluation into standard run/debug cycles, enabling automatic hierarchical trace logging during execution, one-click dataset persistence, and a pluggable, unit-test-like evaluation framework. By minimizing environment setup and context-switching overhead, the solution facilitates routine evaluation and immediate trace visualization. Empirical data from the initial PyCharm release demonstrates high adoption, sustained usage, and low churn, confirming that IDE-integrated tooling effectively lowers the barrier to entry for non-ML developers.

This study addresses the challenge of systematically integrating generative AI into the entire software development lifecycle to enhance productivity while ensuring quality and governance. The authors propose a progressive integration framework centered on an innovative “AI harness” that unifies management of project context, access control, validation, logging, and human approval workflows. This architecture enables seamless co-evolution of technical capabilities, organizational processes, and quality assurance mechanisms. The framework supports a transition from informal AI assistance toward controlled, agent-based development and is empirically validated through a case study in a mid-sized software enterprise, offering both a practical roadmap and evidence-based foundation for AI-driven transformation in software engineering.

This study addresses the high manual overhead faced by DevOps teams in managing multi-interface cloud infrastructures. We propose and systematically evaluate an LLM-driven AI agent framework for automation. Methodologically, the agent unifies heterogeneous interfaces—including SDKs, CLIs, Infrastructure-as-Code (IaC) tools, and web portals—to support core tasks such as configuration deployment, monitoring/alerting, and incident remediation. Key contributions include: (1) the first evaluation framework specifically designed for AI agents in cloud infrastructure management; (2) identification and systematic mitigation of three critical bottlenecks—interface semantic gaps, action execution reliability, and security constraint compliance; and (3) domain-specific optimization strategies validated in real-world deployments, demonstrating both task feasibility and cross-scenario generalizability. Our work establishes a reusable methodology and empirical benchmark for AI-native cloud operations.

This study addresses the unclear adoption of artificial intelligence (AI) libraries in open-source software and their impact on development practices and community engagement. It presents the first systematic analysis of 157.7k Python and Java open-source repositories, comparing projects that adopt AI libraries with those that do not across dimensions such as development activity, community participation, and code complexity. Leveraging repository metadata and software metrics, the research employs a large-scale empirical methodology to uncover the technical and socio-organizational effects of AI library integration within the open-source ecosystem. By bridging a critical gap at the intersection of AI and open-source software engineering, this work provides foundational empirical evidence to inform the design and governance of AI-driven software development practices.

Current AI assistant features in IDEs exhibit a significant misalignment with developers’ authentic needs, necessitating a systematic understanding of heterogeneous user requirements. Method: We conducted semi-structured interviews with 35 practitioners—comprising AI adopters, attriters, and non-users—to empirically construct the first human-AI interaction design space for IDE-integrated AI assistants. Through thematic coding and cross-cohort comparative analysis, we identified fundamental divergences across user groups along five dimensions: reliability, privacy, personalization, proactivity, and ethical concerns. Contribution/Results: We propose a role-driven, five-dimensional design framework—encompassing technical robustness, interaction modality, goal alignment, skill abstraction, and cognitive offloading—alongside 12 actionable design guidelines. This work advances IDE AI tools toward greater reliability, contextual awareness, privacy-by-design, and seamless workflow integration.

This study addresses the challenges of frequent requirement changes, quality assurance, and delivery efficiency in agile software development by systematically investigating the application of artificial intelligence—particularly machine learning and natural language processing—in critical phases such as requirements management, code generation, and testing. Through a comprehensive literature review and empirical research involving industry practitioners, the work demonstrates that AI not only enhances existing agile practices but also fundamentally reshapes software development paradigms by introducing automation and intelligent decision-making capabilities. This transformation significantly improves development efficiency, product quality, and team responsiveness, thereby enabling a synergistic advancement in quality, speed, and innovation.

This study addresses the disruptive impact of large language models and AI agent systems—capable of generating vast volumes of code—on traditional software engineering paradigms. The work proposes a new paradigm centered on agent orchestration, verification of AI-generated code, and structured human-AI collaboration. Through a structured synthesis of literature review and industry practices, it constructs a comprehensive framework encompassing education, toolchains, lifecycle management, and governance. The research reveals a fundamental shift in the nature of code—from a scarce craft artifact to a consumable commodity—and identifies the evolving role of software engineers toward system design, semantic validation, and accountability oversight. It further establishes key directions such as a verification-first software development lifecycle, offering both theoretical grounding and practical pathways for software engineering transformation in the AI era.

This study addresses the inadequacy of the current U.S. Department of Defense software acquisition pathways in effectively managing the unique challenges posed by artificial intelligence systems—particularly their data dynamism, model evolution, and governance requirements. Through scenario-based policy analysis, the authors embed a hypothetical AI-enabled project into critical junctures of the existing acquisition process to systematically evaluate how policies translate into practice. The analysis reveals that core guidance documents lack operational specificity, while AI-related controls are fragmented across supplementary materials, leading programs to rely on inconsistent local interpretations. To bridge this gap, the paper proposes a dedicated AI acquisition sub-pathway alongside targeted documentation enhancements, substantially aligning policy with practice in areas such as data provenance, lifecycle management, and human oversight.

This study addresses the prevailing gap in AI education, which emphasizes model development while neglecting system engineering practices, leaving students ill-equipped to handle real-world challenges such as architectural design, deployment, and monitoring. To bridge this gap, the authors implemented a master’s-level course in which students built a movie recommendation system under realistic constraints, with a focus on integrating AI components into robust software systems, adopting data-driven machine learning practices, and cultivating systems-level thinking. Using a mixed-methods approach—combining analysis of student project artifacts with survey data—the research evaluates learners’ performance in architectural decision-making, integration of heterogeneous models, and adaptation to evolving requirements. Findings reveal common difficulties students encounter in AI system engineering and demonstrate the course’s effectiveness in addressing critical deficiencies in AI engineering education and enhancing systems-aware competencies.