toolchain

This study presents the first empirical investigation into the evolution of CI/CD configurations in machine learning (ML) projects. Addressing the lack of understanding regarding how CI/CD configurations co-evolve with ML components, the authors analyze 508 open-source ML projects, 343 manually annotated commits, and 15,634 automated CI/CD commits. They propose a novel 14-category taxonomy capturing synergistic changes between CI/CD and ML components, develop a dedicated clustering tool to identify recurrent evolutionary patterns, and establish an empirically grounded model linking developer experience to CI/CD configuration modification behavior. Results show that 61.8% of CI/CD-related commits involve build strategy modifications; common anti-patterns—including dependency hardcoding and missing test frameworks—are identified; and senior developers modify CI/CD configurations more frequently and effectively than juniors, confirming the critical role of experience in CI/CD maintenance.

This work addresses the challenge of automated configuration migration across CI platforms—particularly from Travis CI to GitHub Actions—where manual translation is error-prone and labor-intensive. We propose an LLM-based translation framework grounded in empirical analysis of 811 real-world migration cases, enabling the first quantitative characterization of configuration conversion effort. We introduce a four-category taxonomy of translation problems and identify recurring developer pain points. Methodologically, we design a composite prompting strategy integrating documentation-guided instruction, iterative refinement, and in-context learning to enhance LLM robustness and fidelity. Evaluation on GPT-4o shows our approach achieves 75.5% end-to-end build success rate—nearly tripling the performance of baseline prompting—while substantially reducing manual intervention. The framework provides a reproducible, quantitatively evaluable pathway for intelligent CI/CD configuration migration.

This study addresses the quality-efficiency-cost imbalance in industrial CI/CD pipelines caused by heterogeneous failure types. We propose a process refactoring paradigm centered on two critical milestones: code integration (pre-merge) and product release. First, we systematically define “good failures” (early-detected, low-cost) versus “bad failures” (late-occurring, high-blocking). Grounded in empirical studies across four enterprises—including workflow mapping and failure root-cause modeling—we develop a transferable pre-merge failure governance framework. Evaluation results show a 37% reduction in average feedback latency, a 29% decrease in spurious build overhead, significant improvement in developer throughput, and optimized cloud resource utilization. Our core contribution lies in transcending conventional stage-based pipeline segmentation to enable failure-driven, fine-grained process control—marking a paradigm shift toward adaptive, cost-aware CI/CD orchestration.

Existing online feature modeling tools suffer from functional limitations, discontinued maintenance, or reliance on local installations, hindering collaborative development of configurable systems. To address this, we propose the first fully web-based, lightweight, and collaborative feature modeling toolbox. Built upon FeatureIDE’s core library using TypeScript and React, it enables online loading, visual editing, format conversion, persistent storage, and real-time multi-user collaboration of feature models. The tool requires no local installation and executes natively in browsers with low-latency synchronization, significantly improving accessibility and team productivity. A preview version has been open-sourced and empirically validated for usability and extensibility, effectively bridging a critical gap in current online feature engineering infrastructure.

Continuous Integration (CI) practices suffer from severe monitoring deficiencies: developers largely neglect critical metrics such as “build health” and “time-to-fix failed builds,” while mainstream CI services offer only weak native monitoring capabilities, forcing reliance on fragmented and often redundant third-party tools. Method: We conducted a triangulated investigation—including documentation analysis, developer surveys, functional audits of CI platforms, and case studies of open-source projects—to systematically identify cognitive gaps and practical monitoring needs. Contribution/Results: Our study provides the first empirical evidence that although over 80% of developers track test coverage, only a minority monitor build health or timeliness; further, all major CI services lack built-in multidimensional monitoring support. These findings establish an evidence-based foundation for designing next-generation CI monitoring frameworks and prioritizing tooling enhancements.

This study addresses the lack of standardized guidance for effectively integrating technical debt management tools into existing CI/CD practices, which hinders the continuous control of technical debt. By systematically analyzing approximately 600,000 Travis CI configuration files and 50,000 auxiliary scripts from GitHub, the authors identify 3,684 pipelines that integrate technical debt management tools. Their findings reveal that such integrations predominantly rely on external script invocations and frequently exhibit configuration anti-patterns, notably the absence of feedback mechanisms. This work provides empirical evidence of current integration practices and prevalent anti-patterns, offering actionable insights to inform the design of better tooling and improve CI/CD integration strategies for technical debt management.

This work addresses the growing complexity of CI/CD pipelines and the lack of structured analysis capabilities in existing tools for understanding their behavior, failures, and version evolution. The authors propose an innovative approach that uniquely integrates digital twin technology with BPMN-based modeling in DevOps contexts. By automatically parsing raw CI configurations and execution logs, the method constructs structured, high-level process models that enable pipeline visualization, failure traceability, and cross-version comparison. Evaluated across multiple open-source projects, the approach demonstrates effectiveness in monitoring, evolutionary analysis, and fault diagnosis, offering a modular and extensible foundational framework for the analysis and optimization of CI/CD pipelines.

Automatically generating YAML configuration files that are both structurally valid and compliant with multiple continuous integration (CI) service specifications remains a significant challenge, and the capabilities of current large language models (LLMs) on this task are not well understood. This work introduces DOC2CI, the first cross-CI benchmark dataset comprising 3,363 document–YAML pairs, and systematically evaluates 14 open-source models alongside GPT-series models. A novel failure taxonomy is proposed to uncover the root causes of model discrepancies, and this study provides the first empirical evidence that document similarity and structural validity constitute distinct optimization objectives. Experiments reveal that even the largest models achieve an Exact Match rate below 3.1%; while 97% of generated outputs are syntactically parseable, only 71% conform to the target service schema. Schema-guided post-hoc repair without additional training boosts structural validity to 94%, whereas fine-tuning improves document similarity at the expense of standalone structural correctness.

This work addresses the fragility, inefficiency, and strong platform coupling commonly found in CI/CD pipelines for legacy COBOL systems, which often result in high maintenance costs and vendor lock-in. To overcome these challenges, the authors propose a portable CI/CD architecture tailored for highly secure and compliance-driven environments. The approach leverages OCI-compliant container images preloaded with COBOL toolchains, introduces a platform abstraction layer, integrates multiple repositories, and employs Groovy script refactoring to achieve platform-agnostic continuous integration and delivery. Empirical evaluation demonstrates that the proposed solution significantly enhances efficiency—reducing pipeline execution time by 82%—while simultaneously improving system portability, security, and maintainability. This architecture offers a reusable paradigm for modernizing legacy COBOL applications within regulated domains.

This study addresses the lack of systematic, large-scale analyses of structural properties in software feature models, which has hindered the understanding and evolution of variability models. For the first time, it systematically applies large-scale network analysis to 5,709 variability models drawn from 20 repositories. By constructing graphs capturing transitive dependencies and conflicts among features, and integrating graph modeling with network-theoretic and statistical analyses, the work uncovers cross-domain structural commonalities—such as dependency dominance, high centralization, and characteristic degree distributions—as well as domain-specific deviations. These findings provide novel empirical insights and a foundation for identifying pivotal features, guiding modular decomposition, and assessing structural fragility in variability-intensive systems.