containerized deployment

Although Docker is widely assumed to ensure reproducibility of software environments, its practical efficacy remains insufficiently validated. This study presents the first systematic investigation combining a literature review with large-scale empirical analysis of 5,298 real-world GitHub projects. By reconstructing Docker images, performing differential comparisons, and mining workflow patterns, we quantitatively assess the reproducibility of Docker builds and the effectiveness of recommended best practices. Our findings reveal that a significant proportion of Docker builds are not reproducible, and existing best practices offer limited improvements in practice. These results challenge the prevailing assumption that “containers guarantee reproducibility” and provide empirical evidence and actionable insights for enhancing reproducibility in computational research.

This work addresses the challenge of constructing reproducible software stacks in high-performance computing (HPC) and AI convergence scenarios, where constraints such as lack of root privileges, network isolation, and heterogeneous language environments hinder conventional tooling. For the first time, it systematically applies Nix’s fully isolated build model together with its declarative flake configuration system to HPC-AI hybrid environments, enabling unified management of C/C++ and Python dependencies. By automatically generating Apptainer containers, the approach ensures consistency between local development and deployment on supercomputing systems. The method effectively resolves critical issues including dependency discovery, system library leakage, and cross-project composition, achieving highly reproducible deployments across non-root workstations and production clusters. It demonstrates clear advantages over traditional module systems, Conda environments, and manual containerization, while also highlighting current gaps in Nixpkgs’ coverage of machine learning packages.

HPC research suffers from persistent reproducibility challenges due to resource exclusivity, restricted access, and heterogeneous environments. To address this, we propose CORRECT—the first GitHub Actions–based continuous integration framework specifically designed for HPC, enabling secure, automated reproducibility validation on remote supercomputing resources. CORRECT innovatively integrates fine-grained execution provenance tracking, lightweight containerized deployment, and strict permission isolation, thereby overcoming the fundamental incompatibility of conventional CI tools with HPC workload managers (e.g., Slurm). Empirical evaluation across three representative HPC application categories—scientific simulation, AI training, and performance benchmarking—demonstrates that CORRECT significantly improves automation, transparency, and documentation completeness in reproducibility assessment. It provides a scalable, production-ready engineering solution to advance reproducible research in HPC.

In the context of HPC–cloud convergence, containers improve deployment portability but suffer performance degradation due to ABI compatibility constraints that hinder hardware-specific optimization. To address this, we propose XaaS (eXecution-as-a-Service), a performance-portable container framework that defers critical compilation decisions—such as architecture-specific optimizations—to deployment time via source-code and intermediate representation (IR) containerization. Our approach integrates LLM-assisted identification of HPC software specialization mechanisms with IR-level deferred specialization and compiler pipeline analysis, enabling end-to-end automated, system-wide optimization across diverse architectures. Experiments demonstrate that XaaS preserves container deployment agility while achieving performance on par with native, hand-tuned builds—effectively breaking the longstanding “portability-at-the-cost-of-performance” bottleneck of conventional containers in HPC environments.

Research on containerization in multi-cloud environments remains fragmented, lacking a systematic, up-to-date synthesis. Method: We conduct a Systematic Mapping Study (SMS) spanning 2013–2024, analyzing 121 high-quality publications through bibliometric analysis, thematic coding, and ISO/IEC 25010 quality attribute modeling. Contribution/Results: We propose the first four-level classification framework—“Theme–Strategy–Quality Attribute–Tactic”—identifying four core research themes, 98 implementation strategies, 10 critical quality attributes, and 47 corresponding architectural tactics. Innovatively, we introduce a two-dimensional challenge-solution taxonomy organized along Security, Automation, Deployment, and Monitoring dimensions. This yields the first structured, reusable landscape of multi-cloud containerization, bridging theoretical research and industrial practice by supporting architecture design and technology selection—thereby addressing a longstanding gap in systematic knowledge integration for this domain.

This work addresses the pressing need in high-performance computing (HPC) systems to simultaneously support rapidly evolving AI/ML software, seamless scheduler integration, scalable launch capabilities, and production-grade performance—challenges that existing container solutions often fail to meet due to reliance on specialized runtimes incompatible with mainstream cloud-native ecosystems. The authors propose a cloud-native container architecture tailored for HPC, built atop an unmodified upstream Podman engine. By decoupling HPC-specific functionality into an integration layer through declarative runtime configuration, native scheduler execution, distributed shared image access, and standardized host capability injection, the design maintains full compatibility with the Open Container Initiative (OCI) ecosystem while supporting Kubernetes manifests and standard images such as those from NGC. Evaluations on a Cray EX GH200 system demonstrate that workloads including PyFR, SPH-EXA, Megatron-LM, and Pynamic achieve performance and scalability on par with Enroot+Pyxis, alongside significantly faster single-node container startup times.

This study addresses the limited understanding of containerization practices in machine learning (ML) projects, particularly the lack of systematic investigation into how iterative ML workflows affect Docker image size, build performance, and caching behavior. Through a large-scale empirical analysis of Dockerfiles from 1,993 open-source ML projects—integrating static parsing, build log tracing, cache behavior monitoring, and semantic mining of code commits—the work reveals ML-specific container usage patterns and proposes seven ML-tailored Dockerfile refactoring strategies. The findings show that ML images average 10.27 GB in size and require 8.84 minutes to build; 44.4% of commits trigger rebuilds, with 96.4% caused by context changes and 71% exhibiting computational redundancy. The proposed methods substantially reduce image size and improve build efficiency.

While large language models (LLMs) can generate executable multi-service application environments, they often deviate from the architectural and security requirements essential for production deployment. This work proposes a method to automatically generate Dockerfiles and Docker Compose configurations solely from code repositories, evaluating deployment fidelity through end-to-end HTTP testing and structural comparison. It explicitly distinguishes between functional correctness and fidelity to deployment intent, deriving a minimal set of explicit deployment specifications that cannot be inferred automatically from source code alone. Experiments successfully reproduce the topology and dependencies of three heterogeneous multi-service systems, confirming functional feasibility; however, critical production-grade features—such as network isolation and multi-stage builds—are consistently absent, revealing fundamental limitations in current LLMs’ ability to model deployment intent.

This work addresses the widespread irreproducibility of academic Jupyter Notebooks caused by environment drift, missing dependencies, and implicit execution assumptions. The authors propose the first web-oriented, automated reproducibility engineering pipeline that systematically reconstructs and evaluates repository-level execution environments for notebooks hosted on GitHub. By leveraging dependency inference, auto-generated Docker containers, and isolated execution, the pipeline enables large-scale assessment of reproducibility. A novel four-category execution outcome framework is introduced to quantify reproduction fidelity. Evaluation on 443 real-world notebooks shows that containerization resolves 66.7% of dependency-related failures; however, only 46.3% achieve high output fidelity, demonstrating that while containerization is necessary, it is insufficient for bit-for-bit reproducibility. These findings underscore the critical need for systematic reproducibility evaluation in computational research.

This work addresses the challenges in edge and embedded application development—namely, heterogeneous software stacks, multi-language runtimes, and difficult debugging—which lead to rigid deployment workflows and complex fault diagnosis. To overcome these limitations, the paper proposes a novel architecture enabling unified end-edge-cloud development. Its core components include a single programming language, a retargetable runtime system, a local recording and replay mechanism for distributed events, and a cross-platform deployment framework. This design breaks down traditional debugging barriers in edge–cloud collaborative development, facilitating seamless scalability, consistent testing, and flexible deployment across heterogeneous environments. Evaluation of the prototype system demonstrates that the proposed approach significantly simplifies deployment procedures and enhances fault diagnosis efficiency.