integrate models into training pipelines

Medical AI deployment is hindered by insufficient production readiness of machine learning (ML) training pipelines. Method: This paper presents a progressive architectural evolution path—monolithic (chaotic) → modular monolithic → microservices—using SPIRA, a voice-based pre-diagnostic system for respiratory insufficiency, as a case study. It systematically introduces continuous training (CT) and a software-quality-attribute-driven MLOps governance framework tailored to healthcare, integrating modular design, microservice decomposition, and engineered CI/CD pipelines. Contribution/Results: The approach significantly improves pipeline maintainability, fault tolerance, and scalability, enabling stable, iterative evolution of SPIRA. It establishes an “agile ML + robust software engineering” co-design paradigm, delivering a reusable methodology and practical benchmark for engineering medical AI in highly regulated environments.

Existing LLM tool-use methods rely on static data pipelines, decoupling data generation from model training—hindering adaptive focus on model weaknesses and effective removal of noisy labels, thus impairing training efficiency. This paper introduces the first open-source, model-aware data evolution framework, establishing a closed-loop training paradigm comprising three tightly integrated modules: *capability diagnosis*, *label verification*, and *error-driven expansion*. It jointly optimizes data and model through iterative refinement: greedy capability probing identifies model deficiencies; discriminator-guided label verification purifies training data; and error feedback steers targeted data augmentation. The resulting 8B model achieves state-of-the-art performance on BFCL-v3 and ACEBench—surpassing same-scale SOTA models and even outperforming its 32B data generator—marking the first demonstration of data–model co-evolution within an open-source ecosystem.

This work addresses the lack of a general, auditable dynamic control mechanism in existing training systems, which typically rely on framework-specific code. The authors propose the first cross-framework, open-source control plane that exposes training interfaces through a unified protocol, integrating declarative configuration, request validation, and secure control-point scheduling within the Aim workspace to enable metric monitoring, real-time intervention, and operational traceability. The system supports safe human and automated controller interventions during training while fully logging all operational trajectories. Experiments across five NLP and reinforcement learning tasks demonstrate its effectiveness, and the open-source implementation provides a foundation for reproducible human-in-the-loop training.

To address inefficiencies in model incremental updates, unfair policy evaluation, and high retraining costs under continual data growth, this paper proposes an end-to-end adaptive machine learning platform. Methodologically: (1) it introduces a declarative domain-specific language (DSL) to uniformly model data selection strategies (e.g., coreset, uncertainty sampling) and trigger policies (e.g., drift-aware scheduling); (2) it establishes the first composite model evaluation framework enabling fair, cross-policy comparison; and (3) it implements a co-optimization mechanism integrating sample-level fine-grained data selection with high-throughput training. Contributions include an open-source, extensible system architecture, a standardized benchmark ecosystem, and abstracted ML pipeline interfaces. Experiments demonstrate significant improvements in training throughput and substantial reductions in retraining overhead—while preserving model accuracy—and enable reproducible analysis across diverse strategy combinations.

Existing large-scale model training systems struggle to flexibly compose diverse parallelization strategies, often relying on manual expert tuning and lacking generality. This work proposes a programmable distributed training system that enables users to declaratively specify composite parallelism strategies—such as data, pipeline, and expert parallelism—through model annotations and scheduling directives. These specifications are compiled via a unified intermediate representation (IR) into device-level execution plans, fully decoupling strategy definition from runtime execution over a global compute-communication DAG. The system is the first to support automatic compilation of user-defined composite strategies, matching the performance of established approaches like ZeRO while significantly improving both performance and memory efficiency in complex scenarios such as DeepSeek-V3’s DualPipe.

This work addresses the lack of transparent, scalable, and deeply PyTorch-integrated open-source tools for post-training large language models, which hinders research iteration and deployment efficiency. We propose a native PyTorch-based, modular post-training framework centered on the principle of “hackability,” offering composable model builders, training recipes, and a distributed training stack that support diverse fine-tuning strategies and hardware configurations. While maintaining high performance and memory efficiency, the framework significantly enhances code transparency and research flexibility. Empirical evaluations demonstrate that it matches or even surpasses mainstream tools such as Axolotl and Unsloth across multiple post-training scenarios, thereby facilitating efficient and reproducible scientific exploration.

This work investigates whether model ensembles within the 1–3B parameter range can enhance code generation performance through execution feedback and pipeline architectures. We construct a generate-and-refine pipeline based on small language models, incorporate an execution feedback mechanism, and employ a NEAT-inspired evolutionary algorithm to search for optimal topologies. Our experiments reveal that execution feedback is pivotal—yielding performance gains exceeding four standard deviations on HumanEval and MBPP, primarily by correcting runtime errors—whereas increased topological complexity offers no significant benefit. The refinement component’s capability outweighs the identity of the generator, and single-run evaluations tend to overestimate evolutionary improvements; early stopping proves essential to prevent performance degradation. Moreover, specialized code models consistently outperform all combinations of general-purpose models.

This work addresses the lack of structured, verifiable, and reusable decision mechanisms in existing automated machine learning approaches for model selection. It proposes a semantic task profiling–based structured agent framework that leverages retrieval-augmented generation of historical cases and code modules to construct an intermediate representation blueprint encompassing modeling components, composition logic, and execution constraints. By integrating code execution feedback with a failure-aware reinforcement learning strategy, the framework enables memory-driven, traceable, multi-stage search optimization. Evaluated on financial time-series forecasting and generation tasks, the method significantly outperforms both conventional AutoML systems and current agent-based baselines, achieving consistent improvements in task performance, execution success rate, and decision interpretability.