model sdk integration

This work addresses the inefficiency and high cost of migrating deep learning models across frameworks—such as from TensorFlow to JAX—in large-scale AI systems. To tackle this challenge, the authors propose an automated, multi-agent collaborative migration approach that integrates static code analysis with an AI-driven planner to generate precise migration instructions. A coordinator and encoder work in tandem, leveraging AI-generated, example-driven migration guides to achieve high-fidelity translation without requiring test code. Innovatively, an AI-based evaluator assesses migration quality, establishing a self-reinforcing development loop. Evaluated in real-world, large-scale production environments, the method accelerates framework migration by 6.4–8×, substantially expediting model infrastructure evolution.

Scientific computing at the convergence of HPC and AI faces challenges including fragmented toolchains, poor hardware portability, and low cross-paradigm coordination efficiency. To address these, this paper introduces the first PyTorch-level unified abstraction framework enabling tight HPC/AI co-design. Our approach features a hardware-agnostic operator registration mechanism, a dynamic workflow orchestration engine, and an automatic mixed-precision scheduler. Implemented via a C++/Python hybrid architecture, it integrates MPI+NCCL communication, ONNX Runtime extensions, an adaptive graph compiler, and a declarative task-graph DSL. Evaluated on leadership-class supercomputers—including Eagle and Perlmutter—the framework achieves a 3.2× throughput improvement in AI training and reduces end-to-end latency by 67% for coupled HPC simulation and ML inference. It has been deployed in production scientific workloads, including climate modeling and plasma simulation.

Industrial-scale recommendation and ranking models feature highly complex and continuously evolving architectures, rendering traditional optimization approaches—based on manual intervention or module-level rules—difficult to scale. This work proposes the first extensible and customizable operator-level automatic transformation framework integrated into PyTorch 2.x. By leveraging FX intermediate representation, the PT2 compiler, predefined pattern matching, and a greedy search algorithm, the framework achieves general-purpose model optimizations while strictly preserving computational semantics. Evaluated on real-world industrial recommendation models, the approach delivers up to 63% inference speedup, a 6% reduction in peak memory usage, and over 400 seconds of compilation time savings. The implementation has been open-sourced as part of PyTorch 2.x.

This work addresses the limitations of existing agent orchestration frameworks, which rely on external schedulers and incur substantial context overhead, require state-of-the-art large language models, and risk exposing proprietary workflows. To overcome these issues, the authors propose compiling multi-node agent workflows—comprising up to 55 nodes—directly into the weights of a small fine-tuned language model, thereby creating what they term “underground agents.” This approach provides the first systematic demonstration that complex workflows can be effectively internalized within model parameters. By integrating structured workflow representations, task-specific knowledge injection, and decision-hub modeling, the method achieves performance comparable to leading models on tasks such as travel booking, Zoom customer support, and insurance claims processing, while reducing inference costs by two orders of magnitude and substantially diminishing reliance on conventional orchestration frameworks.

The choice between PyTorch and TensorFlow remains a critical decision for AI researchers and practitioners, yet systematic, empirically grounded comparisons across usability, training/inference performance, and production deployment capabilities are lacking. Method: We conduct a comprehensive benchmarking study—including XLA, TensorRT, and other backend accelerators—analyze code complexity, evaluate cross-framework interoperability (ONNX, TorchScript, TFLite), and survey state-of-the-art literature and ecosystem tooling. Contribution/Results: Our analysis reveals fundamental paradigmatic differences: PyTorch’s dynamic computation graph excels in research agility and prototyping flexibility, whereas TensorFlow’s static graph design delivers superior end-to-end deployment maturity, multi-platform support (e.g., mobile, edge), and enterprise service integration. Computationally, both frameworks achieve comparable peak performance; however, their ecosystem roles have significantly diverged. We identify cross-framework interoperability and unified compiler-level optimization as pivotal future directions, providing evidence-based guidance for framework selection in AI development.

This work proposes ExecuTorch, the first end-to-end deployment framework natively integrated with the PyTorch ecosystem, addressing the fragmentation commonly encountered in edge AI deployment. By introducing an extensible backend abstraction, quantization-aware optimizations, and a unified model serialization format, ExecuTorch preserves the original model semantics while seamlessly targeting heterogeneous hardware—from microcontrollers to specialized accelerators—without sacrificing low latency or offline execution capabilities. The framework bridges the gap between research and production workflows, enabling consistent development and efficient deployment across a broad spectrum of devices, ranging from wearables to compute clusters, thereby significantly enhancing both deployment efficiency and cross-platform consistency.

This work addresses key limitations in current AI assistants regarding the coupling of planning and execution, resource scheduling, and tool integration. To overcome these challenges, we propose IronEngine—a general-purpose, efficient, and scalable AI assistant platform featuring a decoupled three-stage pipeline of deliberation, model switching, and execution. The architecture incorporates a unified orchestration core, a hierarchical memory system, and an MCP-compatible framework. IronEngine supports VRAM-aware scheduling across 92 models, intelligent routing with automatic error correction for over 130 tools, and multi-endpoint APIs alongside hardware interfaces. Experimental results demonstrate that IronEngine achieves a 100% task completion rate on file operation benchmarks, with an average execution time of 1,541 seconds, significantly outperforming mainstream systems such as ChatGPT, Claude Desktop, and Cursor.

This study addresses the gap between pre-trained models and their practical adoption by systematically linking real-world usage code from GitHub with Hugging Face model cards. Existing model documentation often lacks concrete, interpretable code examples, hindering effective utilization by developers. To bridge this gap, the authors construct CodeXHug—the first structured dataset capturing code usage patterns of pre-trained models—by collecting 7,325 models and 20,545 associated Python files. Through code parsing, clustering, and statistical analysis, they extract reusable, representative coding paradigms that reflect how models are actually employed in practice. This empirical resource provides actionable insights for enhancing the usability and comprehensibility of model documentation, thereby supporting more effective integration of pre-trained models into real-world applications.

This work addresses the challenge of inefficient programming models for custom AI accelerators, which struggle to support diverse machine learning operators effectively. For the first time, the authors successfully deploy the Triton language on Meta’s in-house MTIA-2i accelerator by introducing a new compiler backend, enhancing TorchInductor’s code generation, and incorporating minimal language extensions tailored to the hardware’s characteristics. This approach bridges the programming gap between high-level ML frameworks and heterogeneous hardware, achieving kernel performance comparable to hand-optimized C++ implementations while preserving developer productivity. The solution has been deployed across approximately 60 model types, covering 50% of network layers and 47% of non-GEMM execution time.

Existing LLM streaming frameworks lack unified support for token-level flow control, queue management, scheduling, and backpressure, often relying on ad hoc callbacks that lead to system complexity and unreliability. This work proposes AiFlow—the first token-native reactive orchestration model—which formalizes LLM generation streams as typed Context<T> events propagated through a directed stream graph. Node guardians uniformly manage queue boundaries, concurrency, ordering, and fault tolerance. AiFlow supports DSL/JSON graph compilation and provides static verification for type safety, stateful concurrency, and injection compatibility. Its reactive architecture, combined with runtime-enforced policies, guarantees bounded memory usage. Experiments demonstrate a 70.9–94.7% reduction in time-to-first-token latency, stable and controllable queue depths, and a 93.7–96.5% decrease in maximum queue length.