joint training

Traditional neural network training relies on fixed optimization pipelines, rendering it inflexible in dynamically addressing training instability and anomalies. To address this limitation, we propose the first interactive training framework enabling real-time human–AI collaborative intervention. Our method employs a lightweight control server that integrates expert human directives with AI agent feedback to dynamically adjust hyperparameters, data sampling strategies, and model checkpoints during training. This framework introduces, for the first time, a closed-loop interactive paradigm into neural network training, establishing a scalable human–machine collaboration interface coupled with automated response mechanisms. Experimental results demonstrate significant improvements in training stability, reduced sensitivity to initial hyperparameter configurations, and enhanced real-time responsiveness to user-specified customization requirements. The effectiveness is validated across multiple benchmark tasks.

Existing neural networks are constrained by hierarchical tree-like architectures, which preclude direct communication among sibling nodes and prohibit backward signal propagation to higher-level modules—resulting in weak inter-module collaboration and inefficient representation learning. To address these limitations, we propose the Synchronous Graph Neural Architecture (SGNA), organizing neural units into a modular, dynamically collaborative graph structure that enables arbitrary node-to-node communication and cross-layer signal transmission. Our key contributions are threefold: (1) introducing the first modular graph-structured paradigm for neural architecture design; (2) developing a systematic regularization framework to enforce module independence and load balancing; and (3) generalizing neural architecture search (NAS) to the space of directed acyclic graphs (DAGs). Extensive multi-task experiments demonstrate that SGNA significantly outperforms deep stacked baselines under parameter constraints, achieving superior collaborative representation capability and more comprehensive coverage of the search space.

Addressing the fundamental limitations of artificial neural networks—namely, poor systematic generalization, catastrophic forgetting, few-shot learning inefficiency, and inadequate multi-step reasoning due to the absence of human-like cognitive development mechanisms—this paper proposes a meta-learning framework explicitly optimizing for “motivation + practice.” Unlike conventional paradigms driven by indirect objectives (e.g., loss minimization), our framework integrates differentiable optimization, curriculum learning, task embedding, and practice trajectory modeling to enable models to autonomously acquire skill-improvement motivation and structured training opportunities during learning. Evaluated across four benchmark task families, it significantly outperforms state-of-the-art methods, demonstrating the efficacy of the motivation-practice mechanism for robust generalization and continual learning. Moreover, it establishes, for the first time, a computationally tractable cognitive development pathway for neural networks—bridging machine learning and cognitive science through a novel, principled paradigm.

To address the non-convex optimization challenge in neural network classification—specifically, susceptibility to poor local minima and flat regions—this paper proposes SAGRAD, a batch-training algorithm integrating Simulated Annealing (SA) with Møller’s Scaled Conjugate Gradient (SCG) method. Its core innovation lies in the first incorporation of SA into the SCG framework, enabling a dynamic restart and escape mechanism that synergistically balances global exploration and local acceleration. Implemented in Fortran 77, SAGRAD incorporates efficient Hessian-vector multiplication, optimized gradient computation, and an adaptive SA weight initialization strategy. Empirical evaluation across multiple classification benchmarks demonstrates significantly improved convergence robustness and generalization performance, while markedly reducing the probability of converging to suboptimal local minima. These results validate SAGRAD’s effectiveness and practicality for non-convex optimization in neural network training.

Existing machine learning approaches for combinatorial optimization problems (COPs) suffer from poor generalization and require problem-specific models, limiting scalability and practical deployment. Method: We propose the first general-purpose combinatorial optimization agent, featuring a hybrid attention block and a multi-type Transformer architecture to jointly encode heterogeneous graph structures and hierarchical features (node-, edge-, and global-level). A parameter-sharing lightweight problem adapter enables zero-shot transfer and cross-problem fine-tuning. Contribution/Results: Our agent is the first to unify solution strategies across diverse NP-hard COPs—including routing, scheduling, and classical graph optimization—without problem-specific architectural modifications. It achieves performance on par with dedicated models across multiple COP benchmarks, significantly outperforms existing transfer-learning methods, and successfully generalizes to unseen problem instances and types.

To address the low training efficiency, poor generalization, and strong hyperparameter sensitivity of neural networks across varying scales, this paper proposes a scale-invariant adaptive optimization framework. The method unifies adaptive optimization, second-order information approximation, learning-rate scaling invariance, and gradient compression, thereby decoupling optimization from model size and hardware configuration. Its core innovation lies in a scale-robust update paradigm that ensures stable optimization dynamics under variations in parameter count, batch size, and device count. Extensive experiments across diverse architectures—including MLPs, CNNs, and Transformers—and benchmarks—including CIFAR-10/100, ImageNet, and WikiText—demonstrate that the framework achieves 1.3–2.1× speedup over baseline optimizers, improved convergence stability, significantly reduced hyperparameter sensitivity, and eliminates the need for scale-specific hyperparameter tuning.

This study addresses the lack of systematic understanding regarding the necessity of joint training in multi-agent approaches for job shop scheduling with transportation resources. Through sensitivity analysis, it quantifies for the first time the coordination gap between joint and modular training paradigms and systematically evaluates their performance under varying conditions of resource scarcity and temporal dominance. The findings reveal that joint training generally outperforms both dispatching rules and modular methods across most scenarios; however, its advantage diminishes significantly in bottleneck environments where either transportation or processing is severely constrained. This suggests that modular training retains practical value in task-dominated settings. The results provide an environment-dependent basis for selecting appropriate multi-agent training paradigms in complex scheduling contexts.

This work addresses the challenge of gradient conflict in multi-task learning, which often degrades performance on certain tasks. While existing dynamic weighting methods such as MGDA aim to mitigate this issue, they suffer from high computational overhead and poor scalability. To overcome these limitations, this paper formulates gradient balancing as a bilevel optimization problem for the first time and introduces a zeroth-order optimization approach to efficiently decouple model training from weight adjustment. The proposed method substantially reduces computational cost while maintaining or even improving multi-task performance across both public benchmarks and industrial-scale datasets, achieving a favorable trade-off between efficiency and effectiveness.

In high-stakes deployment scenarios, machine learning models must simultaneously satisfy regulatory compliance, fairness, and computational constraints—yet existing post-hoc or fine-tuning methods often degrade performance, incur excessive latency, or prove infeasible. To address this, we propose a lightweight model editing framework based on graph-structured meta-networks. Our approach formulates model editing as an end-to-end learnable meta-operation, enabling plug-and-play, single-inference modification of pre-trained models without retraining. It unifies diverse compliance tasks—including data minimization, bias mitigation, and pruning—under a single architecture. Experiments demonstrate that our method reduces editing latency by over 90%, incurs ≤1.2% accuracy degradation on original tasks, and achieves a 94.7% compliance satisfaction rate—substantially outperforming conventional approaches.

Stochastic Gradient Descent (SGD) and its variants lack rigorous theoretical foundations in over-parameterized neural networks, suffering from inefficient training and poor interpretability. Method: This paper proposes a principle-driven guided descent framework that unifies, for the first time, curvature-aware second-order approximations, layer-adaptive preconditioning (calibrated via condition number), and a dynamically parameterized maximum-update learning rate mechanism. It systematically elucidates the synergistic interplay between this framework and exponential moving average (EMA) as well as learning rate scheduling. Contribution/Results: The method achieves both scalability and theoretical interpretability while preserving training stability and significantly accelerating convergence—reducing large-model training time by an order of magnitude. Moreover, it enhances discriminative feature learning, simultaneously improving generalization performance and output consistency.