minibatch optimization

This work addresses the instability and uncontrolled update magnitudes in streaming reinforcement learning caused by fixed step sizes. Inspired by the normalized least mean squares (NLMS) algorithm, we propose a novel approach that adaptively determines step sizes by inversely reasoning from desired update effects. Extending this principle to deep reinforcement learning, we introduce Intentional TD and Intentional Policy Gradient algorithms, which incorporate eligibility traces and diagonal scaling to enable stable and efficient learning in a purely streaming setting—without relying on experience replay buffers. Empirical results demonstrate that our method achieves state-of-the-art streaming performance across multiple benchmark tasks, matching or surpassing batch-based algorithms that depend on experience replay.

This paper addresses the sequential selection problem under streaming heterogeneous inputs, where multi-source data collection and simulation execution must be coordinated under constrained budgets. Method: We propose the first synchronous budget allocation framework that constructs a performance estimator based on temporally aggregated heterogeneous simulation outputs and jointly optimizes data acquisition and simulation resource allocation. Theoretically, we establish asymptotic consistency and asymptotic normality of the estimator. Methodologically, we design a multi-stage stochastic optimization algorithm ensuring both statistical reliability and computational tractability. Results: Numerical experiments demonstrate that our approach significantly outperforms existing benchmarks in selection accuracy and resource utilization efficiency, providing a provably sound, computationally feasible, and practically deployable paradigm for real-time sequential decision-making under streaming heterogeneous environments.

Traditional CNN training relies on random mini-batch sampling, which often leads to rapid saturation of learning signals as most samples quickly become “easy,” thereby slowing convergence. This work proposes A*-inspired Batch Selection (A*-BS), the first approach to integrate A* search into batch selection, introducing a dynamic scoring mechanism that jointly considers sample loss difficulty and reuse penalties to adaptively select informative and diverse batches. Without modifying network architecture or optimizer, A*-BS achieves superior performance on lightweight CNNs across half of the MedMNIST-v2 benchmark tasks, outperforming ResNet-18/50 in both accuracy and AUC—by up to 15% relatively—while significantly accelerating training. These results demonstrate that intelligent batch sequencing can partially substitute for model depth.

This work addresses a key limitation in existing data selection methods, which typically employ fixed selection ratios and overlook the impact of dynamically adjusting data volume on training efficiency and generalization. The authors propose PODS, a plug-and-play oscillating data scheduling framework that extends data selection from “what to select” to “how much to select.” By alternately applying low-ratio regularization phases and high-ratio recovery phases, PODS dynamically balances optimization fidelity and implicit regularization. The framework is lightweight, task-agnostic, and compatible with diverse static and dynamic selection strategies as well as model architectures. Experiments demonstrate that PODS reduces training costs by 50% while improving accuracy on ImageNet-1k and accelerates instruction fine-tuning of large language models by over 2× without any performance degradation.

Modern large-scale distributed training faces sharply diminishing returns in hardware scaling: as GPU counts reach thousands, communication overhead dominates performance bottlenecks, rendering conventional parallelism strategies—data, tensor, and pipeline parallelism—suboptimal. Method: Leveraging real-world LLM training workloads, this project establishes an empirical analytical framework spanning diverse model scales, hardware configurations, and parallelization strategies. It quantifies the nonlinear relationship between accelerator count and performance gain, precisely identifying critical inflection points across model, data, and compute scaling dimensions. Contribution/Results: We discover that low-communication “suboptimal” strategies become optimal at extreme scale; we empirically determine hardware selection criteria, cluster topology requirements, and optimal parallelism combinations for training billion-parameter models. Our findings provide actionable, deployment-ready optimization guidelines for trillion-parameter LLM training infrastructures.

Large language models (LLMs) face significant challenges in task adaptation under resource-constrained and closed-source API settings, where conventional parameter-efficient fine-tuning (PEFT) methods are inapplicable due to their reliance on direct model parameter access and high computational overhead. Method: This paper proposes a lightweight, parameter-free knowledge injection framework that enables task-specific adaptation without accessing the LLM’s internal parameters. Its core innovation is the “Specialized Small Model (SSM) Collaboration Paradigm,” integrating knowledge distillation from the LLM, distribution-aware task modeling, and zero-parameter coupling between the SSM and the LLM. Contribution/Results: Experiments demonstrate that our approach matches PEFT-level performance across diverse downstream tasks while reducing GPU memory consumption by over 90% and inference latency by 85%. Crucially, it operates entirely within black-box API environments—requiring no model weights, gradients, or architectural access—thus enabling seamless integration with proprietary, closed-source LLM APIs.

This work proposes a novel approach to batch acquisition function optimization in multi-objective Bayesian optimization by systematically integrating simulated annealing—a metaheuristic algorithm—as a replacement for conventional gradient-based optimizers such as SLSQP. The method aims to better balance diversity and convergence of the Pareto front approximation, particularly in high-dimensional or complex objective spaces where gradient-based methods often become trapped in local optima. Implemented within the q-Expected Hypervolume Improvement (qEHVI) framework, the proposed approach is evaluated on benchmark problems including ZDT1, DTLZ2, Kursawe, and Latent-Aware. Experimental results demonstrate consistent improvements in hypervolume metrics and Pareto front coverage across most test cases, with especially notable gains observed on DTLZ2 and Latent-Aware problems.

Existing data selection methods introduce bias in heterogeneous data pools due to their reliance on reference trajectories misaligned with downstream task objectives. This work proposes a zeroth-order data selection approach that leverages a compact warm-up trajectory induced by the target validation set as an aligned and decoupled reference path, scoring candidate samples via normalized loss reduction at the trajectory endpoint. The method requires neither gradients nor Hessian approximations and enables reuse of the warm-up trajectory across diverse data pools, substantially reducing computational and storage overhead. Empirical evaluations across logistic regression, vision, and instruction fine-tuning tasks demonstrate performance comparable to strong dynamic attribution baselines while significantly decreasing warm-up time and storage costs.

Industrial-scale vision-language model (VLM) training suffers from significantly lower efficiency compared to unimodal large language models, primarily due to limitations in existing parallelization strategies and insufficient module decoupling. This work proposes a fully decoupled training paradigm that reformulates VLM training as a producer-consumer dataflow over a unified memory pool. By leveraging a global virtual address space, the visual encoder and language backbone can advance independently. The approach introduces a heterogeneous parallel allocator and a dynamic packing scheduler, uniquely integrating throughput matching into parallel strategy design and enabling runtime construction of micro-batches based on actual computational costs. Experiments demonstrate that this method achieves over 50% FLOPs utilization under real-world workloads and up to a 1.7× throughput improvement, substantially narrowing the efficiency gap with pure language model training.

This work addresses the lack of systematic methodologies in model optimization, which often relies on heuristic choices and struggles to accommodate diverse deployment constraints. It formalizes model compression and acceleration as a constraint-aware multi-objective engineering decision problem, establishing a unified and actionable framework grounded in five key dimensions: data availability, latency, memory footprint, accuracy tolerance, and retraining budget. By integrating techniques such as quantization, pruning, knowledge distillation, parameter-efficient fine-tuning (PEFT), and inference optimization, the study proposes tailored optimization pipelines for four representative industrial scenarios, delivering a reproducible and quantifiable guide for technology selection.