machine learning algorithms

Existing machine learning frameworks suffer from insufficient formalization of objective functions and lack a unified, cross-domain behavioral design paradigm. Method: We propose an equation-constrained compositional function modeling approach for learners, constructing task graphs and compositional semantic graphs to enable model-agnostic behavioral specification and optimization. We introduce a novel task-oriented pattern language framework and the “manipulator” task paradigm, supporting end-to-end, architecture-agnostic, and adversarial-training-free minimal editing of data attributes. Contribution/Results: Theoretically, our work integrates formal methods and theoretical computer science principles. Empirically, we demonstrate precise, controllable, and interpretable behavioral editing on small-scale models under stable training—without stochastic sampling or data intervention—yielding significant improvements in deployment efficiency and formal verifiability.

This work proposes a reverse curriculum learning framework tailored for structurally complex and conceptually deep tasks such as advanced mathematical problem solving and code generation. The approach introduces a difficulty scoring mechanism based on structural complexity and conceptual depth, and employs a teacher–student architecture to recursively decompose challenging problems. The teacher model generates progressively simplified examples through step-by-step reasoning, thereby constructing an easy-to-hard curriculum that guides the student model in incremental learning. Experimental results on benchmarks including MATH and AIME demonstrate that this method significantly outperforms standard training strategies, effectively enhancing the model’s capacity to solve complex problems.

Application-driven machine learning (ML) research has been systematically undervalued in academia, leading to a growing disconnect between algorithmic innovation and real-world needs; this marginalization is reinforced by structural biases in peer review, faculty hiring, and pedagogy. Method: This paper introduces, for the first time, a formally defined “application-driven ML research paradigm,” elucidating its complementary relationship with the dominant methodology-driven paradigm. Drawing on interdisciplinary frameworks from education theory, research governance, and ML practice—and substantiated by empirical case studies and institutional critique—it diagnoses three systemic barriers hindering such research. Contribution/Results: The core contribution is a set of actionable, process-level interventions to reform academic evaluation systems, grounded in both theoretical analysis and pragmatic implementation pathways. These proposals have already catalyzed curricular reforms in AI education and adjustments to national funding review criteria across multiple universities, fostering cross-domain collaboration and methodological feedback loops between application domains and core ML research.

Existing meta-learning methods suffer from limited generalizability, often being confined to specific algorithms or requiring differentiability assumptions. This paper proposes a general reinforcement learning–driven meta-learning framework that trains a teacher policy to dynamically guide arbitrary student algorithms—without imposing structural or differentiability constraints on the student. Key contributions include: (i) the first unified pedagogical paradigm for meta-learning; (ii) a parameter-behavior encoder that implicitly infers the student’s internal parameter state from its input-output behavior; and (iii) a reward function grounded in learning progress. Experiments across supervised and reinforcement learning tasks demonstrate that our framework significantly outperforms baselines relying on heuristic rewards and handcrafted state representations, validating its broad generalizability and empirical effectiveness.

Learned optimizers (L2Os) suffer from poor out-of-distribution generalization, limiting their applicability beyond the training data distribution. Method: This paper proposes a novel paradigm integrating classical optimization priors with data-driven modeling. It systematically incorporates fundamental optimization principles—specifically scale invariance and affine covariance—into the architecture design. We introduce a parameterized quasi-Newton update module explicitly constrained to preserve BFGS structure, and jointly optimize it via end-to-end training that unifies optimization-theoretic modeling, neural network architecture design, and meta-learning. Contribution/Results: The resulting enhanced BFGS algorithm significantly outperforms both standard L2Os and conventional solvers on unseen problem classes, dimensions, and condition numbers. It achieves over 40% improvement in cross-distribution generalization performance, establishing a new pathway toward more transferable and robust learned optimizers.

This work proposes the first end-to-end automated artificial intelligence research framework capable of fully automating the development pipeline from algorithmic idea generation to executable machine learning classifiers. The approach integrates structured meta-prompt engineering with large language model–based code generation, augmented by an automated evaluation and iterative optimization mechanism. Experimental results on twenty standard datasets from the Infinity-Bench benchmark demonstrate that multiple novel classifiers autonomously generated by the framework significantly outperform baseline methods implemented in scikit-learn. This study thus achieves, for the first time, complete automation of the entire workflow—from initial algorithmic conception to deployable, runnable code—marking a significant step toward self-driving AI research systems.

This work addresses the significant limitations of large language models (LLMs) in autonomously executing algorithms and performing complex structured reasoning. To overcome these challenges, the authors propose the LLM-DAL framework, which employs a supervised training approach based on Decompositional Algorithmic Learning. This method explicitly guides the model to decompose and internalize algorithmic reasoning steps during training. By doing so, LLM-DAL substantially enhances the model’s capability to execute and generalize on algorithmic tasks—particularly complex arithmetic functions—thereby breaking through inherent bottlenecks in structured reasoning. The framework offers a novel pathway toward improving the systematic reasoning abilities of large language models, demonstrating that explicit decomposition of algorithmic processes can lead to more robust and generalizable performance in tasks requiring precise, stepwise logic.

This study addresses the limitation of existing machine learning methods, which prioritize predictive accuracy while neglecting design-unbiasedness—a critical requirement in official statistics and similar domains. The authors propose a general framework that does not rely on assumptions about the true data-generating model and, for the first time, integrates the known inclusion mechanisms from probability sampling designs into every stage of the learning pipeline: training sample selection, hyperparameter tuning, and performance evaluation. This integration guarantees design-unbiased prediction and classification over finite populations. The approach is compatible with popular algorithms such as k-nearest neighbors and random forests, establishes theoretical conditions under which design-unbiasedness is achieved, and provides practical algorithmic implementations alongside evaluation criteria.

This work addresses the longstanding challenge of reconciling theoretical correctness with practical efficiency by introducing Algorithmist, a multi-agent autonomous research system built upon GitHub Copilot. Through an iterative research-review cycle, Algorithmist collaboratively performs algorithm design, formal verification, proof-guided code generation, and consistency validation. The system establishes a scalable paradigm for provably correct algorithm synthesis by integrating large language models, structured natural-language proof representations, and formal verification techniques to generate algorithms tailored to specific datasets and deployment scenarios. In applications to privacy-preserving data analysis and clustering tasks, Algorithmist automatically produces novel algorithms that simultaneously offer rigorous theoretical guarantees and strong empirical performance, uncovers previously overlooked proof flaws in existing work, and achieves state-of-the-art results in several settings.

This work addresses the longstanding conflation in machine unlearning research between “untraining” and “unlearning,” which has led to ambiguous problem formulations and inadequate evaluation criteria. We formally distinguish these concepts for the first time: untraining aims to remove the influence of specific training samples, whereas true unlearning requires erasing the model’s knowledge of the entire underlying data distribution or concept those samples represent. Through theoretical formalization and a systematic review of existing literature, we establish a clear conceptual framework, reclassify current methods accordingly, and uncover critical challenges that have been overlooked. By clarifying foundational definitions, this study lays the groundwork for rigorous algorithmic evaluation, promotes standardization in the field, and delineates promising directions for future research.