beam search

Existing constrained decoding methods for large language models (LLMs) struggle to balance inference efficiency and constraint satisfaction—e.g., CDLH incurs high computational overhead due to token-wise lookahead, while greedy decoding suffers from low constraint adherence. Method: This paper introduces the first framework integrating speculative decoding into constrained decoding. It employs a lightweight draft model to generate candidate sequences, which are then jointly verified by the target LLM and a task-specific reward function, enabling constraint-aware, offloaded lookahead computation. Contribution/Results: Evaluated across two constrained generation tasks and three major LLM families, our approach achieves 2.2×–12.15× speedup over baseline constrained decoding, with constraint satisfaction rates approaching those of CDLH and substantially outperforming greedy decoding. The core contribution is the establishment of a novel, efficient, general-purpose, and scalable paradigm for constrained decoding—marking the first application of speculative decoding to this domain.

This paper addresses bounded-suboptimal bidirectional search under consistent heuristics. To overcome the lack of theoretical guarantees and systematic design in existing approaches, we generalize the optimal bidirectional algorithm BAE* to the bounded-suboptimal setting, introducing a novel family of variants based on weighted heuristics and bidirectional frontier expansion. Theoretically, we establish an analytical framework linking suboptimality bounds to search efficiency. Empirically, comprehensive experiments demonstrate that our algorithms achieve superior trade-offs among solution quality, expanded node count, and runtime—outperforming state-of-the-art bounded-suboptimal bidirectional methods and weighted A*. Our key contributions are: (1) the first principled generalization of BAE* to bounded-suboptimal search; and (2) a characterization of performance trade-offs induced by bidirectional weighting strategies under varying heuristic accuracy.

Existing language model decoding methods lack a unified theoretical framework and often rely on heuristic hyperparameter tuning. This work formulates the decoding process as a regularized optimization problem over the probability simplex and provides a unified interpretation of multiple mainstream decoding strategies through the introduction of KL-divergence anchoring and analysis of optimality conditions. Building upon this framework, we propose a novel Best-of-K sampler that substantially improves generation quality under high-temperature settings. Experimental results demonstrate that our method achieves an 18.6% absolute accuracy gain on the MATH500 benchmark when applied to the Qwen2.5-Math-7B model, confirming its effectiveness and broad applicability.

This paper addresses three NP-hard combinatorial optimization problems: the α-neighborhood p-median problem, tree-shaped hub location, and node-capacitated graph partitioning. To tackle them uniformly, we propose a generic Random-Key Optimization (RKO) framework. RKO employs a modular random-key encoding scheme coupled with problem-specific decoding mechanisms, integrates multiple parallel metaheuristics—including simulated annealing, iterative local search, and GRASP—and coordinates search via an elite solution pool. Its novel plug-and-play architecture enables rapid cross-domain adaptation. Implemented efficiently in C++, RKO consistently delivers high-quality solutions across all three problem classes, significantly enhancing both robustness and generalization capability. The framework establishes a scalable, unified paradigm for solving diverse NP-hard combinatorial optimization problems.

This work investigates the fundamental impact of permutation pattern avoidance on the computational complexity of combinatorial optimization problems. Focusing on input sequences that avoid specific permutation patterns, we systematically demonstrate how such structural constraints dramatically reduce problem hardness. Methodologically, we integrate the Marcus–Tardos theorem, twin-width theory, and deep combinatorial structural analysis to establish a universal link between pattern avoidance and tractability. Our contributions include: (i) the first complete proof that the dynamic optimality conjecture for binary search trees holds under pattern-avoiding inputs, yielding a tight amortized search cost of *O*(1); (ii) tightening the total *k*-server cost to the optimal bound Ω(*n*/log *k*); (iii) reducing Euclidean TSP path length to the tight bound *O*(log *n*); and (iv) extending these results to Euclidean minimum spanning tree and related problems. This framework transcends conventional input assumptions, providing a new paradigm for optimization complexity theory under structured inputs.

This work addresses the problem of correcting up to $d$ deletion errors in the binary deletion channel by formulating the construction of $d$-deletion-correcting codes as a maximum clique problem. The authors introduce, for the first time, a Penalty-Guided Clique Search (PGCS) algorithm to efficiently construct large codebooks at scale. Furthermore, they design a low-complexity exact decoder based on the Longest Common Subsequence (LCS) distance, incorporating symbol-count filtering and early termination strategies to significantly reduce decoding overhead. Experimental results for block lengths 8–14 and deletion counts 1–3 demonstrate that the constructed codebooks outperform existing graph-based heuristic methods, with some achieving known optimality, thereby validating the efficacy and superiority of the proposed approach.

This work addresses the limitations of existing automated heuristic design approaches, which predominantly rely on bottom-up code search and struggle to extract reusable, transferable high-level knowledge. The authors propose a top-down, knowledge-first search paradigm that treats knowledge as the primary object of search, using code merely for instantiation and validation. By formulating explainable hypotheses, the method enables knowledge reuse across problems and solution trajectories. It establishes, for the first time, a bidirectional bridge between knowledge and code, introduces a statistical learning perspective to characterize the distortion–compression trade-off, and integrates large language models with population-based and tree search mechanisms into a unified, knowledge-driven iterative optimization framework. Experiments demonstrate that this approach significantly outperforms code-centric methods in heuristic discovery efficiency, transferability, and generalization across combinatorial optimization and extended tasks.

This work addresses the finite-block Pareto optimality of injective source coding under the constraint of the four-symbol restricted Markov source introduced by Dalai and Leonardi. By introducing an information-cost function, the authors impose a total order on admissible strings based on information cost, length, and lexicographic order, and construct a canonical injective coding scheme that assigns binary codewords via short lexicographic ordering. This scheme achieves, for the first time on a constrained Markov source, a finite-block average codeword length strictly superior to classical invertible codes: for block lengths \(n \geq 2\), the expected code length satisfies \(\mathbb{E}[|C(X_1^n)|] < 3n/2\) and adheres to the asymptotic upper bound \(\mathbb{E}[|C(X_1^n)|] \leq 3n/2 - c/\sqrt{n}\) for some constant \(c > 0\). Consequently, the Dalai–Leonardi code is shown not to be Pareto optimal.

This work proposes an adaptive hyper-heuristic approach based on stochastic gradient policy to overcome the limitation of traditional selection hyper-heuristics, which require manual tuning of the learning period parameter τ and thus lack adaptability during optimization. The proposed method automatically adjusts τ and dynamically determines the optimal neighborhood size without user intervention. Integrated within a randomized local search (RLS) framework and enhanced with an adaptive learning mechanism, the algorithm achieves the theoretically optimal runtime—up to lower-order terms—on the LeadingOnes benchmark problem. This advancement significantly improves both the efficiency and generality of hyper-heuristic algorithms.