Selected work

Representative Papers

Dynamic LLM Routers are Often Misguided

Oct 01, 2026

This study reveals the failure of dynamic LLM routers in cost optimization. Through benchmarking and Pareto efficiency analysis, we demonstrate that commercial routers underperform random routing due to misaligned standard objective functions and a "difficulty blind spot," while the assumption of requiring large model pools proves invalid. To address these issues, we propose a novel evaluation framework and a dual-model routing paradigm that circumvents prevalent design pitfalls. Experimental results indicate that existing complex routing systems are broadly inefficient, whereas our streamlined dual-model architecture outperforms mainstream commercial solutions. This work establishes principled model selection strategies and provides critical theoretical and practical guidance for the design of LLM routing systems.

0 citationsRead paper

Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers

May 20, 2026

This work addresses two finite-sample biases inherent in preconditioned optimizers during language model training: coupling bias between gradient and preconditioner estimates due to minibatch approximation, and systematic bias introduced by nonlinear inversion operations. For the first time, these biases are jointly identified and corrected through a unified single-batch bias-correction framework. The proposed method employs cross-fitting to decouple gradient and preconditioner estimation and incorporates a Delta-method-based variance correction to mitigate nonlinearity-induced bias. Compatible with mainstream optimizers such as AdamW, Sophia, and Shampoo, the approach reduces validation loss by 0.15, 0.07, and 0.11 nats, respectively, in Qwen2.5-0.5B pretraining, while maintaining or improving downstream instruction-tuning performance—demonstrating markedly enhanced training stability and efficiency.

0 citationsRead paper

GLiNER2-PII: A Multilingual Model for Personally Identifiable Information Extraction

May 11, 2026

Detecting heterogeneous, context-sensitive personally identifiable information (PII) across multiple languages and scenarios faces dual challenges of scarce labeled data and privacy risks. This work proposes a lightweight solution by constructing a 0.3B-parameter model based on the GLiNER2 architecture, trained via character-level span annotation and transfer learning. The approach introduces a constraint-driven generation pipeline to synthesize 4,910 multilingual text samples, marking the first integration of constraint-driven synthetic data with an efficient named entity recognition framework to enable accurate cross-lingual extraction of 42 PII categories. Evaluated on the SPY benchmark, the method achieves significantly higher span-level F1 scores than five existing systems, including the OpenAI Privacy Filter, and the model is publicly released to advance open research in PII detection.

0 citationsRead paper

GLiGuard: Schema-Conditioned Classification for LLM Safeguard

May 08, 2026

This work addresses the high latency and poor scalability of existing large language model (LLM) safety guardrails, which typically rely on large autoregressive decoders. The authors propose a non-autoregressive, multitask classification framework based on a 0.3B-parameter bidirectional encoder. By introducing a novel schema-conditioned input design that embeds task instructions and label semantics directly into the input sequence, the model simultaneously evaluates prompt/response safety, refusal detection, 14 fine-grained harm categories, and 11 jailbreaking strategies in a single forward pass. Evaluated across nine safety benchmarks, this approach matches the F1 performance of much larger 7B–27B decoder-based guardrails while reducing model size by 23–90×, achieving up to 16× higher throughput and up to 17× lower latency.

0 citationsRead paper

Pioneer Agent: Continual Improvement of Small Language Models in Production

Apr 10, 2026

This work addresses the engineering bottlenecks that hinder efficient task adaptation of small language models in production, particularly in data curation, error diagnosis, and iterative control. The authors propose the first fully automated closed-loop adaptation system for small language models: during cold-start, it requires only a natural language task description to automatically collect data, construct evaluation sets, and jointly optimize data selection, hyperparameters, and learning strategies; in production, it diagnoses failure patterns from erroneous samples, synthesizes targeted training data, and re-trains the model under explicit regression constraints. This system achieves end-to-end automation from cold-start to continuous deployment without human intervention, autonomously discovering effective strategies such as chain-of-thought supervision and quality-aware data filtering. It yields performance gains of 1.6–83.8 points across eight cold-start tasks and consistently maintains or improves performance across all seven scenarios in the new AdaptFT-Bench benchmark—where baseline methods degrade by up to 43 points—achieving 99.3% accuracy on intent classification and 0.810 F1 on entity recognition.

0 citationsRead paper

Recent publications

Latest Papers

Dynamic LLM Routers are Often Misguided

Oct 01, 2026

This study reveals the failure of dynamic LLM routers in cost optimization. Through benchmarking and Pareto efficiency analysis, we demonstrate that commercial routers underperform random routing due to misaligned standard objective functions and a "difficulty blind spot," while the assumption of requiring large model pools proves invalid. To address these issues, we propose a novel evaluation framework and a dual-model routing paradigm that circumvents prevalent design pitfalls. Experimental results indicate that existing complex routing systems are broadly inefficient, whereas our streamlined dual-model architecture outperforms mainstream commercial solutions. This work establishes principled model selection strategies and provides critical theoretical and practical guidance for the design of LLM routing systems.

0 citationsRead paper

Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers

May 20, 2026

This work addresses two finite-sample biases inherent in preconditioned optimizers during language model training: coupling bias between gradient and preconditioner estimates due to minibatch approximation, and systematic bias introduced by nonlinear inversion operations. For the first time, these biases are jointly identified and corrected through a unified single-batch bias-correction framework. The proposed method employs cross-fitting to decouple gradient and preconditioner estimation and incorporates a Delta-method-based variance correction to mitigate nonlinearity-induced bias. Compatible with mainstream optimizers such as AdamW, Sophia, and Shampoo, the approach reduces validation loss by 0.15, 0.07, and 0.11 nats, respectively, in Qwen2.5-0.5B pretraining, while maintaining or improving downstream instruction-tuning performance—demonstrating markedly enhanced training stability and efficiency.

0 citationsRead paper

GLiNER2-PII: A Multilingual Model for Personally Identifiable Information Extraction

May 11, 2026

Detecting heterogeneous, context-sensitive personally identifiable information (PII) across multiple languages and scenarios faces dual challenges of scarce labeled data and privacy risks. This work proposes a lightweight solution by constructing a 0.3B-parameter model based on the GLiNER2 architecture, trained via character-level span annotation and transfer learning. The approach introduces a constraint-driven generation pipeline to synthesize 4,910 multilingual text samples, marking the first integration of constraint-driven synthetic data with an efficient named entity recognition framework to enable accurate cross-lingual extraction of 42 PII categories. Evaluated on the SPY benchmark, the method achieves significantly higher span-level F1 scores than five existing systems, including the OpenAI Privacy Filter, and the model is publicly released to advance open research in PII detection.

0 citationsRead paper

GLiGuard: Schema-Conditioned Classification for LLM Safeguard

May 08, 2026

This work addresses the high latency and poor scalability of existing large language model (LLM) safety guardrails, which typically rely on large autoregressive decoders. The authors propose a non-autoregressive, multitask classification framework based on a 0.3B-parameter bidirectional encoder. By introducing a novel schema-conditioned input design that embeds task instructions and label semantics directly into the input sequence, the model simultaneously evaluates prompt/response safety, refusal detection, 14 fine-grained harm categories, and 11 jailbreaking strategies in a single forward pass. Evaluated across nine safety benchmarks, this approach matches the F1 performance of much larger 7B–27B decoder-based guardrails while reducing model size by 23–90×, achieving up to 16× higher throughput and up to 17× lower latency.

0 citationsRead paper

Pioneer Agent: Continual Improvement of Small Language Models in Production

Apr 10, 2026

This work addresses the engineering bottlenecks that hinder efficient task adaptation of small language models in production, particularly in data curation, error diagnosis, and iterative control. The authors propose the first fully automated closed-loop adaptation system for small language models: during cold-start, it requires only a natural language task description to automatically collect data, construct evaluation sets, and jointly optimize data selection, hyperparameters, and learning strategies; in production, it diagnoses failure patterns from erroneous samples, synthesizes targeted training data, and re-trains the model under explicit regression constraints. This system achieves end-to-end automation from cold-start to continuous deployment without human intervention, autonomously discovering effective strategies such as chain-of-thought supervision and quality-aware data filtering. It yields performance gains of 1.6–83.8 points across eight cold-start tasks and consistently maintains or improves performance across all seven scenarios in the new AdaptFT-Bench benchmark—where baseline methods degrade by up to 43 points—achieving 99.3% accuracy on intent classification and 0.810 F1 on entity recognition.

0 citationsRead paper