Selected work

Representative Papers

Stateless Language Agents: Scaling Long-Horizon Automated Research

Oct 05, 2026

This study addresses the efficiency bottlenecks of LLM agents in long-horizon automated research, which arise from history replay, redundant work, and premature stagnation. To overcome these challenges, this work proposes a stateless language agent framework that introduces a novel “stateful search with stateless agents” paradigm. Specifically, it decouples persistent research states from dialogue histories by delegating state management to external tools that dynamically reconstruct context for each invocation. Furthermore, an Advisor-Worker architecture is designed to optimize resource scheduling through parallel execution and centralized evidence summarization. Experimental results demonstrate that the proposed approach achieves state-of-the-art performance on tasks such as software engineering while reducing token consumption by over 84%. Additionally, this work reveals inherent limitations associated with short evaluation horizons.

0 citationsRead paper

SETA: Scaling Environments for Terminal Agents

Jul 12, 2026

This work addresses the challenge of limited large-scale, diverse, and verifiable training environments for terminal-based reinforcement learning (RL). It introduces SETA, a framework featuring two pipelines—SETA-Synth and SETA-Evol—that enable the first scalable and verifiable automatic generation of terminal RL environments. The framework incorporates a unified verification mechanism and a difficulty-adaptive evolution strategy, yielding SETA-Env, an open dataset comprising over 4,500 tasks. By integrating instruction synthesis, environment construction, and automated validation, the authors train agents using the GRPO algorithm on Qwen3-8B and DeepSeek-V4-Flash models. On Terminal-Bench 2.0, these agents achieve 12% pass@1 (state-of-the-art for 8B models) and 43% pass@1 (a 3% improvement), with pass@5 reaching 58% (a 4% gain).

0 citationsRead paper

LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning

May 31, 2026

This work addresses the challenge of balancing efficiency and accuracy in long-context inference (100K+ tokens) by proposing a general, lightweight context compression method. The approach employs a fine-tuned, lightweight cross-attention scoring layer combined with token-level chunking, top-p token budget allocation, positional reordering, and format-agnostic query parsing. A two-stage fine-tuning strategy—first pretraining on Needle-in-a-Haystack (NIAH) tasks followed by task-specific fine-tuning on multi-hop reasoning and code-related benchmarks—enables significant reduction in prefill costs while maintaining or even surpassing full-context performance. The method matches or outperforms full-context baselines on InfiniteBench Code-Debug, substantially narrows performance gaps on LongBench v2 multi-document reasoning tasks, and demonstrates strong transferability across four models spanning three distinct model families, confirming its generality and effectiveness.

0 citationsRead paper

Test-Time Adaptation via Many-Shot Prompting: Benefits, Limits, and Pitfalls

Mar 06, 2026

This study systematically investigates the effectiveness and limitations of test-time adaptation methods that do not require updating model parameters in open-source large language models. Focusing on many-shot in-context learning (ICL), it integrates dynamic and reinforcement-based ICL prompting strategies to evaluate how the number, ordering, and selection mechanisms of examples influence performance across diverse tasks and model architectures. The findings reveal that many-shot prompting substantially improves performance on structured tasks with high information gain but is highly sensitive to example selection, whereas its benefits are limited in open-ended generation tasks. This work delineates the applicability boundaries and potential risks of prompt-based test-time adaptation, offering both theoretical grounding and practical guidance for real-world deployment.

0 citationsRead paper

Cross-Family Speculative Prefill: Training-Free Long-Context Compression with Small Draft Models

Mar 03, 2026

This work addresses the high prefill overhead incurred by long-context prompts in large language model (LLM) agents, a challenge exacerbated by existing compression methods that rely on same-family small draft models. To overcome this limitation, the authors propose a training-free, cross-model-family prompt compression approach that leverages heterogeneous small draft models—such as Qwen, LLaMA, and DeepSeek—to evaluate token importance via attention mechanisms and guide speculative prefilling. This method is the first to demonstrate the effectiveness of cross-family draft models for prompt compression, thereby eliminating the dependency on same-family architectures. It consistently retains 90%–100% of the original model performance across diverse tasks, with occasional improvements due to denoising effects, while significantly reducing time-to-first-token (TTFT) latency.

0 citationsRead paper

Recent publications

Latest Papers

Stateless Language Agents: Scaling Long-Horizon Automated Research

Oct 05, 2026

This study addresses the efficiency bottlenecks of LLM agents in long-horizon automated research, which arise from history replay, redundant work, and premature stagnation. To overcome these challenges, this work proposes a stateless language agent framework that introduces a novel “stateful search with stateless agents” paradigm. Specifically, it decouples persistent research states from dialogue histories by delegating state management to external tools that dynamically reconstruct context for each invocation. Furthermore, an Advisor-Worker architecture is designed to optimize resource scheduling through parallel execution and centralized evidence summarization. Experimental results demonstrate that the proposed approach achieves state-of-the-art performance on tasks such as software engineering while reducing token consumption by over 84%. Additionally, this work reveals inherent limitations associated with short evaluation horizons.

0 citationsRead paper

SETA: Scaling Environments for Terminal Agents

Jul 12, 2026

This work addresses the challenge of limited large-scale, diverse, and verifiable training environments for terminal-based reinforcement learning (RL). It introduces SETA, a framework featuring two pipelines—SETA-Synth and SETA-Evol—that enable the first scalable and verifiable automatic generation of terminal RL environments. The framework incorporates a unified verification mechanism and a difficulty-adaptive evolution strategy, yielding SETA-Env, an open dataset comprising over 4,500 tasks. By integrating instruction synthesis, environment construction, and automated validation, the authors train agents using the GRPO algorithm on Qwen3-8B and DeepSeek-V4-Flash models. On Terminal-Bench 2.0, these agents achieve 12% pass@1 (state-of-the-art for 8B models) and 43% pass@1 (a 3% improvement), with pass@5 reaching 58% (a 4% gain).

0 citationsRead paper

LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning

May 31, 2026

This work addresses the challenge of balancing efficiency and accuracy in long-context inference (100K+ tokens) by proposing a general, lightweight context compression method. The approach employs a fine-tuned, lightweight cross-attention scoring layer combined with token-level chunking, top-p token budget allocation, positional reordering, and format-agnostic query parsing. A two-stage fine-tuning strategy—first pretraining on Needle-in-a-Haystack (NIAH) tasks followed by task-specific fine-tuning on multi-hop reasoning and code-related benchmarks—enables significant reduction in prefill costs while maintaining or even surpassing full-context performance. The method matches or outperforms full-context baselines on InfiniteBench Code-Debug, substantially narrows performance gaps on LongBench v2 multi-document reasoning tasks, and demonstrates strong transferability across four models spanning three distinct model families, confirming its generality and effectiveness.

0 citationsRead paper

Test-Time Adaptation via Many-Shot Prompting: Benefits, Limits, and Pitfalls

Mar 06, 2026

This study systematically investigates the effectiveness and limitations of test-time adaptation methods that do not require updating model parameters in open-source large language models. Focusing on many-shot in-context learning (ICL), it integrates dynamic and reinforcement-based ICL prompting strategies to evaluate how the number, ordering, and selection mechanisms of examples influence performance across diverse tasks and model architectures. The findings reveal that many-shot prompting substantially improves performance on structured tasks with high information gain but is highly sensitive to example selection, whereas its benefits are limited in open-ended generation tasks. This work delineates the applicability boundaries and potential risks of prompt-based test-time adaptation, offering both theoretical grounding and practical guidance for real-world deployment.

0 citationsRead paper

Cross-Family Speculative Prefill: Training-Free Long-Context Compression with Small Draft Models

Mar 03, 2026

This work addresses the high prefill overhead incurred by long-context prompts in large language model (LLM) agents, a challenge exacerbated by existing compression methods that rely on same-family small draft models. To overcome this limitation, the authors propose a training-free, cross-model-family prompt compression approach that leverages heterogeneous small draft models—such as Qwen, LLaMA, and DeepSeek—to evaluate token importance via attention mechanisms and guide speculative prefilling. This method is the first to demonstrate the effectiveness of cross-family draft models for prompt compression, thereby eliminating the dependency on same-family architectures. It consistently retains 90%–100% of the original model performance across diverse tasks, with occasional improvements due to denoising effects, while significantly reducing time-to-first-token (TTFT) latency.

0 citationsRead paper