🤖 AI Summary
This work addresses the challenge that large language models struggle to adapt and self-improve on specific tasks during inference. To this end, the authors propose the MASS framework, which introduces the first end-to-end test-time meta-self-adaptation mechanism. MASS employs a bilevel optimization architecture: in the inner loop, the model is updated using self-generated, task-specific synthetic data, while the outer loop meta-learns an optimal data generation strategy and reward signal. This approach substantially enhances the model’s test-time adaptability and data efficiency on tasks such as mathematical reasoning, enabling the effective generation of instance-level curriculum data and yielding significant performance gains.
📝 Abstract
As strong general reasoners, large language models (LLMs) encounter diverse domains and tasks, where the ability to adapt and self-improve at test time is valuable. We introduce MASS, a meta-learning framework that enables LLMs to self-adapt by generating problem-specific synthetic training data and performing targeted self-updates optimized for downstream performance at inference time. We train this behavior end-to-end via bilevel optimization: an inner loop adapts on self-generated examples while an outer loop meta-learns data-attribution signals and rewards post-update task performance. The synthetic data is optimized with scalable meta-gradients, backpropagating the downstream loss through the inner updates to reward useful generations. Experiments on mathematical reasoning show that MASS learns to synthesize per-instance curricula that yield effective, data-efficient test-time adaptation.