Harness In-Context Operator Learning with Chain of Operators

2026-06-10

AI Summary

This work addresses the limited out-of-distribution (OOD) generalization of existing neural operators, which typically require fine-tuning or retraining. The authors propose the Chain of Operators (CHOP) framework, which, for the first time, adapts prompt engineering concepts from large language models to the neural operator setting. CHOP constructs operator chains composed of explicit elementary transformations and a frozen In-Context Operator Network (ICON), enabling training-free OOD generalization without updating any parameters. The approach yields both interpretability and closed-form expressions, and demonstrates strong generalization capabilities—significantly reducing relative inference errors across diverse tasks, including scalar conservation laws, mean-field control problems, and cross-family partial differential equation settings.

Abstract

Neural operators approximate mappings between function spaces, but often generalize poorly to other operators and usually require fine-tuning or retraining. In-Context Operator Networks (ICON) addresses this issue by prompting the model with numerical context so that the model learns specific operators from prompts and adapt to different operators without fine-tuning. However, ICON may still fail to generalize to out-of-distribution (OOD) operator tasks. Inpired by the success of harness engineering of Large Language models (LLMs), we introduce Chain of Operators (CHOP), a framework that harness a frozen ICON to OOD operator tasks without updating its parameters. Specifically, CHOP constructs a chain of operators consisting of explicit elementary transformations and the frozen ICON. Experiments on a scalar conservation law and a mean-field control problem show that CHOP reduces relative inference error over direct ICON evaluation, while each operator in the chain remains interpretable and in closed form. A chain constructed on one PDE family further generalizes to a different family, indicating shared mechanisms across harness systems.

Problem

• neural operators
• out-of-distribution generalization
• in-context learning
• operator learning
• generalization

Innovation

• Chain of Operators
• In-Context Learning
• Neural Operators
• Out-of-Distribution Generalization
• Prompt Engineering