🤖 AI Summary
This study investigates whether large language model (LLM)-based scientific agents rely primarily on initial task context or experimental feedback when making decisions during neural operator adaptation. Specifically, this work examines the capacity of LLMs to select fine-tuning configurations under limited computational budgets. Through controlled intervention experiments, we decouple the mechanisms underlying prior knowledge and feedback signals, benchmarking the approach against random search and Bayesian optimization. Our findings demonstrate that LLM-driven decision-making responds simultaneously to task descriptions and empirical observations, effectively integrating task priors with feedback sensitivity. In most scenarios, the LLM agent outperforms traditional baselines, achieving lower test errors while utilizing experimental feedback more efficiently. These results establish a novel paradigm for transfer learning in partial differential equations.
📝 Abstract
Do LLM scientific agents rely only on their initial task context, or do they adapt their decisions in response to experimental feedback? We study this question in neural operator adaptation, where a large language model (LLM) selects fine-tuning configurations under a limited trial budget. Across transfers within and between partial differential equation (PDE) families, the LLM achieves lower held-out test nRMSE than random search and Bayesian optimisation in nearly every matched comparison. Endpoint performance alone cannot distinguish what happens, so we verify each attribution with controlled interventions. Before observing any validation score, the LLM's first configuration already ranks near the top of the corresponding random-search pool, indicating a useful initial bias. A complementary cold-start intervention shows that the selected base learning rate shifts with the PDE description. Once feedback becomes available, reassigning validation scores among evaluated configurations changes the next proposal in every case tested, whereas a value-preserving rewrite produces no comparable aggregate effect. These interventions establish that the LLM's decision-level actions respond to the given task and observed outcomes, showing that it combines a task-dependent prior with sensitivity to experimental feedback.