Asymptotic Behavior of Multi--Task Learning: Implicit Regularization and Double Descent Effects

2026-03-05

AI Summary

This work investigates the mechanism by which multi-task learning enhances generalization through inter-task sharing, a phenomenon not yet well understood. Under a misspecified perceptron model, the study combines high-dimensional asymptotic analysis with empirical validation to precisely characterize the implicit regularization effect inherent in multi-task learning. The analysis reveals that combining tasks is equivalent to introducing an additional regularization term, which effectively mitigates—and even asymptotically eliminates—the double descent phenomenon in generalization error. Theoretically, the authors demonstrate that multi-task learning can be reformulated as a conventional learning framework augmented with implicit regularization. Experimental results corroborate the theoretical findings, showing that this implicit regularization substantially improves generalization performance.

Technology Category

• Machine Learning: Learning Theory
• Multiagent Systems: Multiagent Learning
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Multi--task learning seeks to improve the generalization error by leveraging the common information shared by multiple related tasks. One challenge in multi--task learning is identifying formulations capable of uncovering the common information shared between different but related tasks. This paper provides a precise asymptotic analysis of a popular multi--task formulation associated with misspecified perceptron learning models. The main contribution of this paper is to precisely determine the reasons behind the benefits gained from combining multiple related tasks. Specifically, we show that combining multiple tasks is asymptotically equivalent to a traditional formulation with additional regularization terms that help improve the generalization performance. Another contribution is to empirically study the impact of combining tasks on the generalization error. In particular, we empirically show that the combination of multiple tasks postpones the double descent phenomenon and can mitigate it asymptotically.

Problem

• multi-task learning
• generalization error
• double descent
• implicit regularization
• asymptotic analysis

Innovation

• multi-task learning
• implicit regularization
• double descent
• asymptotic analysis
• generalization error