Fundamentals of Regression

2025-11-27

AI Summary

This paper addresses the limited physical interpretability and poor generalizability of conventional data-driven regression methods. To this end, we propose a physics-informed regression modeling paradigm that systematically embeds first-principles constraints, differential equation priors, and conservation laws into statistical regression, curve fitting, and supervised learning frameworks—thereby enabling deep integration of machine learning with classical numerical methods (e.g., finite differences, spectral methods). Our key contributions are threefold: (i) we establish a systematic taxonomy tracing the evolution of regression from purely statistical to physics-guided formulations; (ii) we construct a theoretical bridge linking computational science and scientific machine learning; and (iii) the proposed framework significantly enhances model extrapolation capability, robustness, and physical consistency. As a result, it offers an interpretable, verifiable, and cross-disciplinary modeling paradigm for complex scientific and engineering problems.

Technology Category

• Machine Learning: Classification and Regression
• Computer Vision: Low Level & Physics-based Vision
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Machine learning and data science for the Web

Abstract

This chapter opens with a review of classic tools for regression, a subset of machine learning that seeks to find relationships between variables. With the advent of scientific machine learning this field has moved from a purely data-driven (statistical) formalism to a constrained or ``physics-informed'' formalism, which integrates physical knowledge and methods from traditional computational engineering. In the first part, we introduce the general concepts and the statistical flavor of regression versus other forms of curve fitting. We then move to an overview of traditional methods from machine learning and their classification and ways to link these to traditional computational science. Finally, we close with a note on methods to combine machine learning and numerical methods for physics

Problem

• Regression moves from data-driven to physics-informed formalism
• It integrates physical knowledge with machine learning methods
• Combines machine learning and numerical methods for physics

Innovation

• Physics-informed machine learning integrates physical knowledge
• Combines statistical regression with computational engineering methods
• Links machine learning to traditional numerical methods for physics