Learning a Class of Mixed Linear Regressions: Global Convergence under General Data Conditions

2025-03-24

AI Summary

This paper addresses global convergence and optimal clustering for mixture linear regression (MLR) under weak data conditions, removing reliance on standard i.i.d. or persistent excitation (PE) assumptions. For a two-component stochastic MLR system, we propose a two-stage recursive identification algorithm that integrates least-squares estimation with EM-inspired principles, decoupling direction vector and scaling coefficient estimation. We establish, for the first time without strict excitation conditions, rigorous guarantees of global convergence of parameter estimates, an explicit convergence rate, and asymptotic optimality of both cumulative misclassification error and within-cluster estimation error. Numerical experiments demonstrate the algorithm’s robustness and efficiency under non-ideal data. The core contributions are: (i) a novel theoretical framework for global convergence without PE assumptions, and (ii) asymptotically optimal, data-driven clustering performance guarantees.

Technology Category

• Machine Learning: Clustering
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Mixed linear regression (MLR) has attracted increasing attention because of its great theoretical and practical importance in capturing nonlinear relationships by utilizing a mixture of linear regression sub-models. Although considerable efforts have been devoted to the learning problem of such systems, i.e., estimating data labels and identifying model parameters, most existing investigations employ the offline algorithm, impose the strict independent and identically distributed (i.i.d.) or persistent excitation (PE) conditions on the regressor data, and provide local convergence results only. In this paper, we investigate the recursive estimation and data clustering problems for a class of stochastic MLRs with two components. To address this inherently nonconvex optimization problem, we propose a novel two-step recursive identification algorithm to estimate the true parameters, where the direction vector and the scaling coefficient of the unknown parameters are estimated by the least squares and the expectation-maximization (EM) principles, respectively. Under a general data condition, which is much weaker than the traditional i.i.d. and PE conditions, we establish the global convergence and the convergence rate of the proposed identification algorithm for the first time. Furthermore, we prove that, without any excitation condition on the regressor data, the data clustering performance including the cumulative mis-classification error and the within-cluster error can be optimal asymptotically. Finally, we provide a numerical example to illustrate the performance of the proposed learning algorithm.

Problem

• Global convergence for mixed linear regressions under general data conditions
• Recursive estimation and clustering without strict i.i.d. or PE requirements
• Optimal asymptotic performance in data clustering and parameter estimation

Innovation

• Two-step recursive identification algorithm
• Combines least squares and EM principles
• Global convergence under general data conditions