ANOVA-boosting for Random Fourier Features

2024-04-03

Applied and Computational Harmonic Analysis

AI Summary

Random Fourier Features (RFF) suffer from low approximation accuracy and poor interpretability in high-dimensional function modeling. Method: This paper introduces, for the first time, classical and generalized ANOVA decompositions into the RFF framework, proposing an interpretable iterative sparse modeling approach. It learns low-order variable interaction terms hierarchically to automatically identify key input variables and their coupling structures—even under input dependence—enabling disentanglement of main and interaction effects for enhanced global sensitivity analysis. Contribution/Results: Theoretical analysis and extensive experiments demonstrate that the method significantly reduces approximation error while maintaining strong robustness, scalability, and computational efficiency. It establishes a novel paradigm for interpretable modeling and global sensitivity analysis of high-dimensional black-box functions.

Technology Category

• Machine Learning: Kernel Methods
• Reasoning under Uncertainty: Graphical Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

We propose two algorithms for boosting random Fourier feature models for approximating high-dimensional functions. These methods utilize the classical and generalized analysis of variance (ANOVA) decomposition to learn low-order functions, where there are few interactions between the variables. Our algorithms are able to find an index set of important input variables and variable interactions reliably. Furthermore, we generalize already existing random Fourier feature models to an ANOVA setting, where terms of different order can be used. Our algorithms have the advantage of interpretability, meaning that the influence of every input variable is known in the learned model, even for dependent input variables. We give theoretical as well as numerical results that our algorithms perform well for sensitivity analysis. The ANOVA-boosting step reduces the approximation error of existing methods significantly.

Problem

• Boost random Fourier feature models for high-dimensional functions
• Identify important input variables and their interactions
• Improve interpretability and reduce approximation error

Innovation

• Boosts random Fourier features via ANOVA
• Identifies key variables and interactions
• Enhances interpretability and reduces error