Learning DNF through Generalized Fourier Representations

2025-06-01

AI Summary

This work investigates the learnability of DNF formulas under arbitrary (non-uniform) distributions, particularly those induced by difference-bounded tree-structured Bayesian networks (Tree BNs). We introduce a generalized Fourier representation framework applicable to arbitrary distributions, establishing for the first time polynomial-time learnability of DNFs under difference-bounded Tree BNs. We design a membership-query-based learning algorithm and derive tight upper and lower bounds on the L₁ spectral norm—characterizing both sufficiency and necessity. Furthermore, we propose a distribution-agnostic Bayesian network structure learning method, enabling end-to-end DNF learning without prior knowledge of the underlying distribution. Our core contributions are threefold: (i) the first integration of generalized Fourier analysis with Bayesian modeling for Boolean function representation; (ii) the first provably efficient learnability guarantee for DNFs under difference-bounded Tree BNs; and (iii) a theoretically sound and practically implementable learning framework that bridges representation, structural inference, and query-based learning.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Graphical Models
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

The Fourier representation for the uniform distribution over the Boolean cube has found numerous applications in algorithms and complexity analysis. Notably, in learning theory, learnability of Disjunctive Normal Form (DNF) under uniform as well as product distributions has been established through such representations. This paper makes five main contributions. First, it introduces a generalized Fourier expansion that can be used with any distribution $D$ through the representation of the distribution as a Bayesian network (BN). Second, it shows that the main algorithmic tools for learning with the Fourier representation, that use membership queries to approximate functions by recovering their heavy Fourier coefficients, can be used with slight modifications with the generalized expansion. These results hold for any distribution. Third, it analyzes the $L_1$ spectral norm of conjunctions under the new expansion, showing that it is bounded for a class of distributions which can be represented by difference bounded tree BN, where a parent node in the BN representation can change the conditional expectation of a child node by at most $alpha<0.5$. Lower bounds are presented to show that such constraints are necessary. The fourth contribution uses these results to show the learnability of DNF with membership queries under difference bounded tree BN. The final contribution is to develop an algorithm for learning difference-bounded tree BN distributions, thus extending the DNF learnability result to cases where the distribution is not known in advance.

Problem

• Generalized Fourier expansion for any distribution via Bayesian networks
• Learning DNF with membership queries under difference-bounded tree BN
• Algorithm for learning difference-bounded tree BN distributions

Innovation

• Generalized Fourier expansion via Bayesian networks
• Algorithmic tools for learning with any distribution
• Learnability of DNF under difference-bounded tree BN