Model selection over partially ordered sets

2023-08-20

Proceedings of the National Academy of Sciences of the United States of America

AI Summary

Conventional definitions of false positives (FPs) and false negatives (FNs) fail for model selection in non-Boolean structured domains—such as ranking, clustering, and causal inference—where models lack a natural Boolean logic structure. Method: This paper introduces the first framework that formalizes model classes as partially ordered sets (posets), integrating poset theory, multiple hypothesis testing, and structural risk minimization to define and control generalized false positive error for non-Boolean structures—including permutations and directed acyclic graphs (DAGs). Contribution/Results: (1) It establishes natural, interpretable analogues of FP/FN errors for non-Boolean models; (2) it provides a unified framework for controlling the false positive rate (FPR) under partial orders; and (3) it enables statistically reliable and computationally feasible model selection in high-dimensional, complex structured spaces. By grounding statistical inference in poset-based falsifiability, the framework substantially extends the applicability of classical multiple testing theory beyond Boolean hypotheses.

Abstract

Significance The increasing complexity of modern datasets has been accompanied by the use of sophisticated modeling paradigms in which the task of model selection is a significant challenge. In particular, models specified by structures such as permutations (for ranking) or directed acyclic graphs (for causal inference) are not characterized by an underlying Boolean logical structure, which leads to difficulties with formalizing and controlling false-positive error. We address this challenge by organizing classes of models as partially ordered sets, which leads to systematic approaches for defining natural generalizations of false-positive error and methodology for controlling this error.

Problem

• Extends model selection to partially ordered sets
• Defines error metrics for non-Boolean model classes
• Proposes procedures for false positive error control

Innovation

• Partial order structure for model classes
• Hierarchical organization of models
• False positive error control procedures