decision rule design

Formulating deterministic or probabilistic decision rules and fusion policies that combine signals into actionable choices, including gating, selection, and risk-aware thresholds. Applied to choose best summaries, merge decisions, accept or repair retrieved evidence, and otherwise convert scores into reliable operational actions.

decisionruledesign

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

Harnessing The Collective Wisdom: Fusion Learning Using Decision Sequences From Diverse Sources

Aug 21, 2023
TB
Trambak Banerjee
🏛️ University of Kansas | Fudan University | Yale University

Integrating hypothesis testing results across heterogeneous multi-source studies—some reporting only binary significance decisions, others only FDR control levels—poses a fundamental challenge for rigorous, unified FDR control. Method: We propose the Integrated Ranking and Thresholding (IRT) framework, which operates solely on binary rejection decisions, a prespecified global FDR level, and the set of hypotheses—requiring neither raw data, p-values, nor effect sizes. IRT employs nonparametric evidence aggregation and a ranking-driven thresholding mechanism, circumventing traditional meta-analysis assumptions of statistical homogeneity and reliance on shared summary statistics. Contribution/Results: IRT is the first method to achieve theoretically guaranteed strong FDR control under non-shared statistical summaries. We prove its FDR control property rigorously; simulations demonstrate superior performance over state-of-the-art integration methods; and real-world application to multi-center genome-wide association studies confirms its practical utility and robustness.

Combining findings across diverse data sourcesEnsuring overall false discovery rate controlFusing evidence from multiple testing procedures

On strategies for risk management and decision making under uncertainty shared across multiple fields

Sep 06, 2023
AG
Alexander Gutfraind
🏛️ Loyola University Chicago | University of Illinois at Chicago

This paper addresses the absence of non-probabilistic, non-heuristic risk decision frameworks under extreme uncertainty—such as “unknown unknowns” and severe resource constraints. Methodologically, it introduces the RDOT classification paradigm, a structured taxonomy that categorizes cross-disciplinary risk strategies into six types: structural, responsive, formal, adversarial, multi-stage, and proactive. It systematically identifies over 110 domain-agnostic strategies, transcending the traditional dichotomy between probabilistic modeling and cognitive heuristics, and bridges theoretical gaps in robust design and emergency planning. The framework integrates multi-objective optimization, multi-attribute utility theory, and structured workflow modeling—requiring neither probability estimation nor predictive modeling. Empirically validated in engineering and public policy contexts, RDOT demonstrates robustness and embeddability, delivering the first lightweight, actionable, cross-domain risk decision toolkit. (149 words)

Develops a framework for risk management strategies under uncertainty.Enhances decision-making with RDOT for radical uncertainty and resource constraints.Identifies and categorizes over 110 RDOT strategies across six types.

Decomposable Stochastic Choice

Dec 08, 2023
FS
Fedor Sandomirskiy
🏛️ Princeton University | Caltech

This paper investigates the cross-decision independence of stochastic choice in diversified decision-making—specifically, whether choice probabilities remain invariant when irrelevant alternatives are added to the menu. We develop a menu-dependent stochastic choice model that captures endogenous randomness satisfying the “decomposability” axiom. Theoretically, we establish two key results: first, for general outcome spaces, decomposability is equivalent to the existence of a universal utility function under which choices follow a multinomial logit (MNL) rule; second, for monetary outcomes, it uniquely implies a one-parameter logit family. These characterizations are robust to approximate decomposability and label perturbations. Our framework unifies explanations of intertemporal choice, risky decision-making, and ambiguity preferences, providing the first axiomatized foundation for behavioral modeling grounded in stochastic choice theory.

Characterizing stochastic choice rules unaffected by irrelevant decisionsIdentifying mixed-logit rules as the solution familyModeling decisions as menus with outcome-dependent action probabilities

This paper addresses the modeling challenge of “unconceivable uncertainty”—events unforeseeable yet possible—arising in social and life sciences, where classical probability theory fails due to its inherent reliance on known, enumerable possibilities. We propose an extended evidence-theoretic framework that formally distinguishes between uncertainties within and beyond the agent’s cognitive boundary, integrating imprecise probabilities, subadditive measures, and non-standard information-theoretic approaches. Crucially, we establish a novel interface between this framework and multi-agent systems, rigorously differentiating representable from unrepresentable uncertainty sources. The resulting formalism provides a new mathematical foundation and analytical paradigm for studying complex socio-biological systems, particularly in risk perception and cultural information diffusion. (128 words)

Compares extended Evidence Theory with advanced Probability Theory variantsExplores multi-agent applications of enhanced uncertainty reasoningExtends Evidence Theory to handle unforeseen event uncertainties

What Are the Odds? Improving the foundations of Statistical Model Checking

Apr 08, 2024
TM
Tobias Meggendorfer
🏛️ Lancaster University Leipzig | Institute of Science and Technology Austria | Dresden University of Technology

For Markov decision processes (MDPs) with unknown transition probabilities, existing statistical model checking (SMC) algorithms suffer from high sample complexity and weak theoretical guarantees. Method: We introduce tight concentration inequalities—specifically, the Bretagnolle–Huber and Empirical Bernstein bounds—into the SMC framework for the first time, and design adaptive, structure-aware statistical estimators that exploit MDP topology. Contribution/Results: Theoretically, our approach yields significantly tighter and more general probably approximately correct (PAC) guarantees. Empirically, it reduces required sample sizes by up to two orders of magnitude on standard verification benchmarks. This work establishes a new paradigm for efficient and reliable formal verification of uncertain systems.

Enhancing accuracy of transition probability estimationImproving statistical methods for model checking MDPsReducing sample size requirements for SMC algorithms

Latest Papers

What's happening recently
View more

This work addresses the challenge of enabling auditable decision-making in production AI systems when faced with incomplete or conflicting evidence. The authors propose EvaluatorDPT, a novel model that explicitly treats abstention (To-Be-Determined, TBD) as a learnable outcome, thereby establishing a routable, policy-constrained, and auditable uncertainty-handling mechanism. Built upon a Transformer encoder, the architecture jointly optimizes a primary decision head alongside structured auxiliary semantic channels—such as those encoding values and sentiment—and supports runtime threshold tuning for domain-agnostic deployment. Evaluated on a test set of 44,597 samples, the model achieves an accuracy of 0.8260 and a macro F1-score of 0.8252 (YES: 0.8314, NO: 0.8486, TBD: 0.7956), with a validation expected calibration error (ECE) of only 0.0338, demonstrating both interpretable reasoning pathways and externally verifiable decision evidence.

AI decision controlauditable decision-makinglearned abstention

This work addresses the critical challenge of high-cost errors arising from overconfident predictions of large language models in decision-making systems. To mitigate this risk, the authors propose the Risk-Aware Causal Gating (RACG) framework, which uniquely integrates causal effect estimation with distribution-free risk control. By evaluating the causal pathways from actions to outcomes, RACG dynamically determines whether to execute, defer, or reject a prediction, thereby enabling safe decision-making under minimal privilege. The framework incorporates counterfactual risk estimation, an adaptive gating policy, and a selective prediction mechanism. Empirical results across multiple simulated interventions and real-world decision benchmarks demonstrate that RACG significantly reduces high-cost errors—outperforming existing confidence-based and selective prediction approaches at equivalent abstention rates—while preserving the majority of decision utility.

causal gatinghigh-cost errorsleast-privilege agents

Existing machine learning decision systems offer only marginal safety guarantees, which are insufficient for risk-averse applications. This work proposes an action-conditional conformal prediction framework that explicitly links uncertainty quantification with specific decision actions, thereby constructing feasible decision sets conditioned on actions and establishing their theoretical connection to action-conditional Value-at-Risk (Action-Conditional VaR) optimization. Leveraging the pinball loss, we design an efficient optimization algorithm with finite-sample guarantees. Experimental results demonstrate that the proposed method significantly outperforms existing conformal baselines on two real-world datasets, achieving substantial improvements in action-conditional performance metrics.

action-conditional guaranteeconformal predictionrisk-averse decision making

This study addresses the critical challenge of dynamically allocating AI decision authority in high-stakes environments where evidence quality, uncertainty, and organizational objectives evolve over time. The work proposes the first Bayesian governance framework, modeling adaptive delegation as a governance-aware partially observable Markov decision process (POMDP). By leveraging Bayesian inference to estimate the information state and integrating sequential optimization for real-time authority adjustment, the approach explicitly calibrates AI uncertainty against organizational risk aversion in an interpretable manner. Evaluated in heterogeneous AI quality settings, the method significantly outperforms five benchmark strategies, demonstrating superior generalizability, robustness, graceful degradation, and adaptability across diverse governance scenarios.

AI delegationdecision authoritygovernance

This work addresses the challenge of selecting uncertainty representations that align with decision objectives to achieve optimal and trustworthy decisions under state-variable uncertainty. Drawing on decision theory, it systematically analyzes the optimal forms of uncertainty representation for both risk-neutral and risk-averse agents in known and unknown environments, revealing the minimal uncertainty information required under distinct risk preferences. The study innovatively unifies three approaches to epistemic uncertainty—calibrated prediction, confidence-set robust optimization, and Bayesian inference—establishing a theoretical link between uncertainty representation and decision goals. This integration yields a reliable decision-making framework that provides agents with verifiable utility guarantees.

decision makingepistemic uncertaintyposterior distribution

Hot Scholars

SC

Stephen Casper

PhD student, MIT
AI safetyAI responsibilityred-teamingrobustness
QZ

Quanyan Zhu

Department of Electrical and Computer Engineering, New York University
AIGame and Control TheorySecurity and ResilienceAutonomy
VC

Vincent Conitzer

Professor of CS, Carnegie Mellon University; Professor of CS and Philosophy, University of Oxford
artificial intelligencecomputer scienceeconomicsphilosophy
YC

Yejin Choi

Stanford University / NVIDIA
Natural Language ProcessingDeep LearningArtificial IntelligenceCommonsense Reasoning
HH

Hoda Heidari

Carnegie Mellon University
Responsible AIAI EthicsAI AccountabilityAlgorithmic Fairness