mPOLICE: Provable Enforcement of Multi-Region Affine Constraints in Deep Neural Networks

2025-02-04

AI Summary

This work addresses the challenge of enforcing distinct affine output constraints over multiple—particularly non-convex and disjoint—input regions in safety-critical applications such as climate modeling, robotics, and industrial control. We propose mPOLICE, the first method enabling independent activation pattern assignment across multiple regions. Unlike POLICE—which is restricted to single convex regions—mPOLICE jointly optimizes layer-wise weights and biases, incorporates region-specific activation functions, and employs convex partitioning or non-convex approximation to model complex input domains. It ensures both local affine exactness and global continuity while eliminating constraint conflicts and over-generalization. We provide theoretical guarantees on constraint satisfaction. Experiments demonstrate zero inference overhead, minimal training cost, and effective approximate constraint satisfaction even for non-convex regions, across regression, classification, and function approximation tasks.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Search and Optimization: Non-convex Optimization
• Machine Learning: Hardware-aware ML

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Security and privacy of machine learning and AI applications
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Deep neural networks are increasingly employed in fields such as climate modeling, robotics, and industrial control, where strict output constraints must be upheld. Although prior methods like the POLICE algorithm can enforce affine constraints in a single convex region by adjusting network parameters, they struggle with multiple disjoint regions, often leading to conflicts or unintended affine extensions. We present mPOLICE, a new method that extends POLICE to handle constraints imposed on multiple regions. mPOLICE assigns a distinct activation pattern to each constrained region, preserving exact affine behavior locally while avoiding overreach into other parts of the input domain. We formulate a layer-wise optimization problem that adjusts both the weights and biases to assign unique activation patterns to each convex region, ensuring that constraints are met without conflicts, while maintaining the continuity and smoothness of the learned function. Our experiments show the enforcement of multi-region constraints for multiple scenarios, including regression and classification, function approximation, and non-convex regions through approximation. Notably, mPOLICE adds zero inference overhead and minimal training overhead.

Problem

• Enforces multi-region affine constraints in DNNs
• Prevents conflicts in disjoint constrained regions
• Maintains function continuity and smoothness

Innovation

• Multi-region affine constraints enforcement
• Layer-wise optimization for weights
• Distinct activation patterns assignment