Metric-Guided Synthesis of Class Activation Mapping

2025-04-14

AI Summary

Existing Class Activation Mapping (CAM) methods lack explicit modeling of user intent and domain knowledge, making it difficult to flexibly control heatmap properties such as localization accuracy, faithfulness, and robustness. Method: We propose SyCAM, the first metric-driven framework for automatic synthesis of CAM expressions. Under formal grammar constraints, SyCAM jointly optimizes multiple objectives—including Intersection-over-Union (IoU), faithfulness, and robustness—via differentiable symbolic expression search and grammar-guided program synthesis, generating interpretable and customizable activation expressions. Contribution/Results: SyCAM breaks away from rigid, hand-crafted CAM formulas, enabling on-demand customization of heatmap behavior. Evaluated on ResNet50, VGG16, and VGG19, it achieves an average 12.7% improvement across target metrics. The framework significantly enhances flexibility, controllability, and interpretability of heatmap generation, establishing a new paradigm for adaptive, user-aligned visual explanation.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Class activation mapping (CAM) is a widely adopted class of saliency methods used to explain the behavior of convolutional neural networks (CNNs). These methods generate heatmaps that highlight the parts of the input most relevant to the CNN output. Various CAM methods have been proposed, each distinguished by the expressions used to derive heatmaps. In general, users look for heatmaps with specific properties that reflect different aspects of CNN functionality. These may include similarity to ground truth, robustness, equivariance, and more. Although existing CAM methods implicitly encode some of these properties in their expressions, they do not allow for variability in heatmap generation following the user's intent or domain knowledge. In this paper, we address this limitation by introducing SyCAM, a metric-based approach for synthesizing CAM expressions. Given a predefined evaluation metric for saliency maps, SyCAM automatically generates CAM expressions optimized for that metric. We specifically explore a syntax-guided synthesis instantiation of SyCAM, where CAM expressions are derived based on predefined syntactic constraints and the given metric. Using several established evaluation metrics, we demonstrate the efficacy and flexibility of our approach in generating targeted heatmaps. We compare SyCAM with other well-known CAM methods on three prominent models: ResNet50, VGG16, and VGG19.

Problem

• Automate CAM expression synthesis for user-defined metrics
• Optimize heatmap generation to reflect specific CNN properties
• Enable customizable saliency maps based on domain knowledge

Innovation

• Metric-based synthesis of CAM expressions
• Syntax-guided synthesis with constraints
• Optimizes heatmaps for user-defined metrics