Application of Sensitivity Analysis Methods for Studying Neural Network Models

2025-04-21

AI Summary

Neural network sensitivity to input perturbations remains poorly understood, limiting model interpretability—especially in clinical applications. Method: We propose a hierarchical sensitivity analysis framework: (1) for small feedforward networks on clinical diabetes data, we employ Sobol indices for global sensitivity analysis to identify discriminative features and enable dimensionality reduction with <1.2% accuracy loss; (2) for CNNs (VGG-16, ResNet-18) applied to ultrasound imaging, we integrate local pixel perturbation analysis, activation maximization, and Grad-CAM to quantify spatial consistency between saliency maps and decision-relevant regions (IoU > 0.78). Contribution/Results: Our framework bridges global feature importance and local decision localization, demonstrating anatomically plausible attention patterns in CNNs. It supports diverse model scales and data modalities while ensuring statistical robustness and visual interpretability—constituting the first quantitative validation of spatial alignment between perturbation-based and gradient-based explanations in medical ultrasound.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

This study demonstrates the capabilities of several methods for analyzing the sensitivity of neural networks to perturbations of the input data and interpreting their underlying mechanisms. The investigated approaches include the Sobol global sensitivity analysis, the local sensitivity method for input pixel perturbations and the activation maximization technique. As examples, in this study we consider a small feedforward neural network for analyzing an open tabular dataset of clinical diabetes data, as well as two classical convolutional architectures, VGG-16 and ResNet-18, which are widely used in image processing and classification. Utilization of the global sensitivity analysis allows us to identify the leading input parameters of the chosen tiny neural network and reduce their number without significant loss of the accuracy. As far as global sensitivity analysis is not applicable to larger models we try the local sensitivity analysis and activation maximization method in application to the convolutional neural networks. These methods show interesting patterns for the convolutional models solving the image classification problem. All in all, we compare the results of the activation maximization method with popular Grad-CAM technique in the context of ultrasound data analysis.

Problem

• Analyzing neural network sensitivity to input perturbations
• Comparing methods for interpreting neural network mechanisms
• Reducing input parameters without significant accuracy loss

Innovation

• Uses Sobol global sensitivity analysis
• Applies local sensitivity for pixel perturbations
• Compares activation maximization with Grad-CAM