“Normalized Stress” is Not Normalized: How to Interpret Stress Correctly

2024-08-14

Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization

AI Summary

Normalized stress is a widely used metric for evaluating dimensionality reduction projections; however, it exhibits sensitivity to uniform scaling of projections, introducing scale-dependent bias that violates the physical plausibility of fidelity assessment. This work proposes the first scale-invariant stress correction method: we theoretically derive a scaling-invariance constraint and integrate distance-metric analysis with empirical validation across multiple benchmark datasets. The corrected stress metric fully eliminates unphysical scale dependence, restores expected monotonicity and ranking consistency in small-scale evaluations, and significantly enhances the fairness and reliability of cross-algorithm comparisons. By establishing rigorous scale invariance—grounded in geometric principles and empirically verified—the proposed metric constitutes the first strictly scale-invariant quantitative standard for assessing high-dimensional visualizations and embeddings.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Machine Learning: Dimensionality Reduction/Feature Selection
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Stress is among the most commonly employed quality metrics and optimization criteria for dimension reduction projections of high-dimensional data. Complex, high-dimensional data is ubiquitous across many scientific disciplines, including machine learning, biology, and the social sciences. One of the primary methods of visualizing these datasets is with two-dimensional scatter plots that visually capture some properties of the data. Because visually determining the accuracy of these plots is challenging, researchers often use quality metrics to measure the projection's accuracy or faithfulness to the full data. One of the most commonly employed metrics, normalized stress, is sensitive to uniform scaling (stretching, shrinking) of the projection, despite this act not meaningfully changing anything about the projection. We investigate the effect of scaling on stress and other distance-based quality metrics analytically and empirically by showing just how much the values change and how this affects dimension reduction technique evaluations. We introduce a simple technique to make normalized stress scale-invariant and show that it accurately captures expected behavior on a small benchmark.

Problem

• Normalized stress metric incorrectly responds to projection scaling changes
• Current stress metrics distort evaluation of dimension reduction techniques
• Proposing scale-invariant method to properly assess projection quality

Innovation

• Making normalized stress scale invariant
• Analyzing scaling effects on stress metrics
• Introducing technique for accurate projection evaluation