ICS for complex data with application to outlier detection for density data

📅 2025-05-26
📈 Citations: 0
✨ Influential: 0
📄 PDF
🤖 AI Summary
This paper addresses the challenge of anomaly detection in complex functional and distributional data—particularly under low anomaly prevalence, where detection is inherently difficult. We propose an invariant coordinate selection (ICS) framework that is coordinate-free, enabling its application beyond traditional multivariate vector data. Our method provides, for the first time, a coordinate-free definition of ICS in abstract Euclidean spaces. It leverages Bayes–Hilbert space embedding and maximum penalized likelihood spline smoothing to construct compositional spline-based density representations, thereby achieving robust dimensionality reduction and anomaly identification for distributional data. Evaluated on daily maximum temperature distribution sequences from northern Vietnam (1987–2016), our approach significantly outperforms existing methods, successfully detecting critical climate anomalies—including previously overlooked extreme events—demonstrating both theoretical generalizability and practical efficacy in real-world functional data analysis.

Technology Category

Data Mining & Knowledge Management: Anomaly/Outlier DetectionMachine Learning: Dimensionality Reduction/Feature SelectionConstraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphsWeb Mining and Content Analysis: Robustness and generalizability of Web mining methodsEconomics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
📝 Abstract
Invariant coordinate selection (ICS) is a dimension reduction method, used as a preliminary step for clustering and outlier detection. It has been primarily applied to multivariate data. This work introduces a coordinate-free definition of ICS in an abstract Euclidean space and extends the method to complex data. Functional and distributional data are preprocessed into a finite-dimensional subspace. For example, in the framework of Bayes Hilbert spaces, distributional data are smoothed into compositional spline functions through the Maximum Penalised Likelihood method. We describe an outlier detection procedure for complex data and study the impact of some preprocessing parameters on the results. We compare our approach with other outlier detection methods through simulations, producing promising results in scenarios with a low proportion of outliers. ICS allows detecting abnormal climate events in a sample of daily maximum temperature distributions recorded across the provinces of Northern Vietnam between 1987 and 2016.
Problem

Research questions and friction points this paper is trying to address.

Extends ICS to complex data types like functional and distributional data
Develops outlier detection for complex data with preprocessing impact analysis
Applies ICS to detect climate anomalies in temperature distribution data
Innovation

Methods, ideas, or system contributions that make the work stand out.

Extends ICS to complex data types
Uses compositional spline functions preprocessing
Detects outliers in climate data effectively
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
C
Camille Mondon
Mathematics and Statistics, Toulouse School of Economics, Toulouse, 31000
H
H. T. Trinh
Faculty of Mathematical Economics, Thuongmai University, Hanoi
A
A. Ruiz-Gazen
Mathematics and Statistics, Toulouse School of Economics, Toulouse, 31000
C
C. Thomas-Agnan
Mathematics and Statistics, Toulouse School of Economics, Toulouse, 31000