Outlier Detect using Vector Cosine Similarity by Adding a Dimension

πŸ“… 2024-02-19
πŸ›οΈ Digital Signal Processing and Signal Processing Education Workshop
πŸ“ˆ Citations: 1
✨ Influential: 0
πŸ“„ PDF
πŸ€– AI Summary
This work addresses the challenges of anomaly detection in high-dimensional data, where traditional methods often suffer from performance degradation and sensitivity to parameter settings. The authors propose a novel Multidimensional Outlier Detection (MDOD) algorithm that enhances discriminative power by extending the original data with an additional dimension filled with zeros and constructing vectors rooted at a designated observation point. Anomalies are identified through cosine similarity measures between these vectors. By innovatively integrating dimensionality expansion with an observation-point-based mechanism, MDOD significantly improves detection accuracy in high-dimensional settings. Empirical evaluations across multiple datasets demonstrate the method’s efficiency and effectiveness, and the implementation has been made publicly available as the open-source Python package β€œmdod” on PyPI.

Technology Category

Data Mining & Knowledge Management: Anomaly/Outlier DetectionMachine Learning: Dimensionality Reduction/Feature SelectionSearch and Optimization: Algorithm Configuration

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphsEconomics, Online Markets and Human Computation: Data quality aspects of human-annotated datasetsWeb Mining and Content Analysis: Robustness and generalizability of Web mining methods
πŸ“ Abstract
We propose a new outlier detection method with multi-dimensional data. The method detect outliers based on vector cosine similarity, with a new dataset built by adding a zero value dimension to original data. When a point in the new dataset is chosen as a measured point, an observation point is constructed as an origin with the only difference in the new dimension having a non-zero value when compared to the measured point. The vector from the observation point to the measured point is then formed, followed by another vector from the observation point to another point in the dataset. We compare the cosine similarity of the vectors to find out abnormal data.
Problem

Research questions and friction points this paper is trying to address.

outlier detection
multi-dimensional data
anomaly identification
cosine similarity
Innovation

Methods, ideas, or system contributions that make the work stand out.

outlier detection
cosine similarity
dimension augmentation
vector-based anomaly detection
MDOD
πŸ’Ό Related Jobs
No related jobs found.
Z
Zhongyang Shen