Improving Variational Autoencoder using Random Fourier Transformation: An Aviation Safety Anomaly Detection Case-Study

2026-01-03

arXiv.org

AI Summary

This work addresses the limitations in efficiency and performance of reconstruction-based anomaly detection on high-dimensional aviation safety data. To overcome these challenges, the authors propose integrating Random Fourier Transform (RFT) into the Variational Autoencoder (VAE) framework. Through frequency principle analysis, they demonstrate that RFT enables the model to simultaneously learn both low- and high-frequency features, in contrast to the conventional progressive learning paradigm. Furthermore, a trainable RFT module is designed and embedded within the computational graph to enhance model expressiveness. Experiments on synthetic data and the Dashlink aviation dataset show that the RFT-enhanced VAE outperforms traditional approaches. Although the performance gain from the trainable RFT variant is not yet substantial, the proposed architecture offers a novel perspective for frequency-domain modeling in deep generative models.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements

Abstract

In this study, we focus on the training process and inference improvements of deep neural networks (DNNs), specifically Autoencoders (AEs) and Variational Autoencoders (VAEs), using Random Fourier Transformation (RFT). We further explore the role of RFT in model training behavior using Frequency Principle (F-Principle) analysis and show that models with RFT turn to learn low frequency and high frequency at the same time, whereas conventional DNNs start from low frequency and gradually learn (if successful) high-frequency features. We focus on reconstruction-based anomaly detection using autoencoder and variational autoencoder and investigate the RFT's role. We also introduced a trainable variant of RFT that uses the existing computation graph to train the expansion of RFT instead of it being random. We showcase our findings with two low-dimensional synthetic datasets for data representation, and an aviation safety dataset, called Dashlink, for high-dimensional reconstruction-based anomaly detection. The results indicate the superiority of models with Fourier transformation compared to the conventional counterpart and remain inconclusive regarding the benefits of using trainable Fourier transformation in contrast to the Random variant.

Problem

• anomaly detection
• variational autoencoder
• random Fourier transformation
• frequency principle
• aviation safety

Innovation

• Random Fourier Transformation
• Variational Autoencoder
• Frequency Principle
• Anomaly Detection
• Trainable Fourier Features