Positional Encoding in the Context of Memristor-Based Analog Computation for Automatic Speech Recognition

2026-06-11

AI Summary

This work addresses a critical yet previously unreported issue in memristor-based analog computing: positional encoding induces an excessively large output dynamic range, leading to severe distortion during analog-to-digital conversion (ADC). To mitigate this problem, the study introduces a hardware-aware optimization strategy for positional encoding. In scenarios with tunable ADCs, the approach jointly adjusts the weight scaling of the memristor layer and the ADC bit precision; when the ADC is fixed, it eliminates the linear transformation associated with positional encoding. Notably, the proposed method incurs no additional energy overhead and reduces computational degradation by approximately 50% and 30% in the respective scenarios, substantially improving analog computing accuracy for speech recognition tasks.

Abstract

Memristors provide a new chance for resource-efficient computation of neural models for natural language processing by enabling analog execution of vector-matrix-multiplication. Yet, computations on these devices are currently subject to larger distortion, both in weight programming and execution. In this work, we identify large output values of transformed positional encodings to cause major degradation within analog-to-digital conversion (ADC) as part of memristor-based computation. By adjusting the proportion of weight and precision bits of the ADC of specific memristor layers, we reduce the degradation of the execution by ~50% relative, while keeping the estimated energy consumption stable. Additionally, we investigate scenarios where the ADC cannot be modified. In that case the degradation can be reduced by ~30% relative after removing encoding-related linear transformations.

Problem

• Positional Encoding
• Memristor-Based Analog Computation
• Automatic Speech Recognition
• Analog-to-Digital Conversion
• Output Value Degradation

Innovation

• memristor-based analog computation
• positional encoding
• analog-to-digital conversion (ADC)
• automatic speech recognition
• energy-efficient neural networks