Running Conventional Automatic Speech Recognition on Memristor Hardware: A Simulated Approach

2025-05-30

AI Summary

Existing memristor hardware prototypes suffer from limited scale and precision, restricting deployment to small-scale models (e.g., MNIST) and precluding practical large-scale automatic speech recognition (ASR) systems. Method: This work presents the first end-to-end mapping of a million-parameter Conformer model—trained on TED-LIUMv2—to a high-fidelity memristor simulation platform. We introduce the first PyTorch-based memristor simulator library (built upon Synaptogen) that incorporates realistic non-idealities—including nonlinearity, device noise, and inter-device variability—and integrate it with 3-bit quantization-aware training and architecture-aware adaptation. Contribution/Results: Experimental evaluation demonstrates that, under 3-bit weight precision, memristor-simulated linear layers incur only a 25% relative increase in word error rate—substantially validating the feasibility of deploying large-scale ASR models on low-precision compute-in-memory hardware. This work establishes a foundational methodology and empirical evidence for memristor-accelerated complex temporal modeling.

Technology Category

• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Security and Privacy: Large-scale security measurements

Abstract

Memristor-based hardware offers new possibilities for energy-efficient machine learning (ML) by providing analog in-memory matrix multiplication. Current hardware prototypes cannot fit large neural networks, and related literature covers only small ML models for tasks like MNIST or single word recognition. Simulation can be used to explore how hardware properties affect larger models, but existing software assumes simplified hardware. We propose a PyTorch-based library based on"Synaptogen"to simulate neural network execution with accurately captured memristor hardware properties. For the first time, we show how an ML system with millions of parameters would behave on memristor hardware, using a Conformer trained on the speech recognition task TED-LIUMv2 as example. With adjusted quantization-aware training, we limit the relative degradation in word error rate to 25% when using a 3-bit weight precision to execute linear operations via simulated analog computation.

Problem

• Simulating large neural networks on memristor hardware
• Addressing limitations of current small ML models
• Maintaining performance with low-precision analog computation

Innovation

• Simulates memristor hardware with PyTorch library
• Models large neural networks on memristor systems
• Uses quantization-aware training for 3-bit precision