Adapting Large Language Models for Character-based Augmentative and Alternative Communication

2025-01-17

AI Summary

This work addresses the challenge of real-time, character-level prediction in Augmentative and Alternative Communication (AAC), where large language models (LLMs) struggle to deliver low-latency, high-accuracy sequential character generation. To overcome this, we propose a lightweight character-decoding algorithm built upon subword-based LLMs—avoiding both classification-layer fine-tuning and inefficient byte-level modeling. We curate a high-quality, AAC-domain-specific corpus and introduce a rigorous text quality rating mechanism. Furthermore, we design a dialogue-oriented domain-adaptive curriculum learning strategy to progressively align model behavior with AAC usage patterns. Experiments demonstrate substantial improvements in character-level prediction accuracy for simple conversational scenarios, while maintaining low inference latency and minimal computational overhead. The approach achieves a favorable balance between practical usability and efficiency, establishing a deployable paradigm for adapting LLMs to resource-constrained AAC systems.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Humans and AI: AI for Accessibility

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Users of Augmentative and Alternative Communication (AAC) may write letter-by-letter via an interface that uses a character language model. However, most state-of-the-art large pretrained language models predict subword tokens of variable length. We investigate how to practically use such models to make accurate and efficient character predictions. We fine-tune models using a large dataset of sentences we curated in which each sentence is rated according to how useful it might be for spoken or written AAC communication. We find that using an algorithm to produce character predictions from a subword large language model provides more accurate predictions than adding a classification layer or using a byte-level model. We also find that our domain adaptation curriculum is effective at improving model performance on simple, conversational text.

Problem

• Large Language Models
• Augmentative and Alternative Communication (AAC)
• Letter Prediction

Innovation

• Large Language Models
• Assistive and Alternative Communication (AAC)
• Predictive Letter Typing