SCRIPT: A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models

2026-04-14

AI Summary

This work addresses the limitation of existing Korean pre-trained language models, which predominantly rely on subword tokenization and thus fail to explicitly capture the compositional structure of Hangul characters—formed from Jamo sub-characters—and their morphophonological regularities. To overcome this, the authors propose SCRIPT, a model-agnostic module that injects Jamo-level structural knowledge into existing Korean pre-trained models without requiring architectural modifications or re-pretraining. SCRIPT enhances subword embeddings through compositional Jamo representations and an embedding augmentation mechanism, thereby refining the granularity of linguistic representation. Experimental results demonstrate that SCRIPT consistently outperforms baseline models across a range of Korean natural language understanding and generation tasks, while also yielding embedding spaces that more accurately reflect underlying grammatical and semantic patterns.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Korean is a morphologically rich language with a featural writing system in which each character is systematically composed of subcharacter units known as Jamo. These subcharacters not only determine the visual structure of Korean but also encode frequent and linguistically meaningful morphophonological processes. However, most current Korean language models (LMs) are based on subword tokenization schemes, which are not explicitly designed to capture the internal compositional structure of characters. To address this limitation, we propose SCRIPT, a model-agnostic module that injects subcharacter compositional knowledge into Korean PLMs. SCRIPT allows to enhance subword embeddings with structural granularity, without requiring architectural changes or additional pre-training. As a result, SCRIPT enhances all baselines across various Korean natural language understanding (NLU) and generation (NLG) tasks. Moreover, beyond performance gains, detailed linguistic analyses show that SCRIPT reshapes the embedding space in a way that better captures grammatical regularities and semantically cohesive variations. Our code is available at https://github.com/SungHo3268/SCRIPT.

Problem

• Korean language models
• subcharacter composition
• Jamo
• morphophonology
• tokenization

Innovation

• subcharacter composition
• Korean language modeling
• Jamo representation
• embedding enhancement
• model-agnostic injection