Automatically Suggesting Diverse Example Sentences for L2 Japanese Learners Using Pre-Trained Language Models

2025-06-04

Annual Meeting of the Association for Computational Linguistics

AI Summary

Japanese as a second language (JSL) learners require example sentences that are pedagogically appropriate in linguistic difficulty, semantically diverse, and natural. Method: We propose a retrieval-augmented generation framework for example sentence recommendation. The retrieval path leverages a curated pedagogical sentence corpus and employs PLMs (BERT/GPT) to score candidates across multiple quality dimensions; the generation path uses zero-shot GPT-series models. Contribution/Results: This is the first systematic evaluation of PLMs’ capability to jointly optimize difficulty, diversity, and naturalness in Japanese example sentence recommendation. Human evaluations reveal significant annotator disagreement on diversity and naturalness. Empirically, retrieval-based recommendations achieve highest preference among all raters—especially for beginner and advanced learners—while generation-based outputs, though lower in average scores, demonstrate tunable potential. The study validates PLMs’ feasibility for enhancing personalization and pedagogical effectiveness in example sentence recommendation, establishing a novel paradigm for intelligent language teaching resource generation.

Technology Category

• Natural Language Processing: Generation
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Providing example sentences that are diverse and aligned with learners' proficiency levels is essential for fostering effective language acquisition. This study examines the use of Pre-trained Language Models (PLMs) to produce example sentences targeting L2 Japanese learners. We utilize PLMs in two ways: as quality scoring components in a retrieval system that draws from a newly curated corpus of Japanese sentences, and as direct sentence generators using zero-shot learning. We evaluate the quality of sentences by considering multiple aspects such as difficulty, diversity, and naturalness, with a panel of raters consisting of learners of Japanese, native speakers -- and GPT-4. Our findings suggest that there is inherent disagreement among participants on the ratings of sentence qualities, except for difficulty. Despite that, the retrieval approach was preferred by all evaluators, especially for beginner and advanced target proficiency, while the generative approaches received lower scores on average. Even so, our experiments highlight the potential for using PLMs to enhance the adaptability of sentence suggestion systems and therefore improve the language learning journey.

Problem

• Generating diverse example sentences for L2 Japanese learners
• Assessing sentence quality via difficulty, diversity, and naturalness
• Comparing retrieval and generative PLM approaches for sentence suggestions

Innovation

• Using PLMs for quality scoring in retrieval
• Applying zero-shot learning for sentence generation
• Evaluating sentences via multi-aspect human and AI ratings