OpenThaiGPT 1.6 and R1: Thai-Centric Open Source and Reasoning Large Language Models

2025-04-02

AI Summary

To address the limited generalization and reasoning capabilities of Thai large language models (LLMs), this work proposes two methodological innovations: (1) OTG-1.6 introduces task arithmetic—a model fusion paradigm—adapted for Thai, enabling the first cross-task knowledge transfer in this language and substantially improving generalization; (2) OTG-R1 integrates multi-stage supervised fine-tuning with the Less-Is-More (LIMO) inference hypothesis to enhance logical reasoning. Concurrently, we curate a high-quality Thai instruction dataset and establish a comprehensive evaluation benchmark. Experiments demonstrate that both models outperform larger open-source baselines across diverse Thai understanding, generation, and reasoning tasks, setting new state-of-the-art (SOTA) results for Thai LLMs. This work provides a reproducible methodology and foundational infrastructure for advancing LLM development in low-resource languages.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: (Large) Language ModelsCognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSearch and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We present OpenThaiGPT 1.6 and R1 (OTG-1.6 and OTG-R1), Thai-centric Large Language Models (LLMs) developed through distinct methodologies to enhance generalization and reasoning capabilities. OTG-1.6 employs Task Arithmetic model merging for broad generalization, while OTG-R1 integrates multi-stage training with the Less-Is-More Reasoning Hypothesis (LIMO) for advanced reasoning. Benchmark evaluations demonstrate superior performance across Thai language tasks, achieving competitive results against larger-scale open-source Thai LLMs. This paper details the proposed models, training processes, benchmarks, and results, highlighting improvements over previous models and establishing new performance standards for Thai-centric LLMs.

Problem

• Develop Thai-centric LLMs for enhanced generalization and reasoning
• Improve performance in Thai language tasks via novel methodologies
• Establish new benchmarks for Thai-centric open-source LLMs

Innovation

• Task Arithmetic model merging for generalization
• Multi-stage training with LIMO for reasoning
• Superior performance in Thai language tasks