OpenThaiGPT 1.5: A Thai-Centric Open Source Large Language Model

2024-11-11

arXiv.org

AI Summary

To address the suboptimal performance and lack of multimodal interaction support in existing open-source Thai large language models (LLMs), this work introduces a Thai-specific open-source LLM built upon the Qwen-2.5 architecture. We construct a high-quality Thai instruction dataset comprising over 2 million samples and employ supervised fine-tuning (SFT), retrieval-augmented generation (RAG) integration, and a lightweight structured tool-calling protocol. This is the first open-source Thai LLM to jointly support multi-turn dialogue, RAG-enhanced generation, and extensible tool invocation. Evaluated on multiple Thai-centric benchmarks, the model achieves state-of-the-art (SOTA) performance among open-source models while significantly reducing GPU memory footprint—enabling efficient deployment on consumer-grade GPUs. The approach ensures practical utility, architectural flexibility, and full reproducibility.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: (Large) Language ModelsComputer Vision: Large Vision Models

Application Category

Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

OpenThaiGPT 1.5 is an advanced Thai language chat model based on Qwen v2.5, finetuned on over 2,000,000 Thai instruction pairs. This report provides an engineering perspective on the model's development, capabilities, and performance. We discuss the model's architecture, training process, and key features, including multi-turn conversation support, Retrieval Augmented Generation (RAG) compatibility, and tool-calling functionality. Benchmark results demonstrate OpenThaiGPT 1.5's state-of-the-art performance on various Thai language tasks, outperforming other open-source Thai language models. We also address practical considerations such as GPU memory requirements and deployment strategies.

Problem

• Develops a Thai-centric language model
• Enhances multi-turn conversation capabilities
• Optimizes performance for Thai language tasks

Innovation

• Thai-centric language model
• Retrieval Augmented Generation compatibility
• Advanced multi-turn conversation support