SiamGPT: Quality-First Fine-Tuning for Stable Thai Text Generation

2025-12-22

AI Summary

To address the instability and weak instruction-following capability of open-source large language models (LLMs) under complex Thai instructions, this paper introduces SiamGPT-32B—a Thai-optimized variant built upon the Qwen3-32B architecture. We propose a novel “Quality-First” supervised fine-tuning (SFT) paradigm that entirely forgoes continued pretraining and corpus expansion. Instead, SiamGPT-32B is trained exclusively via high-fidelity English-to-Thai instruction translation data and a Thai-specific AutoIF constraint framework. This approach significantly enhances instruction-following accuracy, multi-turn dialogue robustness, and Thai text generation stability. On the SEA-HELM benchmark, SiamGPT-32B outperforms all comparable open-source Thai LMs of similar scale, achieving consistent state-of-the-art performance across instruction following, multi-turn interaction, and linguistic understanding metrics.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: (Large) Language ModelsSearch and Optimization: Learning to Search

Application Category

Search and Retrieval-Augmented AI: Large language models for searchSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Open-weights large language models remain difficult to deploy for Thai due to unstable generation under complex instructions, despite strong English performance. To mitigate these limitations, We present SiamGPT-32B, an open-weights model based on Qwen3-32B, fine-tuned with a Quality-First strategy emphasizing curated supervision over data scale. The fine-tuning pipeline combines translated high-complexity English instruction data with a Thai-adapted AutoIF framework for instruction and linguistic constraints. Using supervised fine-tuning only, without continual pretraining or corpus expansion, SiamGPT-32B improves instruction adherence, multi-turn robustness, and linguistic stability. Evaluations on the SEA-HELM benchmark show that SiamGPT-32B achieves the strongest overall performance among similar-scale open-weights Thai models, with consistent gains in instruction following, multi-turn dialogue, and natural language understanding.

Problem

• Improves Thai text generation stability under complex instructions
• Enhances instruction adherence and multi-turn dialogue robustness
• Achieves top performance in Thai language understanding benchmarks

Innovation

• Quality-first fine-tuning strategy for Thai language
• Combines translated English data with Thai-adapted AutoIF
• Supervised fine-tuning without continual pretraining or expansion