SS-MPC: A Sequence-Structured Multi-Party Conversation System

2025-02-24

AI Summary

Existing multi-party conversational (MPC) modeling approaches predominantly rely on explicit graph-structured representations, which often incur information loss and hinder end-to-end integration with pretrained language models (PLMs). To address this, we propose the first graph-free MPC response generation method: it implicitly encodes multi-party, multi-turn dialogues as structured sequential inputs, incorporating dialogue topology via role-aware tokens and position-sensitive mechanisms to enable direct fine-tuning of PLMs. By eliminating graph projection, our approach avoids structural information degradation and overcomes key bottlenecks in PLM adaptation for MPC. Experiments on mainstream MPC benchmarks yield BLEU-1 of 15.60% and ROUGE-L of 12.44%, surpassing state-of-the-art methods by +3.91 and +0.62 absolute points, respectively. Human evaluation further confirms significant improvements in response fluency and factual accuracy.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Conversational AI/Dialog Systems
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Recent Multi-Party Conversation (MPC) models typically rely on graph-based approaches to capture dialogue structures. However, these methods have limitations, such as information loss during the projection of utterances into structural embeddings and constraints in leveraging pre-trained language models directly. In this paper, we propose extbf{SS-MPC}, a response generation model for MPC that eliminates the need for explicit graph structures. Unlike existing models that depend on graphs to analyze conversation structures, SS-MPC internally encodes the dialogue structure as a sequential input, enabling direct utilization of pre-trained language models. Experimental results show that extbf{SS-MPC} achieves extbf{15.60% BLEU-1} and extbf{12.44% ROUGE-L} score, outperforming the current state-of-the-art MPC response generation model by extbf{3.91%p} in extbf{BLEU-1} and extbf{0.62%p} in extbf{ROUGE-L}. Additionally, human evaluation confirms that SS-MPC generates more fluent and accurate responses compared to existing MPC models.

Problem

• Improves Multi-Party Conversation models
• Eliminates need for explicit graph structures
• Enhances response fluency and accuracy

Innovation

• Sequence-structured encoding for dialogue
• Direct use of pre-trained language models
• Improved BLEU and ROUGE scores