CobSeg: Coherence Boundary Modeling for Dialogue Topic Segmentation

2026-05-28

AI Summary

This work addresses the challenge in dialogue topic segmentation of simultaneously capturing lexical boundaries at utterance edges and semantic discontinuities across utterances. The authors propose CobSeg, a novel multi-branch architecture that, for the first time, decouples the modeling of semantic coherence and lexical boundary transitions. By integrating directional boundary prediction, information-theoretic boundary weighting, and corpus-derived topical coherence prompts, CobSeg effectively fuses these complementary cues without requiring large language models (LLMs) during inference. The method substantially outperforms existing non-LLM approaches across five benchmarks: under gold-standard supervision, it reduces Pk and WindowDiff (Wd) on VHF by 0.7 and 0.6, respectively, and achieves Pk = 1.0 on DialSeg711; with pseudo-labeling, it lowers Pk on VHF by 14.8 and reduces Pk on DialSeg711 and TIAGE by 1.5 and 1.1, respectively.

Abstract

Dialogue topic segmentation is critical in many human-AI collaborative applications which requires identifying heterogeneous boundary cues, including lexical transitions near utterance edges and semantic discontinuities across utterances. Existing utterance models often dilute these local lexical signals. We propose CobSeg, a novel multi-branch architecture that separates coherence-level semantic continuity from lexical boundary transitions and recovers both through directional boundary prediction. CobSeg further uses boundary informativeness weighting to emphasize high-utility utterance positions, and incorporates a corpus-derived topic coherence cue with learned combination weights. While CobSeg is evaluated as a compact trainable segmenter under supervised gold-boundary training and a pseudo-label setting with automatically induced boundaries, it performs enhanced boundary prediction without LLM calls during inference. Across five benchmarks, it improves $P_k$ and $W_d$ particularly when local lexical cues are prominent: under gold supervision, it reduces $P_k$ by 0.7 points and $W_d$ by 0.6 points on VHF, and reaches $P_k$ of 1.0 on DialSeg711; with induced boundaries, it reduces $P_k$ by 14.8 points on VHF, by 1.5 points on DialSeg711, and by 1.1 points on TIAGE, outperforming prior non-LLM approaches.

Problem

• dialogue topic segmentation
• boundary cues
• lexical transitions
• semantic discontinuities
• coherence modeling

Innovation

• dialogue topic segmentation
• coherence modeling
• lexical boundary transition
• directional boundary prediction
• boundary informativeness weighting