Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewarding

2026-08-06

AI Summary

This work addresses the challenge that large language models face in dynamically balancing long-term goal coordination with rapid adaptation during social interactions. Existing approaches typically apply uniform rewards across all dialogue turns, overlooking the specificity of turn-level objectives and the strategic coherence of responses. To overcome this limitation, the authors propose the Think-Strategy-Response (TSR) framework, which decouples dialogue into high-level strategic planning and low-level response generation. TSR integrates Linearized Hierarchical Reinforcement Learning with a Variance-Gated Reward mechanism (LHRL-VGR), dynamically allocating rewards based on the variance of goal-achievement scores to balance adherence to strategy with successful task completion. Evaluated on the SOTOPIA multi-agent social negotiation benchmark, the TSR model built upon Qwen2.5-7B achieves a 7.32% higher goal success rate than the GPT-4o baseline, establishing a new state-of-the-art performance.

Abstract

Large language models (LLMs) excel in structured tasks but struggle with dynamic social interactions, where success requires long-term goal coordination and rapid adaptation. Current methods often apply uniform goal-based rewards to every utterance, overlooking the specificity of objectives at each dialogue turn and failing to account for the rationale of potential strategies. Inspired by the Theory of Planned Behavior, we propose the Think-Strategy-Response (TSR) framework, which decomposes social dialogue into two hierarchical stages: high-level strategic planning and low-level linguistic execution. To optimize TSR, we introduce Linearized Hierarchical Reinforcement Learning with Variance-Gated Rewards (LHRL-VGR), a novel algorithm that dynamically routes rewards - balancing goal completion and strategy adherence - based on the variance of goal achievement scores. Experiments on the SOTOPIA benchmark show that our approach fine-tunes a Qwen2.5-7B agent to surpass the GPT-4o baseline by 7.32% in goal completion success, demonstrating state-of-the-art performance in multi-agent social negotiation tasks.

Problem

• social intelligence
• large language models
• dynamic social interactions
• goal coordination
• utterance-level rewards

Innovation

• Hierarchical Reasoning
• Utterance-Level Goal Rewarding
• Reinforcement Learning
• Social Intelligence
• Theory of Planned Behavior