🤖 AI Summary
This work addresses the pervasive lack of social intelligence in large language models during prolonged human–AI interaction, particularly their inability to infer mental states, track relationships, understand social norms, and adapt to contextual dynamics. To bridge this gap, the authors propose the Zhijing framework, which systematically integrates evaluation, internalization, and deployment of social intelligence. It introduces SoMBench, a multidimensional psychological benchmark; employs diagnostic-driven, phased training—including supervised fine-tuning, policy distillation, and rule-based reinforcement learning—to achieve parametric internalization; and features the Actio inference architecture that combines Starling mental-state representations, SAGE for experiential reuse, and gated RAG for runtime social knowledge injection. Experiments demonstrate that Zing-series models significantly outperform baselines across five social cognition benchmarks, with Actio improving performance in 14 out of 15 model–benchmark combinations and achieving state-of-the-art or tied-best results on eight.
📝 Abstract
As large language models move from isolated task solving toward long-term service in human environments, they require social intelligence: the ability to infer mental states, track social relations, reason over norms, and adapt behavior under context. This report presents Zhijing, an integrated framework for measuring, internalizing, and grounding social intelligence. For measurement, we introduce SoMBench, a psychology-grounded benchmark spanning 3 primary dimensions, 17 secondary dimensions, and 71 task paradigms. It controls question format, narrative perspective, and context length across 284 shared scenarios and 3,481 expert-verified instances. Evaluation of 20 representative LLMs reveals substantial headroom: the best model achieves only 72.08% overall accuracy, and none of the 17 secondary dimensions reaches the 90% near-ceiling band. For internalization, we develop Zing, a diagnosis-driven training recipe combining supervised fine-tuning, on-policy distillation, and rubric-based reinforcement learning. Across five social-cognition benchmarks, Zing consistently outperforms its base models, with Zing-27B-Stage2 achieving the best average score and Zing-32B-Stage2 remaining competitive with DeepSeek-V4-Pro. For deployment-time grounding, we build Actio, a harness-controlled inference architecture that routes four typed supports into reasoning: PRISM for procedural guidance, Starling for runtime mental-state representation, SAGE for reusable experience, and gated RAG for external social and normative knowledge. Across five base models and three benchmarks, the full harness improves 14 of 15 model-benchmark pairs and is best or tied for best in 8, demonstrating the effectiveness of typed runtime support. Together, these results show that socially intelligent LLMs require coordinated advances in evaluation, parametric internalization, and deployment-time grounding.