Semantic Action Graph: A Shared Representation for Agent Grounding and Human Interpretation of Sports Highlights

2026-09-17

AI Summary

本文提出了一种语义动作图来解决生成代理选择和叙述体育亮点难以验证和个性化的问题，通过连接事件序列、共享词汇表和可寻址帧实现。

Abstract

Generative agents are increasingly used to select and narrate video highlights, but they typically operate over unstructured or frame-level representations. Their output is consequently difficult for a viewer to verify and steer toward individual preferences. We present the semantic action graph, a lightweight domain schema that represents a sports match as performer, action, recipient, moment, and state nodes connected by role, temporal, and outcome edges. The schema demonstrates three key properties: 1) connected event sequences, 2) a shared, closed vocabulary, and 3) frame-addressable moments, making it suitable to serve two consumers at once: an agentic pipeline that composes narrated highlights, and a visual interface through which viewers query and inspect the same structure. We instantiate it in SportSAGE, a design probe pairing a four-module highlight pipeline with a graph interface, and report feedback from 12 soccer fans. Participants were satisfied with the quality of the generated highlights and narratives, and used the graph interface to search, navigate, and interpret the match highlights. These results provide early evidence that one small, human-readable schema can ground agent generation and support human interpretation at the same time.

Problem

• Generative agents
• video highlights
• unstructured representation
• viewer verification
• individual preferences

Innovation

• Semantic Action Graph
• Shared Representation
• Agent Grounding
• Human Interpretation
• Sports Highlights