Problem
Research questions and friction points this paper is trying to address.
on-device AI agents
real-time communication
network contention
multi-flow transport
agent-augmented RTC
Coordinated Networking for On-Device Agent-Augmented Real-Time Communication
🤖 AI Summary
This work addresses the degradation in video quality and increased response latency in on-device agent-augmented real-time communication, caused by competition between video streams and agent context streams for network resources. To resolve this, the paper proposes the HFS framework, which introduces an application-aware multi-stream cooperative transmission mechanism at the application layer. HFS employs a unified coordinator to jointly regulate the sending rates of both stream types, enabling synergistic optimization of high-quality video delivery and low-latency agent responses. A prototype system built upon WebRTC and llama.cpp implements an application-guided, fine-grained bandwidth allocation strategy. Experimental results demonstrate that, compared to baseline approaches, HFS improves video quality by 1.5× and reduces agent response time by 31%.
📝 Abstract
AI agents are enabling a new paradigm of agent-augmented real-time communication (RTC), where humans focus on high-level collaboration, while agents autonomously retrieve, analyze, and generate information in real time to support their interactions. These apps enable new experiences across various domains: for example, when corporate employees co-author a legal document, their agents can discuss and draft on their behalf, sparing them the burden of manually reviewing each other's work. As existing cloud-based agents suffer from privacy risks and unscalable server costs, on-device agent-augmented RTC offers a promising alternative. However, this on-device paradigm introduces a new networking challenge: contention between concurrent traffic flows generated by humans (for live video streaming) and agents (for sending context files for analysis). We design HFS, a framework to ensure both high live video quality and low agent response latency in agent-augmented RTC apps. We achieve the goal through an app-guided multi-flow transport approach, where a unified app-layer orchestrator jointly controls the sending rates of live video and agent context flows based on their heterogeneous app requirements. Our prototype built atop WebRTC and llama.cpp demonstrates that HAFS outperforms baselines, achieving 1.5x higher video quality while reducing agent response time by 31%.
Innovation
Methods, ideas, or system contributions that make the work stand out.
on-device AI agents
real-time communication
multi-flow transport
app-guided orchestration
WebRTC
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
