VineetVC: Adaptive Video Conferencing Under Severe Bandwidth Constraints Using Audio-Driven Talking-Head Reconstruction

2026-02-13

Technology Category

Computer Vision: Generative Adversarial Networks (GANs) for VisionMachine Learning: Deep Generative Models & AutoencodersMultiagent Systems: Agent Communication

Application Category

Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterizationSearch and Retrieval-Augmented AI: Efficiency and scalability of Web search enginesResponsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Intense bandwidth depletion within consumer and constrained networks has the potential to undermine the stability of real-time video conferencing: encoder rate management becomes saturated, packet loss escalates, frame rates deteriorate, and end-to-end latency significantly increases. This work delineates an adaptive conferencing system that integrates WebRTC media delivery with a supplementary audio-driven talking-head reconstruction pathway and telemetry-driven mode regulation. The system consists of a WebSocket signaling service, an optional SFU for multi-party transmission, a browser client capable of real-time WebRTC statistics extraction and CSV telemetry export, and an AI REST service that processes a reference face image and recorded audio to produce a synthesized MP4; the browser can substitute its outbound camera track with the synthesized stream with a median bandwidth of 32.80 kbps. The solution incorporates a bandwidth-mode switching strategy and a client-side mode-state logger.

Problem

• bandwidth constraints
• video conferencing
• real-time communication
• network congestion
• latency

Innovation

• audio-driven talking-head synthesis
• adaptive video conferencing
• bandwidth-constrained networks
• WebRTC integration
• AI-based video reconstruction