Inverting the Hidden: Unveiling Multimodal Privacy Leakage in Collaborative LVLM Inference

2026-08-02

AI Summary

This study addresses the unresolved question of whether deep hidden states in collaborative large vision-language model (LVLM) inference leak recoverable private information. The work provides the first theoretical proof that such hidden states retain sufficient multimodal semantic content to enable reconstruction of the original inputs. To exploit this vulnerability, the authors propose RASR, a coarse-to-fine multimodal reconstruction attack framework that jointly recovers images and text by leveraging modality-specific inversion pathways and optimizing hidden state consistency. Experiments on Qwen3-VL and LLaVA-1.5 demonstrate that RASR reduces image reconstruction mean squared error by approximately 50% and achieves a 99% text recovery accuracy, thereby revealing significant privacy risks inherent in collaborative LVLM inference.

Abstract

Collaborative inference deploys Large Vision-Language Models (LVLMs) by partitioning computation between edge devices and the cloud. While withholding raw inputs supposedly ensures privacy, transmitting intermediate hidden states exposes a critical attack surface. However, it remains unclear whether deep-layer LVLM hidden states retain recoverable private information, given that visual content has been projected into the language embedding space. To address this concern, we theoretically analyze LVLM hidden-state recoverability and show that, under regularity assumptions and a positive semantic--nuisance margin, privacy-relevant visual semantics remain identifiable and stably recoverable. Motivated by this analysis, we propose RASR, a novel coarse-to-fine multimodal reconstruction attack. RASR obtains initial image and text reconstructions through modality-specific inverse paths that follow their respective forward processing pipelines in reverse, and then uses hidden-state consistency to refine both reconstructions. Evaluations on Qwen3-VL-8B-Instruct and LLaVA-1.5-7B across five datasets demonstrate that RASR reduces image reconstruction MSE by \(\sim\)50\% compared to the strongest baselines, while achieving up to 99\% token accuracy for text recovery. These results show that privacy-sensitive visual and textual information can be recovered even from deep-layer LVLM hidden states, exposing the privacy risks of collaborative inference.

Problem

• privacy leakage
• collaborative inference
• Large Vision-Language Models
• hidden states
• multimodal reconstruction

Innovation

• multimodal privacy leakage
• collaborative inference
• hidden-state recoverability
• RASR attack
• LVLM