Beyond Permissions: Investigating Mobile Personalization with Simulated Personas

2025-11-03

AI Summary

Mobile applications widely rely on sensor data to infer user context for personalization, yet such implicit inference logic lacks transparency. To address this, we propose a sandbox-based auditing framework designed to enhance transparency: it employs sensor simulation and structured virtual personas to inject multimodal behavioral data in real time; integrates Android-side real-time data injection, automated screenshot capture, and GPT-4 Vision–driven UI semantic parsing; and establishes an end-to-end behavior–response analysis pipeline. Unlike conventional adversarial sensor spoofing, our approach pioneers the use of sensor forgery for *observable auditing*, enabling dynamic visualization and reproducible validation of personalization mechanisms. We evaluate the framework across fitness, e-commerce, and lifestyle service apps, demonstrating statistically significant response variations under controlled changes in activity level, location, and time-of-day. Our work provides both a novel paradigm and empirical foundation for privacy-enhancing design and user-controllable transparency tools.

Technology Category

• Humans and AI: Brain-Sensing and Analysis
• Search and Optimization: Applications
• Application Domains: Mobility, Driving & Flight

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Security and Privacy: Data transparency and provenance
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

Mobile applications increasingly rely on sensor data to infer user context and deliver personalized experiences. Yet the mechanisms behind this personalization remain opaque to users and researchers alike. This paper presents a sandbox system that uses sensor spoofing and persona simulation to audit and visualize how mobile apps respond to inferred behaviors. Rather than treating spoofing as adversarial, we demonstrate its use as a tool for behavioral transparency and user empowerment. Our system injects multi-sensor profiles - generated from structured, lifestyle-based personas - into Android devices in real time, enabling users to observe app responses to contexts such as high activity, location shifts, or time-of-day changes. With automated screenshot capture and GPT-4 Vision-based UI summarization, our pipeline helps document subtle personalization cues. Preliminary findings show measurable app adaptations across fitness, e-commerce, and everyday service apps such as weather and navigation. We offer this toolkit as a foundation for privacy-enhancing technologies and user-facing transparency interventions.

Problem

• Investigating opaque personalization mechanisms in mobile applications
• Auditing how apps respond to inferred user behaviors and contexts
• Developing tools for behavioral transparency and user empowerment

Innovation

• Sensor spoofing and persona simulation for app auditing
• Real-time injection of multi-sensor profiles on Android
• GPT-4 Vision-based UI summarization for personalization cues