BIV-Priv-Seg: Locating Private Content in Images Taken by People With Visual Impairments

2024-07-25

arXiv.org

AI Summary

Visually impaired individuals often inadvertently capture privacy-sensitive content in photographs, yet existing vision models struggle to detect low-salience, small-scale, or text-free private objects and cannot reliably determine whether an image contains private content. To address this gap, we introduce BIV-Priv-Seg—the first pixel-level private-object segmentation dataset tailored to real-world photographs taken by blind and low-vision users—comprising 1,028 images with fine-grained annotations for 16 categories of private objects. Leveraging this benchmark, we conduct a systematic evaluation of state-of-the-art models including Mask R-CNN and Segment Anything, revealing their significant limitations in localizing minute or non-salient private objects and in binary “privacy present/absent” classification. Both the dataset and an online evaluation platform are publicly released to establish a foundational benchmark for privacy-aware visual assistance technologies.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Machine Learning: Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Security and Privacy: Privacy-enhancing technologies
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

Individuals who are blind or have low vision (BLV) are at a heightened risk of sharing private information if they share photographs they have taken. To facilitate developing technologies that can help them preserve privacy, we introduce BIV-Priv-Seg, the first localization dataset originating from people with visual impairments that shows private content. It contains 1,028 images with segmentation annotations for 16 private object categories. We first characterize BIV-Priv-Seg and then evaluate modern models' performance for locating private content in the dataset. We find modern models struggle most with locating private objects that are not salient, small, and lack text as well as recognizing when private content is absent from an image. We facilitate future extensions by sharing our new dataset with the evaluation server at https://vizwiz.org/tasks-and-datasets/object-localization.

Problem

• Privacy Leakage
• Visual Impairment
• Image Recognition

Innovation

• BIV-Priv-Seg Dataset
• Privacy Protection
• Visual Impairment