Towards Wearable Opportunistic Crowdsensing for Open-Vocabulary Activity Data Collection Through User-Scheduled Trigger-Action Routines

2026-08-04

AI Summary

This study addresses the challenge of obtaining fine-grained annotations for wearable activity data in everyday environments, where retrospective labeling is costly and temporal boundaries are often ambiguous. The authors propose an in-situ annotation approach based on user-defined trigger-action rules: when an audio event occurs, the system prompts the user to provide open-vocabulary activity labels in real time, while simultaneously capturing multimodal sensor data with precise temporal boundaries. By transforming users into active participants, this method integrates opportunistic crowd-sensing with natural language labeling, substantially reducing annotation burden and enhancing ecological validity. Experimental results demonstrate high system log accuracy (97.30% recall, 97.15% precision), and expert evaluations confirm its superior alignment with real-world scenarios, with a field pilot further validating its feasibility in free-living conditions.

Abstract

Collecting richly labeled wearable activity data in everyday settings remains difficult because retrospective annotation is costly and often imprecise. Prior data collection apps rely on a labor-intensive self-reporting strategy and primarily treat participants as crowd labelers. We present Pebbl, a feasibility-stage system that incentivizes in-situ labeling through opportunistic crowdsensing. Pebbl lets users author trigger-action recipes on a smartphone and receive just-in-time reminders for beneficial actions when a trigger is detected. In the prototype, triggers are a limited set with four common audio cues, while actions are described in open-vocabulary natural language. Each confirmed execution yields a short sensor window with explicit start/end boundaries and a user-authored action label. We evaluate Pebbl through an expert workshop with wearable Human Activity Recognition (HAR) researchers (N = 6), a within-subject in-lab study (N = 21), and a pilot deployment (N = 8). Experts viewed the approach as lower burden and more ecologically valid than common labeling workflows. In the lab, Pebbl produced reliable execution logs under controlled conditions (recall = 97.30%, precision = 97.15%) and was preferred over comparison workflows on perceived burden and confidence. The pilot deployment shows that the interaction and sensing pipeline can function in free-living use, while surfacing practical constraints such as false triggers and context dependence. Overall, Pebbl represents a step toward a low-burden, distributable collection approach of user-contributed wearable activity data.

Problem

• wearable sensing
• activity data collection
• open-vocabulary labeling
• opportunistic crowdsensing
• in-situ annotation

Innovation

• opportunistic crowdsensing
• trigger-action routines
• open-vocabulary labeling
• wearable activity recognition
• in-situ annotation