Am I Infected? Lessons from Operating a Large-Scale IoT Security Diagnostic Service

2025-01-13

AI Summary

To address the challenge non-expert users face in autonomously securing home IoT devices, this study deployed a free, web-based automated security diagnostic service in Japan in February 2022. The system enables remote, clientless detection of vulnerabilities and malware by integrating network traffic analysis, device fingerprinting, CVE signature matching, and a lightweight malicious behavior detection model. As the first large-scale empirical study of IoT security scanning designed specifically for non-expert users, it identifies three core design principles: usability, trustworthiness, and actionable feedback. Over two years, the service engaged 114,000 users, achieving a vulnerability detection rate of 0.36%, a malware infection rate of 0.15%, a user satisfaction rate of 96%, and a post-rescan issue resolution rate exceeding 70%. These results empirically validate the effectiveness and scalability of a lightweight, user-centered security intervention paradigm.

Technology Category

• Application Domains: Internet of Things, Sensor Networks & Smart Cities
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Humans and AI: Interaction Techniques and Devices

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

There is an expectation that users of home IoT devices will be able to secure those devices, but they may lack information about what they need to do. In February 2022, we launched a web service that scans users' IoT devices to determine how secure they are. The service aims to diagnose and remediate vulnerabilities and malware infections of IoT devices of Japanese users. This paper reports on findings from operating this service drawn from three studies: (1) the engagement of 114,747 users between February, 2022 - May, 2024; (2) a large-scale evaluation survey among service users (n=4,103), and; (3) an investigation and targeted survey (n=90) around the remediation actions of users of non-secure devices. During the operation, we notified 417 (0.36%) users that one or more of their devices were detected as vulnerable, and 171 (0.15%) users that one of their devices was infected with malware. The service found no issues for 99% of users. Still, 96% of all users evaluated the service positively, most often for it providing reassurance, being free of charge, and short diagnosis time. Of the 171 users with malware infections, 67 returned to the service later for a new check, with 59 showing improvement. Of the 417 users with vulnerable devices, 151 users revisited and re-diagnosed, where 75 showed improvement. We report on lessons learned, including a consideration of the capabilities that non-expert users will assume of a security scan.

Problem

• IoT security
• user-awareness
• non-expert users

Innovation

• IoT Security
• User Experience
• Malware Detection