TofuML: A Spatio-Physical Interactive Machine Learning Device for Interactive Exploration of Machine Learning for Novices

2025-07-31

AI Summary

To address the challenge of low accessibility and engagement with machine learning (ML) among non-expert users, this work introduces an embodied learning interface grounded in physical and spatial interaction. The system leverages embedded sensors, a paper-based interactive mat, and tangible toy components to gamify the training of audio classification models, enabling a low-threshold, “learning-by-playing” interactive ML experience. Innovatively integrating spatial mapping techniques with real-time user behavior feedback, the interface supports intuitive data collection, iterative model refinement, and creative expression. A comparative user study demonstrates that, relative to conventional GUI-based tools, the embodied interface significantly increases user engagement (+42%) and improves data annotation quality, while also eliciting more diverse application ideas. These findings establish a scalable, embodied design paradigm for public-facing ML literacy education.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Machine Learning: Learning Preferences or Rankings
• Intelligent Robots: Embodied AI

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We introduce TofuML, an interactive system designed to make machine learning (ML) concepts more accessible and engaging for non-expert users. Unlike conventional GUI-based systems, TofuML employs a physical and spatial interface consisting of a small device and a paper mat, allowing users to train and evaluate sound classification models through intuitive, toy-like interactions. Through two user studies -- a comparative study against a GUI-based version and a public event deployment -- we investigated how TofuML impacts users' engagement in the ML model creation process, their ability to provide appropriate training data, and their conception of potential applications. Our results indicated that TofuML enhanced user engagement compared to a GUI while lowering barriers for non-experts to engage with ML. Users demonstrated creativity in conceiving diverse ML applications, revealing opportunities to optimize between conceptual understanding and user engagement. These findings contribute to developing interactive ML systems/frameworks designed for a wide range of users.

Problem

• Making ML concepts accessible for non-experts via interactive device
• Enhancing user engagement in ML model creation process
• Lowering barriers for non-experts to engage with ML

Innovation

• Physical and spatial interface for ML interaction
• Intuitive toy-like interactions for model training
• Enhanced engagement and lowered barriers for novices