🤖 AI Summary
This work addresses the current lack of systematic and reusable methodologies for evaluating multimodal user interface toolkits across critical dimensions such as modality support, development efficiency, and experimental integration. The paper introduces the first structured benchmarking framework, enabling a comprehensive comparison of representative toolkits—including Geno, MSP, ReactGenie, WAMI, and EmoSync—along three axes: modality coverage and interaction abstraction, developer experience and workflow, and support for experimentation and system integration. Through document analysis, technical comparisons, and planned developer studies, the authors construct a reusable benchmark template that establishes a standardized foundation for future evaluation, selection, and empirical research on multimodal UI toolkits.
📝 Abstract
Multimodal user interfaces increasingly combine speech, gesture, vision, gaze, touch, biosignals, and other sensor data. Recent toolkits from the past five years, such as Geno, Multisensor-Pipeline (MSP), ReactGenie, and EmoSync, aim to make it easier for developers to prototype such interfaces, while older work such as WAMI shows how early web-based multimodal systems were conceived. Yet the field still lacks a systematic and reusable way to compare what these toolkits actually support, how much implementation work they offload from developers, and which evaluation strategies are appropriate for them. This paper reframes an HCI seminar draft into a benchmarking framework paper for multimodal user interface toolkits. Rather than reporting completed empirical results, it proposes a structured benchmark based on document analysis, technical comparison, and a future developer-based evaluation. The framework is organized around three dimensions: modality coverage and interaction abstraction, developer experience and workflow, and experimental and integration support. The paper illustrates the framework through five representative toolkits: Geno, MSP, ReactGenie, WAMI, and EmoSync. The contribution is a reusable benchmark template that future researchers can instantiate with empirical measurements, developer studies, and additional multimodal toolkits.