AffectEval: A Modular and Customizable Framework for Affective Computing

2025-04-29

AI Summary

Emotion computing systems suffer from inadequate multimodal support, poor cross-domain generalizability, and substantial redundant development efforts. To address these challenges, this paper introduces the first modular, customizable framework specifically designed for emotion computing. It adopts a plugin-based architecture that supports arbitrary modality combinations and domain adaptation. A configuration-driven pipeline engine, unified data interfaces, and pre-integrated emotion model adapters enable zero-code cross-task reuse and automatic pipeline generation. Experimental replication demonstrates that the framework reduces implementation code volume by 90%, significantly improving development efficiency and cross-domain generalization capability. By providing a scalable and transferable infrastructure, it advances practical deployment of multimodal emotion recognition across diverse application domains.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The field of affective computing focuses on recognizing, interpreting, and responding to human emotions, and has broad applications across education, child development, and human health and wellness. However, developing affective computing pipelines remains labor-intensive due to the lack of software frameworks that support multimodal, multi-domain emotion recognition applications. This often results in redundant effort when building pipelines for different applications. While recent frameworks attempt to address these challenges, they remain limited in reducing manual effort and ensuring cross-domain generalizability. We introduce AffectEval, a modular and customizable framework to facilitate the development of affective computing pipelines while reducing the manual effort and duplicate work involved in developing such pipelines. We validate AffectEval by replicating prior affective computing experiments, and we demonstrate that our framework reduces programming effort by up to 90%, as measured by the reduction in raw lines of code.

Problem

• Lack of software frameworks for multimodal emotion recognition
• High manual effort in affective computing pipeline development
• Limited cross-domain generalizability in existing solutions

Innovation

• Modular framework for affective computing pipelines
• Customizable design reduces manual coding effort
• Validated by replicating prior experiments efficiently