Devising Experiments with Interactive Environments

2025-11-14

AI Summary

This study addresses the challenge non-programmers face in designing responsive audiovisual interactions for immersive performance. We propose a no-code authoring system built upon a visual logic layer. Methodologically, the system employs a modular architecture that integrates real-time inputs—including pose, spatial position, and speech—and maps them to lighting and sound outputs. To foster deep integration of technology and theatrical practice, we introduce three ensemble collaboration strategies: role rotation, controlled imperfection to stimulate creativity, and technical metaphor as a dramaturgical scaffold. Through six workshops involving eight professional creators, participants developed an improvised, single-audience immersive performance. Results demonstrate that the system significantly lowers the barrier to interactive theatre creation: it enables real-time adjustment during rehearsal and supports improvisational expression—all without coding. The visual logic layer proves both effective and feasible for cross-disciplinary creative practice. (149 words)

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Humans and AI: Interaction Techniques and Devices
• Multiagent Systems: Teamwork

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

This paper reports a practice-based investigation into authoring responsive light and sound in immersive performance without writing code. A modular system couples live gesture, position, and speech inputs to scenographic outputs through a visual logic layer that performers can operate in rehearsal. Across six workshops with eight professional performance-makers, we staged a progression from parallel ensemble and technical training to integrated dramaturgy, culminating in a single-spectator scratch immersive performance with interactive elements. This paper details the system's building blocks and the workshop arc. A reflexive reading of workshop video logs, post-workshop focus groups, and facilitator notes surfaced three ensemble-level strategies that made the technology workable in a hybrid devising/design practice: rotating roles between operator, performer, and mediator; embracing controlled imperfection as a creative resource; and using technology-describing metaphors to support creative practice.

Problem

• Creating responsive light and sound without coding for immersive performances
• Connecting live inputs to outputs through visual logic for performers
• Developing ensemble strategies for hybrid devising and design practices

Innovation

• Visual logic layer replaces coding for immersive performances
• Modular system connects live inputs to scenographic outputs
• Rotating performer-operator roles enable hybrid devising practice