DiSCoKit: An Open-Source Toolkit for Deploying Live LLM Experiences in Survey Research

2026-02-11

AI Summary

This study addresses technical barriers—such as limited access to chat logs and insufficient control over model behavior—that often hinder social science research deploying real-time large language model (LLM) interaction experiments via online survey platforms. To overcome these challenges, the authors propose DiSCoKit, a lightweight, open-source JavaScript toolkit that enables seamless integration of cloud-based LLM APIs (e.g., from Azure) into mainstream survey platforms like Qualtrics. DiSCoKit offers the first embeddable and reproducible framework for LLM-based interactions in the social sciences, supporting automatic conversation logging, fine-grained control over model behavior, and unified experimental data collection. By significantly lowering the technical门槛 for human–AI interaction studies, this toolkit provides a robust infrastructure for conducting large-scale online social experiments with AI agents.

Technology Category

Humans and AI: Interaction Techniques and DevicesMachine Learning: Large Multimodal Models (LMMs)Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

Economics, Online Markets and Human Computation: LLM based quality controls for crowd workSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Advancing social-scientific research of human-AI interaction dynamics and outcomes often requires researchers to deliver experiences with live large-language models (LLMs) to participants through online survey platforms. However, technical and practical challenges (from logging chat data to manipulating AI behaviors for experimental designs) often inhibit survey-based deployment of AI stimuli. We developed DiSCoKit--an open-source toolkit for deploying live LLM experiences (e.g., ones based on models delivered through Microsoft Azure portal) through JavaScript-enabled survey platforms (e.g., Qualtrics). This paper introduces that toolkit, explaining its scientific impetus, describes its architecture and operation, as well as its deployment possibilities and limitations.

Problem

• large language models
• survey research
• human-AI interaction
• experimental deployment
• AI stimuli

Innovation

• DiSCoKit
• large language models
• survey research
• human-AI interaction
• open-source toolkit