Who Said What WSW 2.0? Enhanced Automated Analysis of Preschool Classroom Speech

2025-05-15

AI Summary

This study addresses the low accuracy and poor scalability of automated speech interaction analysis in authentic preschool classroom settings. We propose WSW2.0, a fully automated speech analysis framework that innovatively integrates wav2vec2-based speaker diarization with multi-version Whisper (large-v2/large-v3) speech-to-text transcription. To our knowledge, this is the first approach to achieve high-consistency quantification of language features in preschool contexts, yielding absolute inter-rater intraclass correlation coefficients (ICCs) of 0.64–0.98. Evaluated on 1,592 hours of real-world classroom audio collected over two years, WSW2.0 achieves a speaker diarization F1-score of 0.845 and word error rates of 0.119 (teacher) and 0.238 (child). Compared to expert human annotation, it delivers both high fidelity and substantial computational efficiency—enabling scalable, evidence-based research on early language development and educational practice.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

This paper introduces an automated framework WSW2.0 for analyzing vocal interactions in preschool classrooms, enhancing both accuracy and scalability through the integration of wav2vec2-based speaker classification and Whisper (large-v2 and large-v3) speech transcription. A total of 235 minutes of audio recordings (160 minutes from 12 children and 75 minutes from 5 teachers), were used to compare system outputs to expert human annotations. WSW2.0 achieves a weighted F1 score of .845, accuracy of .846, and an error-corrected kappa of .672 for speaker classification (child vs. teacher). Transcription quality is moderate to high with word error rates of .119 for teachers and .238 for children. WSW2.0 exhibits relatively high absolute agreement intraclass correlations (ICC) with expert transcriptions for a range of classroom language features. These include teacher and child mean utterance length, lexical diversity, question asking, and responses to questions and other utterances, which show absolute agreement intraclass correlations between .64 and .98. To establish scalability, we apply the framework to an extensive dataset spanning two years and over 1,592 hours of classroom audio recordings, demonstrating the framework's robustness for broad real-world applications. These findings highlight the potential of deep learning and natural language processing techniques to revolutionize educational research by providing accurate measures of key features of preschool classroom speech, ultimately guiding more effective intervention strategies and supporting early childhood language development.

Problem

• Automated analysis of preschool classroom speech interactions
• Improving accuracy and scalability in speaker classification
• Enhancing transcription quality for educational research applications

Innovation

• Uses wav2vec2 for speaker classification
• Employs Whisper models for speech transcription
• Scales to 1,592 hours of audio data