Speech Signals Complement LLMs for Predicting Interpersonal Attraction in Speed Dating

2026-07-25

AI Summary

This study investigates whether vocal signals provide incremental predictive value beyond large language models (LLMs) that rely solely on conversational transcripts to forecast interpersonal attraction. By integrating LLM-based analysis of dialogue text with a supervised learning model of acoustic features, we jointly predict mutual liking among speed-dating participants. Results indicate that the complementarity of vocal information to LLM predictions is not universal but highly contingent on specific contexts and participant subgroups. Notably, among individuals for whom voice-based predictions are more accurate, the fused approach significantly enhances prediction correlation and consistently improves pairwise ranking accuracy across all experimental conditions.

Abstract

Large language models (LLMs) can predict interpersonal attraction from conversation transcripts, but it remains unclear what a speech predictor can add beyond transcript-only LLM prediction. Using Japanese speed-dating conversations, we combine predictions from a transcript-only LLM and a supervised speech predictor to estimate participants' reported liking of their partners. We show that speech can complement transcript-only LLM prediction, but that this complementarity is conditional rather than universal. Combining the two predictions significantly improves pairwise ranking accuracy over the transcript-only LLM alone in all evaluated conditions. By contrast, gains in per-participant Pearson $r$ vary across conversation rounds and rating directions, with none significant after correction. Retrospectively, these $r$ gains are concentrated among participants for whom the speech predictor is more accurate. Speech can therefore retain predictive value even when an LLM predicts attraction from transcripts. The relevant question is not simply whether speech helps, but where its complementarity emerges.

Problem

• speech signals
• large language models
• interpersonal attraction
• speed dating
• predictive complementarity

Innovation

• speech signals
• large language models
• interpersonal attraction
• complementarity
• speed dating