RecruitView: A Multimodal Dataset for Predicting Personality and Interview Performance for Human Resources Applications

2025-11-29

AI Summary

Existing methods struggle to model the intrinsic geometric structure of personality and soft skills, while multimodal behavioral datasets remain scarce. To address these challenges, we introduce the first multimodal benchmark dataset comprising 2,011 real-world video interview clips and 27,000 pairwise comparison annotations. We propose a manifold-fusion-based cross-modal regression framework that—uniquely—jointly models hyperbolic, spherical, and Euclidean manifolds to capture the hierarchical geometric relationships between personality traits and behavioral cues. Furthermore, we design an adaptive routing expert network to enable tangent-space-level modality fusion. Experiments demonstrate significant improvements over state-of-the-art baselines: +11.4% in Spearman correlation coefficient, +6.0% in concordance index, and 40–50% reduction in parameter count, confirming both enhanced predictive accuracy and computational efficiency.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Automated personality and soft skill assessment from multimodal behavioral data remains challenging due to limited datasets and methods that fail to capture geometric structure inherent in human traits. We introduce RecruitView, a dataset of 2,011 naturalistic video interview clips from 300+ participants with 27,000 pairwise comparative judgments across 12 dimensions: Big Five personality traits, overall personality score, and six interview performance metrics. To leverage this data, we propose Cross-Modal Regression with Manifold Fusion (CRMF), a geometric deep learning framework that explicitly models behavioral representations across hyperbolic, spherical, and Euclidean manifolds. CRMF employs geometry-specific expert networks to capture hierarchical trait structures, directional behavioral patterns, and continuous performance variations simultaneously. An adaptive routing mechanism dynamically weights expert contributions based on input characteristics. Through principled tangent space fusion, CRMF achieves superior performance while training 40-50% fewer trainable parameters than large multimodal models. Extensive experiments demonstrate that CRMF substantially outperforms the selected baselines, achieving up to 11.4% improvement in Spearman correlation and 6.0% in concordance index. Our RecruitView dataset is publicly available at https://huggingface.co/datasets/AI4A-lab/RecruitView

Problem

• Predict personality and interview performance from multimodal video data
• Model behavioral traits across hyperbolic, spherical, and Euclidean manifolds
• Reduce trainable parameters while improving prediction accuracy over baselines

Innovation

• Cross-Modal Regression with Manifold Fusion for geometric deep learning
• Expert networks model hyperbolic, spherical, and Euclidean behavioral manifolds
• Adaptive routing mechanism dynamically weights expert contributions