LINC: Supporting Language Independent Communication and Comprehension to Enhance Contribution in Multilingual Collaborative Meetings

2025-04-26

AI Summary

In multilingual academic conferences, non-native English-speaking researchers frequently experience reduced participation, comprehension difficulties, and constrained intellectual contribution due to linguistic barriers. To address this, we propose the first holistic multilingual conference support framework jointly optimizing four dimensions: participation, comprehension, documentation, and feedback. Our dual-modal system comprises a front-end enabling real-time multilingual speech-to-text transcription, simultaneous interpretation, and interactive features; and a back-end leveraging ASR, neural machine translation, and discourse structure parsing to enable intelligent, cross-modal analysis—linking transcripts, audio, and action items—and delivering interactive visual dashboards. Empirical evaluation across six multilingual research teams demonstrates statistically significant improvements: +38% increase in speaking willingness, +42% gain in action-item recall accuracy, and +31% enhancement in pre-meeting preparation efficiency—validating the framework’s effectiveness and novelty in hybrid work environments.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Multimodal Learning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Collaborative research often includes contributors with varied perspectives from diverse linguistic backgrounds. However, English as a Second Language (ESL) researchers often struggle to communicate during meetings in English and comprehend discussions, leading to limited contribution. To investigate these challenges, we surveyed 64 ESL researchers who frequently collaborate in multilingual teams and identified four key design goals around participation, comprehension, documentation, and feedback. Guided by these design goals, we developed LINC, a multimodal Language INdependent Collaboration system with two components: a real-time module for multilingual communication during meetings and a post-meeting dashboard for discussion analysis. We evaluated the system through a two-phased study with six triads of multilingual teams. We found that using LINC, participants benefited from communicating in their preferred language, recalled and reviewed actionable insights, and prepared for upcoming meetings effectively. We discuss external factors that impact multilingual meeting participation beyond language preferences and the implications of multimodal systems in facilitating meetings in hybrid multilingual collaborative settings beyond research.

Problem

• Enhancing ESL researchers' communication in multilingual meetings
• Improving comprehension and contribution in English-dominated discussions
• Addressing language barriers in collaborative research settings

Innovation

• Real-time multilingual communication module
• Post-meeting dashboard for analysis
• Multimodal language-independent collaboration system