Beyond Text-to-Text: An Overview of Multimodal and Generative Artificial Intelligence for Education Using Topic Modeling

2024-09-24

arXiv.org

AI Summary

Current educational research on generative AI is heavily skewed toward text-to-text models (e.g., ChatGPT), while multimodal applications—such as text-to-image and text-to-audio—are markedly underexplored. Method: Drawing on 4,175 publications from the Dimensions database, this study employs a dual-topic modeling approach combining Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF), integrated with systematic text preprocessing and human-validated interpretability checks, to identify 38 fine-grained themes clustered into 14 overarching thematic domains. Results: Text-to-text research constitutes over 68% of the corpus, whereas multimodal educational applications account for less than 5%. The study quantifies pronounced imbalances across modalities and educational levels (K–12 through higher education), and proposes the first empirically grounded, multimodal, and education-level-inclusive research framework—offering actionable evidence for policy formulation and AI development in education.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Generative artificial intelligence (GenAI) can reshape education and learning. While large language models (LLMs) like ChatGPT dominate current educational research, multimodal capabilities, such as text-to-speech and text-to-image, are less explored. This study uses topic modeling to map the research landscape of multimodal and generative AI in education. An extensive literature search using Dimensions yielded 4175 articles. Employing a topic modeling approach, latent topics were extracted, resulting in 38 interpretable topics organized into 14 thematic areas. Findings indicate a predominant focus on text-to-text models in educational contexts, with other modalities underexplored, overlooking the broader potential of multimodal approaches. The results suggest a research gap, stressing the importance of more balanced attention across different AI modalities and educational levels. In summary, this research provides an overview of current trends in generative AI for education, underlining opportunities for future exploration of multimodal technologies to fully realize the transformative potential of artificial intelligence in education.

Problem

• Exploring multimodal GenAI's understudied role in education
• Mapping research gaps in non-text AI education applications
• Identifying imbalance in educational AI modality focus

Innovation

• Topic modeling maps multimodal AI research
• Analyzes 4175 articles for latent topics
• Highlights underexplored non-text AI modalities