GenAIReading: Augmenting Human Cognition with Interactive Digital Textbooks Using Large Language Models and Image Generation Models

2025-03-10

AI Summary

This study addresses low engagement and shallow comprehension in long-text learning by proposing a generative AI (GenAI)-based dual-channel cognitive enhancement method. We integrate large language models (LLMs) with diffusion-based image generation models (IGMs) to establish a semantically aligned text summarization–illustration co-generation mechanism. For the first time, Dual Coding Theory is deeply embedded into GenAI-driven educational applications to enable dynamic content adaptation and personalized reading experiences. Combining cognitive load assessment with human-AI interaction experiments, a 24-participant user study demonstrated a 7.50% improvement in post-test performance, significantly enhancing knowledge retention and conceptual understanding depth. Key contributions include: (1) a theory-grounded text–image co-generation framework; (2) an interpretable, cognition-aware adaptation mechanism; and (3) an empirically validated educational GenAI paradigm.

Technology Category

• Natural Language Processing: Generation
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Cognitive augmentation is a cornerstone in advancing education, particularly through personalized learning. However, personalizing extensive textual materials, such as narratives and academic textbooks, remains challenging due to their heavy use, which can hinder learner engagement and understanding. Building on cognitive theories like Dual Coding Theory -- which posits that combining textual and visual information enhances comprehension and memory -- this study explores the potential of Generative AI (GenAI) to enrich educational materials. We utilized large language models (LLMs) to generate concise text summaries and image generation models (IGMs) to create visually aligned content from textual inputs. After recruiting 24 participants, we verified that integrating AI-generated supplementary materials significantly improved learning outcomes, increasing post-reading test scores by 7.50%. These findings underscore GenAI's transformative potential in creating adaptive learning environments that enhance cognitive augmentation.

Problem

• Personalizing extensive textual materials for better learner engagement.
• Enhancing comprehension and memory using Dual Coding Theory principles.
• Improving learning outcomes with AI-generated text and visual content.

Innovation

• Utilized large language models for text summaries
• Employed image generation models for visual content
• Integrated AI to enhance learning outcomes significantly