🤖 AI Summary
This study addresses the fundamental scientific question of similarities and differences between large language models (LLMs) and human language cognition. Adopting an interdisciplinary approach integrating cognitive psychology, linguistics, and NLP, we design controlled human–machine behavioral comparison experiments to systematically evaluate how well LLMs simulate human linguistic behavior in text understanding and generation tasks. Our key contributions are threefold: (1) We establish LLMs as novel computational tools for investigating human language cognition; (2) We demonstrate that human feedback–based fine-tuning significantly enhances behavioral fidelity; and (3) We propose a new “human–AI co-enhanced language capability” paradigm. The work delineates the potentials and limitations of LLMs in cognitive modeling, providing both theoretical foundations and practical pathways for their trustworthy deployment in psychological assessment, language acquisition research, and educational interventions.
📝 Abstract
This report synthesizes the outcomes of a recent interdisciplinary workshop that brought together leading experts in cognitive psychology, language learning, and artificial intelligence (AI)-based natural language processing (NLP). The workshop, funded by the National Science Foundation, aimed to address a critical knowledge gap in our understanding of the relationship between AI language models and human cognitive processes in text comprehension and composition. Through collaborative dialogue across cognitive, linguistic, and technological perspectives, workshop participants examined the underlying processes involved when humans produce and comprehend text, and how AI can both inform our understanding of these processes and augment human capabilities. The workshop revealed emerging patterns in the relationship between large language models (LLMs) and human cognition, with highlights on both the capabilities of LLMs and their limitations in fully replicating human-like language understanding and generation. Key findings include the potential of LLMs to offer insights into human language processing, the increasing alignment between LLM behavior and human language processing when models are fine-tuned with human feedback, and the opportunities and challenges presented by human-AI collaboration in language tasks. By synthesizing these findings, this report aims to guide future research, development, and implementation of LLMs in cognitive psychology, linguistics, and education. It emphasizes the importance of ethical considerations and responsible use of AI technologies while striving to enhance human capabilities in text comprehension and production through effective human-AI collaboration.