Exploring the Pedagogical Potential of Large Language Models: A Multimodal Study on Student Learning Outcomes

With the rapid advancement of artificial intelligence (AI) technologies, particularly in the field of natural language processing (NLP) and large language models (LLMs), there is a growing opportunity to transform traditional learning methods to create more personalized, interactive, and effective educational experiences that cater to individual learning styles and needs. This pilot study explores the impact of AI technologies on student learning, comparing traditional book learning with AIpowered interactions using LLMs. In a controlled experiment with 27 university students, participants were divided into three equal groups, each exposed to a different learning technique: (1) bookbased learning, (2) LLM-powered interactive typing chat, and (3) LLM-powered virtual agent with voice chat, all containing the same book content. The results showed that the typing chat LLM group achieved the highest knowledge comprehension, outperforming both the book learning and voice chat groups. These findings highlight the potential of AI and LLMs to enhance educational outcomes by providing personalized and interactive learning experiences. This study underscores the importance of integrating AI-powered LLMs into education, paving the way for innovative approaches that leverage technology to improve student learning and retention.