Multimodal Conversational Agent for Real-Time Cognitive and Affective State Classification
August 17, 2026
This system integrates LLMs, KG-BERT, and bidirectional LSTMs with attention to classify student engagement, stress, and understanding using text, prosody, and temporal behavior. A 45-student pilot study reported improved motivation and moderate academic gains over unimodal baselines.
HOW THIS AFFECTS YOU
●
builderYou can combine knowledge graphs and LSTMs with LLMs to build more responsive educational agents.
●
researcherThe multimodal approach provides a framework for real-time affective computing in learning environments.