Fix the Mind, Not the Move: Interpretable AI Assistance via Knowledge-Gap Localization

2026-06-03

AI Summary

This work proposes SENSEI, a novel framework that addresses the limitation of traditional AI assistants—which typically correct only surface-level user errors—by intervening at the level of structured knowledge rather than observable actions. SENSEI infers users’ knowledge gaps through interaction analysis, precisely localizes and disentangles overlapping misconceptions within a structured knowledge graph, and generates minimal yet sufficient explanations to rectify them. The approach achieves zero-shot compositional generalization and effectively identifies real human misunderstandings in three long-horizon tasks. Empirical results demonstrate that 90% of students’ recurrent cognitive errors are successfully corrected, leading to significant improvements in long-term task performance.

Abstract

AI assistants in human-AI collaboration often correct suboptimal human actions through behavioral feedback (e.g., alerts or steering-wheel nudges in assistive driving). Such interventions can mitigate immediate errors, but long-term improvement requires addressing the underlying misconceptions that cause repeated mistakes. We introduce SENSEI, a framework that infers user misconceptions from interaction behavior and provides targeted, minimal yet sufficient suggestions to correct them. Our approach departs from action- or trajectory-level interventions by operating over a structured knowledge representation to localize and correct the sources of erroneous behavior. Across three long-horizon tasks with diverse misconceptions and corresponding behaviors, SENSEI demonstrates zero-shot compositional generalization, disentangling multiple overlapping misconceptions despite training only on single-misconception cases. A user study further shows that our method identifies real human misconceptions and provides effective guidance that improves long-horizon task performance, successfully correcting $90\%$ of student misconceptions. Code and project page are available at https://misoshiruseijin.github.io/SENSEI/.

Problem

• knowledge-gap localization
• human-AI collaboration
• misconception correction
• interpretable AI
• long-horizon tasks

Innovation

• knowledge-gap localization
• interpretable AI assistance
• structured knowledge representation
• zero-shot compositional generalization
• human-AI collaboration