behavioral engagement measurement

As misinformation-driven cognitive attacks grow increasingly sophisticated, existing methods struggle to quantitatively assess user engagement efficacy and the intensity of cognitive impact. Method: This paper proposes a weighted interaction metric framework that introduces, for the first time, a behavioral depth weighting mechanism—integrating interaction type, frequency, scale, and response ratio to attacker-generated content—to enable multidimensional evaluation of cognitive attack participation on social platforms. The approach combines behavioral analysis, weighted indicator modeling, and real-world social media data mining. Contribution/Results: Validated across multiple misinformation case studies, the framework not only differentiates propagation breadth but also precisely identifies high-impact cognitive attack nodes. It significantly enhances the interpretability and operationality of malicious network influence assessment, offering a novel paradigm and practical toolset for cognitive-domain defense strategies.

Existing online learning engagement monitoring often relies on simplistic counting heuristics or supervised predictive models requiring labeled outcomes or extensive training—limiting real-time applicability and interpretability for early intervention. To address this, we propose a course-chapter-aligned cumulative engagement metric derived directly from VLE (Virtual Learning Environment) log data, operating in an unsupervised manner without outcome labels or model training. Our approach enables real-time, fine-grained behavioral tracking while naturally conforming to the pedagogical rhythm of teaching weeks, thereby enhancing temporal alignment and educational interpretability. Validated across three undergraduate statistics courses, the metric achieves high concordance with conventional indicators from Week 3 onward; critically, by mid-semester (Weeks 6–8), it identifies all students who ultimately underperform at term-end, with predictive validity matching or exceeding baseline methods.

A critical challenge in explainable AI (XAI) and human-AI collaboration is determining *when* to provide explanations—i.e., real-time identification of genuine explanation needs—yet existing approaches rely on static, subjective assumptions and fail to dynamically capture users’ contextual demands. Method: We propose the first holistic, real-time explanation-need recognition framework integrating user behavior, system events, and physiological-emotional signals. Through systematic literature synthesis and empirical validation, we identify and verify 39 measurable, generalizable, and triggerable user-side indicators. We organize them into a cross-dimensional taxonomy (behavioral, system-event, and affective/physiological) and develop a demand-type mapping model. Contribution/Results: Grounded in online experiments, self-reports, and qualitative coding, we establish a structured indicator catalog comprising 17 behavioral, 8 system-event, and 14 affective/physiological measures, and design its runtime telemetry integration. The framework enables dynamic, precise, and temporally appropriate automated explanation triggering, validated in both prototype and production environments.

Existing visualization research predominantly focuses on *how to use* interactive features, neglecting the critical question of *how to construct* them. Method: We propose the first three-layer decoupled interaction authoring task model—intent–technique–component—derived from empirical coding and abstraction of 592 interaction units across 47 real-world applications. Contribution/Results: This model provides descriptive, evaluative, and generative capabilities, enabling the first unified formalization of interaction authoring intent, technical implementation, and component instantiation. It yields a reusable, theory-grounded classification framework that supports critical evaluation of existing visualization tools and informs the design and validation of next-generation low-code interaction authoring systems.

Information workers often struggle to translate enterprise-provided productivity metrics into actionable behavioral improvements. To address this, we designed and evaluated a privacy-aware, personalized AI productivity agent powered by GPT-4, grounded in a mixed-methods approach: a survey of 363 knowledge workers and telemetry data from Microsoft Viva Insights. Our method introduces a two-stage “survey-driven + telemetry-informed” paradigm, integrating personified interaction, fine-grained behavioral modeling, and user-controllable privacy mechanisms. In a 40-participant A/B controlled experiment, the agent significantly outperformed conventional dashboards and narrative-based tools—increasing task completion efficiency by 27% and achieving a user satisfaction rating of 4.6/5.0. This work presents the first empirical validation of a human-centered, dual-loop (data + insight) AI agent design for enhancing knowledge worker effectiveness, demonstrating both its feasibility and efficacy in real-world organizational settings.

This study addresses the challenge of designing lightweight digital interventions that can prompt action among low-motivation users within one minute, without requiring registration or sensing. Grounded in the Fogg Behavior Model and four design principles, the work proposes just-in-time micro-interventions targeting physical activity, healthy eating, and mental well-being. A novel mechanism enables users to collaboratively rewrite intervention prompts, facilitating “intentional personalization”—enhancing relevance while maintaining minimal friction. A 14-day field study (N=22) demonstrates that user-rewritten prompts significantly increase both acceptance and willingness to act, thereby validating co-creation as a viable lightweight strategy for effective personalization in behavioral interventions.

Traditional log-based metrics struggle to capture learners’ affective-cognitive states, limiting comprehensive explanations of learning outcomes. This study addresses this gap by integrating trait-like Deep Effortless Concentration (DEC)—a dispositional form of flow—with fine-grained reading strategy behaviors extracted from e-book interactions to construct a more holistic engagement metric. Through questionnaire-based DEC assessment, detailed analysis of reading logs, and regression modeling, the research reveals, for the first time, DEC’s moderating role in the relationship between behavioral indicators and academic performance. Findings show that incorporating DEC and reading strategies explains an additional 21.3% of the variance in academic achievement beyond baseline models, offering both theoretical and methodological innovations for personalized learning analytics.

This work addresses a critical limitation in existing user simulators, which reproduce observable behaviors but fail to capture users’ underlying cognitive states—such as confusion or satisfaction—during search interactions. To bridge this gap, the study introduces, for the first time, a cognitively grounded approach to modeling user behavior from interaction logs. Leveraging information foraging theory and human expert judgments, the authors develop a multi-agent system capable of scalably inferring users’ cognitive trajectories from large-scale behavioral data. The proposed method significantly improves performance on downstream tasks, including conversational outcome prediction and user difficulty recovery. Furthermore, the authors release cognitive annotations and accompanying tools for widely used datasets such as AOL and Stack Overflow, establishing a new paradigm for more human-like user simulation and evaluation of retrieval systems.

Traditional large language model (LLM) benchmarks often fail to capture real-world user experience, leading practitioners to rely on informal “vibe checks” that lack systematicity and reproducibility. This work formalizes vibe checks into a two-stage evaluation framework: personalized prompt generation grounded in user preferences, followed by subjective perception assessment. The authors instantiate this approach in a prototype benchmark and validate its efficacy through user studies, social media data analysis, and programming task experiments. Results demonstrate that the proposed method significantly alters model rankings compared to conventional metrics, effectively bridging the gap between standardized evaluations and actual user experience. By integrating personalized inputs with subjective judgment, this study introduces a novel evaluation paradigm that better reflects how users interact with and perceive LLMs in practice.

This study addresses the unintended long-term consequences of user interventions—such as “sleep reminders”—on recommender systems that dynamically adapt to user feedback. Through a large-scale field experiment on a short-video platform, combined with causal inference, log analysis, and dynamic policy modeling, the authors demonstrate that such interventions can inadvertently “retrain” the recommendation algorithm, inducing systemic shifts in content delivery. Contrary to expectations, the sleep reminder not only failed to reduce usage but increased late-night viewing duration by 14.75% and overall usage by 2.18%, with effects persisting for several weeks. These findings challenge the conventional paradigm of evaluating interventions under static assumptions and underscore the necessity of accounting for algorithmic adaptation in digital well-being policies.