Score
Design and run analyses and experiments to identify and quantify distinct training or operational regimes (their boundaries and phases), including measuring transition points such as forgetting rates and slack, diagnosing visible empirical bottlenecks, and comparing how different policy or model classes perform across those regimes.
In supervised cybersecurity CTF training, evaluating learning outcomes and identifying flaws in training design remain challenging. To address these issues, this paper proposes an evaluation framework integrating process mining with multidimensional visual analytics. It models participants’ operational behavior sequences and implements an open-source, interactive dashboard supporting temporal pattern recognition, multivariate network visualization, and clustering analysis—rigorously adhering to established visualization design principles. Our key innovation lies in deeply embedding process mining into cybersecurity pedagogical assessment, enabling automated discovery of process deviations, bottlenecks, and organizational anomalies directly from system logs. A case study demonstrates that the framework effectively quantifies learner engagement, pinpoints training deficiencies—including task bottlenecks and imbalanced resource allocation—and substantially enhances the interpretability of evaluation results and their utility for instructional improvement.
This paper addresses three key challenges in modeling collaborative learning: difficulty in capturing temporal behavioral patterns, weak visualization capabilities, and low robustness in identifying critical events. To tackle these, we propose the Transition Network Analysis (TNA) framework—a novel probabilistic graphical model that unifies relational structure and temporal dynamics. TNA integrates stochastic process mining with multilayer network analysis, enabling centrality computation, community detection, and temporal clustering. A key innovation is the introduction of a Bootstrap-based transition significance test, which effectively filters spurious transitions. Evaluated on real-world collaborative learning data from 191 students, TNA successfully characterizes the dynamic evolution of regulatory processes, accurately identifies critical learning events and behavior clusters, and significantly enhances both the reliability and interpretability of temporal pattern analysis.
This study addresses the persistent contradictions in empirical assessments of climate and innovation policies by recognizing that the relationship between carbon emissions and economic growth exhibits dynamic heterogeneity during socio-technical transitions—a dimension often overlooked in existing literature. To resolve this, the paper proposes a novel analytical paradigm that operationalizes the theoretical concept of institutional regimes from transition theory by first identifying empirically grounded climate transition mechanisms. Integrating time-varying response analysis, latent variable modeling, and panel data methods within a hybrid econometric–machine learning framework, the authors develop a conditional diagnostic approach. Applying this framework to data from approximately 150 countries over 1991–2022, they successfully uncover distinct mechanisms governing the carbon–economy nexus, each characterized by unique stability and reconfiguration properties, thereby laying a foundation for more precise policy evaluation and forecasting.
This work addresses the challenge of systematically evaluating concept bottleneck models, whose applicability and failure mechanisms remain poorly understood due to the scarcity of real-world datasets with annotated concept labels. To bridge this gap, we introduce the first controllable synthetic benchmark that leverages parametric generation techniques to precisely modulate data modality, concept selection, annotation quality, and label completeness, thereby simulating diverse real-world relationships between concepts and predictions. This benchmark enables comprehensive evaluation of various concept bottleneck models across both decision-support and fully automated tasks, effectively identifying key performance determinants and characteristic failure modes. Our framework fills a critical void in the current evaluation landscape for concept-based interpretability methods.
Current evaluations of AI governance proposals often fall into binary oppositions, overlooking implicit value trade-offs and lacking transparent analytical tools. This work proposes a multidimensional policy analysis framework that integrates expert interviews with computational text analysis to construct an interpretable scoring system across policy attributes, enabling cross-proposal comparison through visualization. Its novelty lies in three aspects: first, a multidimensional evaluation approach that avoids predetermined conclusions and explicitly reveals inherent trade-offs; second, a transparent hybrid methodology combining qualitative expert insights with quantitative computational validation; and third, the introduction of a domain-calibrated model as a benchmark against general-purpose large language models. The framework enables comparable, interpretable assessments of AI governance proposals across multiple attributes, allowing stakeholders to evaluate proposal relevance and coherence according to their own normative priorities.
This study addresses the challenges of collaboration and quality control in open-source deep learning projects stemming from inadequate governance mechanisms. Drawing on the Institutional Analysis and Development (IAD) framework, it employs a mixed-methods empirical approach combining document content analysis and code commit tracking across PyTorch, TensorFlow, and PaddlePaddle. The analysis encompasses 109 governance documents and over 1,700 code commits, systematically uncovering the structure, temporal evolution, and functional dimensions of governance rules. The research identifies 17 rule themes and 7 rule types, revealing a distinct evolutionary pattern wherein operational rules emerge early and undergo frequent revisions, while structural rules appear later and evolve more steadily. Four core governance functions are distilled, culminating in 33 actionable recommendations for effective open-source AI project governance.