human-in-the-loop methods

To address the lack of formal modeling support for human–AI collaborative workflows, this paper proposes a BPMN extension tailored for hybrid collaboration between humans and large language model (LLM)-based agents. We introduce the first BPMN metamodel and corresponding graphical notation explicitly supporting human–agent coordination, formally capturing task responsibility assignment, decision logic ownership, and execution strategy binding—thereby bridging a critical expressiveness gap in existing business process modeling languages for mixed-agent scenarios. Furthermore, we design a domain-specific language (DSL) and implement an open-source web-based modeling tool using TypeScript and React, enabling visual specification and formal verification of collaborative processes. Empirical evaluation through case studies demonstrates that the approach achieves strong expressive power, operational feasibility, and practical applicability in real-world human–LLM workflow engineering.

To address the weak dynamic workflow adaptability and poor fault tolerance of multi-agent systems in complex tasks, this paper proposes an evolvable workflow modeling and runtime re-planning framework based on Activity-on-Vertex (AOV) graphs. We pioneer the representation of workflows as dynamic AOV graphs, integrating dependency analysis with parallelism quantification to assess task dependency complexity and enable modular decoupling. Coupled with large language model (LLM)-based agents, the framework supports real-time rescheduling and autonomous error recovery guided by historical execution performance. Experimental results demonstrate significant improvements: +32.7% in task execution efficiency, +28.4% in goal achievement rate, and enhanced fault tolerance—enabling adaptive adjustment of highly concurrent subtasks under dynamic conditions.

Multi-agent workflow design suffers from an expansive design space, heavy reliance on expert intuition, and a tendency to converge prematurely to suboptimal solutions or engage in inefficient trial-and-error. To address these challenges, this paper proposes a three-layer progressive design-space modeling framework that decouples task planning, agent orchestration, and optimization—integrated with a design pattern library and context-aware recommendation algorithms—to realize an interactive, visual tool supporting in-situ guidance and multi-criteria comparison. Our contributions are threefold: (1) the first systematic, structured formalization of the multi-agent design space; (2) visualization-enabled cognitive scaffolding that closes the “perceive–reason–decide” loop; and (3) context-driven real-time recommendations that enhance both design quality and efficiency. User studies and case validations demonstrate significant reductions in design cycle time, improved systematicity in solution exploration, and higher recommendation adoption rates.

Enterprise operational workflows are notoriously difficult to automate end-to-end due to their heavy reliance on human intervention and limited adaptability to change. This work proposes the first action-centric workflow graph framework, which achieves automated construction, execution, and evolution through a three-stage pipeline: structured workflow graphs are extracted from human operation traces, executed via multi-agent online traversal, and continuously optimized in a closed loop using an Adaptive Traversal Reinforcement (ATR) mechanism. Integrating large-scale offline graph construction, graph-guided retrieval, and large language model reasoning, the approach was deployed across four cloud database services. It substantially outperforms the Trace-RAG baseline in coverage breadth, factual accuracy, and diagnostic throughput, achieving an expert blind-review score of 4.95 out of 5.

To address the insufficient quality and reliability of LLM-generated workflows for complex, multi-intent user queries, this paper proposes Opus, a prompt-based intent framework that introduces a reproducible and customizable intent-capture layer between natural-language queries and workflow generation. Opus decomposes mixed intents into structured intent objects through integrated signal extraction, structured parsing, and intent-driven generation. Its core innovations include formal definitions of “workflow signals” and “structured intents,” along with a lightweight intent abstraction mechanism. Evaluated on a benchmark of 1,000 synthetically generated multi-intent queries, Opus significantly improves the logical coherence, semantic consistency, and semantic similarity of generated workflows—particularly under high-complexity conditions.

Automatically constructing high-quality, reusable skills from heterogeneous, fragmented interaction traces—often missing critical security behaviors—is highly challenging. This work proposes the W2S framework, which introduces a novel intermediate representation called RWSA to decouple skills into workflow structure, execution semantics, and runtime attachments, thereby enabling task decomposition, control-flow modeling, verification, rollback, and state management. W2S achieves efficient skill construction through trajectory segmentation, local skill draft generation, structural alignment, branch fusion, redundancy compression, and confidence-aware retention. Experimental evaluation across 70 skills demonstrates that W2S improves behavioral replay consistency by 10.5% compared to baseline approaches based on summarization and prompting.

This study addresses the limitations of traditional expertise theories, which predominantly conceptualize expertise as a static accumulation of knowledge or skills and thus fail to account for its dynamic cognitive mechanisms. To overcome this, the work proposes a “workflow cognition” framework that reconceptualizes expertise as an emergent cognitive phenomenon arising from the recursive coupling and continuous co-evolution of thought flows and workflow patterns through sustained practice. Drawing on theoretical modeling and cross-domain case comparisons—spanning craftsmanship, creative production, education, and leadership—the study constructs a dynamic cognitive ontology of expertise, offering the first systematic account of the internal grammar linking tacit cognition, aesthetic judgment, and expert workflows. This framework provides both a theoretical foundation and a computational modeling pathway for representing human expertise in AI-enhanced expert systems.

This study addresses the lack of standardized and automated case planning processes in medical social work, which currently relies heavily on individual practitioner experience and suffers from inefficiency. The authors propose a model-agnostic, open-source large language model (LLM) workflow that systematically integrates established social work practice frameworks into LLM prompt design for the first time. The approach decomposes case planning into six sequential stages—assessment, problem analysis, goal setting, intervention planning, risk anticipation, and outcome evaluation—and combines structured client profiling with staged prompt engineering to generate professional, reviewable draft assessment forms and service plans. Designed to be compatible across multiple LLM platforms, the framework ensures cross-model reproducibility, and its code has been publicly released to provide a standardized tool for advancing intelligent support in medical social work.

Existing visual analytics workflows are predominantly described in unstructured textual form, hindering systematic comparison, reuse, and practical guidance. This work proposes ATWL, a formal, declarative language for modeling visual analytics workflows through a modular ontology grounded in eight artifact types and standardized intents. For the first time, this approach enables structured, machine-interpretable representations of such workflows. Leveraging large language models, the authors automatically extract workflows from academic papers to construct a reusable repository comprising 17 annotated instances. Empirical evaluation demonstrates that ATWL effectively uncovers cross-workflow structural patterns and yields more compact, structured, and extensible analytical recommendations than original narrative descriptions, thereby facilitating efficient in-context reuse and adaptation.

This study addresses the lack of explicit contextual representation—encompassing tasks, roles, resources, and constraints—in human-AI collaboration, which often leads to eroded trust, non-traceable processes, and difficulties in accountability. To tackle this challenge, the paper proposes the first ontology-driven Contextual Collaborative AI framework (CCAI), leveraging RDF/OWL to construct machine-readable semantic ontologies that transform prompt-response interactions into structured, queryable collaboration trajectories. By integrating SPARQL-based context retrieval, CCAI enables precise and transparent access to collaborative context. This work pioneers the application of ontology engineering in human-AI collaboration, substantially enhancing the explicitness of task context, traceability of AI outputs, and overall collaboration transparency. The framework’s efficacy is empirically validated in a software development education setting.