Score
Integrating hardware, middleware, control policies, perception, and higher-level reasoning (e.g., MLLM pipelines) to deploy, test, and validate robotic systems on real platforms. Covers end-to-end evaluation protocols to demonstrate task success, synchronization, safety, and autonomous operation across simulated and real-world scenarios.
This study addresses the lack of systematic evaluation of robotic foundation models (RFMs) for industrial applications, where critical requirements such as safety, real-time performance, heterogeneous perception, and edge deployment remain inadequately assessed. To bridge this gap, the work proposes a comprehensive evaluation framework that structures industrial deployment needs into 11 influencing factors and 149 specific criteria, enabling large-scale benchmarking of 324 action-capable RFMs. The authors introduce an innovative assessment pipeline that integrates conservative, large language model–assisted scoring with expert validation to achieve multidimensional, fine-grained capability quantification. Findings reveal that current RFMs exhibit limited and uneven industrial maturity: even the best-performing models only partially satisfy requirements and generally lack cross-dimensional coordination. The results underscore an urgent need for systematic integration of safety, real-time responsiveness, and cost-effectiveness in future RFM development.
This work addresses the challenge that traditional model-based testing is ill-suited for distributed robotic systems due to their high nondeterminism, dynamic reconfiguration, and inherent complexity. To overcome this limitation, the paper proposes the Scenario Specification Language (SCSL), which enables the construction of system-level tests by composing basic scenarios. The approach integrates runtime online test generation and execution with mechanisms for dynamic component joining/leaving and interface reconnection, thereby supporting automated testing and dynamic reconfiguration. The syntax and semantics of SCSL are validated through a robotic salvage mission case study, where automatically generated tests effectively demonstrate the feasibility and advantages of the proposed method.
This study addresses the persistent gap between theoretical control performance and its practical realization in real-world robotic systems, often caused by inadequate discretization, insufficient real-time guarantees, and weak error handling in control software. For the first time from a software engineering perspective, the authors systematically analyze 184 open-source robotic controllers through code review, empirical analysis, and test evaluation, uncovering common deficiencies in application scenarios, implementation details, and verification practices. The findings reveal that most implementations fail to properly account for critical system constraints, and their testing strategies inadequately validate the theoretical assurances they claim. This work highlights a significant disconnect between implementation quality and theoretical promises, offering concrete directions and practical guidelines for developing reliable, verifiable robotic control software.
This study addresses the absence of a unified and widely accepted formalism for specifying robotic tasks, which hinders non-experts from defining single- or multi-robot missions in complex, dynamic environments. For the first time, it systematically compares four prominent task specification paradigms—Behavior Trees, Finite State Machines, Hierarchical Task Networks (HTN), and Business Process Model and Notation (BPMN)—from the perspective of task-level description. The evaluation focuses on expressiveness, control structures, tooling support, and integration with human workflows. Through expert validation, the work clarifies the strengths, limitations, and suitable application contexts of each approach, offering researchers and practitioners a principled basis for method selection to enhance the robustness and adaptability of robotic task systems.
This work addresses the limitations of existing vision-language models in robotic control—namely, poor interpretability, weak generalization, and reliance on cloud-based computation—by proposing a fully onboard multi-agent architecture. The system deploys lightweight vision-language models (3–20B parameters) alongside vision-language-action models on an AMD Ryzen AI mini PC, enabling autonomous mobile manipulation without external support through fine-tuning and hardware-in-the-loop simulation. A novel “Megamind” coordinating agent is introduced to mitigate the challenge of context retention in long-horizon tasks faced by smaller models. The architecture’s feasibility in terms of cost, performance, and real-world transferability is validated across five industrial warehouse tasks, and the associated simulation environment is open-sourced.
To address the challenge of verifying robustness of autonomous robots in unknown environments during design, this paper proposes a formal verification method integrating ROS 2 and behavior trees. The approach models the system using an extended SCXML language that uniformly represents both ROS 2 nodes and behavior tree components. An automated toolchain, AS2FM, is developed to translate such models into the JANI standard format. Verification leverages statistical model checking (SMC) for efficient, scalable property validation. Evaluated on consumer-grade hardware, the method achieves sub-second verification times, with runtime scaling linearly—rather than exponentially—with model size, outperforming existing approaches. Crucially, it successfully identified previously unknown defects in the ROS 2 middleware during case studies, thereby significantly strengthening robustness assurance at the design stage.
This work addresses the validation gap between simulation and real-world deployment of autonomous driving algorithms, particularly the lack of efficient, high-fidelity testing platforms for safety-critical scenarios. To bridge this gap, the authors propose a mixed-reality hardware-in-the-loop testing framework that seamlessly integrates physical mobile robots with high-fidelity virtual environments, enabling multimodal sensing, vehicle-to-everything (V2X) communication, and large-scale multi-agent collaboration. A key innovation is the coexistence of physical and virtual agents within a unified architecture, coupled with an online learning controller based on control barrier functions (CBFs) that establishes an integrated perception-planning-control safety assurance mechanism. Experimental results demonstrate that the platform significantly enhances the reliability and efficiency of sim-to-real transfer and validates its effectiveness across diverse safety-critical scenarios.
This work addresses the challenge of ambiguous natural language instructions leading to deviations from user intent in agricultural robot task planning. The authors propose a novel architecture that integrates large language models (LLMs) with formal verification: two off-the-shelf LLMs are employed—one to generate task specifications and another to verify their correctness—while linear temporal logic (LTL) underpins a multi-layer feedback loop to ensure alignment with the user’s true requirements. This approach uniquely combines LTL-based verification with a dual-LLM collaborative mechanism, effectively mitigating both linguistic ambiguity and model bias. Experimental results demonstrate that the system substantially enhances the accuracy and reliability of task planning while overcoming the limitations of directly using LLMs to generate LTL formulas.
This work addresses the challenges of scaling robotic education in higher education, where high costs of commercial digital twins and the steep learning curve of open-source middleware like ROS hinder widespread adoption. To overcome these barriers, the authors propose an education-oriented, four-layer extensible communication architecture that abstracts complex communication protocols and enables seamless integration between graphical modeling environments and physical robots. Notably, this architecture is the first in an open-source platform to support hardware-agnostic, high-fidelity virtual–physical mapping. By integrating 3D visual modeling, a ROS backend, and efficient data serialization and routing mechanisms, the system significantly reduces deployment complexity. Experimental validation through multi-axis spatial trajectory tasks demonstrates its effectiveness in facilitating practical robotics instruction in engineering education.
This work addresses the critical challenge of hallucination-induced safety violations in large language models (LLMs) deployed within cyber-physical systems, which can compromise system reliability. To mitigate this risk, the authors propose SafePilot, a novel end-to-end assurance framework that uniquely supports both attribute-based and temporal safety constraints. SafePilot employs a hierarchical neuro-symbolic architecture that dynamically guides LLMs toward generating verifiably safe plans through a pipeline comprising task complexity assessment, automatic translation of natural language instructions into formal specifications, hierarchical planning, and iterative refinement. Empirical evaluation across two representative scenarios demonstrates the framework’s effectiveness and adaptability, significantly enhancing the safety and regulatory compliance of LLM-driven cyber-physical systems.
This study addresses the limitations of current robotic system validation, which relies heavily on manual selection of test scenarios, thereby hindering scalability and compromising reproducibility and reliability of conclusions. To overcome these challenges, this work proposes a compositional, scenario-based modeling approach that integrates declarative test specifications, plugin-driven scenario generation, containerized parallel simulation, and unified result analysis to establish the first modular and scalable automated verification framework. The framework enables systematic parameter variation across multiple dimensions and facilitates robust identification of systemic faults versus stochastic anomalies. Evaluated across 5,480 distinct scenario configurations with over 100,000 simulation runs, the approach accumulated 1,800 hours of simulated operation and 1,873 virtual kilometers, demonstrating its efficacy in discerning consistent system deficiencies from random irregularities.