VEHBench: A Stage-Local Diagnostic Benchmark for LLM-Assisted Vibration Energy Harvester Design

2026-07-20

AI Summary

Existing engineering benchmarks struggle to diagnose the behavior of large language models (LLMs) in the multi-stage, coupled design of vibration energy harvesters. To address this gap, this work proposes VEHBench—the first stage-wise diagnostic benchmark grounded in native engineering contexts—comprising 763 literature-derived tasks evaluated via an analytical physics oracle. VEHBench assesses LLM performance across four critical design roles: specification filtering, verification-guided search, error recovery, and strategy selection. Experimental results reveal that LLM capabilities exhibit strong stage dependency, with no single model dominating across the entire workflow. Distinct response control patterns emerge across models, underscoring the necessity of stage-aware evaluation for effective LLM selection, routing, and optimization in engineering applications.

Abstract

Battery-free Internet of Things (IoT) requires iterative design of vibration energy harvesters (VEHs) under coupled physical constraints, while LLMs are emerging as interface layers for engineering workflows. However, existing engineering benchmarks primarily assess final artifact validity, offering limited insights into how LLMs behave across different stages of coupled physical design. We introduce VEHBench, an engineering-native diagnostic benchmark for LLM-assisted VEH design, featuring 763 literature-grounded tasks scored by an analytical physical oracle. VEHBench evaluates four design roles: specification triage, verifier-guided search, corrupted-state recovery, and policy-conditioned selection. Experimental results reveal that LLM capability is strongly stage-dependent: no single model consistently dominates the entire workflow, and response-control profiles expose distinct behavioral patterns across design roles. VEHBench thus provides a stage-aware foundation for evaluating, selecting, routing, and improving verifier-grounded engineering LLMs. The benchmark artifact is available at https://huggingface.co/datasets/AnonymousVehbench/vehbench

Problem

• LLM-assisted design
• vibration energy harvester
• engineering benchmark
• stage-local evaluation
• coupled physical constraints

Innovation

• VEHBench
• stage-local benchmark
• LLM-assisted design
• vibration energy harvester
• diagnostic evaluation