Can LLMs help find Ambiguities in Protocol Specifications?

2026-09-29

AI Summary

This study addresses the semantic under-specification inherent in natural language descriptions of protocol specifications such as 5G, which frequently leads to interoperability failures. To tackle this challenge, we propose SpecLens, a framework that leverages protocol state machines as scaffolding to guide large language models (LLMs) in identifying specification ambiguities and gaps. The approach innovatively integrates ASN.1 formal syntax with natural language semantics to enhance the detection of undefined behaviors, and employs differential testing to comparatively validate implementations including OpenAirInterface and srsRAN. Expert evaluation confirms the discovery of 185 defects, among which 60 result in actual implementation divergences. These findings demonstrate that SpecLens effectively strengthens the robustness of protocol specifications by systematically exposing latent inconsistencies before deployment.

Abstract

Internet protocol specifications written in RFCs are subject to ambiguities and multiple interpretations that can cause interoperability failure. While these have presumably cleared up after years of experience, such ambiguities can bedevil the adoption of newer protocols like 5G. The 5G specifications pair a formal message syntax (ASN.1) with message-handling procedures written in natural language. This creates semantic underspecification: a syntactically valid message can reach a state whose procedures never say how to handle it, so standard-compliant implementations diverge. We frame this as a gap or a fork in a partially specified communicating state machine, and present SpecLens, which puts that view in front of a language model as a scaffold. Stronger models do not remove the need for it: they broaden the search without disciplining it, and fewer than half their findings survive inspection. Across 36 procedures from six 3GPP and O-RAN protocols, experts accept 185 of 197 SpecLens findings, and 60 drive observable divergence between the OpenAirInterface and srsRAN implementations under differential test. While we use 5G as a canonical example of a newer protocol, we also show ambiguity results for the more mature DNS protocol.

Problem

• Protocol Specifications
• Ambiguity Detection
• Semantic Underspecification
• 5G
• Interoperability

Innovation

• Protocol Ambiguity Detection
• Large Language Models
• Communicating State Machine
• Semantic Underspecification
• Differential Testing