Combining Type Checking and Formal Verification for Lightweight OS Correctness

2024-12-31

AI Summary

To address the challenges of ensuring correctness and managing high verification overhead in lightweight operating systems (e.g., Theseus, written in Rust), this paper proposes a hybrid correctness assurance methodology integrating Rust’s type checking with Coq-based formal verification. Innovatively adopting an *intralingual* design paradigm, it leverages Rust’s type system as a trusted extension of Coq’s logical foundation, enforcing critical invariants at compile time. By deliberately relaxing certain correctness guarantees—while preserving essential safety properties—the approach achieves a novel trade-off between verification efficiency and engineering practicality. The method has been successfully applied to verify Theseus’s memory subsystem and its 10 Gb Ethernet driver. It guarantees zero runtime violations of key safety invariants while reducing overall verification effort by over 60% and scaling verification scope by a factor of three.

Technology Category

Constraint Satisfaction and Optimization: Satisfiability Modulo TheoriesMachine Learning: Hardware-aware MLNatural Language Processing: Safety and Robustness

Application Category

Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructuresSecurity and Privacy: Large-scale security measurementsWeb Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

This paper reports our experience of providing lightweight correctness guarantees to an open-source Rust OS, Theseus. First, we report new developments in intralingual design that leverage Rust's type system to enforce additional invariants at compile time, trusting the Rust compiler. Second, we develop a hybrid approach that combines formal verification, type checking, and informal reasoning, showing how the type system can assist in increasing the scope of formally verified invariants. By slightly lessening the strength of correctness guarantees, this hybrid approach substantially reduces the proof effort. We share our experience in applying this approach to the memory subsystem and the 10 Gb Ethernet driver of Theseus, demonstrate its utility, and quantify its reduced proof effort.

Problem

• Lightweight Operating Systems
• Formal Verification
• Memory Management and Ethernet Driver Validation

Innovation

• Hybrid Method
• Formal Verification
• Memory Management