Type-safe and portable support for packed data

2025-04-28

AI Summary

Mainstream programming languages lack native support for packed data, necessitating frequent serialization/deserialization for cross-component data exchange—degrading performance and memory locality. This paper presents the first compiler-unmodified, library-only solution for zero-cost, type-safe packed data in Haskell, built atop type-level programming, GHC generics, and Template Haskell. It enables user-defined types to be represented in memory with layouts identical to their wire formats, eliminating serialization overhead and ensuring cross-language interoperability. By unifying in-memory and network representations, the approach preserves spatial locality and enables direct traversal of compact binary structures. Experimental evaluation shows up to 60% faster traversal of packed data compared to conventional unpacked representations. The work demonstrates that high-performance packed data support can be achieved in existing languages solely through advanced type systems and metaprogramming—without runtime or compiler modifications—thus reconciling strong safety guarantees with practical engineering feasibility and performance benefits.

Technology Category

• Machine Learning: Hardware-aware ML
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Search and Optimization: Distributed Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

When components of a system exchange data, they need to serialise the data so that it can be sent over the network. Then, the recipient has to deserialise the data in order to be able to process it. These steps take time and have an impact on the overall system's performance. A solution to this is to use packed data, which has a unified representation between the memory and the network, removing the need for any marshalling steps. Additionally, using this data representation can improve the program's performance thanks to the data locality enabled by the compact representation of the data in memory. Unfortunately, no mainstream programming languages support packed data, whether it's out-of-the-box or through a compiler extension. We present packed-data, a Haskell library that allows for type safe building and reading of packed data in a functional style. The library does not rely on compiler modifications, making it portable, and leverages meta-programming to allow programmers to pack their own data types effortlessly. We evaluate the usability and performance of the library, and conclude that it allows traversing packed data up to 60% faster than unpacked data. We also reflect on how to enhance the performance of library-based support for packed data. Our implementation approach is general and can easily be used with any programming languages that support higher-kinded types.

Problem

• Eliminates serialization for faster data exchange
• Provides type-safe packed data in functional style
• Improves performance via compact memory representation

Innovation

• Haskell library for type-safe packed data
• Portable solution without compiler modifications
• Meta-programming for effortless data type packing