🤖 AI Summary
This work addresses the growing mismatch between the rapid expansion of genomic data and the limited decompression throughput of conventional formats like gzip, which also lack efficient GPU-based random access. The authors propose the first fully GPU-resident LZ77 decompression pipeline that integrates a compact coordinate index with a range-based decompression strategy, enabling bit-accurate decoding while overcoming GPU memory constraints. Built upon the ACEAPEX parallel LZ77 codec, the approach combines GPU-accelerated entropy decoding and match parsing. Experimental results demonstrate exceptional performance: FASTQ decompression achieves 260 GB/s; single random accesses take only 0.362 ms—six times faster than samtools; the index size is reduced by 6.3×; and range decompression of a 50 GB genome reaches 165.7 GB/s.
📝 Abstract
Genomic archives grow faster than decompression keeps up: the European Nucleotide Archive holds tens of petabytes of fastq.gz, and gzip is fundamentally sequential. GPU decompressors (nvCOMP DEFLATE at ~50GB/s on A100) decode whole files with no random access; CPU genomic tools (CRAM, samtools) support region seeks but only at CPU speed. We extend ACEAPEX, an absolute-offset parallel LZ77 codec included in the official lzbench 2.3 release, with three contributions absent from our prior work. First, a full device-resident GPU decode pipeline (entropy and match resolution both on-device) reaching up to 260GB/s on FASTQ, closing the match-phase-only gap of the earlier paper. Second, position-invariant random access with a compact coordinate index: an arbitrary read decodes in 0.362ms, ~6x faster than warm samtools faidx, with a read-to-block index 6.3x smaller than a .fai. Third, a range-decode strategy that decouples output size from VRAM, sustaining 165.7GB/s on a 50GB genome where whole-file decode runs out of memory. All results are bit-perfect. We also measure Meta's open DietGPU ANS on H100 at 592GB/s decode, faster than the proprietary entropy stage we currently use, showing a fully open high-throughput stack is viable. Code is MIT-licensed.