HiKO: A Hierarchical Framework for Beyond-Second-Order KO Codes

2025-06-11

AI Summary

Traditional Kronecker-product (KP) codes suffer from performance saturation at high code rates (r ≥ 2) and have long lagged behind Reed–Muller (RM) codes. This work introduces HiKO, a hierarchical training framework enabling the first scalable construction of KP codes to third- and fourth-order. Our method addresses key challenges via: (1) a Plotkin-structured neural architecture incorporating dropout and learnable skip connections; (2) hierarchical knowledge transfer coupled with progressive parameter unfreezing; and (3) customized regularization and enhanced CNN/MLP-based encoders. Experiments demonstrate that HiKO consistently outperforms RM codes at r = 3 and r = 4, approaching the Gaussian channel’s Shannon limit while preserving low-complexity decoding—thereby breaking the longstanding theoretical and practical bottleneck restricting KP codes to second-order constructions.

Technology Category

Machine Learning: Kernel MethodsCognitive Modeling & Cognitive Systems: Neural Spike CodingSearch and Optimization: Learning to Search

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper introduces HiKO (Hierarchical Kronecker Operation), a novel framework for training high-rate neural error-correcting codes that enables KO codes to outperform Reed-Muller codes beyond second order. To our knowledge, this is the first attempt to extend KO codes beyond second order. While conventional KO codes show promising results for low-rate regimes ($r<2$), they degrade at higher rates -- a critical limitation for practical deployment. Our framework incorporates three key innovations: (1) a hierarchical training methodology that decomposes complex high-rate codes into simpler constituent codes for efficient knowledge transfer, (2) enhanced neural architectures with dropout regularization and learnable skip connections tailored for the Plotkin structure, and (3) a progressive unfreezing strategy that systematically transitions from pre-trained components to fully optimized integrated codes. Our experiments show that HiKO codes consistently outperform traditional Reed-Muller codes across various configurations, achieving notable performance improvements for third-order ($r = 3$) and fourth-order ($r = 4$) codes. Analysis reveals that HiKO codes successfully approximate Shannon-optimal Gaussian codebooks while preserving efficient decoding properties. This represents the first successful extension of KO codes beyond second order, opening new possibilities for neural code deployment in high-throughput communication systems.

Problem

• Extends KO codes beyond second order for high-rate regimes
• Overcomes degradation of KO codes at higher rates
• Enables neural codes to outperform Reed-Muller codes

Innovation

• Hierarchical training for efficient knowledge transfer
• Enhanced neural architectures with dropout regularization
• Progressive unfreezing for optimized integrated codes