HiRAE: Hierarchical Representation Autoencoding with Residual Budgets
This study addresses the loss of fine-grained details in pretrained visual representations during image generation and the difficulty of latent space modeling caused by multi-layer fusion. To tackle these challenges, we propose a Hierarchical Representation Autoencoder that introduces a residual budget mechanism to learn adaptive fusion across encoder layers. By constraining shallow-layer residual corrections via an upper bound on group norms, the method achieves efficient full-level fusion without complex hyperparameter tuning. Furthermore, integrating deep residual networks with hierarchical feature aggregation significantly enhances reconstruction fidelity while maintaining compatibility with mainstream generative frameworks. Experimental results demonstrate that our approach reduces the FID to 0.209 on ImageNet-256 and achieves a GenEval score of 87.70, substantially outperforming existing baseline models.