Self-Reconstruction Dynamics for Autoencoder Reconstruction Refinement

2026-09-26

AI Summary

This study addresses the suboptimal latent representations inherent in single-pass autoencoder inference, which often lead to reconstruction errors, while noting that conventional test-time optimization incurs prohibitive computational overhead. To overcome this limitation, this work proposes SRD-RR, a method that leverages the self-reconstruction dynamics trajectory of a frozen autoencoder. Specifically, a latent correction network is employed to predict corrective offsets, thereby extracting sample-specific signals to enhance reconstruction quality without requiring test-time optimization. Experimental results demonstrate that the proposed approach recovers approximately 45% of the optimizable error across multiple datasets, yielding an average PSNR improvement of 1.74 dB and significantly outperforming linear correction baselines.

Abstract

Standard autoencoder (AE) inference uses a single encoder-decoder pass, though the latent may not be optimal for each sample under a fixed decoder. We ask whether a trained AE can reveal information for improving its own reconstruction. Repeated application of a frozen AE to its reconstruction produces transient image- and latent-space trajectories, termed Self-Reconstruction Dynamics (SRD). Although this degrades fidelity in the AEs studied here, SRD contains sample-specific information for correcting the reconstruction. We propose SRD-guided Reconstruction Refinement (SRD-RR), which predicts a latent correction from a short SRD with the AE frozen and no per-sample test-time optimization. We also introduce MSE-recov, an MSE recovery ratio relative to an empirical decoder-optimized reference. Across six datasets, SRD-RR recovers 38.6% of the empirically recoverable MSE gap with one transition and 45.3% with two. A two-transition variant trained without direct access to original images, using an SRD-derived pseudo-target, achieves 40.7% recovery and a 1.74 dB average PSNR gain. Removing trajectory information reduces the gain, while cross-sample trajectory assignment causes severe degradation, confirming strong sample specificity. Nonlinear SRD-conditioned refinement consistently outperforms fixed and trained linear latent correction. On a pretrained DINOv2-based representation autoencoder (RAE) with substantially different latent dynamics, SRD conditioning again improves a matched trajectory-free predictor. However, pixel-MSE latent refinement reveals a strong mismatch between pixel fidelity and perceptual quality, while the SRD-derived pseudo-target mitigates this degradation. Overall, SRD is a useful sample-specific refinement signal, while the objective determines how it translates into pixel and perceptual quality.

Problem

• Autoencoder
• Reconstruction Refinement
• Self-Reconstruction Dynamics
• Latent Correction
• Perceptual Quality

Innovation

• Self-Reconstruction Dynamics
• Autoencoder Refinement
• Latent Correction
• Test-Time Optimization Free
• Pseudo-Target