🤖 AI Summary
This work addresses the limited generality of conventional hypernetworks, which are typically constrained to specific architectures and tasks. The authors propose a Universal HyperNetwork (UHN) that employs a fixed-architecture generator to uniformly predict the weights of arbitrary target models based on deterministic encodings of parameters, architectural specifications, and task descriptors. UHN is the first framework to enable a single, fixed hypernetwork to generate models across heterogeneous architectures and diverse tasks, while demonstrating stable three-level recursive generation. Experimental results show that UHN achieves performance comparable to directly trained models across a range of domains—including vision, graph neural networks, text processing, and symbolic regression—significantly enhancing generalization across multiple models and multitask learning capabilities.
📝 Abstract
Conventional hypernetworks are typically engineered around a specific base-model parameterization, so changing the target architecture often entails redesigning the hypernetwork and retraining it from scratch. We introduce the \emph{Universal Hypernetwork} (UHN), a fixed-architecture generator that predicts weights from deterministic parameter, architecture, and task descriptors. This descriptor-based formulation decouples the generator architecture from target-network parameterization, so one generator can instantiate heterogeneous models across the tested architecture and task families. Our empirical claims are threefold: (1) one fixed UHN remains competitive with direct training across vision, graph, text, and formula-regression benchmarks; (2) the same UHN supports both multi-model generalization within a family and multi-task learning across heterogeneous models; and (3) UHN enables stable recursive generation with up to three intermediate generated UHNs before the final base model. Our code is available at https://github.com/Xuanfeng-Zhou/UHN.