A Flexible Plug-and-Play Module for Generating Variable-Length

2024-12-12

arXiv.org

AI Summary

Traditional deep supervised hashing models are constrained by fixed code lengths, necessitating separate model training for each length—leading to a trade-off between efficiency and effectiveness—and neglecting structural correlations among codes of varying lengths. To address this, we propose Nested Hashing Layers (NHL), the first framework enabling a single model to simultaneously generate semantically consistent nested hash codes of multiple lengths. NHL introduces a nested variable-length generation mechanism to preserve semantic coherence across lengths, an adaptive gradient weighting strategy to mitigate conflicts in multi-objective optimization, and a long-to-short cascaded self-distillation framework wherein longer codes structurally guide shorter-code learning. Extensive experiments on multiple benchmark datasets demonstrate that NHL significantly reduces training overhead while improving retrieval accuracy—achieving average mAP gains of 3.2%–5.8%. The source code is publicly available.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Deep supervised hashing has become a pivotal technique in large-scale image retrieval, offering significant benefits in terms of storage and search efficiency. However, existing deep supervised hashing models predominantly focus on generating fixed-length hash codes. This approach fails to address the inherent trade-off between efficiency and effectiveness when using hash codes of varying lengths. To determine the optimal hash code length for a specific task, multiple models must be trained for different lengths, leading to increased training time and computational overhead. Furthermore, the current paradigm overlooks the potential relationships between hash codes of different lengths, limiting the overall effectiveness of the models. To address these challenges, we propose the Nested Hash Layer (NHL), a plug-and-play module designed for existing deep supervised hashing models. The NHL framework introduces a novel mechanism to simultaneously generate hash codes of varying lengths in a nested manner. To tackle the optimization conflicts arising from the multiple learning objectives associated with different code lengths, we further propose an adaptive weights strategy that dynamically monitors and adjusts gradients during training. Additionally, recognizing that the structural information in longer hash codes can provide valuable guidance for shorter hash codes, we develop a long-short cascade self-distillation method within the NHL to enhance the overall quality of the generated hash codes. Extensive experiments demonstrate that NHL not only accelerates the training process but also achieves superior retrieval performance across various deep hashing models. Our code is publicly available at https://github.com/hly1998/NHL.

Problem

• Traditional hashing models require multiple training for different code lengths
• Existing methods ignore relationships between hash codes of varying lengths
• Optimization conflicts arise when learning multiple hash lengths simultaneously

Innovation

• Nested Hash Layer enables multiple-length hash codes
• Dominance-aware weighting resolves optimization conflicts
• Long-short cascade self-distillation improves code quality