Beyond Transformers: Linear Attention Policy for Open-Vocabulary Object Goal Navigation

2026-07-21

AI Summary

This work addresses the performance bottleneck in open-vocabulary object navigation caused by the difficulty of Transformers in modeling long-horizon contextual dependencies through self-attention. To overcome this limitation, the authors propose Linear Attention Navigation (LANav), a novel framework that replaces conventional self-attention with a structured state-update mechanism and introduces Weighted State-expansion Linear Attention (WSLA). WSLA enhances temporal state representations by expanding states into multiple sub-states and applying learnable weighted readouts. Integrated with an open-vocabulary perception module, LANav is trained on HM3D-OVON and HSSD environments and successfully transferred to real-world deployment. Experiments show that LANav achieves an average success rate of 36.4% on HM3D-OVON, outperforming the Transformer baseline by 6.3 percentage points—particularly excelling in long-range tasks—and attains an 82% success rate on physical robots, demonstrating effective sim-to-real transfer.

Abstract

Open-Vocabulary Object Goal Navigation (OVON) requires agents to operate under partial observability, making effective internal state updates critical for navigation performance. This update is implemented by the policy network, where recent approaches adopt Transformer-based backbones with self-attention over a context window to integrate temporal information. However, our controlled experiments show that performance does not scale with context length under Transformer-based policies, questioning the suitability of self-attention for state integration in navigation. To this end, we propose Linear Attention-based Navigation (LANav), which adopts linear attention (LA) as the policy backbone to maintain a structured state update rather than self-attention over the context window. Across multiple LA variants evaluated under identical settings, LANav consistently outperforms Transformer-based baselines. Performance improves as state update mechanisms become more structured and regulated, highlighting the importance of state update design. To improve state update effectiveness, we introduce Weighted State-Expansion Linear Attention (WSLA), which expands each attention head's state into multiple sub-states and uses learnable weighted readout to aggregate expanded sub-states. Equipped with WSLA, LANav achieves 36.4% average success rate (SR) on HM3D-OVON, outperforming Transformer-based counterparts by 6.3 percentage points in macro-averaged SR, while maintaining computational efficiency. Distance-stratified results show larger gains in long-distance episodes, while HSSD transfer and fine-tuning demonstrate robustness across scene distributions. Real-world deployment on a Unitree Go2 further achieves an 82% success rate over 50 trials, supporting the practical feasibility and sim-to-real transfer of LANav.

Problem

• Open-Vocabulary Object Goal Navigation
• state update
• Transformer
• self-attention
• partial observability

Innovation

• Linear Attention
• State Update Mechanism
• Open-Vocabulary Navigation
• Weighted State-Expansion
• Sim-to-Real Transfer