Deploying Foundation Models for Embodied Navigation

2026-09-22

AI Summary

研究解决了基础模型在实体导航中的训练偏差和上下文长度限制问题，通过使用场景中的人类习惯数据预处理及创新的记忆头增强方法来改善。

Abstract

We present and tackle two problems associated with deploying Foundation Models (FMs) on Embodied Agents performing navigation: 1) Training bias in FMs leading to poor personalization in unseen environments, and 2) Limited FM context length hindering success, especially on long horizon tasks. Our solution for the former involves priming the FM with human-habit data mined from the scene and our solution for the latter involves active memory management via a novel `memory head' augmentation. We first present a taxonomy of existing literature on FM-based Embodied Navigation, and highlight these limitations. We then present our approaches, Transit-Aware Planning (TAP) and MemCtrl to address the limitations. With TAP, we present real-world results in a lab environment with a Turtlebot for personalized target finding that shows an average improvement of 18% over a non-TAP baseline. On MemCtrl, we report a 6% average improvement across various embodied tasks, with 20% on long instruction subsets, all while using nearly half the context used in the baseline model. Motivated by these result, we present our stance the deployability of FM-based embodied agents in real-world environments, and highlight open research directions.

Problem

• Foundation Models
• Embodied Navigation
• Training Bias
• Context Length Limitation

Innovation

• Transit-Aware Planning
• MemCtrl
• memory head