Retrospective Open-Vocabulary Memory for Long-Term Object Search

2026-09-29

AI Summary

This study addresses the memory bias problem in long-term object search, where environmental dynamics and uneven observation opportunities distort robotic memory. To this end, we propose the ECROM framework, which introduces a per-opportunity evidence reasoning principle that models unobserved periods as censored data. By weighting inferences according to observation opportunities, the framework estimates long-term object existence probabilities and converts them into active search priors. Additionally, a controlled long-term benchmark is constructed to independently decouple object placement from observation opportunity variables. This work integrates censored data analysis with open-vocabulary memory modeling, achieving improvements of 4.5 points in episode success rate AP and 4.2 points in search SPL on the HM3D dataset. The code and data have been made publicly available.

Abstract

Long-term object search requires learning where objects usually appear from repeated but uneven observations of a changing environment. We formulate retrospective open-vocabulary memory as probabilistic inference from censored observations, where the key idea is to reason with evidence per opportunity: a detection or non-detection should influence belief only in proportion to the robot's opportunity to observe the corresponding location. We introduce ECROM, which uses this principle to estimate long-term prevalence for concepts specified only at query time and converts the resulting belief directly into an active-search prior. To evaluate this problem, we introduce a controlled long-term benchmark in ten HM3D homes that independently varies object placement and observation opportunity across repeated traversals. ECROM improves support-level AP on held-out queries by 4.5 points and search SPL by 4.2 points over the strongest competing memory in each metric. The benchmark, dataset, and code will be open-sourced.

Problem

• Long-term object search
• Open-vocabulary memory
• Censored observations
• Probabilistic inference
• Active search

Innovation

• Open-Vocabulary Memory
• Probabilistic Inference
• Censored Observations
• Long-Term Object Search
• Active Search