Evaluation of Baseline Methods for IDD-based SSD External Memory Search

2026-06-01

AI Summary

This study addresses the lack of systematic evaluation of simple immediate duplicate detection (IDD) methods in external-memory search and the frequent oversight of operating system mechanisms—such as page caching—that significantly influence performance. For the first time, it systematically assesses the practical efficacy of basic IDD baselines within an SSD-based external-memory A* search framework and provides a thorough analysis of how OS-level page caching impacts search efficiency. The findings reveal both the effectiveness and limitations of simple IDD in real-world systems and demonstrate that operating system caching mechanisms play a critical role in algorithmic performance. These insights establish an empirical foundation and offer practical guidance for the design of future high-performance external-memory search algorithms.

Abstract

Many difficult search problems cannot be solved by algorithms such as A* using only RAM. Search algorithms which use external memory such as SSDs and HDDs with much higher capacity than RAM have been proposed in previous work, but previous work has focused on delayed duplicate detection approaches, as well as complex immediate duplicate detection (IDD) methods, and relatively simple methods for IDD have not been systematically studied. In addition, the effect of OS-level mechanisms for managing and speeding up accesses to external memory, such as page caches, has not been studied. This paper addresses these gaps in the literature by evaluating and analyzing the performance of simple baseline approaches for IDD-based A*.

Problem

• external memory search
• immediate duplicate detection
• A*
• page cache
• baseline evaluation

Innovation

• external memory search
• immediate duplicate detection
• A*
• page cache
• SSD