Memory Reallocation with Polylogarithmic Overhead

2026-02-17

AI Summary

This work addresses the online dynamic memory allocation problem, where objects of arbitrary sizes are inserted and deleted while maintaining a non-overlapping contiguous layout, under the constraint that the total size of active objects never exceeds $1 - \varepsilon$ times the memory capacity. The authors propose a novel allocator that, for the first time, achieves an expected reallocation cost of $\mathrm{polylog}(\varepsilon^{-1})$ in the worst case—an exponential improvement over the previous best bound of $\widetilde{O}(\varepsilon^{-1/2})$. They further establish that sub-polynomial reallocation cost is impossible with high probability, thereby demonstrating the necessity of randomization. The key technical ingredient combines the Erdős–Sárközy sunflower lemma with combinatorial structure and probabilistic analysis, yielding a cost that nearly matches the theoretical lower bound of $\Omega(\log \varepsilon^{-1})$.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Distributed Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

The Memory Reallocation problem asks to dynamically maintain an assignment of given objects of various sizes to non-overlapping contiguous chunks of memory, while supporting updates (insertions/deletions) in an online fashion. The total size of live objects at any time is guaranteed to be at most a $1-ε$ fraction of the total memory. To handle an online update, the allocator may rearrange the objects in memory to make space, and the overhead for this update is defined as the total size of moved objects divided by the size of the object being inserted/deleted. Our main result is an allocator with worst-case expected overhead $\mathrm{polylog}(ε^{-1})$. This exponentially improves the previous worst-case expected overhead $\tilde O(ε^{-1/2})$ achieved by Farach-Colton, Kuszmaul, Sheffield, and Westover (2024), narrowing the gap towards the $Ω(\logε^{-1})$ lower bound. Our improvement is based on an application of the sunflower lemma previously used by Erdős and Sárközy (1992) in the context of subset sums. Our allocator achieves polylogarithmic overhead only in expectation, and sometimes performs expensive rebuilds. Our second technical result shows that this is necessary: it is impossible to achieve subpolynomial overhead with high probability.

Problem

• Memory Reallocation
• Online Allocation
• Polylogarithmic Overhead
• Dynamic Memory Management
• Worst-case Expected Overhead

Innovation

• memory reallocation
• polylogarithmic overhead
• sunflower lemma
• online algorithms
• expected amortized cost