On the Hardness Hierarchy for the $O(n sqrt{log n})$ Complexity in the Word RAM

2025-03-26

AI Summary

This paper investigates the intrinsic computational hardness of a class of classic problems—including counting inversions, constructing the Burrows–Wheeler Transform (BWT), computing LZ77 factorizations, finding the longest common substring, orthogonal range counting, and segment intersection—each solvable in $O(nsqrt{log n})$ time on the Word RAM model. Method: We construct a tight reduction network, introduce the new problem String Nesting, prove its equivalence to binary dictionary matching, and establish a three-step reduction from it to inversion counting. Contribution/Results: We identify restricted binary dictionary matching as the minimal hardness benchmark for this complexity class—the first such result. Our framework unifies over ten core problems across six domains, showing they are all computationally equivalent under fine-grained reductions, with their common conditional lower bound rooted in binary dictionary matching. Moreover, we demonstrate that numerous string problems admit efficient reductions to binary input, sharply constraining the space for future algorithmic improvements.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Data Mining & Knowledge Management: Data Compression
• Search and Optimization: Combinatorial Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In this work, we study the relative hardness of fundamental problems with state-of-the-art word RAM algorithms that take $O(nsqrt{log n})$ time for instances described in $Theta(n)$ machine words ($Theta(nlog n)$ bits). This complexity class, one of six hardness levels identified by Chan and Pu{a}trac{s}cu [SODA 2010], includes diverse problems from several domains: Counting Inversions, string processing problems (BWT Construction, LZ77 Factorization, Longest Common Substring, Batched Longest Previous Factor Queries, Batched Inverse Suffix Array Queries), and computational geometry tasks (Orthogonal Range Counting, Orthogonal Segment Intersection). We offer two main contributions: We establish new links between the above string problems and Dictionary Matching, a classic task solvable using the Aho-Corasick automaton. We restrict Dictionary Matching to instances with $O(n)$ binary patterns of length $m = O(log n)$ each, and we prove that, unless these instances can be solved in $o(nsqrt{log n})$ time, the aforementioned string problems cannot be solved faster either. Via further reductions, we extend this hardness to Counting Inversions (a fundamental component in geometric algorithms) and thus to Orthogonal Range Counting and Orthogonal Segment Intersection. This hinges on String Nesting, a new problem which is equivalent to Dictionary Matching and can be reduced to Counting Inversions in three steps. Together, our results unveil a single problem, with two equivalent formulations, that underlies the hardness of nearly all major problems currently occupying the $O(nsqrt{log n})$ level of hardness. These results drastically funnel further efforts to improve the complexity of near-linear problems. As an auxiliary outcome of our framework, we also prove that the alphabet in several central string problems can be efficiently reduced to binary.

Problem

• Study relative hardness of problems with O(n√log n) complexity
• Link string problems to Dictionary Matching for hardness proofs
• Extend hardness to Counting Inversions and geometric tasks

Innovation

• Links string problems to Dictionary Matching
• Reduces hardness via String Nesting equivalence
• Efficient binary alphabet reduction for string problems