Comparison Theorems for the Mixing Times of Systematic and Random Scan Dynamics

2024-10-14

arXiv.org

AI Summary

This work investigates the impact of systematic (fixed-order) versus random (uniformly random-order) scans on the mixing time of Gibbs sampling. For high-dimensional distributions, we establish tight comparative theorems relating their spectral gaps and mixing times. First, we derive the optimal $O(n)$ upper bound on the spectral gap of one systematic-scan iteration relative to that of random scan—proving both its validity and tightness. Second, we establish a reverse polynomial relationship: rapid mixing under *any* scan order implies rapid mixing of the Glauber dynamics. Our analysis relies solely on elementary linear algebra and probabilistic arguments, avoiding advanced coupling techniques or functional inequalities. These results tighten previously known spectral gap ratio bounds to the optimal asymptotic order and fully resolve the open problem—posed by Chlebicka et al.—concerning bidirectional mixing-time equivalences between scan-based samplers and Glauber dynamics.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

A popular method for sampling from high-dimensional distributions is the Gibbs sampler, which iteratively resamples sites from the conditional distribution of the desired measure given the values of the other coordinates. But to what extent does the order of site updates matter in the mixing time? Two natural choices are (i) standard, or random scan, Glauber dynamics where the updated variable is chosen uniformly at random, and (ii) the systematic scan dynamics where variables are updated in a fixed, cyclic order. We first show that for systems of dimension $n$, one round of the systematic scan dynamics has spectral gap at most a factor of order $n$ worse than the corresponding spectral gap of a single step of Glauber dynamics, tightening existing bounds in the literature by He, et al. [NeurIPS '16] and Chlebicka, {L}atuszy'nski, and Miasodejow [Ann. Appl. Probab. '24]. This result is sharp even for simple spin systems by an example of Roberts and Rosenthal [Int. J. Statist. Prob. '15]. We complement this with a converse statement: if all, or even just one scan order rapidly mixes, the Glauber dynamics has a polynomially related mixing time, resolving a question of Chlebicka, {L}atuszy'nski, and Miasodejow. Our arguments are simple and only use elementary linear algebra and probability.

Problem

• Compare mixing times of systematic vs random scan Gibbs sampling
• Bound spectral gap difference between scan orders in high dimensions
• Relate rapid mixing of one scan order to Glauber dynamics

Innovation

• Systematic scan dynamics spectral gap analysis
• Comparison of mixing times between scan methods
• Polynomial relation in mixing time conditions