Bayesian local clustering of functional data via semi-Markovian random partitions

2025-03-11

AI Summary

To address the challenges of partial functional overlap and long-range dependencies—poorly captured by conventional first-order Markov assumptions—in local clustering of functional data, this paper proposes a Bayesian nonparametric framework integrating B-spline expansions with semi-Markov dependent random partitions. Its key contributions are: (1) the first semi-Markov dependent random partition model, which relaxes the first-order Markov constraint and explicitly encodes variable-order dependencies among intervals; and (2) the first alignment of B-splines’ local support with the order of dependency structure, enabling interpretable and adaptive prior construction. Posterior inference is performed via MCMC. Experiments on simulated data and real-world tide-level measurements from the Venice Lagoon demonstrate substantial improvements in accuracy of local structure identification and cross-scenario generalizability over state-of-the-art functional clustering methods.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Local Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

We introduce a Bayesian framework for indirect local clustering of functional data, leveraging B-spline basis expansions and a novel dependent random partition model. By exploiting the local support properties of B-splines, our approach allows partially coincident functional behaviors, achieved when shared basis coefficients span sufficiently contiguous regions. This is accomplished through a cutting-edge dependent random partition model that enforces semi-Markovian dependence across a sequence of partitions. By matching the order of the B-spline basis with the semi-Markovian dependence structure, the proposed model serves as a highly flexible prior, enabling efficient modeling of localized features in functional data. Furthermore, we extend the utility of the dependent random partition model beyond functional data, demonstrating its applicability to a broad class of problems where sequences of dependent partitions are central, and standard Markovian assumptions prove overly restrictive. Empirical illustrations, including analyses of simulated data and tide level measurements from the Venice Lagoon, showcase the effectiveness and versatility of the proposed methodology.

Problem

• Bayesian local clustering of functional data
• Modeling partially coincident functional behaviors
• Extending dependent random partitions beyond functional data

Innovation

• Bayesian framework with B-spline basis expansions
• Semi-Markovian dependent random partition model
• Localized feature modeling via matched basis order