Deviance-style normalization for jointly overdispersed counts

2026-06-24

AI Summary

This work addresses the challenge of normalizing sparse and overdispersed compositional count matrices arising from high-throughput sequencing data by proposing a unified residualization framework based on the Dirichlet–multinomial (DM) distribution and its generalizations, such as the Dirichlet-tree multinomial. The approach models within-sample overdispersion via a single concentration parameter, treats each sample as compositional data with a fixed total count, and naturally extends to features with ordinal or tree-structured relationships. The resulting family of residuals encompasses both joint and feature-wise null models, preserves the original sparsity, integrates seamlessly with existing sparse computational pipelines, and allows each residual to be computed in constant time. Theoretically and empirically, the transformation adaptively shrinks residuals according to the degree of overdispersion under repeated sampling, reduces to multinomial residuals for singleton observations, and converges exactly to the classical multinomial case as the concentration parameter tends to infinity.

Abstract

We introduce a Dirichlet--multinomial (DM) deviance residualization for sparse, jointly overdispersed count matrices, the regime that dominates sequencing-based biochemical assays. The DM null treats each sample's count vector as a fixed-total composition with a single scalar concentration $α_0$ governing overdispersion, and arises exactly by conditioning independent negative-binomial feature counts on the observed sample total -- making the DM the joint conditional analogue of standard feature-wise overdispersed count models. The resulting transform preserves exact sparsity, evaluates in constant time per nonzero entry, agrees with multinomial residuals on singleton counts, shrinks repeated-count residuals according to the overdispersion the null tolerates, and recovers the multinomial residual as $α_0\to\infty$. The same fixed-dispersion comparison principle extends to ordered and tree-structured features via the generalized DM and the Dirichlet-tree multinomial, giving a single residual family that subsumes joint and feature-wise count nulls under a common compositional logic and is computationally lightweight enough to drop into existing sparse pipelines.

Problem

• overdispersed counts
• count normalization
• Dirichlet-multinomial
• compositional data
• sequencing assays

Innovation

• Dirichlet-multinomial
• deviance residualization
• overdispersed counts
• compositional data
• sparsity-preserving transformation