A capture-recapture hidden Markov model framework for register-based inference of population size and dynamics

2026-03-25

AI Summary

Traditional censuses are costly and infrequent, while administrative register data are often compromised by false negatives (omissions) and false positives (erroneous inclusions), hindering accurate inference of population size and dynamics. This work proposes a scalable hidden Markov capture–recapture modeling framework that, for the first time, unifies Cormack–Jolly–Seber–type models within a hidden Markov structure to jointly account for both types of observation error. The framework incorporates individual covariates and unobserved heterogeneity in detection probabilities and accommodates temporary emigration, multi-source data integration, and dynamic population inference. Combining maximum likelihood estimation with the Bag of Little Bootstraps for uncertainty quantification, an application to Swedish register data reveals substantial population bias due to overcoverage—such as individuals remaining registered after emigration—and significantly improves the accuracy of individual trajectory reconstruction and demographic dynamic estimation.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Accurate inference on population dynamics, such as migration and changes in population size, is essential for policymaking, resource allocation and demographic research. Traditional censuses are expensive, infrequent and not timely, leading many countries to adopt register-based approaches to replace or complement them. A primary challenge is that such registers are incomplete: even when individuals are present, their activities may not generate records in specific registers, resulting in false negative observation error. Conversely, some registers arise from administrative or household-level processes, so that individuals may appear in registers despite being absent, leading to false positive observation error. Existing approaches often either rely on ad-hoc decisions that ignore one or both error types, offer inference on population snapshots but not dynamics, or are computationally too slow for practical use. We propose a scalable framework for inferring population size and dynamics from register data, building on Cormack-Jolly-Seber type capture-recapture models formulated as hidden Markov models. Inference is carried out using maximum likelihood estimation, with uncertainty quantified via the Bag of Little Bootstraps. The model accounts for temporary emigration, incorporates an arbitrary number of possibly interacting registers subject to both error types, and allows observation probabilities to vary with individual characteristics and unobservable heterogeneity. We illustrate the approach using Swedish population registers, where overcoverage - individuals registered as living in the country although they are no longer present - provides a motivating example. The application yields new insights into population dynamics and individual trajectories.

Problem

• population dynamics
• register-based inference
• false negative
• false positive
• capture-recapture

Innovation

• capture-recapture
• hidden Markov model
• register-based population inference
• observation error
• population dynamics