Semiparametric Efficient Data Integration Using the Dual-Frame Sampling Framework

2026-01-13

AI Summary

This study addresses the challenges of integrating probability and nonprobability samples, where the unknown sampling mechanism of the nonprobability sample leads to identification difficulties and efficiency loss. Within a dual-frame sampling framework, the authors propose two complementary semiparametric estimators: one achieves the semiparametric efficiency bound by parametrically modeling the inclusion probabilities of the nonprobability sample, while the other enhances robustness through a two-stage sampling approximation that avoids explicit modeling of the sampling mechanism. The work innovatively introduces an identifiability condition based on strong monotonicity, enabling model parameter identification without requiring instrumental variables or record linkage. The proposed methods substantially improve efficiency under correct model specification and remain stable under misspecification or weak identification, as demonstrated through simulations and empirical analyses, and are implemented in the R package dfSEDI.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: State Estimation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Data transparency and provenance

Abstract

Integrating probability and non-probability samples is increasingly important, yet unknown sampling mechanisms in non-probability sources complicate identification and efficient estimation. We develop semiparametric theory for dual-frame data integration and propose two complementary estimators. The first models the non-probability inclusion probability parametrically and attains the semiparametric efficiency bound. We introduce an identifiability condition based on strong monotonicity that identifies sampling-model parameters without instrumental variables, even under informative (non-ignorable) selection, using auxiliary information from the probability sample; it remains valid without record linkage between samples. The second estimator, motivated by a two-stage sampling approximation, avoids explicit modeling of the non-probability mechanism; though not fully efficient, it is efficient within a restricted augmentation class and is robust to misspecification. Simulations and an application to the Culture and Community in a Time of Crisis public simulation dataset show efficiency gains under correct specification and stable performance under misspecification and weak identification. Methods are implemented in the R package \texttt{dfSEDI}.

Problem

• data integration
• dual-frame sampling
• non-probability sampling
• semiparametric efficiency
• identification

Innovation

• semiparametric efficiency
• dual-frame sampling
• non-probability sampling
• identifiability
• data integration