Optimal and Unbiased Fluxes from Up-the-Ramp Detectors under Variable Illumination

2026-01-15

AI Summary

This study addresses the challenge posed by atmospheric scintillation in ground-based near-infrared observations, which induces rapid temporal variations in pixel counts and renders the conventional assumption of constant flux a significant source of bias. To overcome this limitation, the authors propose a Bayesian statistical model that explicitly accounts for temporal variability during flux extraction. By jointly modeling up-the-ramp detector data and leveraging information sharing among neighboring spectral pixels, the method yields unbiased flux estimates with optimal uncertainty quantification. Experiments demonstrate that when flux variations exceed 3.5%, the approach substantially outperforms traditional models, reducing flux biases by up to 120%. Applied to real APOGEE data, the method effectively corrects wavelength-independent time-dependent systematics, with its robustness further validated through cross-checks against meteorological conditions.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Near-infrared (NIR) detectors -- which use non-destructive readouts to measure time-series counts-per-pixel -- play a crucial role in modern astrophysics. Standard NIR flux extraction techniques were developed for space-based observations and assume that source fluxes are constant over an observation. However, ground-based telescopes often see short-timescale atmospheric variations that can dramatically change the number of photons arriving at a pixel. This work presents a new statistical model that shares information between neighboring spectral pixels to characterize time-variable observations and extract unbiased fluxes with optimal uncertainties. We generate realistic synthetic data using a variety of flux and amplitude-of-time-variability conditions to confirm that our model recovers unbiased and optimal estimates of both the true flux and the time-variable signal. We find that the time-variable model should be favored over a constant-flux model when the observed count rates change by more than 3.5%. Ignoring time variability in the data can result in flux-dependent, unknown-sign biases that are as large as ~120% of the flux uncertainty. Using real APOGEE spectra, we find empirical evidence for approximately wavelength-independent, time-dependent variations in count rates with amplitudes much greater than the 3.5% threshold. Our model can robustly measure and remove the time-dependence in real data, improving the quality of data-model comparison. We show several examples where the observed time-dependence quantitatively agrees with independent measurements of observing conditions, such as variable cloud cover and seeing.

Problem

• near-infrared detectors
• time-variable illumination
• flux extraction
• atmospheric variability
• unbiased flux estimation

Innovation

• time-variable flux
• up-the-ramp sampling
• unbiased flux estimation
• near-infrared detectors
• spectral pixel correlation