CellSplat4D: PSF-Aware 4D Gaussian Splatting for Sparse Robotic Live-Cell Imaging

2026-10-03

AI Summary

This study addresses the disruption of live-cell time-lapse recordings and tracking failures caused by sparse sampling in robotic microscopy under phototoxicity constraints. To overcome this, we propose a continuous-time reconstruction framework based on 4D Gaussian Splatting. Methodologically, continuous-time rendering decouples analysis frequency from imaging cost, while a point spread function (PSF)-aware mechanism directly estimates the PSF from measured data. Additionally, a 3D U-Net supervises unobserved time points to accommodate complex dynamics such as cell division. Experiments on *C. elegans* and CHO datasets demonstrate that our approach achieves optimal nuclear fidelity, minimal gap decay, and accurate recovery of unseen focal planes. This work establishes a new paradigm for high-fidelity reconstruction of live-cell dynamics under sparse sampling conditions.

Abstract

A robotic microscope watching living cells cannot afford to look as often as it would like. Every volume it acquires costs photons the specimen does not get back, and time owed to other wells. What such a platform exists to produce is a record of individual cells through time: which cell is which from one volume to the next, and which cell divided into which two. Sampling sparsely breaks that record exactly where it matters, and the fault lies in the acquisition schedule rather than in the analysis software. We fill the gaps by reconstructing them, fitting a 4D Gaussian model to whatever volumes the hardware could afford. The model is a cloud of light-emitting blobs, each carrying a position, a shape and a lifetime. Being continuous in time, it renders any missing volume on demand, decoupling how often the robot analyses from how often it can afford to look. The microscope's point-spread function is measured from the data rather than inherited from acquisition metadata or left to the optimizer, because metadata inflates it and the optimizer cannot recover it at all: a wider blur around a smaller blob fits the images equally well. Each blob's lifetime is stored in frames rather than as a fraction of the recording, so that it denotes a fixed duration on any sequence. Unmeasured timesteps are supervised at coarse scale by a 3D U-Net that predicts the intermediate volume directly and estimates no motion field, since a dividing cell becomes two and no motion describes that. On two Cell Tracking Challenge sequences, a C. elegans embryo and a Chinese Hamster Ovarian (CHO), with fidelity scored per cell nucleus, our reconstruction holds the highest nucleus fidelity at every distance from an acquired frame, has the flattest decay across the gap, and best recovers focal planes it was never shown with graceful degradation across the gap.

Problem

• Live-cell imaging
• Sparse sampling
• 4D reconstruction
• Cell tracking
• Point spread function

Innovation

• 4D Gaussian Splatting
• Point Spread Function
• Sparse Live-Cell Imaging
• 3D U-Net
• Robotic Microscopy