Technology Category
Machine Learning: Large Multimodal Models (LMMs)Computer Vision: Motion & TrackingSearch and Optimization: Sampling/Simulation-based Search
Segment Anything Model for Zero-shot Single Particle Tracking in Liquid Phase Transmission Electron Microscopy
🤖 AI Summary
Robust zero-shot, high-precision nanoparticle tracking in low-dose transmission electron microscopy (LPTEM) videos has long been hindered by the absence of reliable unsupervised methods. This paper introduces SAM4EM—the first fine-tuning-free, end-to-end framework for single-particle analysis in LPTEM videos—marking the first adaptation of the SAM 2 foundation model to video segmentation in liquid-phase EM. SAM4EM establishes a synergistic paradigm integrating promptable video segmentation with physics-informed trajectory modeling. It comprises three core components: zero-shot prompt-driven segmentation, noise-robust trajectory association, and statistical quantification. Evaluated on real LPTEM datasets, SAM4EM achieves segmentation and tracking accuracy approximately 50× higher than state-of-the-art methods. It enables high-confidence automated tracking of nanoscale particles and precise characterization of their diffusion dynamics. By providing a generalizable, AI-powered analytical infrastructure, SAM4EM advances in situ, real-time investigation of nanoscale dynamics in liquid environments.
Technology Category
Application Category
📝 Abstract
Liquid phase transmission electron microscopy (LPTEM) offers an unparalleled combination of spatial and temporal resolution, making it a promising tool for single particle tracking at the nanoscale. However, the absence of a standardized framework for identifying and tracking nanoparticles in noisy LPTEM videos has impeded progress in the field to develop this technique as a single particle tracking tool. To address this, we leveraged Segment Anything Model 2 (SAM 2), released by Meta, which is a foundation model developed for segmenting videos and images. Here, we demonstrate that SAM 2 can successfully segment LPTEM videos in a zero-shot manner and without requiring fine-tuning. Building on this capability, we introduce SAM4EM, a comprehensive framework that integrates promptable video segmentation with particle tracking and statistical analysis, providing an end-to-end LPTEM analysis framework for single particle tracking. SAM4EM achieves nearly 50-fold higher accuracy in segmenting and analyzing LPTEM videos compared to state-of-the-art methods, paving the way for broader applications of LPTEM in nanoscale imaging.
Problem
Research questions and friction points this paper is trying to address.
Liquid Phase Electron Microscopy
Particle Tracking
Image Analysis
Innovation
Methods, ideas, or system contributions that make the work stand out.
Segment Anything Model 2 (SAM 2)
Single Particle Tracking
Liquid Phase Transmission Electron Microscopy (LPTEM)
🔎 Similar Papers
No similar papers found.
