Geometric Feature Prompting of Image Segmentation Models

2025-05-27

AI Summary

Automated segmentation of plant roots in rhizotron/minirhizotron images remains challenging due to structural complexity, while manual annotation is time-consuming and subjective. Method: We propose Geomprompt—a differential-geometry-based, semantic-aware prompt point generation method. Leveraging image gradient ridge detection, it automatically identifies geometrically salient points along slender structures (e.g., roots), providing only 1–3 precisely co-localized prompts to SAM/SAM2 to focus on local ridge-like features. Contribution/Results: This work pioneers the integration of differential geometry into prompt engineering, enhancing both physical interpretability and model robustness. Evaluated on root imagery, Geomprompt achieves a Dice score of 92.4%, significantly outperforming dense manual annotations and random-point baselines. The method is end-to-end compatible with the SAM ecosystem, and the open-source toolkit geomprompt is publicly released.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Computer Vision: Segmentation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Advances in machine learning, especially the introduction of transformer architectures and vision transformers, have led to the development of highly capable computer vision foundation models. The segment anything model (known colloquially as SAM and more recently SAM 2), is a highly capable foundation model for segmentation of natural images and has been further applied to medical and scientific image segmentation tasks. SAM relies on prompts -- points or regions of interest in an image -- to generate associated segmentations. In this manuscript we propose the use of a geometrically motivated prompt generator to produce prompt points that are colocated with particular features of interest. Focused prompting enables the automatic generation of sensitive and specific segmentations in a scientific image analysis task using SAM with relatively few point prompts. The image analysis task examined is the segmentation of plant roots in rhizotron or minirhizotron images, which has historically been a difficult task to automate. Hand annotation of rhizotron images is laborious and often subjective; SAM, initialized with GeomPrompt local ridge prompts has the potential to dramatically improve rhizotron image processing. The authors have concurrently released an open source software suite called geomprompt https://pypi.org/project/geomprompt/ that can produce point prompts in a format that enables direct integration with the segment-anything package.

Problem

• Automating plant root segmentation in rhizotron images
• Improving SAM's segmentation with geometric feature prompts
• Reducing laborious manual annotation in scientific image analysis

Innovation

• Geometric feature prompting for image segmentation
• Automated ridge point prompts for SAM
• Open-source geomprompt software integration