Benchmarking Vision-Language Models on Synapse Detection and Proofreading in Connectomics

2026-09-28

AI Summary

This study evaluates the performance of vision-language models (VLMs) on synapse detection and proofreading tasks in connectomics. Focusing on electron microscopy image analysis, we systematically benchmark three paradigms: zero-shot prompting, few-shot learning, and LoRA-based parameter-efficient fine-tuning. Our results demonstrate that open-source VLMs fine-tuned via LoRA achieve performance comparable to closed-source expert models while surpassing conventional methods in cross-species transfer. Notably, these models outperform expert models trained on identical data when identifying merge errors in unseen species. This work validates the potential and generalization advantages of integrating VLMs with in-context learning and parameter-efficient fine-tuning for neuroscience image analysis.

Abstract

We benchmarked vision-language models (VLMs) on the decisions annotators take when inspecting electron microscopy images in connectomics: synapse detection (presence and polarity) and proofreading (split errors and merge errors). For synapse detection, we evaluated 19 open and 2 closed models across various architectures and sizes under zero-shot, four-shot in-context learning and LoRA settings, against specialist models, on datasets constructed by us using public resources. For proofreading, we evaluated 3 open and 2 closed models on the ConnectomeBench2 dataset, with cross-species transfer from fly and mouse to human and zebrafish. Most models were at chance zero-shot; a few examples helped mainly the closed and largest open ones. LoRA on a few thousand labels brought open models level with specialist models. When evaluated on unseen species, the best adapted VLMs outperformed specialist models trained on the same data in identifying merge errors. The project will be publicly available upon acceptance.

Problem

• Vision-Language Models
• Connectomics
• Synapse Detection
• Proofreading
• Benchmarking

Innovation

• Vision-Language Models
• Connectomics
• LoRA Fine-tuning
• Cross-species Transfer
• Synapse Detection