STSBench: A Large-Scale Dataset for Modeling Neuronal Activity in the Dorsal Stream of Primate Visual Cortex

2026-07-17

AI Summary

This study addresses the limited understanding of the computational mechanisms underlying the primate dorsal visual stream, hindered by the absence of large-scale neural datasets. To overcome this, we introduce STSBench, a high-throughput electrophysiological dataset comprising activity from over 2,000 neurons in the superior temporal sulcus of macaques while they viewed thousands of natural videos. This resource represents a nearly 50-fold increase in scale over existing datasets and provides the first large-scale benchmark for the dorsal stream. STSBench enables robust training and evaluation of neural encoding models and demonstrates its utility by successfully reconstructing visual inputs from neural responses, thereby highlighting its pivotal value for both computational neuroscience and brain-inspired artificial intelligence.

Abstract

The primate visual system is typically divided into two streams - the ventral stream, responsible for object recognition, and the dorsal stream, responsible for encoding spatial relations and motion. Recent studies have shown that convolutional neural networks (CNNs) pretrained on object recognition tasks are remarkably effective at predicting neuronal responses in the ventral stream, shedding light on the neural mechanisms underlying object recognition. However, similar models of the dorsal stream remain underdeveloped due to the lack of large scale datasets encompassing dorsal stream areas. To address this gap, we present STSBench, a dataset of large-scale, single neuron recordings from over 2,000 neurons in the superior temporal sulcus (STS), a nearly 50-fold increase over existing dorsal stream datasets, collected while Rhesus macaques viewed thousands of unique, natural videos. We show that our dataset can be used for benchmarking encoding models of dorsal stream neuronal responses and reconstructing visual input from neural activity.

Problem

• dorsal stream
• neuronal activity
• large-scale dataset
• superior temporal sulcus
• encoding models

Innovation

• STSBench
• dorsal stream
• neural encoding
• superior temporal sulcus
• large-scale dataset