Reference-Free 3D Reconstruction of Brain Dissection Photographs with Machine Learning

2025-03-13

AI Summary

This study addresses the challenging problem of 3D reconstruction from sparse, incomplete stacks of conventional 2D histological photographs—without external reference images—thereby bridging neuropathology and in vivo MRI and enabling mapping of microscopic pathological features into standardized atlas space. We propose the first reference-free, end-to-end deep regression framework: synthetic MRI slices are used to generate appearance-randomized training data, and a joint least-squares geometric fitting loss enables direct regression of 3D coordinates (in atlas space) for each pixel from the input slice stack, while simultaneously producing atlas-guided automatic segmentation. Evaluated on both synthetic and real-world datasets, our method achieves performance comparable to reference-based baselines and demonstrates significantly improved robustness under severe slice dropout. The implementation is publicly available.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Feature Construction/Reformulation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Correlation of neuropathology with MRI has the potential to transfer microscopic signatures of pathology to invivo scans. Recently, a classical registration method has been proposed, to build these correlations from 3D reconstructed stacks of dissection photographs, which are routinely taken at brain banks. These photographs bypass the need for exvivo MRI, which is not widely accessible. However, this method requires a full stack of brain slabs and a reference mask (e.g., acquired with a surface scanner), which severely limits the applicability of the technique. Here we propose RefFree, a dissection photograph reconstruction method without external reference. RefFree is a learning approach that estimates the 3D coordinates in the atlas space for every pixel in every photograph; simple least-squares fitting can then be used to compute the 3D reconstruction. As a by-product, RefFree also produces an atlas-based segmentation of the reconstructed stack. RefFree is trained on synthetic photographs generated from digitally sliced 3D MRI data, with randomized appearance for enhanced generalization ability. Experiments on simulated and real data show that RefFree achieves performance comparable to the baseline method without an explicit reference while also enabling reconstruction of partial stacks. Our code is available at https://github.com/lintian-a/reffree.

Problem

• Reconstruct 3D brain images without external reference
• Correlate neuropathology with MRI using dissection photos
• Enable reconstruction from partial brain slab stacks

Innovation

• Reference-free 3D brain reconstruction using machine learning
• Atlas-based segmentation from dissection photographs
• Training on synthetic MRI data for generalization