MIA: A Visual Analytics System for Multimodal Spectral Imaging Data

2026-05-30

AI Summary

Existing tools struggle to provide unified analysis of high-dimensional, spatially resolved multimodal spectral imaging data, often requiring analysts to switch between software and manually integrate results. This work proposes the first modality-agnostic visualization system that supports end-to-end integrated analysis, incorporating derivative-based preprocessing, hierarchical and landmark-guided embeddings, interactive and automated segmentation with cross-view shared state, spectral similarity search, and multimodal co-registration. By enabling synergistic exploration of multiscale embeddings and joint multimodal representations, the system successfully reproduces tissue compartmentalization, identifies pigment constituents, and fuses molecular and elemental imaging across three real-world chemical analysis cases. This integrated approach substantially reduces software switching and enhances expert analytical efficiency and insight generation.

Abstract

Hyperspectral bioimaging techniques such as infrared (IR) microscopy and laser ablation-inductively coupled plasma-mass spectrometry (LA-ICP-MS) produce high-dimensional, spatially resolved datasets that require sophisticated analysis to reveal chemically and anatomically meaningful structures. Existing software solutions are typically modality-specific and cover only parts of the analytical workflow, forcing researchers to transfer data across multiple tools and manually reconcile results. We present MIA (Multiscale Image Analysis), a modality-agnostic visual analysis environment that integrates the full exploratory workflow -- from spectral preprocessing and dimensionality reduction to interactive segmentation and spectral similarity analysis -- within a single, tightly coupled interface. MIA supports hierarchical and landmark-based embeddings to handle datasets of varying scale and complexity, interactive and automatic segmentation with a shared state across all linked views, and multimodal analysis of co-registered datasets from different instruments. We demonstrate the effectiveness of MIA through three use cases drawn from real analytical chemistry workflows: (1) the recovery of biologically meaningful tissue compartments through derivative preprocessing and hierarchical embedding, (2) pigment identification via spectral similarity search with spatial overview, and (3) multimodal tissue characterization combining molecular IR and elemental LA-ICP-MS data. Qualitative feedback from domain expert collaborators confirms that MIA reduces the need for tool-switching and supports analytical insights that are difficult to obtain with existing software.

Problem

• hyperspectral imaging
• multimodal data
• visual analytics
• spectral analysis
• data integration

Innovation

• visual analytics
• multimodal imaging
• hyperspectral data
• interactive segmentation
• dimensionality reduction