spatial localization

Techniques for localizing relevant spatial regions or signals within images or video—focusing model attention on task-critical areas, estimating object/action/location relevance, and robustly identifying locally dominant terms under noise.

spatiallocalization

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

Existing cross-view geolocalization methods suffer from insufficient inter-view information interaction and coarse-grained spatial relationship modeling, making them vulnerable to edge noise. To address these issues, we propose a dual-attention mechanism: (1) a cross-view cross-attention module enabling bidirectional contextual modeling between aerial and ground views; and (2) a multi-head spatial attention module that fuses multi-scale convolutional features to strengthen implicit correspondence learning. Furthermore, we introduce the first fine-grained Ground-to-Drone (G2D) localization benchmark dataset. Extensive experiments on CVOGL and the proposed G2D dataset demonstrate that our method effectively suppresses irrelevant noise, enhances spatial relational representation, and achieves superior localization accuracy over state-of-the-art approaches.

Addressing ineffective information transfer between different viewsEnhancing cross-view object geo-localization accuracyImproving spatial feature representation with multi-scale attention

Existing self-attention and related models lack a unified theoretical framework, hindering systematic understanding and extension. This work proposes the “localization method”—a general machine learning framework grounded in localization kernels and local averages—that formalizes local modeling through rigorous theoretical constructs and localization techniques. For the first time, it subsumes state-of-the-art architectures such as Transformers under a unified local modeling paradigm. The framework not only reveals intrinsic connections among diverse models—including kernel methods, MeanShift, Hopfield networks, Locally Linear Embedding (LLE), fuzzy inference, denoising autoencoders, and Transformers—but also introduces scalable hierarchical local and non-local models, thereby establishing a novel paradigm for building data-adaptive learning systems.

localizationmachine learningself-attention

Temporal and spatial localization of visual queries in long videos remains challenging due to small object sizes, occlusions, appearance variations, and cluttered backgrounds. Method: We propose a training-free, strong baseline that leverages pre-trained ViT/CLIP for region representation extraction, integrating YOLO-style detection, inter-frame cosine similarity matching, and bidirectional optical flow tracking—augmented with object-aware fine-grained relocalization and visual query enhancement. This design significantly improves robustness to partial visibility and dynamic appearance changes. Contribution/Results: On the Ego4D VQ 2D Localization benchmark, our method achieves a 49% relative improvement in spatiotemporal mean average precision over prior supervised approaches. It establishes the first high-performance, training-free baseline for video grounding, introducing a novel zero-shot paradigm for visual query localization in long videos.

Handle small objects and cluttered scenesImprove accuracy without task-specific trainingLocalize visual queries in long videos

Improving Open-World Object Localization by Discovering Background

Apr 24, 2025
AS
Ashish Singh
🏛️ Univ. of Mass.-Amherst | Mitsubishi Electric Research Labs

This paper addresses open-world object localization: training models with bounding-box supervision for only a limited set of categories, while requiring them to localize *all* objects—including unseen categories—at inference time. To tackle this challenge, we propose a background-driven object proposal learning paradigm, which—uniquely—treats background discovery as explicit supervisory signal. We formally define background as redundant, low-discriminative image regions and model “objectness” via inverse constraints. Our approach comprises three components: (i) a region-discriminativeness-based background discovery module; (ii) a background suppression loss; and (iii) an end-to-end trainable object proposal network. Evaluated on standard benchmarks, our method significantly outperforms state-of-the-art approaches, achieving substantial gains in both overall and unseen-category recall, as well as cross-category generalization capability.

Discovering non-discriminative image regions to enhance object proposalImproving objectness learning by incorporating background informationLocalizing objects in open-world settings with limited training classes

Teaching VLMs to Localize Specific Objects from In-context Examples

Nov 20, 2024
SD
Sivan Doveh
🏛️ IBM Research | Tel Aviv University | JKU Linz | MIT-IBM | MIT CSAIL | Weizmann Institute of Science

This work addresses a core limitation of vision-language models (VLMs): their inability to precisely localize specific objects in few-shot settings relying solely on visual context—especially when textual descriptions are ambiguous or multiple semantically similar objects coexist. To tackle this, we formalize the *personalized few-shot localization* task: given a small set of annotated context images, the model must localize the same object category in a novel query image. Methodologically, we introduce (i) the first dedicated benchmark for this task; (ii) pseudo-name label regularization, which suppresses language priors and strengthens reliance on visual context; and (iii) context-aware instruction tuning data derived from video object tracking sequences. Extensive experiments across VLMs ranging from 7B to 72B parameters demonstrate consistent, significant improvements over state-of-the-art methods on multiple custom benchmarks. Our work is the first to systematically identify and bridge the critical gap in context-driven visual localization capability within modern VLMs.

Enhancing VLMs' context awareness for object localization.Few-shot personalized localization from annotated images.VLMs lack ability to localize objects using context.

Latest Papers

What's happening recently
View more

This work addresses the limitation of existing vision backbones, which prioritize semantic content (“what”) over spatial location (“where”) in image classification, thereby hindering performance on localization tasks. To remedy this, the authors propose a novel Vision Transformer backbone that incorporates a “what–where” disentangled inductive bias: tokens encode semantic representations while attention maps serve as spatial representations. A multi-stream slot architecture processes these two streams in parallel. Remarkably, with only single-label ImageNet supervision, the model directly yields localization-aware features at its final layer. It achieves substantial improvements over current ViT-based methods in zero-shot object discovery and weakly supervised semantic segmentation, and demonstrates strong transferability across diverse localization scenarios.

attention mapsobject localizationspatial representation

This work addresses the challenge of efficiently localizing an unknown number of anomalous regions in large-scale spatially dependent data. The authors propose SPLADE, a two-stage method that integrates intelligent sampling with boundary estimation to simultaneously and consistently estimate both the number and boundaries of multiple axis-aligned anomalous patches under general spatial dependence structures—without requiring full spatial grid segmentation. By leveraging a uniform Gaussian approximation and an efficient search strategy, SPLADE substantially improves computational efficiency and localization accuracy. Experimental results demonstrate that SPLADE outperforms existing approaches on both synthetic and real-world video surveillance datasets, achieving faster runtime, higher localization precision, and robustness to strong spatial dependencies.

anomalous patcheslocalizationmultiple anomalies

Existing object detectors often learn task-driven features that rely on shortcut correlations, failing to adequately capture the underlying annotation structure, which limits their generalization, interpretability, and robustness under task shifts or sparse supervision. To address this, this work proposes an annotation-guided feature enhancement framework that explicitly integrates geometric annotation priors into feature learning for the first time. By constructing a dense spatial feature grid and injecting it into the backbone network—where it fuses with the feature pyramid—the method steers region proposal and detection heads toward representations better aligned with annotation structure. Evaluated on wildlife and remote sensing datasets, the approach significantly improves object focus, reduces background sensitivity, and demonstrates superior generalization and data efficiency in weakly supervised and unseen-task settings.

annotation structureobject detectionrepresentation robustness

This work addresses a critical limitation in existing channel pruning methods, which conflate two orthogonal dimensions—task relevance and local substitutability—thereby constraining performance. For the first time, this study explicitly disentangles these concepts: task relevance quantifies a channel’s contribution to the target objective, while local substitutability measures whether its function can be compensated by other channels within the same layer. Theoretical analysis and empirical evidence demonstrate that these two properties rapidly decouple during training, with local substitutability emerging as a more reliable criterion for pruning. Through comprehensive validation—including input attribution, channel overlap analysis, task information metrics, residual gradient examination, and ablation studies—this approach consistently outperforms conventional pruning strategies across multiple architectures and benchmarks, including CIFAR-100 and ImageNet.

channel importancefeature redundancylocal replaceability

Existing vision models lack subject-awareness, making it difficult to accurately identify and remove distractors in image editing without compromising scene semantic consistency. This work formalizes, for the first time, the task of Subject-Aware Distractor Localization (SADL) and introduces the first real-world benchmark for this task, comprising 1,800 cases with 14,617 annotated candidate objects. The authors propose a two-stage vision-language model (VLM) pipeline grounded in five inclusion factors and three contextual exclusion rules. Evaluation across seven VLMs reveals strong identification capabilities but exposes a systematic over-suppression bias during the exclusion phase. The SADL benchmark serves as a critical diagnostic tool for subject-conditioned reasoning in multimodal systems.

distractor localizationimage compositionsemantic coherence

Hot Scholars

LX

Lihua Xie

Professor of Electrical Engineering, Nanyang Technological University
Robust controlNetworked ControlMult-agent Systems
HV

Holger Voos

University of Luxembourg, SnT Automation & Robotics Research Group
Control EngineeringAutomationMobile Robotics
MP

Marc Pollefeys

Professor of Computer Science, ETH Zurich, and Director Spatial AI Lab, Microsoft
Computer VisionComputer GraphicsRoboticsMachine Learning
MM

Michael Milford

QUT Professor | Director, QUT Robotics Centre | ARC Laureate Fellow | Microsoft Fellow
Roboticscomputational neurosciencenavigationSLAM
XC

Xieyuanli Chen

Associate Professor, NUDT, China
RoboticsSLAMLocalizationLiDAR Perception