Beyond BEV: Optimizing Point-Level Tokens for Collaborative Perception

2025-08-27

AI Summary

Existing collaborative perception methods predominantly adopt 2D bird’s-eye view (BEV) representations for intermediate features, leading to substantial loss of critical 3D geometric and semantic structure information—thereby limiting fine-grained object detection and localization accuracy. To address this, we propose CoPLOT: the first framework to employ point-level tokens as the intermediate representation for collaborative perception, eliminating BEV projection entirely to preserve native 3D geometry and semantics. CoPLOT introduces three key innovations: (i) semantic-aware token reordering, (ii) frequency-domain enhanced state-space modeling, and (iii) a closed-loop-driven neighbor–ego spatial alignment mechanism. Built upon a point-native processing pipeline, CoPLOT achieves significant improvements over state-of-the-art methods on both synthetic and real-world benchmarks, notably enhancing detection and localization accuracy while simultaneously reducing communication bandwidth and computational overhead.

Technology Category

• Computer Vision: Representation Learning for Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Collaborative perception allows agents to enhance their perceptual capabilities by exchanging intermediate features. Existing methods typically organize these intermediate features as 2D bird's-eye-view (BEV) representations, which discard critical fine-grained 3D structural cues essential for accurate object recognition and localization. To this end, we first introduce point-level tokens as intermediate representations for collaborative perception. However, point-cloud data are inherently unordered, massive, and position-sensitive, making it challenging to produce compact and aligned point-level token sequences that preserve detailed structural information. Therefore, we present CoPLOT, a novel Collaborative perception framework that utilizes Point-Level Optimized Tokens. It incorporates a point-native processing pipeline, including token reordering, sequence modeling, and multi-agent spatial alignment. A semantic-aware token reordering module generates adaptive 1D reorderings by leveraging scene-level and token-level semantic information. A frequency-enhanced state space model captures long-range sequence dependencies across both spatial and spectral domains, improving the differentiation between foreground tokens and background clutter. Lastly, a neighbor-to-ego alignment module applies a closed-loop process, combining global agent-level correction with local token-level refinement to mitigate localization noise. Extensive experiments on both simulated and real-world datasets show that CoPLOT outperforms state-of-the-art models, with even lower communication and computation overhead. Code will be available at https://github.com/CheeryLeeyy/CoPLOT.

Problem

• Optimizing point-level tokens for collaborative perception
• Addressing unordered and massive point-cloud data challenges
• Enhancing object recognition with fine-grained 3D structural cues

Innovation

• Point-level tokens preserve 3D structure
• Semantic-aware reordering optimizes token sequences
• Frequency-enhanced model captures spatial-spectral dependencies