image-plane trajectory planning

The field of camera trajectory generation suffers from fragmented knowledge and inconsistent evaluation criteria, lacking a systematic survey. Method: This paper establishes the first unified knowledge framework for the domain, clarifying foundational definitions, a comprehensive technical taxonomy—including rule-based, numerical optimization, supervised/reinforcement learning, and neural motion modeling approaches—and a multidimensional evaluation system. It introduces the first holistic method classification scheme and cross-benchmark evaluation guidelines, integrating mainstream datasets and human-in-the-loop assessment protocols. Contribution/Results: The study identifies critical gaps—insufficient adaptability, low computational efficiency, and limited creative expressivity—and proposes future directions toward efficient modeling, dynamic environment adaptation, and semantics-controllable generation. The framework provides a reusable theoretical foundation and practical paradigm for computer graphics, VR, robotic navigation, and intelligent cinematic production.

Existing image-to-video generation methods suffer from limitations in camera trajectory control, temporal consistency, and geometric completeness. This work proposes an end-to-end framework based on dynamic 3D Gaussian splatting, which— for the first time—employs dynamic 3D Gaussian representations for single-image-driven video synthesis. The method jointly models camera motion and object dynamics within a single forward pass. By leveraging an explicit 3D scene representation, a motion sampling mechanism conditioned on a single input image, and differentiable rendering guided by prescribed camera trajectories, it achieves efficient, controllable, and temporally coherent video generation. Experiments on KITTI, Waymo, RealEstate10K, and DL3DV-10K demonstrate that the proposed approach significantly outperforms existing methods in both video quality and inference efficiency.

Existing video generation methods rely on separate modules to model camera motion, object translation, and local deformation—limiting holistic motion control. To address this, we propose the first unified trajectory-driven motion control framework. Our approach represents multi-granularity motion—including global viewpoint changes and fine-grained local deformations—as editable keypoint trajectories, which are injected into the latent space of pre-trained image-to-video diffusion models (e.g., SVD, PixArt-Video) via a lightweight motion injector. This enables end-to-end, plug-and-play motion conditioning without modifying the backbone architecture. Crucially, our method preserves temporal coherence and semantic alignment while simultaneously controlling both macro-scale camera motions and micro-scale deformations. Extensive experiments demonstrate state-of-the-art performance across motion sketching, dynamic viewpoint synthesis, and precise motion editing—achieving superior controllability, visual fidelity, and cross-model generalization.

Existing video re-rendering methods struggle to simultaneously preserve appearance fidelity, ensure dynamic consistency, and enable precise camera control under novel viewpoints, while lacking explicit spatiotemporal pixel correspondences. To address these limitations, this work proposes a video diffusion Transformer conditioned on paired 3D point trajectories, achieving four-dimensional consistent and camera-controllable generation. The key innovations include a data pipeline that extracts one-to-one trajectory correspondences from multi-view videos and a dual-view trajectory conditioner that integrates geometric operations with temporal aggregation. Evaluated on a benchmark of 400 videos encompassing both static and dynamic scenes, the proposed method substantially outperforms existing approaches, reducing rotation errors by 30–65% and translation errors by 61–72%.

This work addresses the challenge of flexibly supporting multimodal camera motion control in video generation. To this end, the authors propose a modality-agnostic framework that maps video, pose, and text inputs into a unified motion embedding space, enabling consistent and precise viewpoint manipulation. Key contributions include the construction of a Motion Triplet Dataset, the introduction of a geometry-driven motion representation based on camera extrinsics, and the design of a motion consistency objective in the latent space. The proposed method not only unifies multimodal inputs under a single processing pipeline but also enables novel capabilities such as motion sequence composition and cross-modal interpolation. Experiments demonstrate that the approach generates high-quality videos across all three modalities, accurately adhering to target camera trajectories and validating its effectiveness and generalization.

Existing methods for 4D scene synthesis from monocular videos are constrained by view interpolation, often failing to ensure cross-view consistency, particularly in out-of-view regions. This work proposes PanoGaussian, a unified framework that integrates panoramic trajectory guidance with an explicit dynamic Gaussian representation. By incorporating 3D dynamic physical priors into the modeling process, the method effectively mitigates scale and deformation artifacts in unseen regions. PanoGaussian enables camera-conditioned video generation, producing high-quality, temporally coherent 4D dynamic scenes even under large viewpoint changes, significantly outperforming current state-of-the-art approaches.

This work proposes a fully parallelizable, pixel-level distributed visual odometry and depth estimation algorithm that overcomes the inefficiencies of traditional approaches, which rely on transmitting redundant and noisy raw pixel data and struggle with on-sensor deployment. The method introduces, for the first time, an on-chip sensor architecture based on Gaussian Belief Propagation (GBP), where pixels exchange photometric observations and surface normal priors in parallel to reach consensus on camera motion. A keyframe-like anchoring mechanism is incorporated to effectively constrain inter-frame baselines and preserve geometric consistency. Experimental results demonstrate that the proposed approach achieves efficient and robust on-chip visual odometry and depth estimation on real-world datasets.

Traditional visual navigation struggles to balance global geometric consistency with topological generalization, limiting its performance in complex environments. This work proposes a novel map representation based on pixel-level relative 3D connectivity, which constructs a pixel correspondence graph in a relative coordinate frame from image sequences and generates a “WayPixel Costmap” for planning and control. By preserving high-fidelity geometric information without requiring global geometric consistency, the approach overcomes the limitations of conventional topological graphs and dense reconstructions. Experimental results demonstrate that the method significantly outperforms image-level and object-level representations across four simulated tasks and real-world scenarios, validating its accuracy and practicality for visual navigation.