motion compensation

Video editing often suffers from inter-frame inconsistencies, causing flickering and identity drift that degrade visual coherence. To address this, we propose the first object-level consistency restoration framework integrating perceptual metrics with symbolic reasoning. Our method introduces: (1) a learnable adaptive threshold to jointly model perceptual fidelity—via CLIP similarity, LPIPS, histogram statistics, and SAM mask IoU—and temporal logical constraints; (2) a neuro-symbolic verification mechanism that combines an SMT solver with probabilistic model checking to simultaneously ensure low-level stability and high-level temporal logic correctness; and (3) neural adaptive frame interpolation to enhance temporal smoothness. Evaluated on DAVIS and Pexels benchmarks, our approach achieves a +1.4 improvement in CLIP Score and a −6.1 reduction in warp error, significantly outperforming state-of-the-art methods.

This paper proposes a training-free video semantic editing framework that precisely injects semantic features of a user-provided reference object into designated spatiotemporal regions of a video, while rigorously preserving the original motion trajectories and visual context. Methodologically, it introduces a novel motion-aware concept alignment mechanism, integrating structured noise trajectory manipulation, momentum-based semantic correction, gamma-residual noise stabilization, and latent-space object detection and tracking; additionally, diagonal denoising scheduling and class-agnostic segmentation are incorporated to enhance controllability. Key contributions include: (1) the first CASS (Consistency-Aware Semantic Similarity) metric specifically designed for video semantic editing evaluation; (2) superior performance over state-of-the-art baselines on a newly constructed benchmark dataset, with significant improvements in spatial consistency, motion coherence, and CASS scores; and (3) zero-shot, high-fidelity, temporally consistent video editing without any model training.

To address the limited robustness of cross-video stream feature matching under noise, frame misalignment, and cross-modal (e.g., infrared–visible) conditions, this paper proposes a purely temporal, keypoint-free matching method. Instead of relying on spatial keypoint detection, our approach models motion signatures of pixel blocks across consecutive frames by jointly encoding optical flow and block-level temporal correlations, and employs Dynamic Time Warping (DTW) to enhance cross-video motion alignment. The method inherently achieves scale, rotation, and translation invariance, requires no training, and supports cross-modal matching. Experiments demonstrate that it significantly outperforms state-of-the-art methods across diverse challenging scenarios—achieving substantial gains in matching accuracy while reducing computational overhead by over 60%, thereby enabling real-time deployment.

Existing video customization methods rely on reference images or task-specific temporal priors, which struggle to fully exploit the intrinsic spatiotemporal information in videos, thereby limiting generation flexibility and generalization. This work proposes OmniTransfer, a unified framework that enhances appearance consistency through multi-view inter-frame information and integrates temporal cues for fine-grained temporal control. OmniTransfer introduces three key mechanisms: task-aware positional bias, reference-decoupled causal learning, and task-adaptive multimodal alignment. Notably, it achieves high-quality motion transfer without requiring pose annotations—a first in the field—and unifies support for diverse video transfer tasks. Experiments demonstrate that OmniTransfer outperforms existing approaches in identity and style transfer as well as camera motion and visual effect generation, while matching pose-based models in motion transfer fidelity, enabling highly realistic and flexible video synthesis.

Object appearance drift in video generation—causing inter-frame visual discontinuity—remains a critical bottleneck for diffusion-based models. To address this, we propose an end-to-end joint optimization framework that, for the first time within the Stable Video Diffusion architecture, integrates differentiable point tracking directly into the diffusion training objective. Our method leverages inter-frame keypoint trajectories as explicit spatial supervision, jointly optimizing the standard video diffusion loss and a tracking loss enforcing optical flow consistency. Crucially, it requires no auxiliary modules, pretrained models, or manual annotations, unifying video generation and motion tracking within a single network. Experiments demonstrate substantial suppression of appearance drift, yielding significant improvements in motion coherence and temporal stability across multiple benchmarks. Notably, the Fréchet Video Distance (FVD) decreases by up to 32%, establishing a novel paradigm for spatiotemporal consistency modeling in video diffusion.

This work addresses the challenges of ghosting and drift in infrared and visible video fusion, which arise from temporal misalignment, geometric rigidity, and error accumulation in diffusion models. The authors reformulate the fusion task as a history-conditioned motion generation problem and propose a spectral filtering framework that implicitly models motion dynamics to circumvent explicit alignment. Key innovations include stable historical guidance, a soft temporal anchoring mechanism, and a decoupled structure-motion adaptive strategy, complemented by a two-stage training scheme and latent space optimization. The method achieves state-of-the-art performance in both fusion quality and temporal consistency, effectively suppressing artifacts and drift.

Existing slot-based unsupervised video object tracking methods suffer from identity instability and foreground fragmentation under motion, occlusion, or object entry/exit due to the entanglement of appearance and geometric pose. This work proposes STAITUS, a novel framework that explicitly disentangles appearance from pose (position and scale) within slot representations for the first time. By enforcing intra-frame spatial separation constraints and applying temporal alignment solely in the appearance space, STAITUS enhances both mask sharpness and identity consistency. Additionally, an adaptive gating mechanism dynamically adjusts the number of active slots to mitigate oversegmentation. Evaluated on multiple synthetic and real-world video benchmarks, STAITUS significantly outperforms existing approaches, achieving breakthrough improvements in segmentation quality and tracking stability.

Existing video re-rendering methods struggle to simultaneously preserve appearance fidelity, ensure dynamic consistency, and enable precise camera control under novel viewpoints, while lacking explicit spatiotemporal pixel correspondences. To address these limitations, this work proposes a video diffusion Transformer conditioned on paired 3D point trajectories, achieving four-dimensional consistent and camera-controllable generation. The key innovations include a data pipeline that extracts one-to-one trajectory correspondences from multi-view videos and a dual-view trajectory conditioner that integrates geometric operations with temporal aggregation. Evaluated on a benchmark of 400 videos encompassing both static and dynamic scenes, the proposed method substantially outperforms existing approaches, reducing rotation errors by 30–65% and translation errors by 61–72%.

This work addresses the challenge of achieving semantically consistent co-editing across image groups exhibiting significant variations in pose, viewpoint, and spatial layout. To this end, we propose GroupEditing, a novel framework that uniquely integrates explicit geometric correspondences via VGGT with implicit temporal priors derived from pretrained video models. We further introduce an alignment-enhanced RoPE positional encoding to preserve identity consistency during editing. Alongside the method, we present GroupEditData, the first large-scale dataset for group image editing, and GroupEditBench, a comprehensive benchmark for evaluation. Extensive experiments demonstrate that our approach substantially outperforms existing methods in terms of visual quality, cross-view consistency, and semantic alignment.

This work addresses the challenge of transferring image pre-trained models to video tasks, where maintaining temporal consistency across frames and semantic discriminability across videos is difficult to achieve simultaneously. To this end, the authors propose the Co-Settle framework, which introduces a lightweight projection layer atop a frozen image encoder and jointly optimizes the representation space through a temporal cycle-consistency loss and a semantic separability constraint. Co-Settle is the first method to explicitly model and theoretically analyze the trade-off between these two objectives. Remarkably, it achieves significant performance gains across multiple video understanding benchmarks after only five rounds of self-supervised training. Consistent improvements are observed across eight widely used image pre-trained models, demonstrating the framework’s efficiency and broad applicability.