Reshoot-Anything: A Self-Supervised Model for In-the-Wild Video Reshooting

2026-04-23

AI Summary

Dynamic video reenactment in non-rigid scenes is hindered by the scarcity of real multi-view data. This work proposes the first self-supervised framework that requires no real multi-view supervision, operating solely on monocular internet videos. By applying random crops to generate source–target video pairs and leveraging dense optical flow with forward warping to synthesize geometric anchors, the model implicitly learns 4D structure across space and time. Integrated with an adapted diffusion Transformer architecture, the method achieves high-fidelity novel-view synthesis in complex dynamic scenes, significantly improving temporal consistency and robustness to camera control, thereby attaining state-of-the-art performance.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Precise camera control for reshooting dynamic videos is bottlenecked by the severe scarcity of paired multi-view data for non-rigid scenes. We overcome this limitation with a highly scalable self-supervised framework capable of leveraging internet-scale monocular videos. Our core contribution is the generation of pseudo multi-view training triplets, consisting of a source video, a geometric anchor, and a target video. We achieve this by extracting distinct smooth random-walk crop trajectories from a single input video to serve as the source and target views. The anchor is synthetically generated by forward-warping the first frame of the source with a dense tracking field, which effectively simulates the distorted point-cloud inputs expected at inference. Because our independent cropping strategy introduces spatial misalignment and artificial occlusions, the model cannot simply copy information from the current source frame. Instead, it is forced to implicitly learn 4D spatiotemporal structures by actively routing and re-projecting missing high-fidelity textures across distinct times and viewpoints from the source video to reconstruct the target. At inference, our minimally adapted diffusion transformer utilizes a 4D point-cloud derived anchor to achieve state-of-the-art temporal consistency, robust camera control, and high-fidelity novel view synthesis on complex dynamic scenes.

Problem

• video reshooting
• non-rigid scenes
• multi-view data scarcity
• camera control
• dynamic videos

Innovation

• self-supervised learning
• video reshooting
• 4D spatiotemporal modeling
• novel view synthesis
• diffusion transformer