🤖 AI Summary
In controllable video generation, camera motion and object motion are difficult to disentangle due to their shared inverse-depth scaling behavior in optical flow, leading to mutual interference between control signals. This work proposes a geometry–semantics orthogonal attention mechanism that explicitly decouples these factors at the architectural level. Specifically, camera motion is modeled in the geometric branch via norm-preserving rotations based on RoPE phase encoding, while object motion is handled in the semantic branch through gated value injection. A lightweight orthogonality regularization term is introduced to enforce subspace orthogonality between the two response pathways. This constructive design ensures disentanglement by architecture rather than relying on training stochasticity. Experiments demonstrate that the method reduces control crosstalk by over 2.4× while preserving generation fidelity, achieving state-of-the-art accuracy in both camera and object motion control, and generalizing effectively across different backbone architectures.
📝 Abstract
Controllable video generation demands independent command of the camera and the subject, yet 2D conditioning entangles them: camera- and object-induced optical flow share the same inverse-depth (1/Z) scaling and cannot be separated from image evidence alone. We first prove that this entanglement is representational, not architectural -- the 2D camera/object split is a non-identifiable inverse problem -- and therefore reframe decoupling as a question of operator design. We resolve it at the level of the attention operator. OrthoMotion routes camera motion into a geometric channel, a norm-preserving rotation of the rotary position embedding (RoPE) phase, and subject motion into a semantic channel, a gated value injection in cross-attention. Because these sub-operators are algebraically complementary -- a rotation versus a translation of the affine action on tokens -- a lightweight decoupling regularizer provably drives their response subspaces to orthogonality, so the two controls stop interfering. To our knowledge OrthoMotion is the first method to guarantee disentanglement by construction rather than hope for it to emerge. It attains state-of-the-art camera and subject accuracy at once while minimizing cross-talk, which we quantify with a new Cross-Talk Error (CTE) metric, cutting cross-talk by more than 2.4x with no loss in fidelity and generalizing across backbones.