Selected work

Representative Papers

ComplexSync: High-Fidelity and Real-Time Lip Sync in Complex Scenarios

Sep 24, 2026

This work addresses the limitations of diffusion-based lip synchronization models, specifically their degraded quality in complex scenes and high inference latency. To overcome these challenges, we propose a unified framework for high-fidelity, real-time generation. Methodologically, a dual-stream joint training mechanism is designed to prevent information leakage, while knowledge distillation is incorporated to accelerate single-step denoising. Furthermore, priors from vision foundation models and a relation alignment loss are introduced to enhance robustness. Experimental results demonstrate that the proposed method achieves an inference speed exceeding 70 FPS and attains state-of-the-art performance across both standard and complex scenarios. Additionally, a dedicated benchmark is constructed to facilitate evaluation. This study effectively advances the practical deployment of real-time lip synchronization technology.

0 citationsRead paper

High-Fidelity and Long-Duration Human Image Animation with Diffusion Transformer

Dec 26, 2025

Existing methods struggle to simultaneously preserve fine-grained facial and hand details while maintaining spatiotemporal consistency in long-duration (>3-second) human image animation. To address this, we propose a high-fidelity long-video generation framework based on the Diffusion Transformer (DiT). Our approach introduces three key innovations: (1) a novel hybrid implicit guidance signal combined with a sharpness-aware guidance factor; (2) a temporal-aware positional offset adaptation module enabling arbitrary-length video synthesis; and (3) skeleton-aligned modeling coupled with identity-agnostic data augmentation. These components collectively enhance fine-grained structural modeling and inter-frame coherence. Quantitative and qualitative evaluations demonstrate state-of-the-art performance in critical metrics—including facial expression fidelity, hand motion dynamics, and temporal smoothness—while achieving superior visual quality and strong spatiotemporal consistency.

0 citationsRead paper

Detecting Emotional Dynamic Trajectories: An Evaluation Framework for Emotional Support in Language Models

Nov 12, 2025

Existing LLM evaluation for emotional support relies predominantly on static, short-turn dialogues, failing to capture the dynamic evolution and longitudinal nature of human emotions. To address this, we propose the first evaluation framework explicitly designed for emotional dynamic trajectories. Our method introduces a first-order Markov emotional trajectory model, integrating psychologically grounded mechanisms—including contextual selection and cognitive reappraisal—to generate a large-scale benchmark comprising 328 emotion scenarios and 1,152 distractor events. We design a trajectory-level evaluation paradigm featuring constraints on emotion regulation strategies and causal adjustment for emotional state tracking. Furthermore, we introduce three novel metrics: BEL (Emotional Baseline Shift), ETV (Emotional Trajectory Variance), and ECP (Empathic Consistency Probability). Extensive evaluation across diverse LLMs demonstrates that our framework effectively discriminates long-term emotional support capabilities, yielding interpretable and actionable assessments of empathic interaction.

0 citationsRead paper

Playmate2: Training-Free Multi-Character Audio-Driven Animation via Diffusion Transformer with Reward Feedback

Oct 13, 2025

Existing audio-driven video generation methods suffer from critical limitations in lip-sync accuracy, long-term temporal coherence, and multi-character coordinated animation. To address these, we propose Mask-CFG—a training-free framework integrating positional offset inference, LoRA-based efficient fine-tuning, reward-guided optimization, and block-wise diffusion Transformer (DiT) inference—enabling high-fidelity, long-duration, natural dialogue video synthesis for arbitrary numbers of characters without architectural modification or domain-specific data. Our core innovation lies in decoupling character control from speech-driven motion via masked classifier-free guidance and dynamic reward calibration. This significantly improves lip-sync accuracy (+12.7% LSE), temporal consistency (+38.4% TCV), and multi-character motion naturalness. On multi-character benchmarks, Mask-CFG surpasses state-of-the-art methods while maintaining high fidelity, low inference cost, and strong generalization.

0 citationsRead paper

HD-PPT: Hierarchical Decoding of Content- and Prompt-Preference Tokens for Instruction-based TTS

Sep 23, 2025

Existing instructable TTS models suffer from a modality gap between coarse-grained text instructions and fine-grained speech tokens, hindering precise prosodic and phonetic control. To address this, we propose HD-PPT—a Hierarchical Dual-Preference Prompting and Tokenization framework. First, we design a novel hierarchical speech codec that disentangles content-related and instruction-related speech tokens. Second, we introduce a dual-preference token extraction mechanism jointly supervised by ASR and CLAP to align textual instructions with hierarchical speech representations. Third, we establish a layered decoding process enabling controllable generation across semantic, prosodic, and phonemic levels. By integrating large language models, speech codec architectures, ASR, CLAP, and hierarchical modeling, HD-PPT achieves state-of-the-art performance in both instruction adherence and speech naturalness, significantly improving the accuracy and expressiveness of controllable TTS.

0 citationsRead paper

Recent publications

Latest Papers

ComplexSync: High-Fidelity and Real-Time Lip Sync in Complex Scenarios

Sep 24, 2026

This work addresses the limitations of diffusion-based lip synchronization models, specifically their degraded quality in complex scenes and high inference latency. To overcome these challenges, we propose a unified framework for high-fidelity, real-time generation. Methodologically, a dual-stream joint training mechanism is designed to prevent information leakage, while knowledge distillation is incorporated to accelerate single-step denoising. Furthermore, priors from vision foundation models and a relation alignment loss are introduced to enhance robustness. Experimental results demonstrate that the proposed method achieves an inference speed exceeding 70 FPS and attains state-of-the-art performance across both standard and complex scenarios. Additionally, a dedicated benchmark is constructed to facilitate evaluation. This study effectively advances the practical deployment of real-time lip synchronization technology.

0 citationsRead paper

High-Fidelity and Long-Duration Human Image Animation with Diffusion Transformer

Dec 26, 2025

Existing methods struggle to simultaneously preserve fine-grained facial and hand details while maintaining spatiotemporal consistency in long-duration (>3-second) human image animation. To address this, we propose a high-fidelity long-video generation framework based on the Diffusion Transformer (DiT). Our approach introduces three key innovations: (1) a novel hybrid implicit guidance signal combined with a sharpness-aware guidance factor; (2) a temporal-aware positional offset adaptation module enabling arbitrary-length video synthesis; and (3) skeleton-aligned modeling coupled with identity-agnostic data augmentation. These components collectively enhance fine-grained structural modeling and inter-frame coherence. Quantitative and qualitative evaluations demonstrate state-of-the-art performance in critical metrics—including facial expression fidelity, hand motion dynamics, and temporal smoothness—while achieving superior visual quality and strong spatiotemporal consistency.

0 citationsRead paper

Detecting Emotional Dynamic Trajectories: An Evaluation Framework for Emotional Support in Language Models

Nov 12, 2025

Existing LLM evaluation for emotional support relies predominantly on static, short-turn dialogues, failing to capture the dynamic evolution and longitudinal nature of human emotions. To address this, we propose the first evaluation framework explicitly designed for emotional dynamic trajectories. Our method introduces a first-order Markov emotional trajectory model, integrating psychologically grounded mechanisms—including contextual selection and cognitive reappraisal—to generate a large-scale benchmark comprising 328 emotion scenarios and 1,152 distractor events. We design a trajectory-level evaluation paradigm featuring constraints on emotion regulation strategies and causal adjustment for emotional state tracking. Furthermore, we introduce three novel metrics: BEL (Emotional Baseline Shift), ETV (Emotional Trajectory Variance), and ECP (Empathic Consistency Probability). Extensive evaluation across diverse LLMs demonstrates that our framework effectively discriminates long-term emotional support capabilities, yielding interpretable and actionable assessments of empathic interaction.

0 citationsRead paper

Playmate2: Training-Free Multi-Character Audio-Driven Animation via Diffusion Transformer with Reward Feedback

Oct 13, 2025

Existing audio-driven video generation methods suffer from critical limitations in lip-sync accuracy, long-term temporal coherence, and multi-character coordinated animation. To address these, we propose Mask-CFG—a training-free framework integrating positional offset inference, LoRA-based efficient fine-tuning, reward-guided optimization, and block-wise diffusion Transformer (DiT) inference—enabling high-fidelity, long-duration, natural dialogue video synthesis for arbitrary numbers of characters without architectural modification or domain-specific data. Our core innovation lies in decoupling character control from speech-driven motion via masked classifier-free guidance and dynamic reward calibration. This significantly improves lip-sync accuracy (+12.7% LSE), temporal consistency (+38.4% TCV), and multi-character motion naturalness. On multi-character benchmarks, Mask-CFG surpasses state-of-the-art methods while maintaining high fidelity, low inference cost, and strong generalization.

0 citationsRead paper

HD-PPT: Hierarchical Decoding of Content- and Prompt-Preference Tokens for Instruction-based TTS

Sep 23, 2025

Existing instructable TTS models suffer from a modality gap between coarse-grained text instructions and fine-grained speech tokens, hindering precise prosodic and phonetic control. To address this, we propose HD-PPT—a Hierarchical Dual-Preference Prompting and Tokenization framework. First, we design a novel hierarchical speech codec that disentangles content-related and instruction-related speech tokens. Second, we introduce a dual-preference token extraction mechanism jointly supervised by ASR and CLAP to align textual instructions with hierarchical speech representations. Third, we establish a layered decoding process enabling controllable generation across semantic, prosodic, and phonemic levels. By integrating large language models, speech codec architectures, ASR, CLAP, and hierarchical modeling, HD-PPT achieves state-of-the-art performance in both instruction adherence and speech naturalness, significantly improving the accuracy and expressiveness of controllable TTS.

0 citationsRead paper