SkyReels-V3

SkyReels-V3 Technique Report

SkyReels-V3 unifies reference image-to-video synthesis, video extension, and audio-driven talking-avatar generation into one model using multimodal in-context learning. It excels in identity preservation, temporal coherence, and audio-visual alignment across varied video generation tasks.

• multimodal
• talking-head
• avatar
• audio-driven
• lip-sync

Demos

These demos showcase SkyReels-V3's core strengths: multimodal video generation from reference images with strong identity and narrative consistency, video extension maintaining motion and scene coherence, and lifelike talking avatars driven by audio inputs. When evaluating, focus on the fidelity of character identity across frames, smoothness of video transitions and continuity, and natural synchronization of audio with avatar lip movements. The demos highlight SkyReels-V3's unified multimodal in-context learning capability across diverse video generation tasks.

Authors: Debang Li, Zhengcong Fei, Tuanhui Li, Yikun Dou, Zheng Chen, Jiangping Yang, Mingyuan Fan, Jingtao Xu, Jiahua Wang, Baoxuan Gu, Mingshan Chang, Wenjing Cai, Yuqiang Xie, Binjie Mao, Youqiang Zhang, Nuo Pang, Hao Zhang, Yuzhe Jin, Zhiheng Xu, Dixuan Lin, Guibin Chen, Yahui Zhou

Categories: cs.CV

Published 2026-01-24 · Updated 2026-01-29

Links

Paper & demos

• arXiv
• Project page

Code & resources

• GitHub500 stars7 watchers69 forksother
• Hugging Face171 downloads42 likes

Impact

• Semantic Scholar9 citations1 influential citation

Abstract

Video generation serves as a cornerstone for building world models, where multimodal contextual inference stands as the defining test of capability. In this end, we present SkyReels-V3, a conditional video generation model, built upon a unified multimodal in-context learning framework with diffusion Transformers. SkyReels-V3 model supports three core generative paradigms within a single architecture: reference images-to-video synthesis, video-to-video extension and audio-guided video generation. (i) reference images-to-video model is designed to produce high-fidelity videos with strong subject identity preservation, temporal coherence, and narrative consistency. To enhance reference adherence and compositional stability, we design a comprehensive data processing pipeline that leverages cross frame pairing, image editing, and semantic rewriting, effectively mitigating copy paste artifacts. During training, an image video hybrid strategy combined with multi-resolution joint optimization is employed to improve generalization and robustness across diverse scenarios. (ii) video extension model integrates spatio-temporal consistency modeling with large-scale video understanding, enabling both seamless single-shot continuation and intelligent multi-shot switching with professional cinematographic patterns. (iii) Talking avatar model supports minute-level audio-conditioned video generation by training first-and-last frame insertion patterns and reconstructing key-frame inference paradigms. On the basis of ensuring visual quality, synchronization of audio and videos has been optimized. Extensive evaluations demonstrate that SkyReels-V3 achieves state-of-the-art or near state-of-the-art performance on key metrics including visual quality, instruction following, and specific aspect metrics, approaching leading closed-source systems. Github: https://github.com/SkyworkAI/SkyReels-V3.

---

Introduction

SkyReels-V3 is presented as a unified conditional video generation framework built on a multimodal in-context learning paradigm with diffusion Transformers. The central idea is to use one model family to support three distinct but related generation tasks: reference images-to-video synthesis, video-to-video extension, and audio-guided talking-avatar generation. The paper frames video generation as a core capability for world models, emphasizing multimodal contextual inference as the real test of a system's generalization and control.

The contribution is not a single specialized generator, but a shared architecture and training strategy for multiple video-generation regimes. Across the three tasks, the paper repeatedly stresses three design goals: preserving identity and appearance when references are provided, maintaining spatiotemporal coherence over longer horizons, and aligning generated motion with textual or audio conditioning.

In the provided text, the paper does not introduce a new mathematical formulation with explicit losses or a detailed algorithmic derivation. Instead, it describes a system-level design composed of multimodal conditioning, latent fusion, specialized data construction, and task-specific training strategies.

Unified Model Overview

The model family is built around a diffusion Transformer backbone operating in a multimodal in-context learning setting. Inputs can include reference images, video clips, audio, and text, and the model is intended to use these inputs jointly rather than as isolated modalities. The shared framework is meant to bridge short-horizon synthesis and longer, narrative-level generation.

The paper positions this design as an advance over frame-level generation: instead of generating visually plausible snippets only, SkyReels-V3 aims to support coherent continuation, multi-subject composition, controlled instruction following, and audio-visual synchronization in a single system. The same architectural family is used for all three core tasks, but each task introduces its own data processing and conditioning strategy.

The high-level multimodal design can be summarized as follows: reference images are encoded with a video VAE and fused with video latents; video extension uses spatiotemporal consistency modeling plus shot-transition awareness; and talking-avatar generation uses audio-visual alignment strategies, region masking, and key-frame-constrained generation to support long-form synthesis.

Reference Images-to-Video Synthesis

In the reference image-to-video setting, SkyReels-V3 synthesizes temporally coherent videos from one to four reference images and a text prompt. The reference inputs can depict characters, objects, or background scenes. The model is intended to preserve identity attributes, spatial composition, and narrative continuity while following the semantic instruction in the prompt.

Reference-preserving data construction

The paper argues that the quality of reference-to-target training pairs is critical. To improve reference adherence while avoiding trivial copying, it proposes a dedicated pipeline built from in-house video data. The process begins by filtering a large internal dataset to select clips with both strong visual quality and significant motion.

Reference frames are then sampled from continuous video using a cross-frame pairing strategy, which is intended to keep semantic consistency while introducing temporal diversity. The authors additionally use image editing models to extract subject regions and complete backgrounds, and they apply semantic rewriting to create training pairs that reduce copy-and-paste artifacts. Additional filtering removes distorted or inconsistent reference images generated during editing.

The key design goal here is not just identity preservation, but also compositional stability. By constructing data where the reference is informative without being identical to the target frame, the model is trained to learn actual conditional generation rather than mere frame reproduction.

Multi-reference conditioning

To support heterogeneous inputs, SkyReels-V3 uses a unified multi-reference conditioning scheme. Each reference image is encoded using the video VAE, and the resulting latent representation is concatenated with the video latents. The model can accept up to four reference images, enabling flexible composition of multiple subjects and background elements without explicit manual layout engineering.

This is an important implementation detail because it suggests the model treats reference images as latent conditioning tokens that participate directly in the generative process, rather than as external retrieval cues. The paper presents this as a practical mechanism for multi-subject and multi-element control.

Training strategy

Training uses an image-video hybrid strategy that combines large-scale image and video datasets. The motivation is to expose the model to both static appearance cues and dynamic motion patterns so it can generalize better across diverse scenarios.

The paper also describes multi-resolution joint optimization, which is meant to improve robustness across spatial scales and aspect ratios. In practical terms, this lets the model support a wide range of output configurations without being tied to one canonical resolution or framing style.

Evaluation protocol and results

For reference image-to-video evaluation, the paper constructs a test set of 200 data pairs spanning film and television, e-commerce, and advertising scenarios. The reference images include characters, animals, objects, and background scenes. Evaluation is performed along three axes: Reference Consistency, Instruction Following, and Visual Quality.

Reference Consistency covers facial consistency, clothing consistency, object consistency, and background consistency. Visual Quality includes image quality, motion dynamics, aesthetic quality, and motion smoothness. The paper compares SkyReels-V3 against Vidu Q2, Kling 1.6, and PixVerse V5.

On this benchmark, SkyReels-V3 attains the best Reference Consistency and Visual Quality among the listed systems, while PixVerse V5 has the highest Instruction Following score. The paper interprets the result set as evidence that the model is strong in identity preservation and visual fidelity, even when the instruction-following metric is not the best in the comparison.

Video Extension

The video extension model is designed to take an existing video segment and generate a temporally coherent continuation under text guidance. The goal is to preserve motion dynamics, scene structure, and visual style while extending the narrative across longer time horizons.

Capabilities

The paper describes four main capabilities. First, the model supports dual extension modes: single-shot continuation and shot-switching extension. For shot switching, five predefined transition types are supported, and the mode can be selected manually or detected automatically. Second, the model aims to produce high-quality visual synthesis with stable composition, smooth motion, and seamless temporal continuity. Third, it preserves style cues from the input video so the output remains faithful to realistic, cinematic, or domain-specific aesthetics. Fourth, it supports 720p generation with adjustable extension lengths from 5 to 30 seconds for single-shot extension and multiple aspect ratios: 1:1, 3:4, 4:3, 16:9, and 9:16.

Data and modeling components

To make shot-aware video extension workable, the paper introduces a shot switching detector that analyzes long-form videos and identifies whether shot transitions are present, as well as classifying their types. The supported transition categories are single shot, cut-in, cut-out, multi-angle, shot/reverse shot, and cut-away. This detector is used to build effective training data.

The model also uses a unified multi-segment positional encoding together with hierarchical training. The paper presents this as a way to model motion across longer, multi-segment sequences while preserving smooth transitions and allowing the system to handle complex extension scenarios. In addition, the model emphasizes robust spatiotemporal modeling for rapid motion, multi-subject interaction, and abrupt scene changes.

The paper does not provide a quantitative benchmark table for video extension in the supplied text. The evidence reported here is qualitative, shown through representative examples of continuation and shot-switching extension. The main claim is that the model generalizes to cinematic content creation, short-form series production, game cutscenes, and long-form enhancement.

Talking Avatar Generation

The talking-avatar model generates video from a single portrait image and an audio clip. The output is intended to be long-form, visually realistic, and tightly synchronized to speech. The paper emphasizes minute-level generation, multi-character interaction, and support for diverse visual styles.

Targeted capabilities

The model is designed to deliver three key improvements. First, it supports high-fidelity visual synthesis and precise lip synchronization. The paper states that the system can generate 720p videos at 24 fps and that lip movement is aligned to phoneme-level dynamics. Second, it supports multi-style character generalization, including photorealistic humans, cartoons, animals, and stylized characters. Third, it supports long-form coherent generation, with minute-level output in a single forward pass while keeping identity consistency and motion continuity.

The model also supports multi-character scenes, which is important for dialogue, interviews, and other conversational settings. In such scenes, the paper notes that a mask must be used to specify which character is speaking. This explicit speaker specification is part of how the system handles turn-taking and coordinated speaker/listener behavior.

Training and inference design

SkyReels-V3 jointly analyzes audio, visual inputs, and textual cues to infer facial expressions, head movement, and camera dynamics. For synchronization, the paper describes dedicated audio-visual alignment strategies with region masking so the model explicitly learns the correspondence between speech units and facial motion. This is presented as a way to improve robustness across different languages, speaking styles, singing voices, and fast speech.

A second important component is a key-frame-constrained generation framework. Rather than generating the full sequence in an unconstrained way, the model first establishes structurally important key content and then generates smooth transitions between key frames. The intent is to improve temporal stability in long videos and prevent drift in identity or motion over minute-level durations.

The paper also refers to training with first-and-last frame insertion patterns and reconstructing key-frame inference paradigms. The supplied text does not give a formal algorithm, but the description indicates that key boundary frames are used to condition or stabilize long-horizon generation.

Evaluation results

The talking-avatar results are compared against OmniHuman 1.5, KlingAvatar, and HunyuanAvatar on three metrics: Audio-Visual Sync, Visual Quality, and Character Consistency. The paper states that these are internal evaluations against representative mainstream models.

On these numbers, SkyReels-V3 is competitive with the strongest baseline: it matches the best visual quality score, trails OmniHuman 1.5 slightly on audio-visual synchronization and character consistency, and substantially exceeds HunyuanAvatar across all three metrics. The paper's qualitative claim is that the model is particularly strong in stable, perceptually convincing talking-avatar generation.

Overall Assessment of the Method

The core novelty of SkyReels-V3 is the attempt to unify three different generation problems under a single multimodal in-context diffusion-Transformer framework. Rather than training isolated systems for each task, the paper organizes the model around shared latent conditioning, shared spatiotemporal modeling, and task-specific data pipelines.

For reference image-to-video, the main innovation is the pairing of cross-frame reference construction with editing and semantic rewriting to avoid copy-paste behavior while preserving identity. For video extension, the novelty is the incorporation of shot detection, multi-segment positional encoding, and hierarchical training to handle both continuation and shot changes. For talking avatars, the emphasis is on audio-visual alignment, masking, and key-frame-constrained synthesis to support minute-level output and multi-character dialogue.

The paper also makes a practical engineering claim: hybrid image-video training and multi-resolution optimization improve robustness and coverage of real-world use cases. This is consistent with the reported support for several aspect ratios, multiple scene types, and various domains such as film, e-commerce, advertising, live commerce, and entertainment.

Reported Limitations and Missing Details

The supplied paper text does not include a dedicated limitations section, nor does it report formal ablation studies, training hyperparameters, loss definitions, optimization schedules, dataset sizes, or compute costs. As a result, the summary of methodology is necessarily limited to what is explicitly described: system design, data processing strategy, qualitative examples, and the reported benchmark tables.

The extension model is evaluated primarily through qualitative figures in the provided text, so there is no direct numerical comparison for that capability in the excerpt. Likewise, the talking-avatar evaluation is described as internal rather than tied to a fully specified public benchmark protocol. These omissions do not invalidate the reported results, but they mean the paper emphasizes product-style capability demonstrations more than reproducible experimental detail in the provided source.

Conclusion

SkyReels-V3 is presented as a general-purpose multimodal video generation system that spans reference-guided synthesis, long-horizon continuation, and audio-driven avatar generation. The paper's main claim is that a single diffusion-Transformer-based framework, combined with careful conditioning and training design, can achieve strong identity preservation, coherent motion, and competitive audio-visual alignment across multiple tasks.

Based on the reported results, the model achieves the best reference consistency and visual quality in the reference image-to-video benchmark, while remaining highly competitive on talking-avatar metrics and demonstrating qualitatively strong video extension results. The overall framing is that SkyReels-V3 pushes video generation toward narrative-level, controllable, multimodal synthesis rather than isolated clip generation.