Streaming Talking Heads & the Road to Photorealistic Digital Humans
Today's digest covers real-time talking-head generation at higher resolution without sacrificing latency, plus a sweeping survey of controllable 3D avatar creation from body priors to photorealistic animation.
Overview of different research fields in 3D Head Avatars. Representative images are taken from GaussianAvatars~, RGCA~, INSTA~, Cao~ ~, GPHM~, ROME~, HeadStudio~, Next3D~, and GaussianSpeech~. From Technical University of Munich.
Today's two papers zoom in on opposite ends of the digital human pipeline: one tackles the hard real-time constraints of streaming talking-head video, while the other steps back to map the entire landscape of controllable 3D avatar research. Together they paint a vivid picture of where the field stands in mid-2026 — both in shipping low-latency systems and in understanding the full complexity of building photorealistic human representations.
Talking Avatars & Real-Time Interaction
Pushing resolution without blowing up latency in streaming talking-head systems.
Alibaba's Wan-Streamer project returns with a v0.2 update that squarely addresses one of the most persistent tensions in live talking-head generation: can you render higher-fidelity video frames without paying a latency tax that breaks the real-time illusion? The answer here is a resounding yes, thanks to a clean architectural split. A single thinker path handles all perception and state management — the latency-sensitive work — while a performer path parallelizes visual generation across multiple GPUs. The result is 640×368 resolution output at a sustained ~200 ms end-to-end latency, a meaningful step up from prior lower-resolution streaming baselines while keeping the system usable for interactive applications.
Wan-Streamer v0.2: Higher Resolution, Same Latency
A streaming talking-head model that achieves 640×368 resolution while preserving ~200 ms latency through a split thinker-performer architecture. Visual generation is parallelized across multiple GPUs, while perception and state management remain on a single low-latency thinker path.
Abstract
We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker's language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.
Digital Humans & 3D Avatars
A field-wide map of how we get from raw priors to animatable, photorealistic people.
Knowing how to build a streaming talking head is useful, but understanding the broader ecosystem of digital human creation is essential context. Researchers at the Technical University of Munich have produced a comprehensive state-of-the-art report that does exactly that. The survey organizes the field into a clear taxonomy spanning full-body avatars, head avatars, and layered sub-components — hands, hair, and garments — tracing the arc from learning general priors, through personalized avatar generation, all the way to downstream animation. For practitioners trying to navigate a crowded literature, this is an invaluable orientation guide.
How to Build Digital Humans? From Priors to Photorealistic Avatars
A comprehensive state-of-the-art report surveying controllable 3D human avatar creation, from learning priors through personalized avatar generation to animation. It provides a taxonomy organizing methods across full-body avatars, head avatars, and layered components like hands, hair, and garments.
Abstract
This state-of-the-art report provides an overview of controllable 3D human avatar creation. We describe current 3D avatar systems, which typically consist of three stages: (i) learning priors of human appearance and motion, (ii) creating a personalized avatar, and (iii) animating the avatar. To limit the scope, we focus on the prior learning and avatar creation stages. We define current avatar representations and introduce a taxonomy that categorizes existing work along multiple axes, including body regions and employed priors. We review methods for full-body and head avatars, as well as layered representations that decompose the body into components such as hands, hair, and garments. Finally, we outline common underlying principles, reference key literature for newcomers, and discuss open challenges and future research directions.
Trending on Hugging Face
Vidu S1: A Real-Time Interactive Video Generation Model
Vidu S1 is a real-time interactive video generation model that supports voice-controlled digital character animation with infinite-length output and high frame rate on consumer hardware.
MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
MiniCPM-o 4.5 enables real-time full-duplex interaction where perception and response happen simultaneously across vision, audio, and speech. Its Omni-Flow framework aligns multimodal signals on a shared timeline, supporting both reactive and proactive behaviors in continuous, streaming conversation.
OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation
A novel GPT-based model, OmniFlatten, enables real-time natural full-duplex spoken dialogue through a multi-stage post-training technique that integrates speech and text without altering the original model's architecture.
Fish Audio S2 Technical Report
Fish Audio S2 is an open-source text-to-speech system with multi-speaker capabilities, multi-turn generation, and instruction-following control through natural-language descriptions, utilizing a multi-stage training approach and production-ready inference engine.
IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System
IndexTTS, an enhanced text-to-speech system combining XTTS and Tortoise models, offers improved naturalness, enhanced voice cloning, and controllable usage through hybrid character-pinyin modeling and optimized vector quantization.