Real-Time Avatars, Smarter Speech, and More Expressive Faces
Today's digest spans talking heads, full-body digital humans, and interactive speech systems. Highlights include text-guided head stylization, real-time avatar animation, relightable telepresence capture, and agentic ASR that revises transcripts in dialogue.
Overview of . % % From EgoRelight.
Talking Avatars & Talking Heads
CapTalk
CapTalk: Text-Guided Stylization and Speech-Driven 3D Head Animation
CapTalk is a framework for audio-driven 3D head animation that enables separate, text-guided control of speaking style and emotion. It generates real-time, expressive 3D facial animations with synchronized lip movements, adapting styles dynamically from natural language and audio inputs.
CogPortrait
CogPortrait: Fine-Grained Eye-Region Control in Portrait Animation via Hierarchical Agent Planning
CogPortrait provides fine-grained eye-region control in portrait animation from high-level labels, using multimodal large language model agents for detailed facial keypoint planning and a keypoint-driven video synthesis method, enabling precise and natural eye and head motion beyond typical emotion-driven methods.
Real-Time Full-Body Avatars & Digital Humans
EchoAvatar
EchoAvatar: Real-time Generative Avatar Animation from Audio Streams
EchoAvatar is a real-time system that generates continuous full-body avatar motion from streaming speech and music audio without explicit mode switching. It combines audio-driven motion synthesis with semantic control from large language models to enable expressive, interactive humanoid avatars.
DiscoForcing
DiscoForcing: A Unified Framework for Real-Time Audio-Driven Character Control with Diffusion Forcing
DiscoForcing is a real-time audio-driven system for full-body character control that operates with strict causality and low latency. It balances responsiveness with long-term stability under changing audio streams, enabling interactive avatar animation and humanoid robot control in real time.
EgoRelight
EgoRelight: Egocentric Human Capture and Illumination Recovery for Relightable and Photoreal Avatar Rendering
EgoRelight captures full-body motion and photoreal avatars with relightable appearance and HDR lighting from a single head-mounted display. It integrates egocentric stereo cameras and neural rendering to enable immersive telepresence with realistic motion and lighting, without studio setups at inference.
