Modern Video Generation Models (Sora, Veo, Kling, Wan) rely on 3D causal VAEs that compress both space and time (typically 4× temporal × 8× spatial), letting DiT backbones attend across multi-second clips without exhausting GPU memory. In ComfyUI, the VAE is loaded as an explicit modular node alongside model Checkpoints.
Latent Space & VAE
Also known as: VAE · Variational Autoencoder · 3D Causal VAE
Plain English
A learned compression layer that shrinks high-resolution images or videos into a compact numerical map (latent space) so generative models can work faster, then expands the result back into crisp pixels.
Technical Overview
In latent Diffusion Models, a Variational Autoencoder (VAE) encodes an H×W×3 image (or T×H×W×3 video via a 3D causal VAE) into a lower-dimensional continuous latent tensor—typically 8× spatially and 4× temporally compressed. Diffusion operates solely in this latent space; VAE channel count (e.g., 4-channel vs. 16-channel) directly governs fine text, hands, and texture fidelity.
Chronological Entries (2)
Upgrading from 4-channel VAEs (used in SD 1.5 and SDXL) to 16-channel VAEs in SD3 and FLUX dramatically reduced reconstruction artifacts around small typography, eyes, and fine fabric textures—proving that VAE quality caps the visual ceiling of any latent generator.
Direct Relations (3)
- EnablesDiffusion Models
- EnablesDiffusion Transformer (DiT)
- UsesComfyUI