IndexMapAbout
ConceptVideoCross-Modal

World Models

Also known as: Interactive World Models · Genie · Dreamer

Plain English

AI models that learn an internal simulator of how an environment works—predicting future visual frames and physics in response to user actions so agents or players can interact in real time.

Technical Overview

World models learn compressed latent dynamics conditioned on actions, enabling policy training in imagination (Dreamer) or real-time interactive video simulation (Google DeepMind's Genie series). Modern generative world models combine spatiotemporal Transformers, diffusion, and autoregressive frame prediction with synchronized audio and low-latency streaming.

Timeline (4)

RSS

2018

  1. Paper

    Ha and Schmidhuber published World Models, pairing a VAE vision encoder with a recurrent dynamics predictor so reinforcement-learning agents could train policies entirely inside a learned latent dream environment.

2023

  1. Paper

    Hafner et al. introduced DreamerV3, a general-purpose model-based RL agent using a discrete-latent world model that mastered diverse domains—including collecting diamonds in Minecraft from scratch—with fixed hyperparameters.

2024

2026

See also (bkz.) (5)

Open full graph