Welcome to Learn World Models

Learn World Models is a project-driven world models curriculum covering VAE encoders and latent dynamics, Dreamer, TD-MPC, STORM, and frontier debates on language versus the physical world.

Suggested learning order: Follow the interleaved roadmap from L01 and L02 through P01 and P02, continue with L03 and P03/P04, then use L04 during P05 and P06 before finishing with L05.

Lectures

Lecture	Topic
Lecture 1	Internal Simulation and Historical Context
Lecture 2	Observation Encoding and Latent Dynamics
Lecture 3	Architecture Patterns, Learning Paradigms, and Planning
Lecture 4	Diagnosing World Models
Lecture 5	Frontier Debates

Projects

Build a VAE encoder, latent dynamics model, full Dreamer pipeline, TD-MPC planner, STORM evaluation dashboard, and counterfactual world model hands-on.

What You Will Learn

• Why world models are needed: historical context from Craik (1943) to JEPA.
• Observation encoding: VAE, CNN encoder, ELBO, and the bridge to Dreamer.
• Latent dynamics: GRU, MDN-RNN, and RSSM (deterministic + stochastic).
• Architecture families: nine architecture families, RNN/RSSM, Transformer, diffusion models, JEPA, RWM, spatial 3D/4D models, Genie, LoopWM, and WAM.
• Planning mechanisms: CEM-MPC, latent actor-critic, and TD-MPC.
• Evaluation metrics: FID, reward correlation, consistency loss, PSNR, and horizon drift.
• Frontier debates: language vs. the physical world, Bitter Lesson, AGI as a research target.