When Does LeJEPA Learn a World Model?

Klindt, David, LeCun, Yann, Balestriero, Randall (May 2026) When Does LeJEPA Learn a World Model? arXiv. ISSN 2331-8422 (Submitted)

PDF 10.48550.arXiv.2605.26379.pdf - Submitted Version Available under License Creative Commons Attribution. Download (7MB)

Abstract

A representation that scrambles the true degrees of freedom of the world cannot support reliable planning or compositional generalization. We prove that LeJEPA (alignment plus Gaussian regularization) linearly recovers the world's latent variables from nonlinear observations, a property known as linear identifiability, in a broad class of worlds where latents evolve under stationary, additive-noise transitions. Our main result is that among all such worlds, the Gaussian is the unique latent distribution for which this guarantee holds. The forward direction rests on a spectral decomposition in which each degree of nonlinearity is strictly penalized by alignment, making the linear map the optimum; the converse rules out every non-Gaussian alternative. We further prove an approximate identifiability result where the guarantee degrades gracefully, and show that linear, orthogonal identifiability enables optimal latent-space planning. We validate the theory with experiments ranging from 2D examples to 1024-dimensional latents, including distributional ablations and pixel-based robotic control. Our theory turns an empirically successful recipe into a mathematical guarantee, providing the foundation for building World Models that provably recover the structure of the world.

Item Type:	Paper
Subjects:	bioinformatics
CSHL Authors:	Klindt, David
Communities:	CSHL labs > Klindt lab
SWORD Depositor:	CSHL Elements
Depositing User:	CSHL Elements
Date:	25 May 2026
Date Deposited:	28 May 2026 12:50
Last Modified:	28 May 2026 12:50
Related URLs:	Publisher
URI:	https://repository.cshl.edu/id/eprint/42215

Actions (login required)

Administrator's edit/view item