Complete or Sparse: A Tale of Two Identifiabilities

Zhou, Junze, Klindt, David (October 2026) Complete or Sparse: A Tale of Two Identifiabilities. arXiv. ISSN 2331-8422 (Submitted)

PDF 10.48550.arXiv.2610.20851.pdf - Submitted Version Available under License Creative Commons Attribution. Download (2MB)

Abstract

Understanding when neural networks learn interpretable representations is central to mechanistic interpretability. We study this through identifiability: when can supervised learning recover the true latent variables underlying the data? We introduce two data generating processes distinguished by their causal direction where either labels cause latents (generative) or latents cause labels (discriminative). When the representation has at least as many dimensions as the latent space, we prove that the optimal encoders are linear in the latents. Conversely, when dimensions are insufficient, identifiability generically fails and encoders become nonlinear. We then show that additional constraints yield recovery of individual latent variables, not just linear mixtures. In the generative setting, non-negative activations and energy efficiency force each neuron to encode exactly one latent, providing a precise account of selective grandmother cell-like responses. In the discriminative setting, when latents are sparse and the true measurement matrix satisfies the spark condition, the learned encoder inherits compressed sensing structure, enabling sparse dictionary learning to perfectly recover individual latents. This constitutes, to our knowledge, the first identifiability result for undercomplete nonlinear independent component analysis. Our framework unifies nonlinear ICA, compressed sensing, and neural network interpretability, delineating exactly when representations transition from nonlinear distortions to linear representations and, under additional constraints or steps, to clean, axis-aligned codes. Theoretical predictions are validated on synthetic benchmarks and on activations drawn from vision models, large language models, and macaque inferotemporal cortex.

Item Type:	Paper
Subjects:	neurobiology
CSHL Authors:	Klindt, David Zhou, Junze
Communities:	CSHL labs > Klindt lab School of Biological Sciences > Publications
SWORD Depositor:	CSHL Elements
Depositing User:	CSHL Elements
Date:	14 October 2026
Date Deposited:	03 Nov 2026 16:41
Last Modified:	03 Nov 2026 16:41
URI:	https://repository.cshl.edu/id/eprint/42356

Actions (login required)

Administrator's edit/view item