Critical Behavior and Universality Classes for an Algorithmic Phase Transition in Sparse Reconstruction

Ramezanali, M., Mitra, P. P., Sengupta, A. M. (May 2019) Critical Behavior and Universality Classes for an Algorithmic Phase Transition in Sparse Reconstruction. Journal of Statistical Physics, 175 (3-4). pp. 764-788. ISSN 00224715 (ISSN)

DOI: 10.1007/s10955-019-02292-6


Recovery of an N-dimensional, K-sparse solution x from an M-dimensional vector of measurements y for multivariate linear regression can be accomplished by minimizing a suitably penalized least-mean-square cost ||y-Hx||22+λV(x). Here H is a known matrix and V(x) is an algorithm-dependent sparsity-inducing penalty. For ‘random’ H, in the limit λ→ 0 and M, N, K→ ∞, keeping ρ= K/ N and α= M/ N fixed, exact recovery is possible for α past a critical value α c = α(ρ). Assuming x has iid entries, the critical curve exhibits some universality, in that its shape does not depend on the distribution of x. However, the algorithmic phase transition occurring at α= α c and associated universality classes remain ill-understood from a statistical physics perspective, i.e. in terms of scaling exponents near the critical curve. In this article, we analyze the mean-field equations for two algorithms, Basis Pursuit (V(x) = | | x| | 1 ) and Elastic Net (V(x)=||x||1+g2||x||22) and show that they belong to different universality classes in the sense of scaling exponents, with mean squared error (MSE) of the recovered vector scaling as λ43 and λ respectively, for small λ on the critical line. In the presence of additive noise, we find that, when α> α c , MSE is minimized at a non-zero value for λ, whereas at α= α c , MSE always increases with λ. © 2019, Springer Science+Business Media, LLC, part of Springer Nature.

Item Type: Paper
Subjects: bioinformatics
CSHL Authors:
Communities: CSHL labs > Mitra lab
Depositing User: Matthew Dunn
Date: 15 May 2019
Date Deposited: 29 May 2019 19:38
Last Modified: 29 May 2019 19:38

Actions (login required)

Administrator's edit/view item Administrator's edit/view item
CSHL HomeAbout CSHLResearchEducationNews & FeaturesCampus & Public EventsCareersGiving