Ramezanali, M., Mitra, P. P., Sengupta, A. M. (May 2019) Critical Behavior and Universality Classes for an Algorithmic Phase Transition in Sparse Reconstruction. Journal of Statistical Physics, 175 (3-4). pp. 764-788. ISSN 00224715 (ISSN)
Abstract
Recovery of an N-dimensional, K-sparse solution x from an M-dimensional vector of measurements y for multivariate linear regression can be accomplished by minimizing a suitably penalized least-mean-square cost ||y-Hx||22+λV(x). Here H is a known matrix and V(x) is an algorithm-dependent sparsity-inducing penalty. For ‘random’ H, in the limit λ→ 0 and M, N, K→ ∞, keeping ρ= K/ N and α= M/ N fixed, exact recovery is possible for α past a critical value α c = α(ρ). Assuming x has iid entries, the critical curve exhibits some universality, in that its shape does not depend on the distribution of x. However, the algorithmic phase transition occurring at α= α c and associated universality classes remain ill-understood from a statistical physics perspective, i.e. in terms of scaling exponents near the critical curve. In this article, we analyze the mean-field equations for two algorithms, Basis Pursuit (V(x) = | | x| | 1 ) and Elastic Net (V(x)=||x||1+g2||x||22) and show that they belong to different universality classes in the sense of scaling exponents, with mean squared error (MSE) of the recovered vector scaling as λ43 and λ respectively, for small λ on the critical line. In the presence of additive noise, we find that, when α> α c , MSE is minimized at a non-zero value for λ, whereas at α= α c , MSE always increases with λ. © 2019, Springer Science+Business Media, LLC, part of Springer Nature.
| Item Type: | Paper |
|---|---|
| Subjects: | bioinformatics |
| CSHL Authors: | |
| Communities: | CSHL labs > Mitra lab |
| Depositing User: | Matthew Dunn |
| Date: | 15 May 2019 |
| Date Deposited: | 29 May 2019 19:38 |
| Last Modified: | 29 May 2019 19:38 |
| URI: | https://repository.cshl.edu/id/eprint/37984 |
Actions (login required)
 |
Administrator's edit/view item |