Ramezanali, M., Mitra, P. P., Sengupta, A. M.
(May 2019)
*Critical Behavior and Universality Classes for an Algorithmic Phase Transition in Sparse Reconstruction.*
Journal of Statistical Physics, 175 (3-4).
pp. 764-788.
ISSN 00224715 (ISSN)

## Abstract

Recovery of an N-dimensional, K-sparse solution x from an M-dimensional vector of measurements y for multivariate linear regression can be accomplished by minimizing a suitably penalized least-mean-square cost ||y-Hx||22+λV(x). Here H is a known matrix and V(x) is an algorithm-dependent sparsity-inducing penalty. For ‘random’ H, in the limit λ→ 0 and M, N, K→ ∞, keeping ρ= K/ N and α= M/ N fixed, exact recovery is possible for α past a critical value α c = α(ρ). Assuming x has iid entries, the critical curve exhibits some universality, in that its shape does not depend on the distribution of x. However, the algorithmic phase transition occurring at α= α c and associated universality classes remain ill-understood from a statistical physics perspective, i.e. in terms of scaling exponents near the critical curve. In this article, we analyze the mean-field equations for two algorithms, Basis Pursuit (V(x) = | | x| | 1 ) and Elastic Net (V(x)=||x||1+g2||x||22) and show that they belong to different universality classes in the sense of scaling exponents, with mean squared error (MSE) of the recovered vector scaling as λ43 and λ respectively, for small λ on the critical line. In the presence of additive noise, we find that, when α> α c , MSE is minimized at a non-zero value for λ, whereas at α= α c , MSE always increases with λ. © 2019, Springer Science+Business Media, LLC, part of Springer Nature.

Item Type: | Paper |
---|---|

Subjects: | bioinformatics |

CSHL Authors: | |

Communities: | CSHL labs > Mitra lab |

Depositing User: | Matthew Dunn |

Date: | 15 May 2019 |

Date Deposited: | 29 May 2019 19:38 |

Last Modified: | 29 May 2019 19:38 |

URI: | http://repository.cshl.edu/id/eprint/37984 |

### Actions (login required)

Administrator's edit/view item |