Unification of field theory and maximum entropy methods for learning probability densities

Kinney, J. B. (September 2015) Unification of field theory and maximum entropy methods for learning probability densities. Physical Review E, 92 (3). ISSN 1539-3755

URL: https://www.ncbi.nlm.nih.gov/pubmed/26465426
DOI: 10.1103/PhysRevE.92.032107


The need to estimate smooth probability distributions (a.k.a. probability densities) from finite sampled data is ubiquitous in science. Many approaches to this problem have been described, but none is yet regarded as providing a definitive solution. Maximum entropy estimation and Bayesian field theory are two such approaches. Both have origins in statistical physics, but the relationship between them has remained unclear. Here I unify these two methods by showing that every maximum entropy density estimate can be recovered in the infinite smoothness limit of an appropriate Bayesian field theory. I also show that Bayesian field theory estimation can be performed without imposing any boundary conditions on candidate densities, and that the infinite smoothness limit of these theories recovers the most common types of maximum entropy estimates. Bayesian field theory thus provides a natural test of the maximum entropy null hypothesis and, furthermore, returns an alternative ( lower entropy) density estimate when the maximum entropy hypothesis is falsified. The computations necessary for this approach can be performed rapidly for one-dimensional data, and software for doing this is provided.

Item Type: Paper
Subjects: bioinformatics > computational biology > statistical analysis
CSHL Authors:
Communities: CSHL Cancer Center Program > Gene Regulation and Cell Proliferation
CSHL labs > Kinney lab
Depositing User: Matt Covey
Date: 9 September 2015
Date Deposited: 01 Oct 2015 21:24
Last Modified: 18 Jan 2017 16:51
Related URLs:
URI: https://repository.cshl.edu/id/eprint/31904

Actions (login required)

Administrator's edit/view item Administrator's edit/view item
CSHL HomeAbout CSHLResearchEducationNews & FeaturesCampus & Public EventsCareersGiving