Verleyen, W., Ballouz, S., Gillis, J. (April 2016) Positive and negative forms of replicability in gene network analysis. Bioinformatics, 32 (7). pp. 1065-73. ISSN 1367-4811 (Electronic)1367-4803 (Linking)
Abstract
MOTIVATION: Gene networks have become a central tool in the analysis of genomic data but are widely regarded as hard to interpret. This has motivated a great deal of comparative evaluation and research into best practices. We explore the possibility that this may lead to overfitting in the field as a whole. RESULTS: We construct a model of 'research communities' sampling from real gene network data and machine learning methods to characterize performance trends. Our analysis reveals an important principle limiting the value of replication, namely that targeting it directly causes 'easy' or uninformative replication to dominate analyses. We find that when sampling across network data and algorithms with similar variability, the relationship between replicability and accuracy is positive (Spearman's correlation, rs ~ 0.33) but where no such constraint is imposed, the relationship becomes negative for a given gene function (rs ~ -0.13). We predict factors driving replicability in some prior analyses of gene networks and show that they are unconnected with the correctness of the original result, instead reflecting replicable biases. Without these biases, the original results also vanish replicably. We show these effects can occur quite far upstream in network data, and that there is a strong tendency within protein-protein interaction data for highly replicable interactions to be associated with poor quality-control. AVAILABILITY AND IMPLEMENTATION: Algorithms, network data, and a guide to the code available at: https://github.com/wimverleyen/AggregateGeneFunctionPrediction. CONTACT: jgillis@cshl.edu.
| Item Type: | Paper |
|---|---|
| Subjects: | bioinformatics > genomics and proteomics bioinformatics > computational biology bioinformatics > genomics and proteomics > genetics & nucleic acid processing > DNA, RNA structure, function, modification > genes, structure and function > gene network |
| CSHL Authors: | |
| Communities: | CSHL labs > Gillis Lab |
| Depositing User: | Matt Covey |
| Date: | 1 April 2016 |
| Date Deposited: | 23 Dec 2015 18:10 |
| Last Modified: | 09 May 2016 19:06 |
| Related URLs: | |
| URI: | https://repository.cshl.edu/id/eprint/32198 |
Actions (login required)
 |
Administrator's edit/view item |