Improving representations of genomic sequence motifs in convolutional networks with exponential activations

Koo, Peter K, Ploenzke, Matt (March 2021) Improving representations of genomic sequence motifs in convolutional networks with exponential activations. Nature Machine Intelligence, 3 (3). pp. 258-266. ISSN 2522-5839

DOI: 10.1038/s42256-020-00291-x


Deep convolutional neural networks (CNNs) trained on regulatory genomic sequences tend to build representations in a distributed manner, making it a challenge to extract learned features that are biologically meaningful, such as sequence motifs. Here we perform a comprehensive analysis of synthetic sequences to investigate the role that CNN activations have on model interpretability. We show that employing an exponential activation in the first layer filters consistently leads to interpretable and robust representations of motifs compared with other commonly used activations. Strikingly, we demonstrate that CNNs with better test performance do not necessarily imply more interpretable representations with attribution methods. We find that CNNs with exponential activations significantly improve the efficacy of recovering biologically meaningful representations with attribution methods. We demonstrate that these results generalize to real DNA sequences across several in vivo datasets. Together, this work demonstrates how a small modification to existing CNNs (that is, setting exponential activations in the first layer) can substantially improve the robustness and interpretabilty of learned representations directly in convolutional filters and indirectly with attribution methods.

Item Type: Paper
Subjects: bioinformatics
bioinformatics > computational biology > algorithms
bioinformatics > computational biology
bioinformatics > genomics and proteomics > genetics & nucleic acid processing > genomes
bioinformatics > computational biology > algorithms > machine learning
CSHL Authors:
Communities: CSHL labs > Koo Lab
CSHL Cancer Center Program
CSHL Cancer Center Program > Gene Regulation and Inheritance Program
SWORD Depositor: CSHL Elements
Depositing User: CSHL Elements
Date: March 2021
Date Deposited: 07 May 2021 20:10
Last Modified: 13 Feb 2024 18:39
PMCID: PMC8315445
Related URLs:

Actions (login required)

Administrator's edit/view item Administrator's edit/view item
CSHL HomeAbout CSHLResearchEducationNews & FeaturesCampus & Public EventsCareersGiving