Improving representations of genomic sequence motifs in convolutional networks with exponential activations.

Koo, Peter, Ploenzke, Matt (June 2020) Improving representations of genomic sequence motifs in convolutional networks with exponential activations. BioRxiv. (Unpublished)

PDF 2020.Koo.convolutional_networks.pdf Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (29MB)

Abstract

Deep convolutional neural networks (CNNs) trained on regulatory genomic sequences tend to build representations in a distributed manner, making it a challenge to extract learned features that are biologically meaningful, such as sequence motifs. Here we perform a comprehensive analysis on synthetic sequences to investigate the role that CNN activations have on model interpretability. We show that employing an exponential activation to first layer filters consistently leads to interpretable and robust representations of motifs compared to other commonly used activations. Strikingly, we demonstrate that CNNs with better test performance do not necessarily imply more interpretable representations with attribution methods. We find that CNNs with exponential activations significantly improve the efficacy of recovering biologically meaningful representations with attribution methods. We demonstrate these results generalise to real DNA sequences across several in vivo datasets. Together, this work demonstrates how a small modification to existing CNNs, i.e. setting exponential activations in the first layer, can significantly improve the robustness and interpretabilty of learned representations directly in convolutional filters and indirectly with attribution methods.

Item Type:	Paper
Subjects:	bioinformatics > genomics and proteomics bioinformatics > computational biology
CSHL Authors:	Koo, Peter K
Communities:	CSHL labs > Koo Lab
SWORD Depositor:	CSHL Elements
Depositing User:	CSHL Elements
Date:	15 June 2020
Date Deposited:	21 May 2021 20:51
Last Modified:	29 Apr 2024 15:44
URI:	https://repository.cshl.edu/id/eprint/40128

Actions (login required)

Administrator's edit/view item