Uncertainty-aware genomic deep learning with knowledge distillation

Zhou, Jessica, Rizzo, Kaeli, Tang, Ziqi, Koo, Peter K (November 2024) Uncertainty-aware genomic deep learning with knowledge distillation. bioRxiv. ISSN 2692-8205 (Public Dataset) (Submitted)

PDF 10.1101.2024.11.13.623485.pdf - Submitted Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (20MB)

Abstract

Deep neural networks (DNNs) have advanced predictive modeling for regulatory genomics, but challenges remain in ensuring the reliability of their predictions and understanding the key factors behind their decision making. Here we introduce DEGU (Distilling Ensembles for Genomic Uncertainty-aware models), a method that integrates ensemble learning and knowledge distillation to improve the robustness and explainability of DNN predictions. DEGU distills the predictions of an ensemble of DNNs into a single model, capturing both the average of the ensemble's predictions and the variability across them, with the latter representing epistemic (or model-based) uncertainty. DEGU also includes an optional auxiliary task to estimate aleatoric, or data-based, uncertainty by modeling variability across experimental replicates. By applying DEGU across various functional genomic prediction tasks, we demonstrate that DEGU-trained models inherit the performance benefits of ensembles in a single model, with improved generalization to out-of-distribution sequences and more consistent explanations of cis-regulatory mechanisms through attribution analysis. Moreover, DEGU-trained models provide calibrated uncertainty estimates, with conformal prediction offering coverage guarantees under minimal assumptions. Overall, DEGU paves the way for robust and trustworthy applications of deep learning in genomics research.

Item Type:	Paper
Subjects:	organs, tissues, organelles, cell types and functions > tissues types and functions > neural networks organs, tissues, organelles, cell types and functions organs, tissues, organelles, cell types and functions > tissues types and functions
CSHL Authors:	Tang, Ziqi Koo, Peter K Zhou, Jessica Rizzo, Kaeli
Communities:	CSHL labs > Koo Lab School of Biological Sciences > Publications
SWORD Depositor:	CSHL Elements
Depositing User:	CSHL Elements
Date:	15 November 2024
Date Deposited:	18 Nov 2024 15:53
Last Modified:	10 Dec 2024 13:50
Related URLs:	Publisher
Dataset ID:	https://zenodo.org/records/14145285
URI:	https://repository.cshl.edu/id/eprint/41736

Actions (login required)

Administrator's edit/view item