Compact deep neural network models of visual cortex

Cowley, Benjamin R, Stan, Patricia L, Pillow, Jonathan W, Smith, Matthew A (November 2023) Compact deep neural network models of visual cortex. bioRxiv. (Submitted)

PDF 2023_Cowley_Compact_deep_neural_network_preprint.pdf - Submitted Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (39MB)

Abstract

A powerful approach to understanding the computations carried out in visual cortex is to develop models that predict neural responses to arbitrary images. Deep neural network (DNN) models have worked remarkably well at predicting neural responses [1, 2, 3], yet their underlying computations remain buried in millions of parameters. Have we simply replaced one complicated system in vivo with another in silico? Here, we train a data-driven deep ensemble model that predicts macaque V4 responses ∼50% more accurately than currently-used task-driven DNN models. We then compress this deep ensemble to identify compact models that have 5,000x fewer parameters yet equivalent accuracy as the deep ensemble. We verified that the stimulus preferences of the compact models matched those of the real V4 neurons by measuring V4 responses to both ‘maximizing’ and adversarial images generated using compact models. We then analyzed the inner workings of the compact models and discovered a common circuit motif: Compact models share a similar set of filters in early stages of processing but then specialize by heavily consolidating this shared representation with a precise readout. This suggests that a V4 neuron’s stimulus preference is determined entirely by its consolidation step. To demonstrate this, we investigated the compression step of a dot-detecting compact model and found a set of simple computations that may be carried out by dot-selective V4 neurons. Overall, our work demonstrates that the DNN models currently used in computational neuroscience are needlessly large; our approach provides a new way forward for obtaining explainable, high-accuracy models of visual cortical neurons.

Item Type:	Paper
Subjects:	organs, tissues, organelles, cell types and functions > tissues types and functions > neural networks organs, tissues, organelles, cell types and functions organs, tissues, organelles, cell types and functions > tissues types and functions organs, tissues, organelles, cell types and functions > tissues types and functions > visual cortex
CSHL Authors:	Cowley, Benjamin
Communities:	CSHL labs > Cowley lab
SWORD Depositor:	CSHL Elements
Depositing User:	CSHL Elements
Date:	23 November 2023
Date Deposited:	20 Dec 2023 16:55
Last Modified:	20 Dec 2023 16:55
PMCID:	PMC10690296
Related URLs:	Publisher
URI:	https://repository.cshl.edu/id/eprint/41327

Actions (login required)

Administrator's edit/view item