Moving beyond generalization to accurate interpretation of flexible models

Genkin, Mikhail, Engel, Tatiana A (October 2020) Moving beyond generalization to accurate interpretation of flexible models. NATURE MACHINE INTELLIGENCE, 2 (11). pp. 674-683. ISSN 2522-5839

DOI: 10.1038/s42256-020-00242-6

Abstract

© 2020, The Author(s), under exclusive licence to Springer Nature Limited. Machine learning optimizes flexible models to predict data. In scientific applications, there is a rising interest in interpreting these flexible models to derive hypotheses from data. However, it is unknown whether good data prediction guarantees the accurate interpretation of flexible models. Here, we test this connection using a flexible, yet intrinsically interpretable framework for modelling neural dynamics. We find that many models discovered during optimization predict data equally well, yet they fail to match the correct hypothesis. We develop an alternative approach that identifies models with correct interpretation by comparing model features across data samples to separate true features from noise. We illustrate our findings using recordings of spiking activity from the visual cortex of monkeys performing a fixation task. Our results reveal that good predictions cannot substitute for accurate interpretation of flexible models and offer a principled approach to identify models with correct interpretation.

Item Type: Paper
CSHL Authors:
Communities: CSHL labs > Engel lab
SWORD Depositor: CSHL Elements
Depositing User: CSHL Elements
Date: 26 October 2020
Date Deposited: 03 Feb 2021 16:48
Last Modified: 03 Feb 2021 16:48
Related URLs:
URI: https://repository.cshl.edu/id/eprint/39835

Actions (login required)

Administrator's edit/view item Administrator's edit/view item
CSHL HomeAbout CSHLResearchEducationNews & FeaturesCampus & Public EventsCareersGiving