Neural Networks With Motivation.

Shuvaev, Sergey A, Tran, Ngoc B, Stephenson-Jones, Marcus, Li, Bo, Koulakov, Alexei A (January 2021) Neural Networks With Motivation. Frontiers in Systems Neuroscience, 14. p. 609316. ISSN 1662-5137

[img] PDF
2021.Shuvaev.NeuralNetworksMotivation.pdf

Download (3MB)
URL: https://www.ncbi.nlm.nih.gov/pubmed/33536879
DOI: 10.3389/fnsys.2020.609316

Abstract

Animals rely on internal motivational states to make decisions. The role of motivational salience in decision making is in early stages of mathematical understanding. Here, we propose a reinforcement learning framework that relies on neural networks to learn optimal ongoing behavior for dynamically changing motivation values. First, we show that neural networks implementing Q-learning with motivational salience can navigate in environment with dynamic rewards without adjustments in synaptic strengths when the needs of an agent shift. In this setting, our networks may display elements of addictive behaviors. Second, we use a similar framework in hierarchical manager-agent system to implement a reinforcement learning algorithm with motivation that both infers motivational states and behaves. Finally, we show that, when trained in the Pavlovian conditioning setting, the responses of the neurons in our model resemble previously published neuronal recordings in the ventral pallidum, a basal ganglia structure involved in motivated behaviors. We conclude that motivation allows Q-learning networks to quickly adapt their behavior to conditions when expected reward is modulated by agent's dynamic needs. Our approach addresses the algorithmic rationale of motivation and makes a step toward better interpretability of behavioral data via inference of motivational dynamics in the brain.

Item Type: Paper
Subjects: organism description > animal behavior > addiction
bioinformatics > computational biology
organism description > animal behavior > learning
bioinformatics > computational biology > algorithms > machine learning
CSHL Authors:
Communities: CSHL labs > Koulakov lab
CSHL labs > Li lab
SWORD Depositor: CSHL Elements
Depositing User: CSHL Elements
Date: 11 January 2021
Date Deposited: 06 May 2021 15:46
Last Modified: 17 Jun 2021 19:46
PMCID: PMC7848953
URI: https://repository.cshl.edu/id/eprint/40007

Actions (login required)

Administrator's edit/view item Administrator's edit/view item
CSHL HomeAbout CSHLResearchEducationNews & FeaturesCampus & Public EventsCareersGiving