Neural Networks With Motivation.

Shuvaev, Sergey A, Tran, Ngoc B, Stephenson-Jones, Marcus, Li, Bo, Koulakov, Alexei A (January 2021) Neural Networks With Motivation. Frontiers in Systems Neuroscience, 14. p. 609316. ISSN 1662-5137

PDF 2021.Shuvaev.NeuralNetworksMotivation.pdf Download (3MB)

Abstract

Animals rely on internal motivational states to make decisions. The role of motivational salience in decision making is in early stages of mathematical understanding. Here, we propose a reinforcement learning framework that relies on neural networks to learn optimal ongoing behavior for dynamically changing motivation values. First, we show that neural networks implementing Q-learning with motivational salience can navigate in environment with dynamic rewards without adjustments in synaptic strengths when the needs of an agent shift. In this setting, our networks may display elements of addictive behaviors. Second, we use a similar framework in hierarchical manager-agent system to implement a reinforcement learning algorithm with motivation that both infers motivational states and behaves. Finally, we show that, when trained in the Pavlovian conditioning setting, the responses of the neurons in our model resemble previously published neuronal recordings in the ventral pallidum, a basal ganglia structure involved in motivated behaviors. We conclude that motivation allows Q-learning networks to quickly adapt their behavior to conditions when expected reward is modulated by agent's dynamic needs. Our approach addresses the algorithmic rationale of motivation and makes a step toward better interpretability of behavioral data via inference of motivational dynamics in the brain.

Item Type:	Paper
Subjects:	bioinformatics organism description > animal behavior > addiction bioinformatics > computational biology > algorithms organism description > animal behavior bioinformatics > computational biology organism description > animal behavior > learning bioinformatics > computational biology > algorithms > machine learning
CSHL Authors:	Shuvaev, Sergey Tran, Ngoc Stephenson-Jones, Marcus Koulakov, Alexei A. Li, Bo
Communities:	CSHL labs > Koulakov lab CSHL labs > Li lab School of Biological Sciences > Publications
SWORD Depositor:	CSHL Elements
Depositing User:	CSHL Elements
Date:	11 January 2021
Date Deposited:	06 May 2021 15:46
Last Modified:	25 Jan 2024 16:42
PMCID:	PMC7848953
URI:	https://repository.cshl.edu/id/eprint/40007

Actions (login required)

Administrator's edit/view item