Neural Memory for Reinforcement Learning and Temporal Credit Assignment
Date of Award
Spring 2023
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Neuroscience
First Advisor
Seo, Hyojung
Abstract
In this thesis, I describe results from three sets of experiments investigating the neural basis of memory used for decision-making and temporal credit assignment (TCA). In the first study, I ask whether mnemonic working memory representations are encoded in a similar way to mnemonic representations used for reinforcement learning. I perform novel analysis of previously collected data from dorsolateral prefrontal (DLPFC) and lateral intraparietal cortex (LIP), where the same neurons were recorded during multiple tasks. I find evidence for overlap of mnemonic encoding between working memory and decision-making. In the second study, I investigate the temporal dynamics of neural memory for TCA through delays. I develop a paradigm based on the matching pennies task in which variable delays occurred between choices and outcomes. I analyze neural activity from dorsomedial prefrontal cortex (DMPFC), DLPFC, and dorsal striatum. I find evidence that DLPFC may uniquely encode a neural eligibility trace signal through temporal delays, as choice-related activity was correlated with reduced behavioral credit assignment following longer delays in DLPFC only. In the third and final study, I present the lagged bandits paradigm, which was designed to study TCA under interference. I find that neither DMPFC nor dorsal striatum encodes the hypothesized dynamic eligibility trace which would be computationally optimal to solve the credit assignment problem despite interference from irrelevant actions and outcomes.
Recommended Citation
Murray, Shanna Kathleen, "Neural Memory for Reinforcement Learning and Temporal Credit Assignment" (2023). Yale Graduate School of Arts and Sciences Dissertations. 1228.
https://elischolar.library.yale.edu/gsas_dissertations/1228
