Context RL: Motivational states shape brain and behavior during learning
Imperative and interrogative motivational states lead to different choices, memories, and neural activity
Motivation influences goals, decisions, and memory formation. Imperative motivation links urgent goals to actions, narrowing the focus of attention and memory. Conversely, interrogative motivation integrates goals over time and space, supporting rich memory encoding for flexible future use.
We developed a paradigm to induce these motivational states by varying cover stories for a reinforcement learning task. In prior behavior studies, participants in the imperative group imagined executing a museum heist, whereas the interrogative group imagined planning a future heist. Participants repeatedly chose among four doors, representing different museum rooms, to sample trial-unique paintings with variable rewards (later converted to bonus payments). The next day, participants performed a surprise memory test. Crucially, only the cover stories differed between the imperative and interrogative groups; the reinforcement learning task was identical, and all participants had the same expectations about how and when bonus payments would be awarded.
We found that imperative motivation increased exploitation during reinforcement learning, leading to bigger bonuses. In congrast, interrogative motivation increased strategic exploration to learn more about uncertain options, and enhanced future memory for the paintings (peripheral details associated with reward), especially the high-value paintings . Overall, we demonstrate that these motivational states lead to a trade-off between immediate action and long-term associative memory formation.
In subsequent work, we conducted an fMRI study with this paradigm to test predictions about neural mechanisms. The theoretical framework proposes that imperative motivation is supported by noradrenergic modulation, originating in the locus coeruleus. In contrast, interrogative motivation is supported by dopaminergic modulation, originating in the ventral tegmental area. We found that these different motivational states are associated with activation in these key neuromodulatory regions, and the neural differences correspond to the behavioral differences (in choices and memory) we observed between groups. For preliminary fMRI findings, you can check out the SfN poster PDF below.
Project Contributors:
Allie Sinclair, University of Pennsylvania / Rice University
R. Alison Adcock, Duke University
Yuxi Candice Wang, Duke University
We developed a paradigm to induce these motivational states by varying cover stories for a reinforcement learning task. In prior behavior studies, participants in the imperative group imagined executing a museum heist, whereas the interrogative group imagined planning a future heist. Participants repeatedly chose among four doors, representing different museum rooms, to sample trial-unique paintings with variable rewards (later converted to bonus payments). The next day, participants performed a surprise memory test. Crucially, only the cover stories differed between the imperative and interrogative groups; the reinforcement learning task was identical, and all participants had the same expectations about how and when bonus payments would be awarded.
We found that imperative motivation increased exploitation during reinforcement learning, leading to bigger bonuses. In congrast, interrogative motivation increased strategic exploration to learn more about uncertain options, and enhanced future memory for the paintings (peripheral details associated with reward), especially the high-value paintings . Overall, we demonstrate that these motivational states lead to a trade-off between immediate action and long-term associative memory formation.
In subsequent work, we conducted an fMRI study with this paradigm to test predictions about neural mechanisms. The theoretical framework proposes that imperative motivation is supported by noradrenergic modulation, originating in the locus coeruleus. In contrast, interrogative motivation is supported by dopaminergic modulation, originating in the ventral tegmental area. We found that these different motivational states are associated with activation in these key neuromodulatory regions, and the neural differences correspond to the behavioral differences (in choices and memory) we observed between groups. For preliminary fMRI findings, you can check out the SfN poster PDF below.
Project Contributors:
Allie Sinclair, University of Pennsylvania / Rice University
R. Alison Adcock, Duke University
Yuxi Candice Wang, Duke University