By John Mikhael

Several dopamine-related conditions, such as attention-deficit/hyperactivity disorder, schizophrenia, and addiction, feature dysregulations of impulsive, risk-seeking, and explorative behaviors. But how dopamine actually shapes these behaviors has remained unclear. Consider, for example, the intertemporal choice task, where animals must choose between a small reward delivered soon and a large reward delivered later. This task is often taken to measure an animal’s impulsivity, or its preference for smaller/sooner rewards at the expense of larger/later rewards. Thus, manipulations that shift animal behavior in a systematic way can be interpreted as affecting an animal’s impulsivity. For example, if administering dopamine agonists makes animals more likely to select the smaller/sooner reward, then one may draw the conclusion that higher dopamine levels promote impulsive choices.

A rat chooses between a small/certain reward and a large/risky reward (St. Onge et al., 2010).

Though dopamine manipulations in this task have been studied in various labs and across species, the exact role of dopamine has remained elusive, as it seems to exert opposite effects with minor variations of the task. We ask here whether animals in this task act impulsively at all, or whether they simply seek to maximize their total accumulated rewards over the course of many trials. We hypothesize that an animal’s subjective estimates of reward magnitudes and delay durations ordinarily regress toward their contextual means because of the imprecision of stimulus encoding, and that dopamine reduces this imprecision. This causes the estimated reward magnitudes and delays to become more extreme, thereby increasing both the perceived benefit and cost of the larger/later reward. We show that, when reward magnitudes are encoded more imprecisely than delays, reward estimates will be more strongly subject to the effect of dopamine, and the net effect of dopamine agonists will be to favor larger/later rewards. On the other hand, dopamine has the opposite effect in conditions where the temporal imprecision dominates. We show that this framework captures dopamine’s seemingly contradictory behaviors in tasks that are taken to measure impulsivity, risk preferences, and the exploration-exploitation trade-off. Importantly, under our hypothesis, animals do not necessarily act in an impulsive, risk-seeking, or explorative way at all, and dopamine does not influence these behaviors, but rather gates the relative influence of context when making decisions.

John Mikhael is an MD-PhD candidate in the lab of Sam Gershman in the Department of Psychology at Harvard.

Learn more in the original research article:
Impulsivity and risk-seeking as Bayesian inference under dopaminergic control.
Mikhael JG, Gershman SJ. Neuropsychopharmacology. 2021 Aug 10. doi: 10.1038/s41386-021-01125-z. Epub ahead of print. PMID: 34376813.