Drugs of abuse and impulsive choice

High levels of impulsive choice in drug users are evident on personality inventories, as well as on laboratory tests such as intertemporal choice (delay discounting) tasks (Bickel and Marsch, 2001; de Wit and Richards, 2003; Kirby and Petry, 2004; Heil et al., 2006; Reynolds, 2006). Delay discounting refers to the fact that rewards decrease in subjective value (i.e. - are discounted) if their delivery is delayed. Such discounting is observed in a range of species, including humans, in tasks in which subjects must choose between two alternatives – one which produces a small reward, and another which produces a large reward. When the delay to the two rewards is equal, subjects invariably choose the large reward. However, as the delay to the large reward is increased relative to the delay to the small reward, subjects begin to switch their preference to the smaller, more immediate reward, demonstrating that the value of the large reward has been reduced, or “discounted” relative to the small reward (Rachlin and Green, 1972; Ainslie, 1975; Mazur, 1987; Logue, 1988; Evenden and Ryan, 1996; Woolverton and Anderson, 2006). All subjects will discount future rewards in a delay-dependent manner to some degree; however, discounting of delayed rewards is considerably greater in drug users compared to non-users (Bickel and Marsch, 2001; Kirby and Petry, 2004; Reynolds, 2006). Importantly, abnormal preference for small immediate vs. large delayed rewards can also be caused by changes in sensitivity to reward magnitude (Ho et al., 1999). However, with few exceptions (Roesch et al., 2007) the effects of chronic drug administration specifically on reward sensitivity in decision-making tasks have not been examined.

It is well established that acute administration of a wide range of drugs of abuse can modulate impulsive choice in both human and animal subjects (although these effects have tended to be more robust in animal studies). Briefly, depressants such as ethanol and morphine tend to increase impulsive choice (i.e. – promote selection of small immediate over large delayed rewards), whereas the effects of stimulants such as caffeine, nicotine, and amphetamine have been more mixed, with reports of different effects on impulsive choice depending on the drug, task procedures (e.g. – the presence or absence of a reward-predictive cue during the delay period), and even baseline levels of impulsive choice (Cardinal et al., 2000; de Wit et al., 2002; Dallery and Locey, 2005; Pitts and McKinney, 2005; Olmstead et al., 2006; Winstanley et al., 2007; Barbelivien et al., 2008; Diller et al., 2008a; Pattij and Vanderschuren, 2008; Perry and Carroll, 2008; Stanis et al., 2008b). In cases of acute drug-induced increases in impulsive choice, such alterations have the potential to impair regulation of drug intake, rendering further immediate drug use more likely (de Wit, 2009). However, drug addiction (characterized by a desire to seek out and use drugs despite adverse consequences) can well outlast the period of acute intoxication and withdrawal. If prior drug use can increase impulsive choice even at long withdrawal time points, then such alterations could contribute to the addiction process by rendering individuals more likely to choose the immediate rewards of further drug use over the delayed but ultimately more beneficial rewards (such as health, employment, and family) of abstinence (Jentsch and Taylor, 1999; Bickel and Marsch, 2001; Monterosso et al., 2001; Bechara, 2005; Kalivas and Volkow, 2005; Olmstead, 2006), and could account in part for the disturbingly high relapse rates that follow even the best available addiction therapies (O'Brien, 2005). In the sections below, we review evidence that chronic exposure to stimulant drugs of abuse (particularly cocaine) can cause lasting alterations in impulsive choice in animal models, and discuss potential mechanisms of and treatment for such alterations.

Effects of chronic stimulant administration on impulsive choice

A number of studies have investigated the effects of chronic stimulant exposure on impulsive choice in rats, using intertemporal choice tasks that involve decisions between smaller sooner and larger delayed rewards. In one of the earliest of these studies, chronic administration of a high dose of methamphetamine (4 mg/kg daily for two weeks, 22 h prior to each test session) caused an increase in impulsive choice (i.e. – elevated preference for the smaller sooner reward) using a within-subjects design (Richards et al., 1999), an effect similar to that observed in human methamphetamine users (Monterosso et al., 2007; Hoffman et al., 2008). However, the extent to which this effect persisted following cessation of methamphetamine administration was not reported. The effects of chronic administration of a high dose of caffeine (30 mg/kg daily for at least 15 sessions, immediately prior to each test session) stand in contrast to those of chronic methamphetamine, in that chronic caffeine decreased impulsive choice (Diller et al., 2008b). However, because acute caffeine administration was also shown to decrease impulsive choice in the same study, it is not clear to what extent decreased impulsive choice during chronic administration was due to acute effects of caffeine.

The effects of chronic nicotine (0.3 or 1 mg/kg daily for over 65 sessions, immediately prior to each test session) were similar to those of methamphetamine in that treatment increased impulsive choice (Dallery and Locey, 2005). Importantly, the effects of chronic nicotine in this study were not solely due to acute effects of the drug (which also increased impulsive choice), as rats continued to display increased impulsive choice for at least a month after nicotine cessation, followed by a gradual return to pre-drug baseline. The fact that a regimen of chronic nicotine continued to promote choice of smaller sooner over larger delayed rewards well after the drug was removed suggests that nicotine-induced alterations in impulsive choice could contribute to relapse (e.g. – by promoting choice of smoking over the delayed but ultimately larger rewards of abstinence). In addition, the fact that impulsive choice eventually returned to pre-drug baseline, following nicotine cessation, accords with findings in human subjects that although levels of impulsive choice are higher in current smokers than in non-smokers, levels in ex-smokers are similar to those in non-smokers (Bickel et al., 1999; Mitchell, 1999).

A recent study by Stanis et al. (2008a) examined the effects on impulsive choice of a chronic regimen of D-amphetamine administration (3 mg/kg, i.p., every other day for 20 days, three weeks prior to the start of testing). In contrast to nicotine, there was no effect of chronic amphetamine on impulsive choice on an intertemporal choice task after amphetamine cessation, despite the fact that amphetamine-exposed rats showed robust psychomotor sensitization to amphetamine both before and after testing in the task. The absence of an effect of amphetamine in this study was somewhat surprising, given that this amphetamine regimen was shown to produce cognitive deficits in other behavioral tasks in rats (Belcher et al., 2005; Briand et al., 2005); however, these data are consistent with studies in humans in which amphetamine users display fewer cognitive deficits than chronic users of other drugs (particularly cocaine) (Ersche et al., 2008). In addition, the use of a reward-predictive cue during the delay prior to large reward delivery (illumination of a light above the large reward choice lever) may have altered the effects of amphetamine on task performance (Cardinal et al., 2000).

Effects of chronic cocaine administration on impulsive choice

Cocaine has been the most well studied drug of abuse in terms of its chronic effects on impulsive choice in animals, a focus that accords well with the human literature in which cocaine users show reliably greater impulsive choice compared to non-users (Coffey et al., 2003; Kirby and Petry, 2004; Bornovalova et al., 2005; Heil et al., 2006). In the earliest such study of which we are aware, Logue et al. (1992) used a within-subjects design to examine the effects of daily cocaine injections (15 mg/kg, i.p.) on impulsive choice using an adjusting delay procedure in rats, in which for each choice, the delay to the large reward was modified based on the rat's previous choices. Each rat received at least 10 cocaine injections, administered immediately prior to each test session, and behavior was assessed across a block of 5 sessions during which the mean adjusted delay (i.e. – the delay to the large reward that produced preference equivalent to that for the small reward) was stable. Cocaine caused an increase in impulsive choice (a decrease in the mean adjusted delay), relative to pre-cocaine baseline sessions in which rats received saline vehicle injections. However, because rats were given cocaine immediately prior to each test session, it is not clear whether this increased impulsive choice was due to acute or chronic effects of the drug. An additional feature of this study was that impulsive choice was shown to return to baseline (pre-cocaine) levels after cessation of cocaine injections, suggesting that there were no long-lasting effects of chronic cocaine administration. However, behavioral measures for the sessions immediately following cocaine cessation were not provided (only values for “stable” behavior – at least 10 sessions post-cocaine – were reported), and thus it is not clear whether cocaine withdrawal caused any temporary alterations in impulsive choice.

Paine et al. (2003) examined the effects of chronic cocaine administration on impulsive choice in rats using the fixed delays procedure developed by Evenden and Ryan (1996), in which each test session consists of five blocks of ten trials of choices between a small immediate and a large delayed reward, and in which the delay to the large reward increases across blocks (0, 10, 20, 40, 60 s) (Evenden and Ryan, 1996; Cardinal et al., 2000). Following pretraining on the task, rats received daily injections of cocaine (15 mg/kg, i.p., 3 times/day) or saline vehicle for 14 days. Behavioral testing continued during this period of drug administration, but took place prior to each day's injections, so as to avoid acute effects of the drug on behavior. Cocaine caused a small increase in impulsive choice (decreased choice of the large delayed reward in cocaine exposed rats compared to saline controls) that was statistically significant during the 7^th and 8^th day of cocaine administration. However, this effect appeared transient, as impulsive choice in cocaine exposed rats was no different from that in saline controls by the 14^th day of cocaine administration, suggesting that, at least during the period of drug administration, there were no lasting effects of cocaine.

Winstanley et al. (2007) employed an experimental design similar to that of Paine et al. (2003), in that following pretraining on the fixed delay task, they examined the effects of 3 weeks of daily injections of cocaine (15 mg/kg, i.p., 2 times/day) or saline vehicle on performance during this same time period (the injections were timed such that performance would not be disrupted by acute effects of the drug). In contrast to Paine et al. (2003) however, they found no effects of chronic cocaine administration on impulsive choice, either during the period of cocaine injections or during two weeks of drug-free testing thereafter. The reasons for this difference are not clear, although the lower cumulative dose used by Winstanley et al. (30 vs. 45 mg/kg/day) could account for their failure to find effects of cocaine on impulsive choice.

Two other recent studies have taken a somewhat different approach to examining the effects of chronic cocaine on impulsive choice, in that testing took place only at long time points (at least 3 weeks) following cocaine administration (Roesch et al., 2007; Simon et al., 2007a). This approach was motivated by observations that some of the consequences of chronic psychostimulant exposure (e.g. - psychomotor sensitization and cue-induced reinstatement) undergo an “incubation” period, in that they may become more robust with the passage of time (Paulson and Robinson, 1995; Lu et al., 2004), as well as studies of human cocaine abusers showing that increased impulsive choice can last well into abstinence (Heil et al., 2006). In both of these studies, rats were given daily injections of cocaine (30 mg/kg, i.p., 1 time/day) or saline vehicle for 14 days, followed by at least a 3-week withdrawal period during which the rats were undisturbed (this same cocaine administration and withdrawal regimen was found previously to produce deficits in reversal learning, reinforcer devaluation, water maze learning, and extinction of Pavlovian fear conditioning (Schoenbaum et al., 2004; Schoenbaum and Setlow, 2005; Burke et al., 2006; Mendez et al., 2008).

Following the withdrawal period, Roesch et al. (2007) trained the rats in a novel two-choice task in which reward magnitude and delay to reward delivery were manipulated independently. Both manipulations caused changes in choice preference (e.g. – an increase in the delay to reward delivery associated with one choice caused an increase in preference for the immediately available reward associated with the other choice), but cocaine-exposed rats changed their choice preference more rapidly than controls, suggesting that prior cocaine administration caused an increase in sensitivity to both reward delay and reward magnitude. How such increases in delay and reward sensitivity might affect reward preference in an intertemporal choice task may depend on the test parameters (delay duration and reward magnitude) used; however, these data are consistent with the idea that cocaine administration increases impulsive decision making (Ho et al., 1999; Roesch et al., 2007).

In the study by Simon et al. (2007a), impulsive choice was assessed after the three-week withdrawal period with the fixed delays procedure used by other investigators (Evenden and Ryan, 1996; Cardinal et al., 2000). Rats given the two week cocaine administration regimen described above exhibited decreased preference for the large delayed reward (i.e. – increased impulsive choice). This effect was evident 3 months after cocaine cessation (due to the long duration of training in the fixed delays procedure), suggesting that chronic cocaine-induced alterations in impulsive choice can be relatively stable. This long time point also suggests a possible reason for the small or negative effects of chronic cocaine observed in previous studies, in that there may be an “incubation period” for cocaine-induced increases in impulsive choice, such that they do not become robustly evident until relatively late withdrawal time points (Grimm et al., 2001). However, an alternative explanation for these differences is that, unlike rats in the Paine et al. and Winstanley et al. studies described above, rats in the Simon et al. study were not pretrained on the delay discounting task prior to cocaine administration. Pretraining may modulate the impact of neurobiological manipulations on delay discounting (Mobini et al., 2002; Winstanley et al., 2004), and thus this difference could account for the different results across these experiments.

Despite the success of the studies of Roesch et al. (2007) and Simon et al. (2007a) in modeling the elevated impulsive choice observed in human cocaine users, an important consideration regarding these findings is the fact that they used experimenter-administered rather than self-administered cocaine. There is ample evidence that the route of administration can have a large impact on neurobiological consequences of cocaine administration that could potentially influence impulsive choice (McFarland et al., 2003; Kalivas et al., 2009). Although there is some limited evidence from animal studies that experimenter- and self-administered cocaine are equivalent in terms of their long-term effects on cognition (Schoenbaum et al., 2004; Calu et al., 2007), the fact remains that self-administration is the best available animal model of patterns of human drug intake. With this issue in mind, we recently examined the effects on impulsive choice of a two-week period of intravenous cocaine self-administration, in rats with a daily intake of at least 30 mg/kg/day, followed by a three-week withdrawal period (Mendez et al., in preparation). As in the Simon et al. study, rats self-administering cocaine showed increased impulsive choice, compared to yoked saline controls, three months after cocaine cessation, suggesting that the dose of cocaine and/or duration of withdrawal is more important than the method of drug delivery in determining its long-term effects on impulsive choice. Interestingly, this same cocaine self-administration regimen did not appear to affect reward preference in a “probabilistic discounting” task (Mendez et al., in preparation), in which rats chose between a small guaranteed reward and a large reward for which the probability of delivery varied across blocks within a session (Cardinal and Howes, 2005; Floresco et al., 2008). Although these data come with the caveat that the rats were tested at a longer withdrawal time point (four months) than rats in the intertemporal choice task, they are consistent with the idea that delay and probabilistic decision making are mediated by distinct (though similar) mechanisms, and that they can be affected independently by behavioral and neurobiological manipulations (Cardinal, 2006; Weber and Huettel, 2008; Adriani et al., 2009; Simon et al., 2009b) – see Rachlin et al. (1991), Green et al. (1999), and Kheramin et al. (2003) for further discussion of these issues.

Neurobiological substrates of drug-induced alterations in impulsive choice

Neurobiological experiments in animals and neuroimaging studies in humans have implicated a network of limbic-striatal brain structures in regulation of decision making and impulsive choice (McClure et al., 2004; Cardinal, 2006; Winstanley et al., 2006a; Bezzina et al., 2008). This evidence is perhaps strongest for the ventral striatum (VS) and its dopaminergic innervation from the ventral tegmental area, and for the prefrontal cortex (PFC). Experimentally-induced lesions within both of these systems produce alterations (either increases or decreases) in impulsive choice (Cardinal et al., 2001; Kheramin et al., 2002; Mobini et al., 2002; Winstanley et al., 2004; Rudebeck et al., 2006; Bezzina et al., 2007), and impulsive choice behavior is correlated with neural activity as assessed in human imaging studies, and with electrophysiological and neurochemical activity in animal studies (McClure et al., 2004; Roesch et al., 2006; Winstanley et al., 2006b). Functional imaging studies in humans have been perhaps most revealing, suggesting that PFC (especially lateral PFC) is particularly important for choices of delayed rewards, whereas VS and its dopaminergic afferents (along with some portions of medial PFC) are particularly important for choices of immediate rewards. Indeed, it has been suggested that it is activity within these two brain systems that determines the balance between choices of delayed vs. immediate rewards, and that alterations in their relative influence could bias behavior toward increased preference for immediate over delayed rewards (Jentsch and Taylor, 1999; McClure et al., 2004; Bechara, 2005; Bickel et al., 2007). This idea is of particular relevance to the issue of drug-induced alterations in impulsive choice, as both PFC and VS/ventral tegmental area undergo robust structural and functional plasticity following chronic drug (particularly stimulant) use, suggesting that drug-induced alterations in these systems may account in part for the effects of chronic drug administration on impulsive choice (Nestler, 2004; Robinson and Kolb, 2004; Kalivas and Volkow, 2005).

Implications for pharmacological treatment of drug-induced alterations in impulsive choice

Since the inception of research on delay discounting in drug users, it has been suggested that the elevated impulsive choice observed in these individuals could promote continued drug use, and that targeting this form of impulsivity might serve to reduce drug use and decrease the likelihood of relapse (Perry and Carroll, 2008). Current pharmacological treatments for impulsivity (as a symptom of other conditions such as ADHD) include stimulant medications such as amphetamine and methylphenidate (Dalley et al., 2008). There are obvious concerns about the potential for abuse and diversion of such drugs in patients with a history of drug abuse (Vocci, 2008). However, aside from these concerns, there is some evidence from animal models that such drugs might actually be ineffective, at least in individuals with a history of cocaine use.

Winstanley et al. (2007) examined the effects of acute cocaine administration in rats tested on an intertemporal choice task, using the fixed delays procedure. Acute cocaine caused a decrease in impulsive choice in previously drug-naïve rats. In rats with a history of chronic cocaine administration, however, (15 mg/kg, i.p., 2 times/day, × 3 weeks), acute cocaine had no effect, suggesting that chronic cocaine rendered the rats tolerant to the cognitive effects of acute cocaine administration (Winstanley et al., 2007). The ability of prior chronic cocaine administration to prevent the decreased impulsive choice caused by acute cocaine was reproduced by viral vector-induced overexpression of the constituitively-active transcription factor delta-FosB in OFC. As increased delta-FosB expression in OFC can be caused by chronic cocaine administration, these data suggest that chronic cocaine-induced up-regulation of delta-FosB may cause tolerance to the cognitive effects of subsequent acute cocaine administration.

It is unlikely that cocaine would be used therapeutically to treat impulsive choice; however, a preliminary experiment from our laboratory suggests that chronic cocaine administration similarly blocks the effects of D-amphetamine on impulsive choice (Simon et al., 2007b). Rats were given i.p. injections of cocaine (20 mg/kg, i.p. × 14 days) or saline vehicle, and three weeks later began testing on an intertemporal choice task, using the fixed delays procedure. Several months after cessation of drug treatment, the effects of acute administration of amphetamine or saline vehicle on task performance were examined. Consistent with some previous findings (Wade et al., 2000; van Gaalen et al., 2006), acute D-amphetamine (1.0 mg/kg) decreased impulsive choice in cocaine-naïve saline control rats; however, the same dose had no effect on impulsive choice in the rats with a history of cocaine administration (Figure 1), suggesting that, as in the Winstanley et al. (2007) study, chronic cocaine attenuated the effects of subsequent acute stimulant administration on impulsive choice (see also Dalley et al., 2004). Interestingly, a similar phenomenon was observed following acute administration of the non-specific dopamine receptor antagonist flupenthixol (which increased impulsive choice in the cocaine-naïve but not chronic cocaine-exposed rats), suggesting that chronic cocaine administration results in a general reduction in sensitivity to dopaminergic manipulations of impulsive choice (Figure 1). It remains to be determined whether drugs targeting other neurotransmitter systems (e.g. - atomoxetine targeting the norepinephrine transporter: Robinson et al., 2007) are effective in cocaine-exposed subjects, but the results described above argue against therapeutic targeting of the dopaminergic system for elevated impulsive choice in (at least cocaine) addiction.

Summary and Future Directions

The data reviewed above demonstrate that chronic administration of several stimulant drugs of abuse can increase impulsive choice, as assessed in several varieties of intertemporal choice tasks in rats. These effects appear to be most pronounced and long-lasting following chronic cocaine administration, consistent with observations in human cocaine users (Coffey et al., 2003; Kirby and Petry, 2004; Bornovalova et al., 2005; Heil et al., 2006; Roesch et al., 2007; Simon et al., 2007a). However, the effects on impulsive choice of chronic administration of other stimulants have been considerably less well investigated, and it therefore remains to be determined whether apparent differences between the effects of cocaine and other stimulants are due simply to the doses and regimens tested or to true differences in the drugs’ actions. This issue highlights several important shortcomings of the current literature on the effects of chronic drug administration on impulsive choice. First, there has been little or no systematic comparison of the effects of different chronic drug doses and administration regimens on impulsive choice; therefore, even for cocaine, it is not clear to what factor(s) differences in the effects of chronic administration between studies should be attributed. Second, to our knowledge, there are no animal studies investigating the effects of chronic administration of non-stimulant drugs of abuse on impulsive choice; such research would appear to be important in light of the steep delay discounting observed in, for example, alcoholics and heroin users (Vuchinich and Simpson, 1998; Kirby et al., 1999). Third, there has been little consideration of the role of withdrawal duration on chronic drug effects on impulsive choice. A comparison of the chronic cocaine administration studies described above suggests that increased impulsive choice may be more evident at longer withdrawal times, in agreement with a recent study in humans suggesting that current cocaine use may mask cognitive deficits that become evident at longer withdrawal times (Ardila et al., 1991; Berry et al., 1993; Woicik et al., 2008; Volkow et al., 2009). However, the absence of systematic comparisons of different withdrawal times in animal studies renders this idea speculative at present.

In addition to these shortcomings of the existing literature described above, there are several other research areas surrounding chronic drug-induced alterations in impulsive choice that deserve future scrutiny. One concerns the relationship between pre-existing levels of impulsive choice and chronic drug effects on impulsive choice. Studies in human subjects have suggested that a pre-existing tendency to discount delayed rewards more steeply may predispose individuals toward drug use (Bechara, 2005), and studies in rats have shown that greater pre-existing levels of impulsive choice are associated with subsequent faster acquisition of cocaine self-administration and escalation of intake, one of the hallmarks of addictive behavior (Perry and Carroll, 2008). However, the additive effects on impulsive choice of such pre-existing differences combined with chronic drug administration are currently unknown (for example, does chronic cocaine administration selectively increase impulsive choice in some vulnerable subpopulations but not others?). Moreover, it is important to keep in mind that such correlational studies do not reveal whether high levels of impulsive choice actually cause an increase in drug use (e.g. – via preference for immediate drug rewards over the delayed rewards of abstinence), or whether high levels of impulsive choice are merely correlated, but bear no causal relationship with, predisposition for drug use. Another area of future research concerns the relationship between drug-induced alterations in impulsive choice and other aspects of decision-making. Little or nothing is known about how chronic drug administration influences the effects of reward probability, effort, or punishment on decision-making (Mitchell, 1999, 2004; Floresco and Whelan, 2009; Simon et al., 2009b; Mendez et al., in preparation).

Finally, despite an enormous body of literature on neurobiological changes induced by chronic administration of drugs of abuse (particularly in brain regions implicated in regulating impulsive choice), it remains difficult to link these changes directly with drug-induced alterations in impulsive choice, as the majority of these neurobiological changes have been examined with less intense drug administration regimens and/or at shorter withdrawal times than those shown to affect impulsive choice. Clearly, more research is needed in this and all of the areas described above, in order to understand the relationships between chronic drug use and elevated impulsive choice. Such research could have considerable potential for our understanding of the causes and treatment of drug addiction, as well as other conditions such as gambling disorders and ADHD in which altered impulsive choice plays a role (Reynolds, 2006; Winstanley et al., 2006a).