2.1. Our goals

The goal of this target article is to lay out a novel explanation for addiction as “vulnerabilities” in an established decision-making system. Although many of the vulnerabilities that we describe can be identified closely with current theories of addiction (see, e.g., Table 5), those theories have generally arisen from explanations of specific experiments and have all been attacked as incomplete. Our article is the first to identify them as “failure points” in a unified decision-making system. This theory has implications for the taxonomy of addiction, both drug-related and behavioral, as well as implications for prevention and treatment. These implications are addressed at the end of the article.

Although we do not directly address the social, environmental, or policy-level theories, we believe that our proposed framework will have implications for these viewpoints on addiction. For example, changes in drug price, taxes, legality, and level of policing can change the costs required to reach the addictive substance or behavior (Becker et al. 1994; Grossman & Chaloupka 1998; Liu et al. 1999). The presence of casinos can provide cues triggering learned associations (Dickerson & O’Connor 2006). Acceptability of use and punishments for use will affect the relationship between rewards and costs (Goldman et al. 1987; 1999). Genetics will shape the person’s vulnerabilities to the potential failure points noted further on and will have to be an important part of the individual’s treatment plan (Goldman et al. 2005; Hiroi & Agatsuma 2005).

Before proceeding to the implications of this theory, we first need to lay out the unified model of the decision-making system (sect. 3). As we go through the components of this system, we point out the identifiable vulnerabilities as they arise. In section 4, we then return to each identified vulnerability in turn and discuss the interactions between that vulnerability and specific drugs and problematic behaviors. In section 5, we discuss the implications of this theory for individual susceptibility to addiction, for multiple pathways to relapse, and for the necessity of making available multiple appropriately guided treatment regimens. In section 6, we turn to social, political, and clinical implications, lay out open questions, and suggest future directions for addiction research. Finally, we include an appendix reviewing the known effects of six drugs and problematic behaviors, discussed in the light of the vulnerabilities identified in this article (A: cocaine; B: opiates; C: nicotine; D: alcohol; E: caffeine; and F: gambling).

3.1. Transitions between decision systems

Behavior generally begins with flexible planning systems but, for repeated behaviors, can become driven by the less-flexible (but also less computationally expensive) habit systems. Examples of this development are well known from our experiences. For example, the first time we drive to a new job, we need a travel plan; we pay attention to street-signs and other landmarks. But after driving that same trip every day for years, the trip requires less and less attention, freeing up resources for other cognitive processes such as planning classes, papers, or dinner. The flexible system, however, generally remains available, as when road construction closes one’s primary route to work and one now needs to identify a new route. Errors can exist within both systems, as for example, a misjudged plan or a trip so automatic, that if one is not paying attention, one might accidentally find oneself having driven to work even though one planned to go somewhere else. This interaction is well-studied in the animal literature, including the overlaying of planning by habit systems (Dickinson 1980; Hikosaka et al. 1999; Packard & McGaugh 1996; Schmitzer-Torbert & Redish 2002), restoration of planning in the face of changes (Gray & McNaughton 2000; Isoda & Hikosaka 2007; Sakagami et al. 2006), and conflict between the two systems (Gold 2004; McDonald & White 1994; Packard 1999; Poldrack & Packard 2003; Redish et al. 2000).

Four well-studied examples in the animal literature are the transfer of place strategies to response strategies in the plus-maze (Chang & Gold 2004; Packard & McGaugh 1996; Yin & Knowlton 2004), the development of the regularity of behavioral paths (Barnes et al. 2005; Jog et al. 1999; Schmitzer-Torbert & Redish 2002), the disappearance of devaluation in animal learning studies (Adams & Dickinson 1981; Balleine & Dickinson 1998; Colwill & Rescorla 1985; Tang et al. 2007), and the inhibition of habitual responses in go/no-go tasks (Gray & McNaughton 2000; Husain et al. 2003; Isoda & Hikosaka 2007).

In the plus-maze, animals are trained to take an action that can be solved either by going to a specific place (Tolman et al. 1946) or by taking an action in response to being placed on the maze (Hull 1952). These algorithms can be differentiated by an appropriately designed probe trial (Barnes et al. 1980; Packard & McGaugh 1996; Restle 1957). Rats on this task (and on other similar tasks) first use a place strategy, which then evolves into a response strategy (McDonald & White 1994; Packard & McGaugh 1996; Yin & Knowlton 2004). Place strategies depend on hippocampal, as well as ventral and dorsomedial striatal integrity, while response strategies depend on dorsolateral striatal integrity (Packard & McGaugh 1996; Yin & Knowlton 2004; 2006; Yin et al. 2005).

In tasks in which animals are provided a general task with specific cases that change from day to day or session to session, animals can learn the specific instantiations very quickly. In these tasks, behavioral accuracy improves quickly, followed by a slower development of a regularity in the actions taken by the animal (rats, Jog et al. 1999; Schmitzer-Torbert & Redish 2002; monkeys, Hikosaka et al. 1999; Rand et al. 1998; 2000; humans, Nissen & Bullemer 1987; Willingham et al. 1989). In these tasks, the early (accurate, flexible, and slower) behavior is dependent on hippocampal integrity and correlated to hippocampal activity (Ferraro et al. 1993; Johnson & Redish 2007; Knopman & Nissen 1987), whereas later (also accurate, but inflexible and faster) behavior is dependent on dorsolateral striatal integrity and correlated to dorsolateral striatal activity (Barnes et al. 2005; Doyon et al. 1998; Hikosaka et al. 1998; Jackson et al. 1995; Jog et al. 1999; Knopman & Nissen 1991).

The implication of multiple decision-making systems on the calculation of value can also be seen in the effect of these two decision systems on changes in the valuation of a reward (Adams & Dickinson 1981; Balleine & Dickinson 1998; Colwill & Rescorla 1985; Dickinson 1980; 1985). Classically, these differences are measured by (1) training an agent to take an action (or a sequence of actions) to receive a reward R, and then, (2) changing the value of reward R to the agent, usually in a different context. The value of a reward can be changed by providing excess amounts of the reward (satiation, Balleine & Dickinson 1998) or by pairing the reward with an aversive stimulus, such as lithium chloride (devaluation, Adams & Dickinson 1981; Colwill & Rescorla 1985; Holland & Rescorla 1975; Holland & Straub 1979; Nelson & Killcross 2006; Schoenbaum et al. 2006a). Finally, (3) the agent is provided the chance to take the action. If the action-selection process takes into account the current value of the reward, then the agent will modify its actions in response to the change, but if the action-selection process is an association between the situation and the action (hence does not take into account the value of the reward), the agent will not modify its response. Lesions to ventral striatum (Corbit et al. 2001; Schoenbaum et al. 2006c) and prelimbic medial prefrontal cortex (Killcross & Coutureau 2003) or orbitofrontal cortex (Ostlund & Balleine 2007; Schoenbaum et al. 2006a; 2006b) discourage devaluation, whereas lesions to dorsolateral striatum (Yin et al. 2004; Yin & Knowlton 2004; Yin et al. 2006) and infralimbic cortex (Coutureau & Killcross 2003; Killcross & Coutureau 2003) encourage devaluation processes. Lesions to entorhinal cortex (Corbit & Balleine 2000) and dorsomedial striatum (Adams et al. 2001; Ragozzino et al. 2002a; 2002b; Yin et al. 2005) disrupt flexibility in the face of predictability changes (contingency degradation), whereas lesions to dorsolateral striatum do not (Yin & Knowlton 2006).

It is important to note that not all transitions need be from planning strategies to habit strategies. Because planning strategies are flexible and learned quickly, while habit-based strategies are more rigid and learned more slowly, many tasks are solved in their early stages through the planning system and in their late stages through the habit system (Dickinson 1980; Hikosaka et al. 1999; Packard & McGaugh 1996; Restle 1957). But the habit system can also learn in the absence of an available planning system (Cohen & Squire 1980; Day et al. 1999; Knowlton et al. 1994; Mishkin et al. 1984). Under appropriate conditions, well-developed automated responses can be overridden by controlled (planning-like) systems as in go/no-go tasks (Goldman et al. 1970; Gray & McNaughton 2000; Isoda & Hikosaka 2007; Iversen & Mishkin 1970) or reversal learning (Hirsh 1974; Mackintosh 1974; Ragozzino et al. 2002a). Which system drives behavior at which time depends on parameters of the specific task (Curran 1995; McDonald & White 1994; O’Keefe & Nadel 1978; Redish 1999) and may even differ between individuals under identical experimental conditions. In many cases, identical behaviors can be driven by the two systems, and only specialized probe trials can differentiate them (Barnes 1979; Curran 2001; Hikosaka et al. 1999).

3.2. The planning system

The planning system requires recognition of a situation and/or context S, identification of the consequences of taking action a in situation S (recognition of a means of achieving outcome O), and the evaluation of the value of achieving outcome O. This system selects the most appropriate action by considering the potential consequences of that action. The key behavioral parameters involved in this system are fast storage and slow retrieval. As noted earlier, retrieval within this system can be slow because the calculation of value at each step requires processing through the consideration of possibilities. Because the consideration of possibilities does not commit one to a single choice, this system is flexible in its behavioral choices. Because the value of taking action a in situation S is calculated from the value of achieving expected outcome O, which is calculated online from the current needs of the agent, if the desire (need) for the outcome is changed (even in another context), the value calculation can reflect that change.

Computationally, the planning system is likely to require three interacting components: a situation-recognition component, which classifies the complex interaction of contexts and stimuli to identify the situation in which the animal finds itself; a prediction component, which calculates the consequences of potential actions; and an evaluative component, which calculates the value of those consequences (taking into account the time, effort, and probability of receiving reward).

The situation-recognition component entails a categorization process, in which the set of available cues and contexts must be integrated with the agent’s memory so as to produce a classification of the situation. This system is likely to arise in cortical sensory and association systems through competitive learning (Arbib 1995; Grossberg 1976; Redish et al. 2007; Rumelhart & McClelland 1986). Mathematically, the cortical recognition system can be modeled with attractor network dynamics (Durstewitz et al. 1999; 2000; Kohonen 1984; Laing & Chow 2001; Redish 1999; Seamans & Yang 2004; Wilson & Cowan 1972; 1973), in which a partial pattern can be completed to form a remembered pattern through recurrent connections within the structure (Hebb 1949/2002; Hertz et al. 1991; Hopfield 1982). This content addressable memory provides a categorization process transforming the observed set of cues to a defined (remembered) situation that can be reasoned from (Redish et al. 2007).

The prediction component entails a prediction of the probability that the agent will reach situation s_t+1, given that it takes action a in situation s_t: P(s_t+1|s_t, a). This functionality has been suggested to lie in the hippocampus (Jensen & Lisman 1998; 2005; Johnson & Redish 2007; Koene et al. 2003) or frontal cortex (Daw et al. 2005). The hippocampus has been identified with stimulus-stimulus associations (Devenport 1979; 1980;Devenport & Holloway 1980; Hirsh 1974; Mackintosh 1974; White & McDonald 2002), episodic memory (Cohen & Eichenbaum 1993; Ferbinteanu & Shapiro 2003; Ferbinteanu et al. 2006; Squire 1987), flexible behavior (Devenport et al. 1981b; Gray & McNaughton 2000), including flexible navigation behavior (the cognitive map, i.e., spatial associations between stimuli; O’Keefe & Nadel 1978), as well as in sequence learning (Agster et al. 2002; Cohen & Eichenbaum 1993; Fortin et al. 2002; Levy 1996; Levy et al. 2005) (see Redish [1999] for review). Similar functionality has been proposed to lie in the frontal cortex (Daw et al. 2005), which has long been associated with the ability to recategorize situations (Baddeley 1986; Clark & Robbins 2002; Dalley et al. 2004; Isoda & Hikosaka 2007; Jentsch & Taylor 1999; Quirk et al. 2006; Rushworth et al. 2007) with the storage of delayed events (Baddeley 1986; Fuster 1997; Goldman-Rakic et al. 1990) and the anticipation of reward (Davis et al. 2006; Fuster 1997; Watanabe 2007), as well as with sequence planning (Averbeck & Lee 2007; Kolb 1990; Mushiake et al. 2006; Owen 1997).

The evaluative component allows the calculation of value with each predicted outcome. Anatomically, the evaluative component is likely to include the amygdala (Aggleton 1993; Dayan & Balleine 2002; Phelps & LeDoux 2005; Rodrigues et al. 2004; Schoenbaum et al. 2003), the ventral striatum (nucleus accumbens) (Daw 2003; Kelley 1999a; 1999b; Kelley & Berridge 2002; Mogenson 1984; Pennartz et al. 1994; Stefani & Moghaddam 2006; Wilson & Bowman 2005) and associated structures (Tindell et al. 2004; 2006), and/or the orbitofrontal cortex (Feierstein et al. 2006; Padoa-Schioppa & Assad 2006; Plassmann et al. 2007; Sakagami & Pan 2007; Schoenbaum et al. 2003; 2006a; Volkow et al. 2003). Neurons in the ventral striatum show reward correlates (Carelli 2002; Carelli et al. 2000; Carelli & Wondolowski 2003; Lavoie & Mizumori 1994; Martin & Ono 2000; Miyazaki et al. 1998) and anticipate predicted reward (Martin & Ono 2000; Miyazaki et al. 1998; Schultz et al. 1992; Yun et al. 2004). Neurons in the ventral pallidum are associated with the identification of hedonic signals (Tindell et al. 2004; 2006). Both the hippocampus and prefrontal cortex project to ventral striatum (Finch 1996; McGeorge & Faull 1989; Swanson 2000), and ventral striatal firing patterns reflect hippocampal and prefrontal neural activity (Goto & Grace 2005a; 2005b; Kalivas et al. 2005; Martin 2001; Pennartz et al. 2004). Neurons in the orbitofrontal cortex encode parameters relating the value of potential choices (Padoa-Schioppa & Assad 2006; Schoenbaum & Roesch 2005).

These structures all receive strong dopaminergic input from the ventral tegmental area. Neurophysiologically, dopamine signals in the ventral striatum – measured by neural recordings from dopaminergic projection neurons (Schultz 1998; 2002) and from voltammetry signals in the ventral striatum itself (Roitman et al. 2004; Stuber et al. 2005) – show increased firing to unexpected rewards and to unexpected cues predicting rewards. In computational models of the habit system, these signals have been hypothesized to carry value-prediction error information (see further on). Much of the data seems to support a similar role for dopamine from ventral tegmental sources (de la Fuente-Fernandez et al. 2002; Ljungberg et al. 1992; Roitman et al. 2004; Stuber et al. 2005; Ungless et al. 2004). However, detailed, anatomically instantiated computational models are not yet available for the planning system. Theories addressing dopamine’s role in the planning system have included motivation and effort (Berridge 2006; Berridge & Robinson 1998; 2003; Niv et al. 2007; Robbins & Everitt 2006; Salamone & Correa 2002; Salamone et al. 2005; 2007) and learning (Ikemoto & Panksepp 1999; Reynolds et al. 2001). An important open question, however, is to what extent dopamine is carrying the actual signal of motivation (Berridge 2007) and to what extent dopamine’s effects are dependent on corticostriatal synapses (Anagnostaras et al. 2002; Li et al. 2004; McFarland & Kalivas 2001; McFarland et al. 2003; Nicola & Malenka 1998; Reynolds & Wickens 2002). Finally, dopamine in the prefrontal cortex has also been hypothesized to have a role in controlling the depth of the categorization process (Durstewitz et al. 1999; 2000; Redish et al. 2007; Seamans et al. 2001; Seamans & Yang 2004; Tanaka 2002; 2006).

Neuropharmacologically, these systems, particularly the ventral striatum, are also highly dependent on mechanisms involving opioid signaling. Opioid signaling has been hypothesized to be involved in hedonic processes (Berridge & Robinson 1998; 2003; Kelley et al. 2002). Consistent with these ideas, Levine and colleagues (Arbisi et al. 1999; Levine & Billington 2004) report that opioid antagonists directly interfere with the reported qualia of hedonic pleasure in humans eating sweet liquids, without interfering in taste discrimination. We have suggested that the multiple opioid receptors in the mammalian brain (μ, κ, δ; De Vries & Shippenberg 2002; Herz 1997; 1998) are associated with an evaluation process identifying positive (euphorigenic, signaled by μ-opioid activation) and negative (dysphorigenic, signaled by κ-opioid signaling) evaluations (Redish & Johnson 2007). Whereas μ-receptor agonists are rewarding, euphorigenic, and support self-administration, κ-receptor agonists are aversive, dysphoric, and interfere with self-administration (Bals-Kubik et al. 1989; Chavkin et al. 1982; De Vries & Shippenberg 2002; Herz 1997, 1998; Kieffer 1999; Matthes et al. 1996; Meyer & Mirin 1979; Mucha & Herz 1985).⁶

We have also proposed that part of the evaluation mechanism occurring during the search process (calculating the expected value from the agent’s needs and the expected outcomes given a $S\stackrel{(a)}{\to }O$ relation) may also involve the opioid system (Redish & Johnson 2007). This would predict a release of μ-opioid agonists (e.g., enkephalins) in anticipation of extreme rewards. Rats placed in a drug-associated location show a dramatic increase in released enkephalin in the nucleus accumbens relative to being placed in a saline-associated compartment, presumably in anticipation arising from the drug-associated compartment (Mas-Nieto et al. 2002).⁷

3.3. The habit system

In contrast to the complexity of the planning system, the habit system entails a simple association between situation and action. Thus, the habit system requires recognition of a situation S, and a single, identified action to take within that situation. This simplicity allows the habit system to react quickly. However, this simplicity also makes the habit system rigid. A learned association essentially commits the agent to take action a in situation S. This means that it would be dangerous to store an association that was not reliable. Therefore, habit associations should only be stored after extensive experience with a consistent relation.

In contrast to the planning system, the habit system does not include any consideration of the potential outcome (i.e., there is no O term in the $S\stackrel{a}{\to }$ relation; Table 2). Therefore, in contrast to the planning system, the habit system does not include a prediction of available outcomes and cannot evaluate those potential outcomes online. Hence, it cannot take into account the current perceived needs (desires) of the agent. The habit system is still sensitive to the overall arousal levels of the agent. Thus, a hungry rat will run faster and work harder than a satiated rat (Bolles 1967; Munn 1950; Niv et al. 2007). However, because the habit system does not reflect the current desires of the agent, the habit system will not modify responses when a reward is devalued. Similarly, the habit system cannot select multiple actions in response to a single situation.⁹ This means that in navigation, the habit system can only take a single action in response to a given situation. For example, a rat with a damaged planning system cannot decide to turn left for water when it is thirsty on one day and turn right for food when it is hungry on the next day.¹⁰

Computationally, there are very good models of how the habit system might work. These models have generally been based on the temporal difference instantiation of reinforcement learning (Daw 2003; Daw et al. 2005; 2006; Dayan et al. 2000; Doya 2000b; Montague et al. 1996; Redish 2004; Schultz et al. 1997; Suri & Schultz 1999; Sutton & Barto 1998). In the simplest version of this model, each situation-action pair is associated with a value (termed Q(S, a); Sutton & Barto 1998).¹¹ When an agent takes an action from a situation, the agent can compare the expected value of taking action a in situation S (i.e., Q(S, a)) with the observed value (the reward received minus the cost spent plus the value of being in the state the agent ended up in):

where R(t) is the reward observed, C(t) the cost spent, max[Q(S_new, a)] the most value one can get from the situation the agent finds itself in (S_new), and Q(S_old, a) is the estimated value of taking action a in the situation the agent was in (S_old). By updating Q(S_old, a) by δ, the agent moves its estimate Q(S_old, a) closer to its true value; γ is a discounting parameter (γ < 1) which ensures that the time required to reach future rewards is taken into account (Daw 2003; Sutton & Barto 1998). These models can be trained to learn complex situation-action sequences.

The slow development of a habit association has been most studied in contrast to the fast planning system. Lesions or inactivation of the dorsolateral striatum (Yin & Knowlton 2004; Yin et al. 2004; Yin et al. 2006) and the infralimbic cortex (Coutureau & Killcross 2003; Killcross & Coutureau 2003) prevent the loss of devaluation with experience. Lesions or inactivation of the dorsolateral striatum (McDonald &White 1994; Packard & McGaugh 1992; 1996; Potegal 1972; White & McDonald 2002; Yin & Knowlton 2004) or the parietal cortices (DiMattia & Kesner 1988; Kesner et al. 1989) shift rats from response strategy to map strategies in navigation tasks.

As in the planning system, the habit system requires a situation-recognition component, in which the set of cues and contexts is integrated with the agent’s memory to classify the situation, to allow retrieval of the correct $S\stackrel{a}{\to }$ association. As with our discussion in the planning system, we suggest that cortical sensory and association systems classify situations through competitive learning processes (Arbib 1995; Grossberg 1976; Redish et al. 2007; Rumelhart & McClelland 1986). Although there are no neurophysiological data suggesting that this situation categorization system is anatomically separate from that used for the planning system, the identification of the habit system with networks involving dorsal and lateral aspects of striatum, and the planning system with networks involving more ventral and medial aspects of striatum, suggest that the specific cortical systems involved may differ. The $S\stackrel{a}{\to }$ association itself, including the mechanisms by which the situation signals are finally categorized to achieve a single action decision, have been hypothesized to include the afferent connections from cortex to dorsolateral striatum (Beiser et al. 1997; Houk et al. 1995; Samejima et al. 2005; Wickens 1993; Wickens et al. 2003).

Neuropharmacologically, the habit system receives strong dopamine inputs from the substantia nigra pars compacta (SNpc). The dopamine signal has been well studied in the primate (Bayer & Glimcher 2005; Ljungberg et al. 1992; Mirenowicz & Schultz 1994; Schultz 1998; 2002; Waelti et al. 2001). Like the dopamine neurons in the ventral tegmental area, dopamine neurons in SNpc increase firing in response to unexpected increases in expected value (via unexpected rewards or via cues that lead to an increased expectation of reward), and decrease firing in response to unexpected decreases in value (via lack of expected reward or via cues that lead to a decrease in the expectation of reward). This signal has been identified with the value-error signal δ in the temporal difference reinforcement learning algorithm (Barto 1995; Montague et al. 1995; 1996; Schultz et al. 1997), which can provide dopamine a role in training up $S\stackrel{a}{\to }$ associations. Dopamine has been shown to be critical to the learning of habitual $S\stackrel{a}{\to }$ associations (Faure et al. 2005). However, the role of dopamine in learning and performance is still controversial (Berridge 2007; Cagniard et al. 2006; Frank et al. 2004; Niv et al. 2007).

Cellularly, neurons in the dorsal striatum have been found to represent key parameters of the temporal difference reinforcement learning algorithm: for example, situation-action associations (Barnes et al. 2005; Carelli & West 1991; Daw 2003; Gardiner & Kitai 1992; Hikosaka et al. 1999; Hikosaka et al. 2006; Jog et al. 1999; Kermadi et al. 1993; Kermadi & Joseph 1995; Matsumoto et al. 1999; Miyachi et al. 1997; Schmitzer-Torbert & Redish 2004; Tremblay et al. 1998); reward delivery (Daw 2003; Schmitzer-Torbert & Redish 2004; White & Hiroi 1998;); and value signals (Daw 2003; Kawagoe et al. 2004; Nakahara et al. 2004). These signals develop in parallel with the development of automated behaviors (Barnes et al. 2005; Itoh et al. 2003; Jog et al. 1999; Samejima et al. 2005; Schmitzer-Torbert & Redish 2004; Tang et al. 2007), through an interaction with dopamine signals (Arbuthnott & Wickens 2007; Centonze et al. 1999; Picconi et al. 2003; Reynolds & Wickens 2002). Functional imaging data from humans playing sequential games show similar correlates to value, δ, and other parameters of these models (McClure et al. 2003; 2004; O’Doherty 2004; O’Doherty et al. 2004; Seymour et al. 2004; Tanaka et al. 2004a).

3.4. Interactions between planning and habit systems

Generally, the planning system is engaged early; but with experience, behavioral control in repetitive tasks is transferred to the habit system. This has been observed in the navigation (O’Keefe & Nadel 1978; Packard & McGaugh 1996; Redish 1999), animal conditioning (Balleine & Dickinson 1998; Dickinson 1985; Yin et al. 2006), and human learning (Jackson et al. 1995; Knopman & Nissen 1991; Poldrack et al. 2001) literatures. However, in tasks which require behavioral flexibility, behavioral control can remain with the planning system, even in highly trained animals (Gray & McNaughton 2000; Killcross & Coutureau 2003; McDonald & White 1994; Morris et al. 1982; White & McDonald 2002).

For many tasks, both systems can drive behavior in the absence of the other (Cohen & Squire 1980; Nadel 1994; O’Keefe & Nadel 1978; Squire 1987), but some tasks can only be solved by one system or the other. For example, the water maze requires a flexible response to reach a hidden platform and requires hippocampal integrity to be learned quickly (Morris et al. 1982; for a review, see Redish 1999). If the flexibility of the required response is decreased (by, for example, starting the animal in the same location each trial), then the hippocampus no longer becomes necessary to reach the platform (Eichenbaum et al. 1990). Other tasks, such as mirror-writing or the serial reaction time task, require the slow recognition of regularities in situation-action associations, and they are learned at similar speeds by patients with damaged and intact planning systems (Cohen & Squire 1980; Ferraro et al. 1993; Knopman & Nissen 1987). Patients with damaged lateral striatal systems are impaired on these habit-based tasks (Doyon et al. 1998; Ferraro et al. 1993; Knopman & Nissen 1991; Smith et al. 2000; Yin & Knowlton 2006). For some tasks, the planning system can “train up” the habit system, potentially through replay events occurring during subsequent sleep states (Buzsáki 1996; Hoffmann & McNaughton 2002; Marr 1971; Pavlides & Winson 1989; Redish 1999; Redish & Touretzky 1998; Wilson & McNaughton 1994). This transfer of information between systems can explain observations of incomplete retrograde amnesia with certain lesions (consolidation, Cohen & Squire 1980; Nadel & Bohbot 2001; Nadel & Moscovitch 1997; Redish 1999; Squire 1987) but predicts that “consolidated memories” will be less flexible than unconsolidated memories (Redish & Touretzky 1998).

The question of which system drives behavior when the two are put into conflict has only begun to be addressed computationally (Daw et al. 2005) and experimentally (Isoda & Hikosaka 2007), but there is a large literature on behavioral inhibition, in which a changed, novel, or potentially dangerous or costly behavior is inhibited (Gray & McNaughton 2000). This system seems to involve the prefrontal (Sakagami et al. 2006) and/or the hippocampal system (Gray & McNaughton 2000), depending on the specific conditions involved. Whether the interaction entails the planning system overriding a developed habit (Gray & McNaughton 2000) or an external system that mediates control between the two (Isoda & Hikosaka 2007) is still unresolved. Whether such an external mediator can be identified with executive control (presumably, in the prefrontal cortex, Baddeley 1986; Barkley 2001; Barkley et al. 2001) is still a matter of open research.

Vulnerability 8: Selective inhibition of the planning system

The habit system is inflexible, reacts quickly, “without thinking,” whereas the planning system is highly flexible and allows the consideration of possibilities. The habit and planning systems consist of different anatomical substrates. Pharmacological agents that preferentially impair function in structures involved in the planning system or preferentially enhance function in structures involved in the habit system would encourage the automation of behaviors. A shift back from habits to planned behaviors is known to involve the prefrontal cortex (Dalley et al. 2004; Husain et al. 2003; Isoda & Hikosaka 2007; Iversen & Mishkin 1970), and has been hypothesized to involve executive function (Barkley 2001; Barkley et al. 2001; Tomita et al. 1999). If pre-existing dysfunction exists within the inter-system control or if pharmacological agents disrupt this inter-system control, then the agent would develop habits quickly and would have difficulty disrupting those established habits. This vulnerability is distinguishable from the specific planning and habit vulnerabilities by its disruption of function of the planning system and/or its disruption of the inter-system conflict resolution mechanism. Thus, the other vulnerabilities affecting the planning system lead the planning system to make the incorrect choice; Vulnerability 8 makes it difficult for the planning system to correct a misguided habit system (Bechara 2005; Bechara et al. 2001; Bickel et al. 2007; Gray & McNaughton 2000; Jentsch & Taylor 1999; Lubman et al. 2004; Verdejo-Garcia et al. 2006).

3.5. Summary: Decision-making systems

The decision-making system in the mammal is hypothesized to consist of two subsystems: a planning system, based on the evaluation of potential possibilities (e.g., $S\stackrel{(a)}{\to }O$ relationships), and a habit system, based on the association of specific actions with specific situations (e.g., $S\stackrel{a}{\to }$ associations), both of which require a situation- recognition system, in which observed cues are categorized into situations (e.g., the S terms in the previous formulations). Correct decision-making depends on the integrity of each of these systems (see Figure 1).

3.6. Additional failure points

The eight vulnerabilities identified so far are certainly an incomplete list of the potential failure points of the decision-making system. The description of the decision-making system is, by necessity, incomplete. For example, we have not addressed the question of discounting and impulsivity. Nor have we addressed the question of learning rates.

Vulnerability 9: Over-fast discounting processes

Both the planning and habit systems need to take into account the probability and the delay before an expected goal will be achieved (Ainslie 1992; 2001; Mazur 2001; Stephens & Krebs 1987). In the planning system, this can be calculated online from the expected value of the expected goal given the searched sequence; in the habit system, this would have to be cached as part of the stored value function. The specific mechanism (and even the specific discounting function) are still a source of much controversy (for a review, see Redish & Kurth-Nelson [in press] in Madden et al. [in press]), but the long-term discounting of future rewards is well established. If an agent discounts too strongly, it will overemphasize near-term rewards and underemphasize far-future costs. Because addictions often involve near-term pleasures and far-future costs, faster-than-normal discounting could drive an agent to underestimate the far-future costs and to choose those near-term pleasures. A number of studies have found that addicts discount faster than non-addicts (Alessi & Petry 2003; Bickel & Marsch 2001; Kirby et al. 1999; Madden et al. 1997; Madden et al. 1999; Odum et al. 2002; Petry 2001; Petry & Bickel 1998; Petry et al. 1998; Vuchinich & Simpson 1998).

Vulnerability 10: Changes in learning processes

Other unincorporated components include changes in learning processes (such as over-attention to cues or learning rates being too high or too low). More detailed models of each of the systems will be required before it will be possible to make strong claims about the consequences of such a potential failure point.

However, the hypothesis put forward in this paper that addiction is a consequence of falling victim to vulnerabilities (failure modes) in the decision-making system lays out a research paradigm with important consequences both for what (and how) research questions should be addressed as well as for drug-treatment paradigms and drug-control policies.

These vulnerabilities in the decision-making system may arise from individual predisposition (either due to genetic or social/environmental factors) as well as from drug- or behavior-driven interactions with the decision system. In the second half of this article, we address the interactions and implications of each of the previously identified vulnerabilities with drugs and behaviors of abuse as well as the policy and treatment consequences of the theory.

4.1. Vulnerability 1: Deviations from homeostasis

A classic example of deviations from homeostasis that will produce changing needs is the well-known “crash” after the euphoria of an opiate experience (Koob & Le Moal 2006). These negative effects can occur after even a single dose of morphine (Azolosa et al. 1994; Harris & Gewirtz 2005; Koob & Le Moal 2006). Such a negative affect would drive an agent to attempt to compensate by returning to the positive qualia occurring during the drug use. Deviations from homeostasis also lead to the well-known withdrawal symptoms (Altman et al. 1996; Lowinson et al. 1997) seen in reaction to nicotine (Benowitz 1996; Hanson et al. 2003; Hughes & Hatsukami 1986), alcohol (Kiefer & Mann 2005; Littleton 1998; Moak & Anton 1999), opiates (Altman et al. 1996; Koob & Bloom 1988; Schulteis et al. 1997), and caffeine (Daly & Fredholm 1998; Evans 1998) addictions.

4.2. Vulnerability 2: Changes in allostatic set-points

Drug use, particularly repeated drug use, can also produce changes in the set-point itself (referred to as “changes in allostasis,” most likely through long-term changes in receptor levels and changes in levels of endogenous ligands released during normal behaviors; Koob & Le Moal 2006). Animals given prolonged access to drugs, particularly access over many days to long periods of drug availability, develop greatly increased drug-intake levels (Ahmed & Koob 1998; 1999). This has been hypothesized to arise from developing allostatic changes (Ahmed & Koob 2005; Koob & Le Moal 2006).

Pharmacologically, chronic nicotine use changes levels of cholinergic receptors in the brain (Flores et al. 1997; Marks et al. 1992). Chronic alcohol use changes function and expression of gamma-aminobutyric acid (GABA_A) and N-methyl-D-aspartate (NMDA) receptors (Hunt 1998; Littleton 1998; Valenzuela & Harris 1997). Repeated cocaine (Hurd & Herkenham 1993; Steiner & Gerfen 1998), alcohol (Ciraulo et al. 2003), and opiate (Cappendijk et al. 1999; Weissman & Zamir 1987) treatment all produce changes in endogenous opioid release and in opiate receptor expression. Many smokers titrate the number of cigarettes smoked throughout the day, ensuring a relatively constant blood-plasma level of nicotine (Schmitz et al. 1997).

These neurobiological changes change the identified needs of the agent, and thus imply changes in the evaluation of expected outcomes of drug-taking (or abstinence), which will change action-selection in the planning system.¹² This vulnerability is identifiable by changes in long-term set-points of physiological variables.

4.3. Vulnerability 3: Overvaluation of the expected value of a predicted outcome – mimicking reward

The planning system requires a signal that directly evaluates the successful achievement of a perceived need (thus leading to the qualia of pleasure). A number of authors have suggested that this may reside within the opiate system (Berridge & Robinson 2003; Redish & Johnson 2007). μ-Opiate agonists (such as heroin, morphine, etc.) are generally highly euphorigenic (Jaffe et al. 1997; Mark et al. 2001; Meyer & Mirin 1979). Even though exogenously delivered μ-opiate agonists (such as heroin or morphine) are not a true reward that the system evolved to recognize, they can mimic the reward system and trick the system into believing that it just received a strong reward, which it will learn to return to. Drugs accessing this vulnerability are likely to be highly euphorigenic, particularly with initial use. Heroin and morphine produce profound euphoria very quickly after injection (Koob & Le Moal 2006). This reward signal will be stored in memories associated with the planning system, which would lead to the recall of highly euphoric signals when the planning system recognizes a path to achieve these reward-mimicking drugs. This vulnerability is recognizable by strong craving when agents recall those euphoric events.

4.4. Vulnerability 4: Overvaluation in the planning system

As reviewed earlier, the planning system consists of recognition (memory), search through, and evaluation of $S\stackrel{(a)}{\to }O$ relationships. A fundamental vulnerability of this relationship is in the valuation of the outcome, which is calculated from the level of “need” and the “value” of the outcome satisfying that perceived need, presumably learned through dopaminergic signals (Robinson & Berridge 1993; 2001; 2003; 2004) projecting to the ventral striatum and orbitofrontal cortex. Dopamine firing patterns in the ventral tegmental area (projecting to the ventral striatum and orbitofrontal cortex) indicate changes in the amount of expected or just-received reward (Pan et al. 2005; Roesch et al. 2007; Schultz 2002; Schultz et al. 1997), analogous to the δ signal required for the Q-learning algorithm described in the habit system. Although no computational theories are as yet available describing how these dopaminergic signals translate into changes in evaluation in the planning system, voltammetry recordings from the ventral striatum have shown dopamine signals occurring before both cued and self-initiated actions leading to drug receipt (Phillips et al. 2003; Roitman et al. 2004; Stuber et al. 2004; 2005). These changes presumably modulate the cortico-and hippocampo-ventral-striatal synapses, both during learning (Thomas et al. 2001) and during performance (Goto & Grace 2005a; 2005b; Lisman & Grace 2005; Yun et al. 2004).

Other researchers have suggested that this evaluation process may arise in the orbitofrontal cortex (Padoa-Schioppa & Assad 2006; Plassmann et al. 2007; Schoenbaum et al. 2006a) and that overvaluation in the orbito-frontal cortex can lead to overvaluation of expected rewards (Kalivas & Volkow 2005; Volkow et al. 2003). In rats with a past history of cocaine intake, the orbitofrontal cortex becomes less capable of predicting adverse outcomes than in normal rats (Stalnaker et al. 2006), implying a potential difficulty in identifying negative consequences. Nevertheless, an overvaluation of expected drug outcomes would produce craving (Redish & Johnson 2007) and an increased likelihood of taking actions leading to those expected drug outcomes (German & Fields 2007a).

4.5. Vulnerability 5: Incorrect search of $S\stackrel{(a)}{\to }O$ relationships

As noted earlier, the prediction component of the planning system is also a memory process, requiring the exploration of multiple consequences from a given situation S. This prediction process has been suggested to require the hippocampus (Jensen & Lisman 1998; 2005; Johnson & Redish 2007) and the prefrontal cortex (Daw et al. 2005), but the specific mechanism is still unknown.

Although the specific mechanisms of storage and access of $S\stackrel{(a)}{\to }O$ relationships are unknown within the hippocampus and prefrontal cortex, Vulnerability 5 can occur due to maladaptive and often subtle modifications of system storage and access functions rather than system failure. These systemic modifications may occur as a result of changes in the cell morphology within these areas and plasticity mechanisms within the hippocampus and prefrontal cortex. A number of addictive substances produce such changes. Both hippocampus and prefrontal cortex receive dopaminergic inputs, which are known to change the sensitivity to synaptic plasticity (Huang et al. 1995; Seamans & Yang 2004) and to modulate representations (Kentros et al. 2004; Seamans & Yang 2004) and performance (Arnsten et al. 1994; Murphy et al. 1996). Similarly, morphine and other opiate agonists increase synaptic spine formation in cell culture (Liao et al. 2005) and development (Hauser et al. 1987; 1989). Long-term drug exposure also increases the spine formation in the hippocampus and prefrontal cortex in vivo (Robinson & Kolb 1999; Robinson et al. 2001; 2002). These changes may affect the prediction process, possibly driving it to preferentially search the drug-related potential choices, which would appear clinically as an over-sensitivity to drug-related cues and obsessive consideration of choices leading to drug receipt.

4.6. Vulnerability 6: The illusion of control

When faced with reward distributions that change over time, agents can react in one of two ways: The agent can identify itself as being in a new situation with a different reward distribution, or the agent can identify itself as being in the same situation, but change the expectation of the likelihood of receiving reward. If the agent incorrectly classifies the same situation as different or different situations as the same, the agent may find itself making incorrect decisions.

Misclassification of situations has been primarily identified as a potential cause for problem gambling, in which agents incorrectly identify a statistically unlikely sequence of wins as a separate situation from more-commonly experienced losses (Custer 1984; Langer & Roth 1975; Redish et al. 2007; Wagenaar 1988). This provides the agent with the illusion that certain cues can identify winning situations while other cues identify losing situations (referred to as the “illusion of control”; Langer & Roth 1975; Redish et al. 2007; Sylvain et al. 1997; Wagenaar 1988). Problem gamblers tend to have experienced a statistically unlikely sequence of wins followed by devastating losses (Custer 1984; Wagenaar 1988). This misclassification may arise from excessive recognition of cue changes between the winning and losing experiences (Redish et al. 2007). Problem gamblers are often observed to “explain away” losses by post hoc identification of differences in cues between the losses and memories of wins (also referred to as “hindsight bias”; Custer 1984; Dickerson & O’Connor 2006; Wagenaar 1988). Similarly, near misses, in which gamblers lose but come close to the winning situation, encourage continued play (Parke & Griffiths 2004). These near misses may provide illusory support for the hypothesis that certain noisy cues have a relationship to the predictability of the reward.

4.7. Vulnerability 7: Overvaluation in the habit system

In the habit ($S\stackrel{a}{\to }$) system, phasic (bursting) dopamine signals are correlated with the value-prediction error signal δ, needed by the temporal difference reinforcement learning algorithm to learn situation-action sequences (Barto 1995; Daw 2003; Montague et al. 1995; 1996). With natural rewards, as the value-prediction system learns to predict those rewards correctly, the value-prediction compensates for the reward, and dopamine at the time of correctly predicted reward decreases to zero with learning (Schultz 1998). Drugs that produce dopamine neuropharmacologically (like cocaine or amphetamine) will bypass that value-compensation system, providing a constant “better-than-expected” δ signal. This non-compensable dopamine signal leads to overvaluation in the $S\stackrel{a}{\to }$ system (Bernheim & Rangel 2004; Redish 2004). Cocaine and many other abused drugs produce large increases of dopamine pharmacologically throughout the striatum (Ito et al. 2002; Kuhar et al. 1988; Roitman et al. 2004; Stuber et al. 2005). This mechanism can lead to the formation of habits, which have been suggested as a key process in late stages of drug addiction (Altman et al. 1996; Di Chiara 1999; Everitt & Robbins 2005; Robbins & Everitt 1999; Tiffany 1990). Clinically, such users would be unlikely to show strong craving and would manifest a robotic drug use, without conscious planning or statements of drug-seeking. Habit-based drug use could well be uncorrelated to the qualia of pleasure.

4.8. Vulnerability 8: Selective inhibition of the planning system

Exposure to drugs can shift the normal balance between systems, emphasizing one system over the other. For example, pretreatment with amphetamine shifts rats to preferentially use systems that do not show devaluation (i.e., habit-based over planning-based systems) (Nelson & Killcross 2006). Alcohol, as another example, has been hypothesized to preferentially impair hippocampal (Hunt 1998; White 2003) and prefrontal (Oscar-Berman & Marinkovic 2003) function, which would shift the normal balance from the planning to the habit systems. Dickinson et al. (2002) found that alcohol-seeking in rats is driven primarily by $S\stackrel{a}{\to }$ mechanisms and does not show devaluation. Similarly, Miles et al. (2003) found that including cocaine in a sucrose solution prevents devaluation. Such distinctions would appear as a fast increase in habitual responses over planning-based responses.

Following from the hypothesis that prefrontal (executive) systems are involved in the shift from habit back to planning systems (Baddeley 1986; Barkley 2001; Barkley et al. 2001; Dalley et al. 2004; Isoda & Hikosaka 2007), deficits in this executive system would lead to a difficulty in breaking habits (Bechara et al. 2001; Jentsch & Taylor 1999; Lubman et al. 2004). Following from the hypothesis that extinction follows from a reinterpretation of situations (Bouton 2002; Capaldi 1957; Quirk et al. 2006; Redish et al. 2007), this would suggest a difficulty in extinguishing drug-taking. It is certainly possible to extinguish drug-taking in animals (Kalivas et al. 2006; Olmstead et al. 2001), but those extinguished behaviors are particularly susceptible to relapse and reinstatement (McFarland & Kalivas 2001; Shalev et al. 2002). Whether it is more difficult to extinguish some drug-taking behavior in certain agents due to selective inhibition of the planning system or excitation of the habit system is still unknown. Agents falling victim to this vulnerability would show a particularly strong, uncontrolled relapse, likely cue-dependent, and possibly independent of explicitly identified cravings.

4.9. Vulnerability 9: Overfast discounting

As reviewed earlier, there is strong evidence that addicts discount faster than non-addicts (Bickel & Marsch 2001; Reynolds 2006). An important question that is still unresolved is whether these faster discounting factors exist as preconditions or develop with experience. Impulsivity shows a strong heritability that has been hypothesized to underlie a pre-existing factor in addiction (Kreek et al. 2005). Impulsivity has been identified with changes in neuromodulators, particularly serotonin¹³ (Chamberlain et al. 2006); changing serotonin levels can lead to online changes in discounting rates (Schweighofer et al. 2004; Tanaka et al. 2004b). (Computationally, serotonin has been explicitly modeled as controlling the discounting factor in temporal-difference learning; Doya 2000a; 2002.) Many drugs of abuse, such as cocaine, directly affect serotonin levels (Paine et al. 2003; Ritz et al. 1987), while in other substances, such as alcohol, self-administration levels reflect serotonin levels (Chastain 2006; Valenzuela & Harris 1997). It is currently unknown whether the excess discounting seen in addicts is a pre-existing condition or a consequence of the addictive process itself (Reynolds 2006). As with many of these vulnerabilities, it is possible to have a positive feedback, in which pre-existing conditions support the entrance to addiction (Kreek et al. 2005; Perry et al. 2005; Poulos et al. 1995), and then post-addictive consequences arising from pharmacology or experience exacerbate it (Paine et al. 2003).

4.10. Vulnerability 10: Changes in learning rates

The decision-making system reviewed earlier depends on learning associations among situations, outcomes, and actions. These systems depend on specific learning rates. Neuromodulators such as acetylcholine and dopamine have been hypothesized to control learning rate parameters (Doya 2000a; 2002; Gutkin et al. 2006; Hasselmo 1993; Hasselmo & Bower 1993; Yu & Dayan 2005). Pharmacological substances that manipulate these learning rates can produce enhanced associations, leading to overdeveloped expectations or habits. For example, nicotine enhances the presence of already available phasic dopaminergic signals in vitro (Rice & Cragg 2004). Following from a hypothesized role of phasic dopamine signals in identifying high-value associations to be stored (Montague et al. 1996; Schultz et al. 1997), this would predict that nicotine would enhance small learning signals, further increasing the likelihood of making cue-related associations. Although there is as yet no direct evidence for a general role of nicotine in learning, if nicotine did generally enhance learning signals, this would make smokers particularly susceptible to cue-driven associations (Chiamulera 2005). Multiple drugs taken simultaneously may interact with each other, and drugs may interact with natural rewards as well.

5.1. Individual vulnerabilities

One of the main thrusts of addiction research right now is the question of why some people become addicted and some do not (Deroche-Gamonet et al. 2004; Koob & Le Moal 2006; Tarter et al. 1998; Volkow & Li 2005b). These individual differences arise from an interaction between the genetics of the individual, the development environment (social and physical), the developmental stage of the individual, and the behavioral experience with the addictive substance (Koob & Le Moal 2006; Kreek et al. 2005; Volkow & Li 2005a; 2005b). Laying out the complete details of individual vulnerabilities are beyond the scope of this article (and are in large part still unknown), but the multiple-vulnerabilities hypothesis put forward here suggests a plan of attack to the problem. Addiction research has been historically aimed at problems with a single drug (e.g., nicotine, heroin, alcohol, etc.) or at unifying parameters across drugs (e.g., the role of dopamine). The multiple-vulnerabilities hypothesis suggests that we should look, instead, at the potential vulnerabilities within the natural learning system.

These vulnerabilities are likely to depend on a number of specific individual parameters. For example, imagine an individual who was particularly sensitive to rewards and punishments. That individual would be more susceptible to Vulnerability 3 in which a drug of abuse produced a euphorigenic signal. Imagine an individual who was more likely to treat a slightly new situation as new. Such an individual would be more susceptible to Vulnerability 6 in which wins and losses are not matched. Or imagine an individual in which the effect of nicotine on dopamine was increased. Such an individual would receive a strong dopamine kick with each puff of a cigarette and become particularly vulnerable to Vulnerability 7. Perhaps, in some individuals, nicotine produces excessive dopamine release (Vulnerability 7), leading to a habit-like addiction, whereas in others, nicotine produces allostatic changes (Vulnerability 2), leading to a maintenance-of-levels addiction. Other individuals may have neither of these vulnerabilities, leaving them more resistant to becoming addicted to nicotine.

5.2. Interactions among vulnerabilities

The failure points identified here are not mutually exclusive; they can co-occur. For example, excess dopamine delivered simultaneously to the ventral striatal regions (hypothesized to be involved in the planning system), to the dorsal striatal regions (hypothesized to be involved in the habit system), and to the frontal cortices and hippocampus (hypothesized to be involved in situation-categorization mechanisms) could drive an individual into a host of vulnerabilities. Increased dopamine in the planning system has been hypothesized to lead to increased motivational salience (Robinson & Berridge 2001; 2003; 2004; Vulnerability 4). Increased dopamine to the habit system has been hypothesized to lead to overvaluation of $S\stackrel{a}{\to }$ associations (Bernheim & Rangel 2004; Redish 2004; Vulnerability 7). Increased dopamine to the situation-categorization system has been hypothesized to change the stability of categorization systems (Redish et al. 2007; Seamans & Yang, 2004; Vulnerability 6). Thus, a single effect of a single drug can access multiple failure points.

Drugs can also produce multiple effects, which can lead to multiple vulnerabilities all leading to maladaptive decisions. For example, cocaine, amphetamine, and methamphetamine pharmacologically block the dopamine transporter (Kuhar et al. 1988; Ritz et al. 1987) leading to increases in dopamine in the ventral striatum (Stuber et al. 2005), but long-term exposure also leads to changes in dopamine receptor levels (Letchworth et al. 2001; Porrino et al. 2004a; 2004b), to a decrease in dopamine release caused by other mechanisms (Martinez et al. 2007), and to changes in long-term depression (LTD) and long-term potentiation (LTP) in ventral (Thomas et al. 2001) and dorsal (Nishioku et al. 1999) striatum. But long-term exposure to cocaine also produces changes in opioid-receptor distributions (Shippenberg et al. 2001). Each of these effects can lead to an agent falling victim to a different vulnerability, which can lead to separate mechanisms driving maladaptive decisions.

Similarly, nicotine has multiple access points throughout the brain (Ikemoto et al. 2006). Repeated nicotine use can lead to allostatic changes in response to the flooding of the system with cholinergic agonists (Benowitz 1996; Koob & Le Moal 2006), but it also leads directly to dopamine release (Pidoplichko et al. 1997), increases the effect of glutamatergic inputs to the ventral tegmental area (Mansvelder & McGehee 2000), and strengthens the effect of already present phasic dopaminergic signals (Rice & Cragg 2004). This combination of vulnerabilities could lead to the subject falling victim to Vulnerability 2 (allostasis), to Vulnerability 7 (overvaluation in the habit system), and to Vulnerability 10 (increased learning rates of drug-related cues) simultaneously. Treatment of only the allostatic component (such as through nicotine replacement therapy, Hanson et al. 2003; Rose et al. 1985) would not treat the simultaneous problem arising from the other vulnerabilities.

The fact that these vulnerabilities can interact implies interactions between drugs which can lead to polydrug abuse. Drugs may be synergistic in their effects on a single vulnerability or they may involve multiple vulnerabilities simultaneously. Cocaine and heroin both affect the opiate system, and allostatic changes made in response to one drug may affect the neurobiological response to another (Leri et al. 2003). Nicotine enhances already present dopaminergic signals (Rice & Cragg 2004), thus the presence of nicotine (potentially changing learning rates, Vulnerability 10) may enhance the ability of other drugs to drive dopamine-produced overvaluation in the planning (Vulnerability 4) or habit systems (Vulnerability 7). Similarly, amphetamine can sensitize cue-driven motivational signals (Wyvell & Berridge 2000), which may explain some of the interaction between cocaine and methamphetamine addiction and sexual behavior (Schneider & Irons 2001). Our theory suggests that polydrug abuse arises from the same causes as drug abuse: Interactions between the agent’s environment (drugs, cues, and experience) and the agent’s internal decision-making system (genetics, planning, and habit systems) lead to the agent falling victim to vulnerabilities in the decision-making system, leading to the continued use of problematic drugs and behaviors.

We are not the first to suggest that decisions to take drugs or to gamble can arise as a consequence of multiple processes. These multiple-process theories are generally discussed in terms of a transition sequence from more cognitive, “planned” processes to less cognitive, more “automatic” processes. For example, Everitt and Robbins (2005) suggest a transition sequence of “actions to habits to compulsion.” Oei and Baldwin (2002) suggest a transition in alcohol consumption from a controlled process to a more automatic habit-based process.

In contrast, it is our contention that there are many paths through these vulnerabilities. It is not always a transition from flexible planning strategies to automated habitual strategies.

Animal experiments have found numerous methods through which animals can appear to lose control over drug-taking, including escalation due to extended exposure to drug availability (Ahmed & Koob 1998; 1999; 2004; 2005; Vanderschuren & Everitt 2004), incubation by separation after exposure to drugs (Bossert et al. 2005; Grimm et al. 2001), relapse due to stress (Shaham et al. 2000; Shalev et al. 2000), relapse due to reinstatement (de Wit & Stewart 1981; McFarland & Kalivas 2001), and even that susceptibility time-courses can change between individuals due to unknown (potentially genetic) factors (Deroche-Gamonet et al. 2004; Goldman et al. 2005; Hiroi & Agatsuma 2005; Ranaldi et al. 2001).

Agents can show addictive decisions through vulnerabilities in planning systems or through vulnerabilities in habit systems or through vulnerabilities in the interaction between them. Our suggestion is that there are many vulnerabilities in the decision-making system and thus many ways for an agent to become addicted. This means that there are many transition sequences as well.

5.3. Transitions

Clinically, the transition to addiction is usually described in terms of three stages: initial exploratory or trial use, subsequent maintenance of drug use associated with the beginning of strong desires (craving), followed in some users by a strong, habitual use in which the user loses control of the drug use (Altman et al. 1996; Everitt & Robbins 2005; Kalivas & Volkow 2005; Lowinson et al. 1997; Oei & Baldwin 2002; Robbins & Everitt 1999). This sequence can be described as a path through the vulnerabilities of the decision systems: once the drug or behavior has been sampled, it will be repeated due to euphorigenic, pharmacological, or socially positive effects. Euphorigenic effects will drive repeated use due to associated reward signals (Vulnerability 3). Pharmacological effects will drive repeated use due to fast homeostatic changes (Vulnerability 1). It is also possible for drugs that are not euphorigenic to be driven by associated socially positive associations, such as has been hypothesized for tobacco (Bobo & Husten 2001; Cummings 2002), alcohol (Bobo & Husten, 2001; Goldman et al. 1987; 1999), and caffeine (Greden & Walters 1997), which we might categorize under Vulnerability 3. However, repeated use will lead to potentiation of the $S\stackrel{(a)}{\to }O$ relationship in the planning system (Vulnerability 4) and to the development of allostatic changes (Vulnerability 2), which will lead to strong desires and craving. With sufficient habitual use, actions leading to drug use can become over-valued in the habit system through increased value associated with an $S\stackrel{a}{\to }$ relationship (Vulnerability 7). This sequence parallels many examples of normal learning, proceeding from ventral to dorsal striatal systems (Balleine & Dickinson 1998; Everitt et al. 2001; Haber et al. 2000; Letchworth et al. 2001; Packard & McGaugh 1996).

This sequence will not be followed by all individuals or via all drugs of abuse. The timeline with which individuals make these transitions from vulnerability to vulnerability likely depends on a complex interaction between the genetics, development, and drug experience of the individual. We do not expect all addicts to take the same path through this maze of vulnerabilities.

Just as different tasks entail different interactions between planning and habit systems (some tasks entail transitions from planning to habit, other tasks always require the planning system, other tasks require the habit system, and other tasks can entail transitions from habit to more flexible planning systems), we expect different agents (with different genetics, different experiences, etc.) to take different paths through these vulnerabilities. In addition, just as some tasks entail an overlaying of automated habit-like strategies on top of planning-based strategies (e.g., Packard & McGaugh 1996), treating the habit-based vulnerability of a patient may uncover earlier planning-based vulnerabilities. Other agents can show addictive decisions through vulnerabilities in habit or interactive systems without ever passing through planning systems. It is also possible that habit-based addictive decisions may shift to planning-based addictive decisions (e.g., when obstacles are put into place). We argue that, in order to understand and treat the issue of addiction, we need to know not only where the patient is in his or her trajectory through these vulnerabilities, but also which vulnerability (vulnerabilities) the patient has fallen victim to.

5.4. Relapse

The fundamental issue with addiction is that of relapse, which can be defined as drug-seeking or the making of the addictive choice, even after a period of abstinence.

Relapse has been studied both clinically by measuring populations remaining abstinent from drug use and in animals identifying the return to responding for drug after forced removal (extinction, forced abstinence). In humans, relapse can occur after re-exposure to the drug, to cues associated with drug-taking and drug-seeking, and to stress (Self & Nestler 1998; Shalev et al. 2002). Relapse to behavioral addictions (such as gambling) has not been studied in the same detail, but we have suggested that gambling addiction may be related to the reinstatement of responding seen after extinction of normal rewards (Redish et al. 2007, see also Bouton 2002; 2004). In animals, a return to responding can occur due to acute re-exposure to the drug, to cues associated with drug-taking and drug-seeking, and to stress (Bossert et al. 2005; Kalivas et al. 2006; Shaham et al. 2003). The validity of the reinstatement paradigm as a model of abstinence and relapse is still controversial (Kalivas et al. 2006; Katz & Higgins 2003); nevertheless, the reinstatement paradigm can provide an understanding of mechanisms by which relapse could occur (Epstein & Preston 2003).

All of the vulnerabilities noted earlier can potentially drive relapse to the addictive behavior, but the path to relapse will differ depending on the vulnerabilities involved.

For example, relapse driven via homeostatic needs (Vulnerability 1) should occur through the natural time-course of the homeostatic change. Relapse driven through allostatic needs (Vulnerability 2) can be driven by changes in physiological set-points, driven in part by cues or by circadian or other rhythmic changes. The natural time-course of these changes can be seen in some smokers who show a circadian time-course of craving (Benowitz 1996; Perkins 2001; Schmitz et al. 1997). Allostatic set-points driving expectation can also be cue-driven (Ehrman et al. 1992; Hunt 1998; Meyer & Mirin 1979; Siegel 1988). Experienced users can show preparation tolerance with cues associated with heroin (Meyer & Mirin 1979; O’Brien et al. 1977; Siegel 1988). Similarly, alcohol users show fewer coordination deficits under the influence of alcohol in alcohol-associated environments (such as bars) than in non-alcohol-associated environments (such as offices) (Hunt 1998). A number of authors have suggested that relapse under stress may be due to cue-driven deviations from homeostatic set-points (Ahmed & Koob 1997; Shaham et al. 2000; Shalev et al. 2000; Weiss et al. 2001).

Relapse caused by expectation (overvaluation in the planning system, Vulnerabilities 3 and 4) can be identified by “craving.” Such relapse can be triggered by a recall of an $S\stackrel{(a)}{\to }O$ association, in which the agent recognizes an action-sequence that can get the agent from the situation the agent is currently in (S) to an over-valued outcome (O). The recognition can be cue-driven or may arise spontaneously, but in either case will entail an expectancy of the outcome. As noted earlier, correct decision-making within the planning component requires a search of multiple possibilities. It is likely that once a pathway to a highly valued outcome is recognized, the search will keep returning to that possibility, producing cognitive blinding and obsession (Vulnerability 5). Both craving and obsession are common to pre-relapse conditions in some (but not all) patients (Altman et al. 1996; Childress et al. 1988; Grant et al. 2006; MacKillop & Monti 2007; O’Brien 2005; Sayette et al. 2000).

Note that our theory predicts that craving should be clinically separable from relapse: Because the planning system is flexible, recognition of a path to an outcome (in our theory, craving is recognition of a path to a high-valued outcome) does not necessarily lead to taking that path. Thus, craving can occur without relapse. Because the habit system does not include recognition of an outcome, in our theory, it does not produce craving. Thus, relapse caused by overvaluation in the habit system (Vulnerability 7) may be robotic, without craving, perhaps without even conscious recognition (Everitt & Robbins 2005; Robbins & Everitt 1999). (Retrospectively, the addict may believe he or she craved the drug, whether or not any actual craving occurred prospectively; Sayette et al. 2000.)

Multiple vulnerabilities can cause a relapse to the addictive choice, but the pathway to that relapse may be different, depending on the vulnerability involved. Therefore, prevention of relapse will also depend on treating the vulnerabilities involved.

5.5. Treatment

Each vulnerability drives the decision-making process towards the addictive choice and provides a potential access-point for the addiction to relapse, but each vulnerability is a different failure-point of the decision process and leads decision-making to error through a different mechanism. Thus, each vulnerability is likely to require a different treatment regimen. This concept (of different treatments for different vulnerabilities) has enjoyed some recent success and been used to explain historical treatment failures. For example, in a recent study (Grant et al. 2006), significant success was found in treating a subset of pathological gamblers that showed strong urges (craving). Irvin and colleagues (Irvin & Brandon 2000; Irvin et al. 2003) suggest that the well-documented decreasing success of smoking cessation in tobacco studies is due to the presence of available over-the-counter cessation-aid products and thus a changing distribution of smokers participating in the studies.

Treatment of the homeostatic deviations and allostatic changes in nicotine through nicotine replacement therapy has been extremely successful (Balfour & Fagerström 1996; Benowitz 1996; Hanson et al. 2003; O’Brien 2005; Rose et al. 1985). However, long-term relapse after these treatments is notoriously high (Balfour & Fagerström 1996; Hanson et al. 2003; Monti & MacKillop 2007). This is likely a consequence of the fact that nicotine replacement therapy does not address the other vulnerabilities involved in nicotine addiction (e.g., Vulnerability 7).

Treatment of the planning vulnerabilities (Vulnerabilities 3, 4, and 5), which lead to excess expectation of positive outcomes (see above) and may be identifiable through craving and obsession (Redish & Johnson 2007), may depend on blocking the misevaluation process. Opiate antagonists have been used to reduce craving in alcohol addictions (Kiefer & Mann 2005; O’Brien et al. 1996) and in gambling (Grant et al. 2006). Many heroin abusers on naltrexone report no craving (Meyer & Mirin 1979; O’Brien 2005, but see Halikas 1997 for another view). Whether this is due to controlling allostatic effects (Koob & Le Moal 2006) or to the blocking of craving and the recognition of future rewards (O’Brien 2005) is still unresolved. Naltrexone treatment of cocaine addicts failed to find a significant effect on craving (Schmitz et al. 2001). It is clear that there is still work to be done to completely elucidate the specific relationship among clinically tested treatments, the qualia identified as craving, and the potential vulnerabilities identified in this article.

Treatment of each vulnerability requires a regimen specifically designed to address that vulnerability. For example, the homeostatic and allostatic vulnerabilities (Vulnerabilities 1 and 2) likely require pharmacological treatment to rebalance the system. Overvaluation in the planning system (Vulnerabilities 3, 4, and 5) likely requires treatment to change the recall and re-evaluation processes, either through pharmacological means or through cognitive behavioral re-training, or some combination of the two. Overvaluation and over-strengthening in the habit system (Vulnerabilities 7 and 8) likely require mechanisms with which to strengthen alternative choices available in the planning system. Miscategorization of situations (Vulnerability 6) likely requires treatments aimed at executive function and its role in re-categorizing situations. Although we have not proposed specific treatments for any of these vulnerabilities, it is our contention that these failure modes are treatable and that treatments aimed at these specific modes are more likely to be successful than general treatments aimed at the general addicted population.

In general, we propose that the clinical treatment of addiction should not be addressed to the general addicted population, or to specific drugs of abuse. Instead, we propose that treatment should first entail the identification of which vulnerabilities have been triggered within the individual, and then treatment should be addressed to the specific constellation of vulnerabilities into which the addicted patient has fallen.