Acute stressor effects on goal-directed action in rats
- Institute of Biology, Department of Animal Physiology, University of Stuttgart, D-70550 Stuttgart, Germany
Abstract
Here we examined effects of acute stressors that involve either systemic coadministration of corticosterone/yohimbine (3 mg/kg each) to increase glucocorticoid/noradrenaline activity (denoted as “pharmacological” stressor) or one or several distinct restraint stressors (denoted as “single” vs. “multiple” stressor) on performance of goal-directed actions. Rats were trained over 11 d to perform two instrumental actions, one for food pellets the other for sucrose solution, followed by two consecutive tests days. On each test day, rats were first sated in a counterbalanced manner on one of the two outcomes by prefeeding (selective outcome devaluation), then subjected to an acute stressor, and tested afterward in a two-lever choice task in extinction to assess whether instrumental performance is goal-directed, i.e., sensitive to changes in outcome value. Like in control rats, in rats subjected to the pharmacological or single restraint stressor prior to the choice test, performance of instrumental action was goal-directed, i.e., sensitive to outcome devaluation. By contrast, in rats exposed to the multiple stressor prior to the choice test, performance of instrumental action was habitual, i.e., insensitive to outcome devaluation. Pretreatment with diazepam (1 and 2 mg/kg) did not alleviate (or only marginally) this multiple stressor-induced effect. Thus, an intense acute stressor can render performance of previously acquired instrumental action habitual, possibly due to a compromised retrieval of encoded relationships between actions and their outcome value. Our observation in rats that an acute stressor can shift instrumental responding from goal-directed to habitual control is consistent with similar findings in humans.
Two interacting learning systems govern the performance of instrumental action, one controls the acquisition of goal-directed actions and the other the acquisition of habits (Dickinson 1985). The acquisition of goal-directed actions is driven by causal relations between actions and outcomes and, hence, sensitive to changes of action–outcome contingencies and outcome devaluation. By contrast, the acquisition of habits is driven by stimulus–response relations and insensitive to changes of action–outcome contingencies or outcome devaluation (Balleine and Dickinson 1998). Goal-directed actions are flexible and can be adapted to changing environments, whereas habits are inflexible and performed almost automatically, allowing attention to be focused elsewhere (for comprehensive reviews, see Graybiel 2008; Balleine and O'Doherty 2010).
Accumulating evidence from rodent studies suggests that the medial prefrontal cortex (mPFC), the dorsomedial striatum (DMS), and the basolateral amygdala play key roles in the acquisition and/or performance of goal-directed action (Balleine et al. 2003; Ostlund and Balleine 2005; Yin et al. 2005; Naneix et al. 2009; Balleine and O'Doherty 2010; Lex and Hauber 2010a; Braun and Hauber 2011), whereas the dorsolateral striatum (DLS) and the central nucleus of the amygdala support habit learning (Yin et al. 2004; Lingawi and Balleine 2012). Similarly, studies in humans implicated the mPFC and DMS in encoding the causal effects of actions and the DLS in habit formation (Tanaka et al. 2008; Valentin and O'Doherty 2009), indicating that brain systems involved in controlling goal-directed action and habit formation are highly conserved across mammalian species (Tanaka et al. 2008).
Importantly, acute physical/psychosocial stress in humans can modulate processes governing learning and expression of action–outcome contingencies in a manner that favors the habit system over the goal-directed system (Schwabe and Wolf 2009). These acute stress effects on instrumental responding can be mimicked by an increased glucocorticoid/noradrenergic activity elicited through coadministration of the glucocorticoid hydrocortisone and the α2-adrenoceptor antagonist yohimbine (Schwabe et al. 2010). A reduced capacity of the prefrontal cortex to assess changes in the incentive value of actions may account for the stress-induced shift from goal-directed action to habit behavior (Schwabe et al. 2012).
In animal studies, to date stress effects on goal-directed action have rarely been investigated. Chronic stress in rats facilitated a shift from goal-directed to habitual control of instrumental action (Dias-Ferreira et al. 2009; Gourley et al. 2012) and caused a neuronal atrophy in the DMS as well as a neuronal hypertrophy in the DLS (Dias-Ferreira et al. 2009). However, it is as yet unknown whether acute stress in rats can shift instrumental responding from goal-directed to habitual control, as shown in humans. Here we examined the effects of acute stressors that involve either systemic coadministration of corticosterone/yohimbine to increase glucocorticoid/noradrenaline activity (denoted here as “pharmacological” stressor) or one or several distinct restraint stressors (denoted here as “single” vs. “multiple” stressor) on performance of goal-directed actions guided by previously acquired action–outcome associations. In four experiments, separate groups of rats were trained over 11 d to press two levers for distinct outcomes followed by two consecutive test days. One each test day, rats were first sated in a counterbalanced manner on one of these two outcomes by prefeeding (selective outcome devaluation). Then, rats were subjected to an acute stressor and tested afterward in a two-lever choice task in extinction to assess whether instrumental performance was goal-directed, i.e., sensitive to changes in the outcome value. We examined the effects of a pharmacological stressor (Experiment 1), a single restraint stressor (Experiment 2), and a multiple restraint stressor (Experiment 3) applied prior to the choice test on instrumental performance. Furthermore, we examined whether multiple stressor effects on instrumental performance can be reversed by pretreatment with a low (Experiment 4A) or high (Experiment 4B) dose of the standard anxiolytic drug diazepam. Previous studies revealed that diazepam can decrease hypothalamus-pituitary-adrenal (HPA) axis activity after acute stress induced by single or repeated restraint (Kalman et al. 1997). To assess stressor-induced HPA axis activity, in another group of rats that did not undergo behavioral training and testing, we also measured the effects of stressors applied in Experiments 2–4 on plasma corticosterone levels (Experiment 5).
Results
Experiment 1: corticosterone/yohimbine effects on goal-directed action
Lever-press training
All animals received instrumental training on two levers, one delivering food pellets, the other sucrose solution. Both treatment groups (to be subjected or not subjected to corticosterone/yohimbine administration) showed increasing response rates during instrumental training as the ratio schedule parameter increased across consecutive training days (data not shown). An analysis of variance (ANOVA) revealed an effect of day (F(1,20) = 14.1, P < 0.001), but no effect of treatment group (F(1,20) < 0.7, n.s.) and no day × treatment group interaction (F(1,20) < 0.6, n.s.). Mean lever press rates/min (±SEM) on the last day of training were 17.7 ± 3.1 (controls) and 18.3 ± 3.2 (to be subjected to drugs).
Outcome devaluation and choice test in extinction
During outcome devaluation on two subsequent test days, individual rats consumed on average 5.2 ± 0.3 g pellets and 13.3 ± 0.4 g sucrose. In the choice tests on two subsequent days, control rats and rats subjected to corticosterone/yohimbine administration displayed a selective devaluation effect, that is, fewer responses were emitted to the lever that in training delivered the now devaluated outcome (Fig. 1). This observation was confirmed by the statistical analysis. An ANOVA showed an effect of day (F(1,20) = 30.51, P < 0.001) and devaluation (F(1,20) = 22.43, P < 0.001), a day × devaluation interaction (F(1,20) = 4.67, P < 0.05), but no effect of treatment (F(1,20) < 2.8, n.s.).
Effects of corticosterone/yohimbine (3 mg/kg, each, i.p.) (n = 11) or vehicle (n = 11) applied prior to the two-lever choice tests on lever preference (Experiment 1). Response rates for the lever that delivered valued and devalued outcomes on test days 1 and 2 are shown. On both test days, the outcomes (pellets vs. sucrose) were devaluated in a counterbalanced manner. On days 1 and 2, respectively, there was a significant devaluation effect, but no outcome devaluation × treatment interaction.
For a more detailed analysis of treatment effects across days, we performed separate ANOVA for outcome devaluation effects on days 1 and 2. On day 1 there was a devaluation effect (F(1,20) = 16.59, P < 0.001), but no effect of treatment (F(1,20) < 1.6, n.s.) and no treatment × devaluation interaction (F(1,20) < 0.004, n.s.). Likewise, on day 2 there was an effect of devaluation (F(1,20) = 7.01, P < 0.05) and a trend for a significant treatment effect (F(1,20) < 3.6, P = 0.072), but no treatment × devaluation interaction (F(1,20) < 0.7, n.s.). Thus, on both test days, rats subjected to corticosterone/yohimbine treatment, like controls, were able to discriminate the lever that delivered the valued outcome from the lever that delivered the devaluated outcome.
Experiment 2: single stressor effects on goal-directed action
Lever-press training
Rats to be exposed to the single stressor as well as controls showed increasing response rates to both levers during instrumental training (data not shown). An ANOVA revealed an effect of day (F(1,16) = 8.13, P < 0.05), but no effect of treatment (F(1,16) < 0.01, n.s.) and no day × treatment interaction (F(1,16) < 0.4, n.s.). Mean lever press rates/min on the last day of training were 16.2 ± 4.0 (controls) and 14.6 ± 3.0 (to be stressor exposed).
Outcome devaluation and choice test in extinction
During outcome devaluation on two subsequent test days, rats consumed on average 5.8 ± 0.2 g pellets and 13.9 ± 0.5 g sucrose. In the choice test, control rats and rats subjected to the single stressor displayed a selective devaluation effect (Fig. 2). Accordingly, an ANOVA revealed an effect of devaluation (F(1,16) = 14.21, P < 0.01), but not of treatment (F(1,16) < 3.4, n.s.), and no devaluation × treatment interaction (F(1,16) < 0.6, n.s.). Furthermore, treatment effects were influenced by day (treatment × day interaction [F(1,16) = 5.59, P < 0.05]).
Effects of single stressor exposure (stress-exposed rats n = 9, controls n = 9) prior to the two-lever choice tests on lever preference (Experiment 2). Response rates for the lever that delivered valued and devalued outcomes on test days 1 and 2 are shown. On both test days, the outcomes (pellets vs. sucrose) were devaluated in a counterbalanced manner. On day 1, there was a significant devaluation effect, but no devaluation × treatment interaction. On day 2, there was a significant effect of devaluation and treatment, but no devaluation × treatment interaction.
A separate ANOVA for treatment effects on day 1 revealed an effect of devaluation (F(1,16) = 8.62, P < 0.01), but no effect of treatment (F(1,16) < 0.3, n.s.) and no devaluation × treatment interaction (F(1,16) < 0.1, n.s.). On day 2, the ANOVA detected an effect of devaluation (F(1,16) = 7.16, P < 0.05) and treatment (F(1,16) = 8.24, P < 0.05), but no devaluation × treatment interaction (F(1,16) < 1.4, n.s.). Thus, on both test days, rats exposed to the single stressor, like controls, displayed a selective devaluation effect.
Experiment 3: multiple stressor effects on goal-directed action
Lever-press training
Rats to be exposed to the multiple stressor as well as controls showed increasing response rates to both levers during instrumental training (data not shown). The ANOVA revealed an effect of day (F(1,22) = 12.91, P < 0.002), but no effect of treatment (F(1,22) < 0.5, n.s.) and no day × treatment interaction (F(1,22) < 0.2, n.s.). Mean lever press rates/min on the last day of training were 20.2 ± 2.9 (controls) and 18.5 ± 3.9 (to be stressor exposed).
Outcome devaluation and choice test in extinction
During the outcome devaluation on two subsequent test days, rats consumed on average 5.8 ± 0.3 g pellets and 11.8 ± 0.5 g sucrose. In the choice test, control rats emitted fewer responses to the devaluated compared to the valued outcome (Fig. 3), however this effect was markedly reduced in rats exposed to the multiple stressor. Accordingly, there was an effect of treatment (F(1,22) = 35.06, P < 0.0001), devaluation (F(1,22) = 40.82, P < 0.0001), and day (F(1,22) = 7.35, P < 0.05), and a devaluation × treatment interaction (F(1,22) = 10.40, P < 0.004). Furthermore, an ANOVA on data from day 1 showed an effect of devaluation (F(1,22) = 10.00, P < 0.005) and treatment (F(1,22) = 14.39, P < 0.001), but no devaluation × treatment interaction (F(1,22) < 0.76, n.s.). Importantly, on day 2, there was an effect of devaluation (F(1,22) = 12.55, P < 0.002) and treatment (F(1,22) = 31.02, P < 0.0001), and a devaluation × treatment interaction (F(1,22) = 7.44, P < 0.05). Subsequent planned contrasts revealed a selective devaluation effect in controls (P < 0.0001), but not in stressed rats (P > 0.5). Therefore, the selective devaluation effect observed in rats exposed to the multiple stressor on test day 1 was abolished on test day 2.
Effects of multiple stressor exposure (stress-exposed rats n = 12, controls n = 12) prior to the two-lever choice tests on lever preference (Experiment 3). Response rates for the lever that delivered valued and devalued outcomes on test days 1 and 2 are shown. On both test days, the outcomes (pellets vs. sucrose) were devaluated in a counterbalanced manner. On day 1, there was a significant effect of devaluation and treatment, but no devaluation × treatment interaction. On day 2, there was a significant effect of devaluation and treatment, and, importantly, a significant devaluation × treatment interaction. Subsequent planned contrasts revealed a significant devaluation effect in controls (P < 0.001), but not in stressed rats (P > 0.05).
Experiment 4: diazepam modulation of multiple stressor effects on goal-directed action
Lever-press training
Experiment 4A (lower dose diazepam)
All groups showed increasing response rates to both levers during instrumental training (data not shown). An ANOVA revealed an effect of day (F(1,20) = 37.73, P < 0.001), but no effect of treatment group (F(2,20) < 0.5 n.s.) and no day × treatment group interaction (F(2,20) < 0.9, n.s.). Mean lever press rates/min on the last day of training were 24.9 ± 3.4 (to be exposed to diazepam + no stressor), 22.0 ± 2.8 (to be exposed to vehicle + multiple stressor), and 28.2 ± 4.5 (to be exposed to diazepam + multiple stressor).
Experiment 4B (higher dose diazepam)
Likewise, all groups showed increasing response rates to both levers during instrumental training (data not shown). There was an effect of day (F(1,19) = 28.91, P < 0.001), but no effect of treatment group (F(2,19) < 0.5, n.s.) and no day × treatment group interaction (F(2,19) < 0.8, n.s.). Mean lever press rates/min on the last day of training were 21.3 ± 2.8 (to be exposed to diazepam + no stressor), 19.8 ± 3.1 (to be exposed to vehicle + multiple stressor), and 22.3 ± 1.1 (to be exposed to diazepam + multiple stressor).
Outcome devaluation and choice test in extinction
Experiment 4A (lower dose diazepam)
During outcome devaluation rats consumed on average 5.9 ± 0.4 g pellets and 13.3 ± 0.7 g sucrose. In the choice test, control rats subjected to diazepam treatment displayed a selective devaluation effect, which was reduced in rats subjected to the multiple stressor/vehicle or diazepam treatment (Fig. 4A). Consistent with this description, an ANOVA revealed an effect of day (F(2,21) = 14.14, P < 0.001), treatment (F(2,21) = 14.81, P < 0.0001), and devaluation (F(2,21) = 47.4, P < 0.0001), and a devaluation × treatment interaction (F(2,21) = 5.86, P < 0.01). A separate ANOVA for treatment effects on day 1 indicated an effect of devaluation (F(1,21) = 55.85, P < 0.0001) and treatment (F(2,21) = 22.12, P < 0.0001), and a devaluation × treatment interaction (F(2,21) = 3.904, P < 0.05). Planned contrasts further revealed selective devaluation effects in all treatment groups (P < 0.05, each). On day 2, there was an effect of devaluation (F(1,20) = 19.8, P < 0.001) and of treatment (F(2,21) = 13.62, P < 0.001), and, importantly, a devaluation × treatment interaction (F(2,21) = 4.06, P < 0.05). Planned contrasts demonstrated selective devaluation effects in control rats that received diazepam and stressed rats that received diazepam (P < 0.05, each), but not in stressed rats that received vehicle (P > 0.5). Thus, on test day 1 all treatment groups were able to discriminate the lever that delivered the valued outcome from the one that delivered the devalued outcome. By contrast, on day 2, this selective devaluation effect was abolished specifically in stress-exposed rats that received vehicle treatment.
Effects of multiple stressor exposure prior to the two-lever choice tests on lever preference in rats pretreated with diazepam (A, 1 mg/kg, i.p.; B, 2 mg/kg, i.p.) (Experiments 4A and 4B). Response rates for the lever that delivered valued and devalued outcomes on test days 1 and 2 are shown. On both test days, the outcomes (pellets vs. sucrose) were devaluated in a counterbalanced manner. (A) On days 1 and 2, respectively, there was a significant effect of devaluation and treatment, and a devaluation × treatment interaction. Planned contrasts further revealed selective devaluation effects on day 1 in all treatment groups (P < 0.05, each), while on day 2 selective devaluation effects were found in control and stressed rats that received diazepam (P < 0.05, each), but not in stressed rats that received vehicle (P > 0.05). (B) On day 1, there was a significant effect of devaluation and treatment, but no devaluation × treatment interaction. On day 2, there was a significant effect of devaluation and treatment, and, importantly, a significant devaluation × treatment interaction. Planned contrasts revealed a selective devaluation effect in control rats that received diazepam (P < 0.001), but not in stressed rats that received diazepam or vehicle (P > 0.05, each). Sample sizes: (A) diazepam (1 mg/kg) + no stressor (n = 7), diazepam + multiple stressor (n = 8), vehicle (2 mL/kg) + multiple stressor (n = 8); (B) diazepam (2 mg/kg) + no stressor (n = 8), diazepam + multiple stressor (n = 7), vehicle (2 mL/kg) + multiple stressor (n = 7).
Experiment 4B (higher dose of diazepam)
Rats consumed on average 5.8 ± 0.6 g pellets and 14.1 ± 1.0 g sucrose during the devaluation procedure. In the choice test, control rats subjected to diazepam treatment showed a selective devaluation effect that was reduced in rats subjected to the multiple stressor/vehicle or diazepam treatment (Fig. 4B). Accordingly, an ANOVA revealed an effect of day (F(1,19) = 21.54, P < 0.001), treatment (F(2,19) = 16.87, P < 0.001), and devaluation (F(1,19) = 62,79, P < 0.0001), and an interaction of devaluation × treatment (F(2,19) = 6.96, P < 0.005). Subsequent ANOVA for data from day 1 detected an effect of devaluation (F(1,19) = 22.65, P < 0.001) and treatment (F(2,19) = 16.82, P < 0.0001), but no devaluation × treatment interaction (F(2,19) < 0.3, n.s.). On day 2, there was an effect of devaluation (F(1,19) = 10.01, P < 0.005) and treatment (F(2,19) = 10.44, P < 0.001) and, importantly, a devaluation × treatment interaction (F(2,19) = 7.18, P < 0.005). Planned contrasts revealed selective devaluation effects in control rats that received diazepam (P < 0.001), but not in stressed rats that received diazepam or vehicle (P > 0.05, each). Therefore, on day 1 rats of all treatment groups displayed a selective devaluation effect, however on day 2 this effect was abolished in stress-exposed rats that received vehicle or diazepam treatment.
Experiment 5: single and multiple stressor effects on plasma corticosterone
Single and multiple stressor exposition markedly elevated plasma corticosterone levels regardless of stressor identity (Table 1). Consistent with this description, a repeated measures ANOVA with stressor identity (single, multiple stressor), group (control, stressed), and point in time (before outcome devaluation, before stressor exposition, after stressor exposition) as variables demonstrated no significant effect of stressor identity (F(1,7) = 0.01, P = 0.91), significant effects of group (F(1,7) = 23.35, P < 0.001), and point in time (F(2,14) = 108.65, P < 0.001) and a stressor identity × group × point in time interaction (F(2,14) = 5.22, P < 0.05). Post hoc comparisons revealed that corticosterone levels in control rats and rats exposed to the single stressor did not differ before outcome devaluation and stressor exposition, but after stressor exposition (P < 0.01, Tukey HSD-test). Likewise, in control rats and rats exposed to the multiple stressor corticosterone levels did not differ before outcome devaluation and stressor exposition, but after stressor exposition (P < 0.01, Tukey HSD-test).
Plasma corticosterone levels
Discussion
Here we show that rats subjected to the pharmacological or single restraint stressor prior to the choice test displayed goal-directed responding sensitive to outcome devaluation. By contrast, rats exposed to the multiple restraint stressor were biased toward habitual responding, i.e., they showed a selective devaluation effect on test day 1, but no longer on test day 2. Diazepam pretreatment did not alleviate (or only marginally) this multiple stressor-induced impairment. Thus, in rats an acute intense stressor, such as multiple restraint, can render performance of previously acquired instrumental action habitual.
Experiment 1 demonstrated that, unlike in humans (Schwabe et al. 2010), in rats concurrent glucocorticoid/noradrenaline activity did not shift instrumental responding from goal-directed to habitual control. Furthermore, as in all other experiments, we observed a decrease of overall lever press rates across both test days, an effect that predominantly reflects repeated testing in extinction. The failure of corticosterone/yohimbine to alter responding might not be accounted for by inappropriate drug dosing or timing of the injections. For instance, corticosterone given at similar doses 30 min before test (as done here) resulted in increased plasma corticosterone levels that resemble those measured during moderate-to-severe stress and impaired memory retrieval, in a similar manner as acute stress (de Quervain et al. 1998). Furthermore, in rats tested in a maze task, comparable doses of yohimbine favored caudate-dependent habit learning over hippocampus-dependent cognitive learning (Packard and Wingard 2004).
Concerted glucocorticoid/noradrenergic activity plays a key role in mediating stressor effects on learning and memory (Roozendaal et al. 2009); however, acute administration of corticosterone and yohimbine only partially mimics the complex neurochemical responses to acute physical stressors that involve the release of multiple stress mediators (Joels and Baram 2009). Hence, we also investigated effects of acute physical stressors on goal-directed action. Experiment 2 revealed that a typical stressor, i.e., restraint in a cylinder for 60 min, known to alter learning and memory (Conrad et al. 2004) increased plasma corticosterone levels, as shown by earlier studies (Keim and Sigg 1976; Ulrich-Lai and Herman 2009). Yet this single stressor, though behaviorally effective, as indicated by decreased overall lever press rates on choice test day 2, did not render the rats’ performance habitual. Of note, plasma corticosterone levels after single stressor exposure in the light cycle (Experiment 2) and multiple stressor exposure in the dark cycle (Experiment 3) were increased significantly, but only multiple stressor exposure produced a bias toward habitual responding. Importantly, acute stressor-induced plasma corticosterone elevations reported in a number of previous studies were not or only partially related to the light–dark cycle (Pielock et al. 2013), stressor intensity, and the magnitude of the stressor-induced cognitive impairments (Woodson et al. 2003). For instance, male rats exposed to a predator, such as a cat, or to a female receptive rat showed equivalent increases in plasma corticosterone levels, but only cat-exposed rats exhibited working memory deficits (Woodson et al. 2003). Together, these findings provide further evidence to the notion that plasma corticosterone levels are not always a sensitive index for stressor-induced behavioral effects (e.g., Natelson et al. 1987; Pielock et al. 2013).
It is well known that the effect size of acute stressors is critically determined by a number of variables, including time of day of exposition or predictability. For instance, in rats, restraint stressors elicit more pronounced physiological and behavioral effects in the dark (activity) period as compared with the light (inactivity) period (Koolhaas et al. 2011). Therefore, in Experiment 3, we examined the effects of an acute stressor that involved an unpredictable combination of typical restraint stressors (Keim and Sigg 1976; Howland and Cazakoff 2010; Musazzi et al. 2010) on goal-directed action and performed the experiment in the dark period. Results show that the multiple stressor increased plasma corticosterone and produced a bias toward habitual responding, i.e., rats showed a devaluation effect on test day 1, but no longer on test day 2, pointing to sequential effects of twofold multiple stressor exposure. In view of lowered lever press rates across test days, one could argue that the failure to detect a devaluation effect simply reflects impaired instrumental performance. However, a selective devaluation effect on choice performance is a robust phenomenon that does not require high rates of performance (Yin et al. 2005) and can be observed if baseline responding is as low as observed here (Shiflett et al. 2010). Consistent with this notion, a selective devaluation effect was also detected in rats exposed to the pharmacological (Experiment 1) or single stressor (Experiment 2), though baseline responding was low. In an earlier study we showed that instrumental responding was altered by the single and multiple stressor exposure in the dark but not in the light period, suggesting that their efficacy is determined in particular by the time of day of administration (Pielock et al. 2013). Furthermore, the multiple stressor, in behavioral terms, was moderately more effective in the dark period than the single stressor (Pielock et al. 2013); therefore, we used the former stressor in the current study. However, it is conceivable that the single stressor if applied in dark period could also induce a bias toward habitual responding, a possibility that we did not test in the current study.
As baseline responding and flexibility of instrumental responding was not reduced significantly after combined corticosterone/yohimbine injections (Experiment 1), the effects observed after multiple stressor exposition (Experiment 3) may not, or not exclusively, be related to an increased corticosterone/glucocorticoid activity. However, acute stressors can produce fear and anxiety (Davis 2006; McEwen et al. 2012), thereby suppressing instrumental responding (Hunt and Brady 1951; Blanchard and Blanchard 1969; Minor et al. 2001) and cognitive functions (Packard 2009). To examine the possibility that reduced baseline lever pressing and flexibility of responding were related to multiple stressor-induced anxiety, we investigated effects of prestress administration of the standard anxiolytic drug diazepam. In Experiments 4A and 4B we were able to replicate the multiple stressor effects on instrumental performance seen in control animals of Experiment 3. Furthermore, our results demonstrate that diazepam at doses of 1 and 2 mg/kg, which are devoid of sedative effects interfering with lever pressing by its own (George et al. 2009), did not alleviate reduced baseline responding and had no (Experiment 4B) or only marginal (Experiment 4A) effects on habitual responding induced by prior multiple stressor exposition. In acute stress paradigms, even low doses of diazepam (<3 mg/kg) produced anxiolytic effects on a variety of behavioral measures (Grahn et al. 1995; Kalman et al. 1997), suggesting that diazepam dosing might have been adequate. Taken together, Experiments 4A and 4B confirmed that rats exposed to the multiple restraint stressor were biased toward habitual responding and displayed reduced baseline responding, but gave no clues to the neurochemical and psychological mechanisms underlying these multiple stressor-induced effects. Ongoing studies further revealed that the selective β-1 antagonist betaxolol (5 mg/kg, i.p.; S Braun and W Hauber, unpubl.) did not reduce habitual responding induced by prior multiple stressor exposition. Higher doses reduced lever pressing by its own and are, therefore, not appropriate. Thus, an analysis of the neurochemical mechanisms underlying stressor-induced effects on goal-directed action is hampered by the fact that systemic administration of relevant drugs at higher doses can produce motor inhibition.
According to the dual system concept, action control is mediated by the balance between a goal-directed and a habitual system (Dickinson 1985). Previous studies in humans revealed that acute stressors can shift this balance in favor of the habit system, thereby affecting both instrumental learning as well as the expression of previously acquired actions without compromising processes involved in instrumental learning (Schwabe and Wolf 2011). For instance, acute physical/psychosocial stress administered after instrumental learning and outcome devaluation immediately prior to the extinction test prompted habitual responding, suggesting acute stress in humans may render instrumental behavior habitual without any effects on instrumental learning (Schwabe and Wolf 2010). Our present study in rodents provides partial support to this notion and shows that at least an intense stressor can render performance of previously acquired instrumental action habitual. Performance of goal-directed actions relies on an animal's previous learning of the outcomes that results from its action. Thus, it is likely that multiple stressor exposition interferes with the retrieval of encoded relationships between actions and their outcome values guiding goal-directed action in the choice test. Unexpectedly, unlike in humans (Schwabe and Wolf 2009; Schwabe et al. 2010), in rats corticosterone/noradrenaline activity and a moderate stressor were unable to induce a shift from goal-directed to habitual responding. Methodological discrepancies might account for this difference. In studies in humans, the acute stressor or the pharmacological manipulation was often administered before the initial acquisition of instrumental learning task, and thus not only the retrieval, but also the acquisition, might have been mediated by the habitual system. Furthermore, in humans, choice behavior in the extinction test is being tested immediately after brief instrumental training over one session (Schwabe and Wolf 2011), whereas in rats instrumental training requires many trials over several days. Therefore, repeated action–outcome learning in rats may increase memory and render retrieval of action–outcome contingencies in the choice test less sensitive to prior exposition to mild but not to intense acute stressors. In turn, as in humans, in rats the expression of learned responses that has been acquired through brief training is highly sensitive even to mild acute stressors. For instance, in spatial memory tasks in water mazes, acute stressors were able to impair retrieval of goal approach responses acquired through short training, e.g., in 8–12 trials over 1–3 d (de Quervain et al. 1998; Roozendaal et al. 2003; Diamond et al. 2006). Our present findings in rodents that an intense acute stressor makes rats less able to perform actions based on their outcome values extend previous evidence in rodents that chronic stress also leads to a shift from goal-directed to habitual instrumental responding (Dias-Ferreira et al. 2009). Together, these findings suggest that habits, though inflexible, seem to be advantageous both under acute and chronic stress as they are performed almost automatically and thus allow the mobilization of attentional and decision making resources.
Notably, rats exposed to the multiple stressor in Experiments 3 and 4 consistently displayed a significantly impaired expression of learned action–outcome contingencies on the second, but not the first, test day. This latter finding is critical as it suggests that twofold exposure to the multiple stressor is necessary to abolish the retrieval of action–outcome contingencies. A number of inter-related mechanisms might account for this sequential effect. First, it is well known that morphological neuronal changes and memory actions induced by acute stressors increase with the number of stressor exposures (Cerqueira et al. 2007; Sousa and Almeida 2012). Second, single exposure to an acute stressor not only triggers immediate short-lasting effects but also delayed effects in an hours-to-days range that involve neurochemical and genomic actions of stress mediators (Joels and Baram 2009). For instance, an acute restraint stressor can cause an immediate increase in striatal dopamine release followed by a delayed down-regulation of midbrain dopamine neuron activity 24 h after cessation of restraint (Chang and Grace 2013). Furthermore, multiple stressor exposure could compromise reconsolidation after retrieval of action–outcome contingencies on test day 1 and, in turn, retrieval on test day 2. Thus, it is likely that several mechanisms activated by twofold stressor exposure mediate the shift toward habitual responding, however their precise role is open to question. Studies utilizing rats exposed to the multiple stressor only on day 1 may help to clarify the issue whether delayed effects of stressor exposition on day 1 or additive of stressor exposition on both days may cause the observed shift to habitual responding on day 2. A preliminary experiment tentatively suggests that delayed effects of stressor exposition on day 1 alone may not produce a shift toward habitual responding, however these data require further experimental support.
Interestingly, in a recent study we examined the effects of acute stress using a Pavlovian-instrumental transfer task. Results indicated that single exposure of the same multiple stressor as used in the current study reduced the overall lever press rate, but left intact the ability of a reward-predictive stimulus to invigorate instrumental responding, thereby suggesting that the acute stressor did not impair the retrieval of stimulus–reward associations (Pielock et al. 2013). Our present results point to the possibility that twofold instead of single multiple stressor exposure could impair this type of stimulus modulation of instrumental performance.
At the neural level, converging lines of evidence implicated the mPFC and DMS in goal-directed instrumental actions (Ostlund and Balleine 2005; Yin et al. 2005). Importantly, the mPFC, a brain region particularly sensitive to stress (Roozendaal et al. 2009), is selectively involved in the acquisition of, but not permanent storage or expression of, action–outcome associations in instrumental conditioning (Ostlund and Balleine 2005). By contrast, the DMS supports both acquisition and expression of action–outcome learning (Yin et al. 2005), suggesting that this brain region is critical for the expression of action–outcome associations guiding action selection in the choice test used here. Furthermore, the DMS is highly sensitive to chronic stress (Dias-Ferreira et al. 2009) and, thus, may be one key region mediating the detrimental effects of the mutiple stressor on expression of action–outcome learning seen here.
In summary, a growing literature from studies in laboratory rodents (Holmes and Wellman 2009; Cazakoff and Howland 2010) demonstrates that acute stressor exposition induces multifaceted alterations in a variety of cognitive and executive processes underlying acquisition and performance of reward-related behavior, including working memory (Butts et al. 2011), memory retrieval (Wong et al. 2007), and decision-making (Shafiei et al. 2012), as well as cognitive flexibility (Butts et al. 2013; Thai et al. 2013). For instance, under certain conditions acute stressor exposition can reduce the ability to suppress a previously learned cognitive set, thus preventing the adoption of new behavioral strategies appropriate to changing circumstances (Butts et al. 2013). Here, we examined for the first time acute stressors effects on performance of goal-directed action in animals. Our data show that an intense acute stressor can render performance of previously acquired instrumental action habitual, possibly due to a compromised retrieval of encoded relationships between actions and their outcome values. Our observation in rats that an acute stressor can shift instrumental responding from goal-directed to habitual control is reminiscent of similar findings in humans (Schwabe et al. 2011). However, as in rats only twofold exposure to an intense physical stressor prompted habitual responding, but not, for instance, concurrent glucocorticoid/noradrenaline activity as observed in humans (Schwabe et al. 2011), the underlying stress signaling pathways may differ across species.
Materials and Methods
All experiments comply with the German Law on Animal Protection and were approved by the proper authorities in Stuttgart, Germany.
Subjects
Experimentally naive male Lister-hooded rats (Harlan-Winkelmann, Borchen, Germany; Charles River, Sulzfeld, Germany), weighing between 190 g and 280 g at the beginning of the experiments, were used. They were housed in groups of four animals in transparent plastic cages (55 × 39 × 27 cm, Ferplast). Throughout the experiments water was available ad libitum and standard laboratory maintenance chow (Altromin) was restricted to 15 g per animal and day to maintain them at ∼85% of their free-feeding weight. All animals were fed at the same time each day after the last training session. Temperature (22°C ± 2°C) and humidity (50% ± 10%) were kept constant in the animal house. In Experiments 1 and 2, animals were kept on a 12-h light–dark cycle (lights on at 8:00 a.m.). Behavioral training and tests were conducted in the light phase. In Experiments 3 and 4, animals were kept on an inverted 12-h light–dark cycle (lights off at 8:00 a.m.). Behavioral training and tests were performed in the dark phase.
Apparatus
Training and tests took place in six identical operant chambers (24 cm × 21 cm × 30 cm, Med Associates) housed within sound-attenuating cubicles. Each operant chamber was equipped with a pellet dispenser that delivered 45-mg grain-based pellets (formula A/I, TestDiet) into a dual pellet/liquid cup receptacle, which was positioned in the middle of the right wall, and a syringe pump that delivered 0.1 mL of a 20% sucrose solution into the same receptacle. Each chamber also contained two retractable levers located on either side of the dual pellet/liquid cup receptacle. A 24-V/3-W houselight mounted on the top center of the left wall illuminated the chambers and an electric fan integrated into the cubicle provided a constant background noise (∼70 dB). A computer with the PC program MED-PC IV controlled the equipment and recorded the data.
Experiment 1: corticosterone/yohimbine effects on goal-directed action
Behavioral procedures
Each magazine and lever-press training session lasted for 30 min and began with the illumination of the houselight and the insertion of the appropriate lever. The sessions ended with the retraction of the lever and the turning off of the houselight.
Magazine and lever-press training
On the first day of training all animals received two magazine training sessions in which both outcomes (pellets and 20% sucrose solution) were delivered on independent random-time schedules (RT-60) with both levers withdrawn. Thereafter, lever-press training began; for half of the animals of each treatment group pressing the left lever earned one pellet and pressing the right lever earned 0.1-mL 20% sucrose solution. The remaining animals were assigned to the opposite action–outcome pairings. All animals received two daily lever-press training sessions in which only one lever and one outcome was available, respectively. After the first training session, there was at least a 2-h break before the second training session began with the opposite lever–outcome pairing. The order of pellet and sucrose sessions was alternated each day. Lever-press training was conducted for 11 consecutive days with progressively leaner random ratio (RR) schedules of reinforcement, except for the first 2 d in which a continuous reinforcement schedule (CRF) was used, as follows: days 1–2 = CRF, P (O|A) = 1.0; days 3–5 = RR-5, P (O|A) = 0.2; days 6–8 = RR-10, P (O|A) = 0.1; days 9–11 = RR-20, P (O|A) = 0.05.
Outcome devaluation, drug administration, and choice test in extinction
On each of the following 2 d, rats were first sated in a counterbalanced manner on one of these two outcomes by prefeeding (selective outcome devaluation), then subjected to the pharmacological stressor, and tested afterward in a two-lever choice task in extinction.
For outcome devaluation, the animals were placed individually in separate transparent plastic cages (42 × 26 × 18 cm, Ferplast) and got ad libitum access to one food outcome for 60 min. On the first test day, half of the animals received 40-g pellets in a feeding bowl and the other half received 150-mL 20% sucrose solution in a drinking bottle. On the second test day, the same procedure was used, but the two types of outcome were switched (animals that received pellets on day 1 received sucrose on day 2, and vice versa), so that finally each animal got both outcomes devalued on consecutive days. Rats received injections of vehicle solution (n = 11) or yohimbine and corticosterone (n = 11) 30 min after the onset of the outcome devaluation (duration 60 min), i.e., rats were removed from the cage with ad libitum access to the food outcome, received i.p. injections, and were placed back into their individual cage for another 30 min with ad libitum access to the food outcome. This protocol was used as it takes about 30 min for yohimbine (Shepard et al. 2004) and corticosterone (Koot et al. 2013) to become behaviorally active after systemic administration. Upon completion of outcome devaluation, animals were tested in the two-lever choice extinction test for 10 min to test for their lever preference, with both levers inserted but without outcome delivery.
Drug administration
Corticosterone and yohimbine drugs were coadministered systemically at a dose of 3 mg/kg (i.p.), respectively, in a volume of 2 mL/kg. Yohimbine was dissolved in distilled water, and corticosterone in 5% DMSO in isotonic saline. Control animals received vehicle injections (2 mL/kg, i.p.).
Data analysis and statistics
Lever training
Lever presses, but not magazine entries, were recorded in each session. Data from lever-press training were subjected to repeated-measures analysis of variance (ANOVA) with training days as within-subject factor and group (to be subjected to vehicle vs. drug treatment) as between-group factor. According to Levene's test, data meet the criterion of homogeneity of variances.
Outcome devaluation
Mean lever press rates for levers that delivered valued and devalued outcome on choice tests days 1 and 2 were subjected to repeated-measures ANOVA with two within-subject factors (devaluation, choice test day) and one between-subject factor (treatment). We included test day as with-subject factor to detect possible order effects of acute stress applied on both choice test days, although in most previous studies, outcome devaluation effects were analyzed across both choice test days (e.g., Lex and Hauber 2010b). A square-root transformation was used for outcome devaluation data to meet the criterion of homogeneity of variances analyzed by means of Levene's test. All statistical computations were carried out with STATISTICA (version 7.1, StatSoft, Inc.). The level of statistical significance (α-level) was set at P < 0.05. Mean lever presses per minute (± SEM) are reported throughout the paper.
Experiment 2: single stressor effects on goal-directed action
Unless otherwise noted, the same procedures as in Experiment 1 were used.
Behavioral procedures
Outcome devaluation, stressor exposure, and choice test in extinction
Outcome devaluation was conducted in a separate cage for 45 min. On the first test day, half of the animals received ad libitum access to 40-g pellets in a feeding bowl and the other half received ad libitum access to 150-mL 20% sucrose solution in a drinking bottle. On the second test day, the same procedure was used, switching the two types of outcome (animals that received pellets on day 1 received sucrose on day 2, and vice versa), so that finally each animal got both outcomes devalued on consecutive days.
Both on the first and second test days, animals exposed to the single stressor (n = 9) received 60-min restraint stress in transparent plastic tubes (height 63.5 mm, min. length 58 mm, max. length 195 mm) after outcome devaluation and prior to the choice test. Control animals not exposed to the single stressor (n = 9) remained upon completion of outcome devaluation in the outcome devaluation cage for an additional 60 min with feeding bowls and drinking bottles removed. Subsequently, animals exposed to the single stressor and controls were tested in the choice test in extinction.
Experiment 3: multiple stressor effects on goal-directed action
Unlike Experiments 1 and 2, Experiment 3 was performed in the dark period of the light–dark cycle because this multiple stressor is highly effective to alter reward-directed responding if applied in the dark period (Pielock et al. 2013). Unless otherwise noted, the same procedures as in Experiment 1 and 2 were used.
Behavioral procedures
Outcome devaluation, stressor exposure, and choice test in extinction
Prefeeding was conducted in a separate cage for 45 min. After outcome devaluation, control animals (n = 12) remained therein for an additional 60 min with feeding bowls and drinking bottles removed. By contrast, other animals (n = 12) were exposed to the multiple stressor for 60 min. This stressor consisted of a combination of acute stressors applied for 60 min in total in a brightly lit novel room with music from a local radio station (∼83-dB sound pressure level [SPL]) (Wong et al. 2007; Chen et al. 2010). First, rats were exposed to restraint for 20 min by placing them into Plexiglas cylinders. Thereafter, rats were placed on an elevated platform (12 × 12 cm, 1 m tall) for 20 min (Chen et al. 2010). Eventually, rats were reexposed to restraint for 20 min by placing them into Plexiglas cylinders. During this restraint, five tail shocks (1 mA, 1 sec) were applied (e.g., Waddell et al. 2010). Subsequently, animals were tested in the extinction test. Note that for these rats examined in the dark phase, stressor exposure in the bright room represents a light pulse that might act per se as additional stressor (Song and Rusak 2000; Zubidat et al. 2007).
Experiment 4: diazepam modulation of multiple stressor effects on goal-directed action
As Experiment 3, Experiment 4 was performed in the dark period of the light–dark cycle. Unless otherwise noted, the same procedures as in Experiments 1–3 were used.
Drug administration
Diazepam (1 mg/kg or 2 mg/kg) (Sigma Aldrich) was dissolved in a vehicle solution of 10% absolute ethanol, 40% polyethylene glycol, and 50% isotonic saline, and administered at a volume of 1 mL/kg (i.p.) 5 min before outcome devaluation, lasting for 45 min (i.e., 50 min before multiple stressor exposure for 60 min). Control animals received vehicle solution at a volume of 1 mL/kg. Previous studies revealed that diazepam at 1 or 2 mg/kg i.p. produced anxiolytic-like effects without marked locomotor inhibition (Johnson 1978; Chaouloff et al. 1997) and did not reduce instrumental responding (Poncelet et al. 1983; George et al. 2009). The following treatment groups were used:
-
Experiment 4A: low-dose diazepam (1 mg/kg) + no stressor (n = 7), low-dose diazepam + multiple stressor (n = 8), vehicle (2 mL/kg) + multiple stressor (n = 8);
-
Experiment 4B: high-dose diazepam (2 mg/kg) + no stressor (n = 8), high-dose diazepam + multiple stressor (n = 7), vehicle (2 mL/kg) + multiple stressor (n = 7).
Experiment 5: single and multiple stressor effects on plasma corticosterone
To assess stressor effects applied in Experiments 2 and 4 on HPA axis activity, we measured plasma corticosterone levels in 36 experimentally naive rats without prior behavioral training and test. Rats were assigned to four treatment groups and exposed to either no stressor (n = 9)/single stressor (n = 9) in the light period (see Experiment 2) or no stressor (n = 9)/multiple stressor (n = 9) in the dark period (see Experiment 3). Trunk blood was collected in rats exposed to the single or multiple stressor (1) before the onset of the specific satiety procedure (n = 3 per group), (2) before the onset of the stressor exposure (n = 3 per group), and (3) after termination stressor exposure (n = 3 per group); respective control rats were left in the home cage rather than exposed to a stressor (see Table 1). The specific satiety procedure (outcome devaluation) and stress protocols were used as described for Experiments 2 and 3.
Rats were anesthetized with an overdose of isoflurane (cp-pharma) and decapitated. Short-term treatment with isoflurane per se did not alter corticosterone levels (Zardooz et al. 2010). Trunk blood was collected into a Falcon tube (50 mL) and after 15-min incubation at room temperature the sample was centrifuged (4000 rpm, 10 min, 8°C). The supernatant was transferred and divided into two 1.5-mL Eppendorf tubes and recentrifuged (4000 rpm, 10 min, 8°C). The blood serum was aliquoted into 0.5-mL Eppendorf tubes and stored at −20°C. Serum corticosterone levels (ng/mL) were measured using the corticosterone ELISA kit RE52211 (IBL International GmbH).
Data analysis and statistics
Plasma corticosterone levels in rats exposed to no, single, or multiple stressors were subjected to an ANOVA with treatment and time point of blood collection as factors. The level of statistical significance (α-level) was always set at P < 0.05. Mean plasma corticosterone levels (± SEM) are reported.
Acknowledgments
This research was supported by the Deutsche Forschungsgemeinschaft (HA2340/8-2) and a grant of the Landesgraduiertenförderung Baden-Württemberg to S.B. We are grateful to I. Nevar for assistance in animal husbandry and S. Mueller for technical assistance.
Footnotes
-
↵1 Corresponding author
E-mail hauber{at}bio.uni-stuttgart.de
- Received August 28, 2013.
- Accepted September 16, 2013.
This article is distributed exclusively by Cold Spring Harbor Laboratory Press for the first 12 months after the full-issue publication date (see http://learnmem.cshlp.org/site/misc/terms.xhtml). After 12 months, it is available under a Creative Commons License (Attribution-NonCommercial 3.0 Unported), as described at http://creativecommons.org/licenses/by-nc/3.0/.
