Behavioral microanalyses refine sign-tracking characterization and uncover different response dynamics during omission and extinction learning

Sign-tracking is not well characterized by CS lever presses alone

An attraction to reward-predictive CSs does not always result in vigorous, physical CS engagement. Thus, we first investigated the relationship between standard sign-tracking measures of lever pressing and individual behaviors gathered from video monitoring data. After 12 Pavlovian conditioning sessions, many animals acquired the sign-tracking response, which is displayed by an increase in lever presses per minute (ppm) (Fig. 1A). Lever presses increased significantly over sessions. Subjects also differentiated between lever types, with significantly lower press rates on the CS− lever than the CS+ lever (Fig. 1A; linear mixed effects [LME] model; effect of session: est: 2.98; confidence interval [CI]: 2.17– 3.78; P < 0.001; effect of lever: est: −21.13; CI: −28.73 to −13.52; P < 0.001; interaction between session and lever: est: −3.82; CI: −4.85 to −2.79; P < 0.001). Over Pavlovian conditioning sessions, food cup entries during CS+ lever presentations significantly decreased (Fig 1B; LME model; effect of session: est: −1.53; CI: −2.40 to −0.67; P = 0.0011). We then calculated the Pavlovian conditioned approach (PCA) (Meyer et al. 2012) index of each animal (Fig. 1C) based on the number, latency, and probability of lever deflections and food cup entries during the 12th and final session of Pavlovian conditioning. Many of the animals in this experiment were categorized as sign-tracking, however, there was still a high concentration of identified intermediates.

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Figure 1.

Detailed behavioral analyses are a more suitable means to study sign-tracking responses. (A) Lever ppm on the CS+ (teal) and CS− (orange) levers throughout Pavlovian conditioning. Error ribbons display ± SEM. (B) Average food cup entries during CS+ presentations throughout Pavlovian conditioning. Error ribbons display ± SEM. (C) Histogram of calculated PCA indexes during session 12. Standard PCA index cutoffs for goal-tracking and sign-tracking are displayed by dotted lines at −0.5 and 0.5, respectively. (D) Illustrations of behaviors that were scored in session 12. Behaviors range from non-sign-tracking to increased sign-tracking intensity from left to right. (E) Heatmap depicting the detailed behavioral fingerprints of all animals in session 12. Darker blue regions indicate higher engagement with those behaviors, and lighter green to white represents the least engagement with those behaviors. Subjects are arranged in order of their calculated PCA index, from −1 to 1. PCA indexes of −0.5, 0, and 0.5 are marked by dotted black lines. (F) Scatterplot of behavioral intensity (behaviors in D scaled from −1 to 6, −1 representing food cup directed, and 6 representing lever bite) and CS+ lever ppm in session 12. (G) Scatterplot of the percentage of lever-directed behaviors scored (behaviors from lever orient through lever bite in D vs. food cup–directed and non-task-directed behaviors) and lever ppm in session 12. For all plots, asterisks (*) represent statistical significance (P < 0.05).

We next sought to dig deeper into the detailed behaviors to characterize the structure and variability of the responses. Several individual behaviors were observed (Fig. 1D) from video recordings of the animals during their final Pavlovian conditioning session. These behaviors ranged from more “goal-tracking”-like or non-task directed (e.g., grooming), to quite vigorous, more consummatory sign-tracking, such as repeated biting of the lever. We mapped the behavioral “fingerprints” (i.e., charting the topography of the individual behaviors measured through video-guided analysis of each animal) of the CRs from the final session, then aligned them with their calculated PCA index (Fig. 1E). Notably, most of the animals that were identified as intermediates using the PCA index (PCA index of −0.5 to 0.5) appeared to engage primarily in sign-tracking behaviors, including quite intense behaviors such as biting and grabbing, in addition to some that do not involve physically touching the lever (e.g., sniffing). Most PCA-classified intermediates did not engage in many food-cup directed behaviors, which led us to believe that many of these animals were miscategorized, and instead were truly sign-trackers.

To explore this discrepancy between video-captured behavior and the PCA index further, we examined the relationship between an animal's physical and consummatory “intensity” of behavior. Intensity was given a scale in which goal-tracking behavior was assigned to −1, and sign-tracking behaviors ranged from 1 to 6 with the most physical and consummatory behaviors assigned as 6 (Fig. 1F). While the intensity score and lever presses were significantly correlated (Spearman rank-order correlation: ρ = 0.383; P = 0.0159), this effect may be driven by animals who goal-track that do not press the lever and engage only with the food cup. Meanwhile, many animals who engage in highly appetitive and more vigorous sign-tracking behaviors had a vast range of lever press rates ranging from 0 to >40 ppm. This could potentially be due to smaller variabilities in force or vigor within each individual behavior, in which some animals will engage in more intense behaviors but are still not forceful enough to cause lever deflection.

Finally, to confirm that the “intermediate” animals were truly sign-trackers, we assessed the proportion of behaviors scored that were lever directed versus non-task directed or food cup directed (Fig. 1G). A similar pattern was discovered, in which higher proportions of lever-directed behaviors were positively correlated with higher lever press rates (Pearson correlation: r = 0.401; P = 0.0115). A large range of press rates was once again found across animals, and even in cases when an animal's recorded behaviors were almost entirely lever-directed. This result could occur for a couple of reasons. One is that the animal's response was lever-directed, but its lever interaction was not vigorous enough to cause deflections. For example, one animal in the data set had extremely low press rates in addition to zero food cup entries during the cue. This animal could be mistaken as one that has failed to develop a CR. However, video data revealed that the animal was sniffing, orienting to, and lightly contacting the lever, which is instead indicative of a sign-tracking CR. This instance contrasts with another example animal that pressed at a much higher rate but engaged almost entirely in biting and grabbing. In this case, the detailed behaviors were aligned with automated measures. Interestingly, many animals who were categorized as “intermediates” through the PCA index also engaged in behaviors that were still just as vigorous (i.e., biting and grabbing) as animals who were considered as sign-trackers with a PCA index of at least 0.5. This observation could further imply that animals could engage in very intense behaviors, yet still not deflect the lever as much as others who have similar sign-tracking topographies. Therefore, lever deflections alone cannot reveal whether an animal is or is not engaging in sign-tracking responses. We recommend, rather than categorizing animals by purely their lever-cue deflection rates and latencies (and magazine/food cup entries), that animals instead could be categorized as sign-trackers if their behaviors are cue-directed at least 75% of the time. This approach is comparable to the PCA index cutoff of +0.5 to be considered a sign-tracker, but would lead to more accuracy in behavioral categorization. True categorization of Pavlovian CRs can only be gleaned from video analyses (which could include detailed behavior analyses, or perhaps more simply, animal position and orientation tracking).

Individual behaviors cannot explain lever deflections

We next set out to assess whether any particular behaviors resulted in more or fewer lever deflections (Fig. 2A). We only observed that animals who engaged in more grabbing behaviors had significantly more lever deflections (Pearson correlation: r = 0.416; P = 0.00837), and that more magazine directed behaviors resulted in significantly fewer lever deflections (Pearson correlation: r = −0.399; P = 0.0118). Biting (Pearson correlation: r = 0.180; P = 0.276), heavy contacting (Pearson correlation: r = 0.0961; P = 0.560), light contacting (Pearson correlation: r = 0.211; P = 0.197), orienting (Pearson correlation: r = −0.0962; P = 0.560), sniffing (Pearson correlation: r = −0.297; P = 0.0677), and non-task-directed behaviors (Pearson correlation: r = −0.236; P = 0.148) did not significantly correlate with lever deflections. We collapsed these behaviors to reduce noise and ran similar linear regressions with lever presses per minute. Biting, grabbing, and heavy contacting were collapsed into “high” vigor behaviors, while light contacting, sniffing, and orienting were compiled into “low” vigor behaviors (Fig. 2B). High-vigor behaviors were correlated with lever pressing (Pearson correlation: r = 0.389; P = 0.0142). No correlation was found between lower vigor behaviors and lever pressing (Pearson correlation: r = −0.0602; P = 0.715).

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Figure 2.

Individual behaviors cannot reliably predict lever pressing. (A) Scatterplots of individual behaviors (left to right, top to bottom: lever bite, lever grab, heavy contact, light contact, lever orient, lever sniff, non-task directed, food-cup directed) and lever presses per minute during session 12 of Pavlovian conditioning. Linear models are depicted in each plot. (B) Scatterplots of collapsed high (bite, grab, heavy contact; top) and low (light contact, orient, sniff; bottom) vigor lever-directed behaviors and lever presses per minute during session 12 of Pavlovian conditioning. Linear models are depicted in each plot. For all plots, asterisks indicate significant results. (C) Pearson correlation matrix of each individual behavior. Dark purple indicates R-values closer to 1, and dark green indicates R-values closer to −1. For all plots, asterisks (*) represent statistical significance (P < 0.05).

As noted, animals may have specific choreographies of individual behaviors that are strung together when they are sign-tracking. Choreographies vary and are not identical between all sign-tracking animals (see Fig. 1E). For example, some animals engage in more approach, rather than consummatory, behavior (e.g., sniffing, rather than biting). In line with a phenotype of less vigorous, “approach”-style sign-tracking responding, we found that the less vigorous behaviors such as lighter contacts, sniffing, and orienting, tended to be strongly correlated with each other (Fig. 2C). The most intense or consummatory behaviors (biting, grabbing, and heavy contacting) were weakly correlated with each other, but were negatively associated with all other behaviors. This may indicate a separate phenotype of “intense” or “consummatory” sign-tracking. We further uncovered potential intermediate behavioral choreographies, as food cup directed behaviors were strongly correlated with both lever sniffing and orienting.

Overall, even detailed behavior analyses could not fully explain if an animal will press lever CSs more or less. In other words, while lever deflections are generally correlated with sign-tracking responding, they are not easily predicted by any particular type of sign-tracking behavior. Even some very intense behaviors such as biting and grabbing the lever cue will not reliably result in deflection all the time (Fig. 2A; see also Fig. 1E). Thus, it is difficult to know if animals are engaging with the lever CS or not based on lever deflection alone, which can sometimes render it as an unreliable measure. Instead, as stated, incorporating a more detailed array of behavioral factors (or, possibly, the number of cue-directed behaviors or amount of time spent attending to cues) could lead to more accurate representations of animal behavior as regards sign-tracking. Detailed mapping of behaviors is also more favorable when discussing whether an animal is “more” or “less” intensely sign-tracking, as pressing alone sometimes can miss less intense behaviors and, in some examples, miss even the most vigorous sign-trackers.

Behavioral details provide insight into underlying, dynamic sign-tracking structures

Omission (known also as negative automaintenance) (Stiers and Silberberg 1974; Locurto et al. 1976), in which lever deflections result in the cancellation of reward delivery on that trial (Fig. 3A), is a striking example of how cues retain motivational value even when engagement can be detrimental. It is also a situation in which inaccurate conclusions will result when only lever-cue pressing is considered as the primary measure of sign-tracking. Behavioral details are important to studying sign-tracking animals in this omission task, as they reveal that animals do not cease responding, but rather alter the structure of responding to meet the new contingency while continuing to sign-track (Davey et al. 1981; Eldridge and Pear 1987; Chang and Smith 2016; Townsend et al. 2023). When only automatic readouts of lever deflection are observed in this task, an incorrect conclusion would be reached that sign-tracking decreases because lever pressing diminishes. Thus, here we use an omission task to (1) replicate prior findings, (2) further evaluate behavioral sign-tracking details, and (3) provide a foundation of comparison with extinction conditions as below.

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Figure 3.

Detailed behavior analysis is integral to accurately capturing behavioral changes in omission. (A) Visualization of omission task. (B) Lever presses per minute (ppm) on the CS+ (teal) and CS− (orange) levers from the final Pavlovian conditioning session (12) through omission (13–17). Error ribbons display ± SEM. (C) Average food cup entries during CS+ presentations from the final Pavlovian conditioning session (12) through omission (13–17). Error ribbons display ± SEM. (D) Line plot of the PCA index change for each individual subject starting in the final Pavlovian conditioning session (session 12) through the final omission session (17). (E) Line plots of each individual animal's percentage of lever directed behaviors (behaviors from lever orient through lever bite in Fig. 1D vs. food cup directed and non-task directed behaviors) starting from the final Pavlovian conditioning session (session 12) through omission (sessions 13–17). (F) Line plots of each individual animal's sign-tracking intensity (behaviors in Fig. 1D scaled from −1 to 6, −1 representing food cup directed, and 6 representing lever bite) starting from the final Pavlovian conditioning session (session 12) through omission (sessions 13–17). (G) Scatterplot of the intensity of sign-tracking behavior and the amount of cancelled (deflected) trials. (H) Heatmap depicting the detailed behavioral fingerprints of animals in the final Pavlovian conditioning session (12). Darker blue regions indicate higher engagement with those behaviors, and lighter green to white represents the least engagement with those behaviors. Subjects are arranged in order of their calculated PCA index, from −1 to 1. PCA indexes of −0.5, 0, and 0.5 are marked by dotted black lines. (I) Heatmap depicting the detailed behavioral fingerprints of all animals in the final omission session (17). Darker blue regions indicate higher engagement with those behaviors, and lighter green to white represents the least engagement with those behaviors. Subjects are arranged in order of their calculated PCA index during this session (session 17), from −1 to 1. PCA indexes of −0.5, 0, and 0.5 are marked by dotted black lines. (J) Heatmap depicting the percent change in individual behaviors of all animals from session 12 to session 17. Orange indicates negative percent change, and blue represents positive percent change. Subjects are grouped by their initial, session 12 PCA-derived categorization (<−0.5: left; >−0.5 and <0.5: middle; >0.5: right), and ordered from −1 to 1 from left to right, as in H. For all plots, asterisks (*) represent statistical significance (P < 0.05).

As expected, animals decreased their lever deflection rate as they learned the omission rules over five sessions (Fig 3B; LME model; effect of session: est: −7.16; CI: −9.95 to −4.38; P < 0.001; effect of lever: est: −143.93; CI: −196.30 to −91.56; P < 0.001; interaction between session and lever: est: 7.50; CI: 3.91–11.08; P < 0.001). Food cup entries moderately increased over sessions (Fig. 3C; LME model; effect of session: est: 2.78; CI: 0.55–5.01; P = 0.015), in line with previous reports of a food cup “checking” behavior that develops in some sign-trackers during this omission task (Townsend et al. 2023). The PCA indexes tended to decrease after animals completed omission (Fig. 3D) (t = 3.67; P = 0.002), reflecting the reduction in lever pressing and increasing food cup entries. However, there was a highly variable range of PCA scores that erroneously indicated that many of these animals were not sign-tracking, during the final session (session 12) of Pavlovian conditioning as well as after the completion of the task (session 17) (Fig. 3D). Upon closer look at the proportion of behaviors scored, most of these animals remained at the lever at least 80% of the time, even as omission experience progressed over days (Fig. 3E). This reveals that animals were indeed continuing to sign-track throughout omission learning, despite the significant drop in lever press rates and the ranging PCA indexes. Detailed behavior analyses can explain this decrease. Subjects did not cease sign-tracking, but instead tended to decrease the intensity of their sign-tracking responses to incorporate less vigorous behaviors, such as sniffing, that do not lead as often to lever deflection (Fig. 3F).

The behavioral intensity of sign-tracking responses did not correlate with how many cancelled trials (i.e., a trial in which the lever was pressed) an animal would accumulate over a session (Spearman rank-order correlation: ρ = −0.086; P = 0.718) (Fig. 3G). This was expected given that sign-tracking behaviors before omission were found to be unreliable predictors of lever pressing (Fig. 2). This finding of a similar lack of correlation during omission underscores the conclusion that animals can robustly sign-track without causing lever deflection.

We further charted the topography of the individual behaviors measured through video-guided analysis of each animal, creating fingerprints of the CRs that we aligned with their calculated PCA index on their final Pavlovian conditioning session (session 12) (Fig. 3H) and the final omission session (session 17) (Fig. 3I). The individual behaviors observed between these two sessions were markedly different. A strong bias toward more intense behaviors, such as biting and grabbing, was seen before omission in session 12 (Fig. 3H). However, once animals learned the omission contingency, animals tended to avoid touching the lever while still maintaining their attraction to it. Less intense behaviors such as orienting, sniffing, and light lever contacts thus became more prominent (session 17) (Fig. 3I). Of note, a larger range of animals were classified as intermediates rather than sign-trackers based on the PCA index during omission (Fig. 3D,I,J) compared to the last acquisition session (Fig. 3H). These animals did still overwhelmingly display lever-directed behaviors (Fig. 3J).

Behavioral details can confirm the presence or absence of sign-tracking responses

Extinction, in which cues no longer predict reward delivery (Fig. 4A), can also be used to test sign- and goal-tracking response persistence when task conditions change. When sign-tracking animals undergo extinction, the presumption is that animals will decrease responding entirely, as the cue no longer holds any predictive value for reward. In a prior study, sign-trackers were found to slowly extinguish their response as a form of persistence (Fitzpatrick et al. 2019). Here, we found that lever-pressing events were readily extinguished over the five extinction sessions (Fig. 4A,B; LME model; effect of session: est: −11.02; CI: −14.09 to −7.95; P < 0.001; effect of lever: est: −190.14; CI: −240.74 to −139.54; P < 0.001; and interaction between session and lever: est: 10.84; CI: 7.38–14.30; P < 0.001). This reduction in lever presses during extinction was indistinguishable to the reduction in lever presses during omission (see Fig. 3B). A slight decline in food cup entries on the first session of extinction was also observed, which was likely driven by the few goal-tracking animals (Fig. 4C). However, there were no significant effects of session on food cup entries over all of extinction (LME model; no effect of session: est: −1.84; CI: −4.76 to 1.08; P = 0.215). As with omission, PCA indexes were variable on the final Pavlovian conditioning session (session 12) (Fig. 4D). However, the PCA indices of animals after completion of the omission task differed significantly from their initially calculated PCA index on the last acquisition day (t = −2.73; P = 0.014) (Fig. 4D), with many animals trending closer to a PCA index of 0.

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Figure 4.

Detailed behavior analyses align with lever presses in extinction. (A) Visualization of extinction task. (B) Lever ppm on the CS+ (teal) and CS− (orange) levers from the final Pavlovian conditioning session (12) through extinction (13–17). Error ribbons display ± SEM. (C) Average food cup entries during CS+ presentations from the final Pavlovian conditioning session (12) through extinction (13–17). Error ribbons display ± SEM. (D) Line plot of the PCA index change for each individual subject starting in the final Pavlovian conditioning session (session 12) through the final extinction session (17). (E) Line plots of each individual animal's percentage of lever directed behaviors (behaviors from lever orient through lever bite in Fig. 1D vs. food cup directed and non-task directed behaviors) starting from the final Pavlovian conditioning session (session 12) through extinction (sessions 13–17). (F) Line plots of each individual animal's sign-tracking intensity (behaviors in Fig. 1D scaled from −1 to 6, −1 representing food cup directed, and 6 representing lever bite) starting from the final Pavlovian conditioning session (session 12) through extinction (sessions 13–17). (G) Heatmap depicting the detailed behavioral fingerprints of all animals in the final Pavlovian session (12). Darker blue regions indicate higher engagement with those behaviors, and lighter green to white represents the least engagement with those behaviors. Subjects are arranged in order of their calculated PCA index, from −1 to 1. PCA indexes of −0.5, 0, and 0.5 are marked by dotted black lines. (H) Heatmap depicting the detailed behavioral fingerprints of all animals in the final extinction session (17). Darker purple regions indicate higher engagement with those behaviors, and lighter pink to white represents the least engagement with those behaviors. Subjects are arranged in order of their calculated PCA index, from −1 to 1. PCA indexes of −0.5, 0, and 0.5 are marked by dotted black lines. (I) Heatmap depicting the percent change in individual behaviors of all animals from session 12 to session 17. Orange indicates a negative percent change, and blue represents a positive percent change. Subjects are grouped by their initial, session 12 PCA-derived categorization (<−0.5: left; >−0.5 and <0.5: middle; >0.5: right), and ordered from −1 to 1 from left to right, as in G. For all plots, asterisks (*) represent statistical significance (P < 0.05).

Due to the similarities in lever pressing and PCA index dynamics to omission, we decided to then investigate behavioral details to further confirm that animals did indeed extinguish responding rather than continuing to be cue-directed without deflections as seen in omission. The proportion of lever-directed behaviors scored from video monitoring declined in most animals during extinction, suggesting that the animals did indeed extinguish their sign-tracking responses (Fig. 4E). In line with this result, the intensity of sign-tracking diminished in nearly all animals, ultimately hovering around the low score of 0-to-1 (i.e., these are non-lever directed and lever orienting behaviors) (Fig. 4F).

Behavioral fingerprints, or a mapping of the topography of the individual behaviors measured through video-guided analysis of each animal, were assessed on the final Pavlovian conditioning session (Fig. 4G) and the final extinction sessions (Fig. 4H). These analyses showed that animals do not alter their structure of sign-tracking as they do in omission (Fig. 3H), but instead, they cease lever-directed behaviors entirely (Fig. 3I). By the final extinction session (17), animals were almost entirely engaging in non-task directed behaviors during cue presentations (Fig. 4H,I). Peculiarly, the PCA index still identified some animals as sign-trackers despite the fact that they did not engage in cue-directed behaviors much, if at all. This could occur, for example, when the animal deflects the lever only once or twice during the whole session and never enters the magazine, leading to a skewed and thus misleading PCA calculation. In short, the decrease in lever pressing in extinction did accurately reflect the reduction of sign-tracking, while the similarly reduced pressing in omission did not capture the underlying persistence in sign-tracking.