Projects

Current Research

Spring 2016

           In the Spring I rotated in the Isaacson lab. The main project I worked on was developing a behavioral protocol for mouse auditory reversal learning. My data summary can be found here.

Fall 2016

Acute Stress-Induced Reinstatement of Goal-Directed Behavior

Introduction
Confluent evidence suggests that decision-making involves the recruitment of memories formed by two parallel learning processes, one that governs the development of goal-directed action and another for the acquisition of habits. More specifically, goal-directed behavior reflects the ability to enact purposive action sensitive to outcome-revaluation, or in other words, change behavior appropriately following alterations in the value associated with that action. This contrasts habitual action, which is characterized by an insensitivity to outcome-revaluation and is more akin to stimulus-response behavior independent of purpose; it is based on the history of reinforcement instead of the immediate expected value [1]. Indeed, the pathological inability to alter behavior after outcome-revaluation is thought to underlie the repetitive, compulsive behavior seen in drug addiction [2, 3]. Interestingly, stress, which is thought to potentiate the vulnerability to drug addiction, has also been associated with increases of habitual behavior in humans, however, the mechanism by which this occurs remains to be determined [4, 5].
The rapid change in neuronal chemistry as a result of environmental stress, while beneficial for the bias towards reflexive, survival-oriented actions in the face of danger, may come at a cost to higher cognitive ability. For instance, evidence supports stress-induced disruptions of hippocampal-prefrontal cortex (PFC) synaptic plasticity as a mechanism for concurrent deficits in working memory and behavioral flexibility [6]. Furthermore, as seen previously in humans, chronic stress has been shown to bias decision-making strategies in rodents, such that habitual responses persist when rats are exposed to changes in action-outcome contingency or outcome devaluation [7]. However, the effects of stress on an individual can be highly variable, depending on the severity, predictability and timing of the stressful experience. For example, contrasting what was seen after chronic stress exposure, a single instance of restraint stress led to lever pressing behavior that was sensitive to changes in outcome value, reflecting a goal-oriented decision-making strategy [8]. Therefore, to disentangle the contrasting effects of acute versus chronic stress on decision-making strategies, further studies must be conducted with stressors of varying intensity and length. In addition, it would be interesting to see whether stress would reinstate habitual lever pressing after extinction, along the lines of what might be seen with relapsing drug addiction patients that return to a stressful environment after rehabilitation. Surprisingly, our preliminary data indicated that after a period of extinction, goal-directed, but not habitual behavior, may be reinstated in female mice exposed to acute restraint-stress

Methods
Animals
Sixteen C57BL/6J mice (eleven female) aged 7-10 weeks were housed in groups of one to four with mouse chow and water ad libitum unless stated otherwise, and were kept on a 13/10 hr light/dark cycle. All behavioral experiments were performed during the light portion of the cycle.
Lever Press Training
Before training, mice were food restricted to 90% of their baseline weight, which was maintained for the duration of the experimental procedures. Two days before training, the mice were each placed in individual empty cages for 1 h to habituate the animal to future exposure during devaluation testing. For the training portion, mice were placed in operant chambers in sound attenuating boxes in which they pressed a single lever to the left or right of the feeding chamber in a self-paced manner for a chow pellet outcome.
Each training day, mice were placed in two separate operant chambers with either black-and-white stripes or clear Plexiglas walls to visually cue and differentiate between two contexts. After a training session in one, they were immediately placed in the next. For each mouse, the order of schedule exposure, lever position, and outcome obtained upon lever pressing were kept constant across contexts. However, the context, schedule order, lever position, and outcome earned were counterbalanced across mice. Each training session commenced with illumination of the house light and lever extension and ended following schedule completion or after 60 min, with the lever retracting and the house light turning off.
On the first day, mice were trained to approach the food magazine in absence of a lever within each context on a random time (RT) schedule, in which a pellet reinforcer (20 mg pellet per reinforcer, Bio-Serve formula F05684) was delivered on average every 60 s for a total of 15 min. On the following 3days, mice were trained in each context on a continuous reinforcement schedule (continuous ratio of reinforcement [CRF]]), in which every lever press was reinforced, with the possible number of earned reinforcers increasing across these 3 training days (CRF5, CRF15, and CRF30). In the absence of any predictive cue signaling reward delivery, mice acquired lever-press behavior within 3 days. After acquiring lever-pressing behavior, mice were trained on RI and RR schedules of reinforcement [9] with schedules differentiated by context and the session ending in each context after 15 reinforcers were earned or after 60 min had elapsed. Mice initially pressed under RI30 (on average, one reinforcer following the first press after an average of 30 s) and RR10 (on average, one reinforcer every 10 lever presses) schedules for 2 days, followed by 4 days of RI60 and RR20 training.
Outcome devaluation testing occurred across 2 consecutive days. On the valued day, mice had ad libitum access to the pellet reinforcers for 1 hr in an empty cage before serial, brief, non-reinforced test sessions in the previous RI and RR training contexts. On the Valued day, mice were placed in an empty cage and in the absence of pellet reinforcers, and then underwent
serial, non-reinforced test sessions in each training context. Prefeeding in the Devalued day took
place in a cage separate, but similar from the one in which mice were previously habituated, and the amount consumed was recorded. Order of context exposure during testing was the same as training exposure, with order of devaluation day counterbalanced across mice. Tests in each context were 5 min in duration.
            For the next 4 days after this initial devaluation testing session, mice were placed in serial, non-reinforced extinction sessions in each training context. These extinction sessions were 1 h long in duration. A second outcome devaluation testing session occurred across the next 2 consecutive days. For mice designated to be in the Non-stress group, the devaluation testing was conducted as previously described. Mice that were designated to be in the Stress group were placed in a modified confocal tube for 30 min after prefeeding to induce stress responsivity [10], followed by a 15 min interval, before being placed in serial, non-reinforced extinction sessions in each training context for 5 mins each.

Data Analysis
To investigate the within-subject distribution of lever presses between valued and devalued states, we normalized lever presses for valued and devalued states to total lever pressing (Valued + Devalued) in each context. In addition, we examined the magnitude of outcome devaluation by creating a devaluation index [(Valued lever presses - Devalued lever presses)/Total lever presses] for each mouse in the RR and the RI contexts. Behavioral data was analyzed using repeated-measures ANOVA (Extinction as Repeated Measures Factor) and two-way ANOVA (Value * Context) with post hoc analyses performed using Bonferroni-corrected paired t tests where appropriate.

Results
To assess the effects of extinction and stress over action selection, we used a recently developed within-subject, self-paced, instrumental lever-press task that allowed for the quantification of goal-directed and habitual action in the same animal throughout our experiment [11]. In this task, food-deprived mice are trained to press a lever for a food pellet reward using either a goal-directed or habitual action strategy, the use of which depends on the context cues in their environment. In essence, two contexts were individually paired with one of two different schedules of reinforcement known to bias towards goal-directed or habitual action, random ratio (RR) and random interval (RI), respectively (Fig. 1). Training produced different amounts of lever pressing between the two schedules, however lever-pressing rates were mostly similar (Fig. 2).

Figure 1. Schematic of Experimental Schedule.
Figure 2. Lever Pressing Behavior Throughout Training. (n=16/g)

            In order to assess the existence of a bias towards goal-directed behavior as a result of training, we performed an initial satiety-outcome devaluation test across 2 days (valued [V] and devalued [DV]). In the valued day, mice were placed in an empty-cage for an hour before the test to control for the effects that a lack of satiation may have on lever-press behavior. During the devalued day, outcome devaluation is achieved by placing a mouse in an empty cage for an hour of ad-libitum pre-feeding of the pellet reinforcers, at which point mice have decreased motivation for the future outcome of lever-pressing due to their now sated state. After pre-feeding or not in respective devalued or valued days, mice were placed in each training context in the same serial order in which they were trained, and non-rewarded lever pressing was measured in a 5-minute-long “devaluation test”.
As previously reported, goal-directed behavior is characterized by a sensitivity to a change in motivation for an expected outcome, which contrasts habitual behavior where the opposite is true. Therefore, comparing lever pressing behavior between both days will demonstrate the degree to which mice were sensitive to outcome devaluation; more lever pressing seen in the valued day compared to the devalued day would indicate goal-directed behavior elicited by a motivated state due to a lack of pre-feeding, whereas no difference in lever pressing would be indicative of habitual action.
Contrary to previous reports, initial devaluation testing produced only goal-directed behavior. A two-way ANOVA on our lever pressing measures revealed a main effect of value (F1,15 = 33.43, p < 0.001), with no interaction. This demonstrates a greater amount of lever pressing in both RR and RI contexts during the valued day compared to the devalued day (Bonferroni corrected p < 0.001; Fig. 3). This is further demonstrated by a lack of shift in the distribution of goal-directed behavior between the two contexts, as determined by our devaluation index measure ([(Valued lever presses - Devalued lever presses)/Total lever presses]), in which values closer to 1 are indicative of greater goal-directedness for each training context (Fig. 4). These measures may be demonstrative of a lack of association between the RI reinforcement schedule and its context, hence the persistence of goal-directed lever-pressing in botch RR and RI contexts.

Figure 3. Raw and Normalized Lever Pressing Behavior During First Devaluation Testing Session, Pre-Extinction. (n=16/g)
Figure 4. Devaluation Index During First Devaluation Testing Session, Pre-Extinction. (n=16/g)

            While we could not replicate the bias of RR schedule towards goal-directed action and RI schedule bias towards habitual behavior in this cohort, we nonetheless continued our experiment to assess the effects of an extinction period on the observed goal-directed lever-pressing. The next 4 days were comprised of hour-long sessions of non-rewarding lever-pressing in both RR and RI schedule-associated contexts. A difference in lever-pressing between the two contexts was observed only in the first day of extinction, with the following 3 days demonstrating similar levels of lever-pressing (Fig. 5). To assess the magnitude of extinction, we quantified lever-pressing behavior as a percentage of initial lever-pressing during the first day of schedule training and saw a return to similar levels (Fig. 6).

Figure 5. Lever Pressing Behavior Throughout Extinction Sessions. (n=16/g)
Figure 6. Lever Pressing Behavior Throughout Extinction Using First 60 Min Schedule Training Day as Baseline. (n=16/g)

            After extinguishing lever-pressing behavior to levels seen during the first day of training in both contexts, we performed a second devaluation test session to see whether goal-directed behavior, as indicated by greater lever-pressing during the valued day, would be reinstated in a subset of stressed animals. Following the 4 extinction sessions, our original 16 mouse cohort was subdivided into two groups: Stressed and Non-Stressed. Mice in the Stressed group underwent a 30-minute long period of restraint in a modified confocal tube, followed by a 15-minute break in their home cage, before being placed in the empty cage where pre-feeding occurred or not, depending on the day of the devaluation testing session.
            Comparisons within the pre- and post-extinction devaluation testing sessions in the Non-Stress cohort revealed, via a repeated measures ANOVA, a main effect of Extinction (F1, 16 = 64.329, p < 0.001), a main effect of Value (F1, 16 = 9.246, p = 0.005), and an interaction between Extinction * Value (F1, 16 = 7.247, p = 0.012). The difference between responding on Valued and Devalued days (independent of context) was not apparent following extinction training (Fig. 7), suggesting that extinction training disrupted the effects of outcome devaluation on lever pressing.
            Similarly, comparisons within the pre- and post-extinction devaluation testing sessions in the Stress cohort revealed, via a repeated measures ANOVA, a main effect of Extinction (F1, 16 = 146.879, p < 0.001), a main effect of Value (F1, 16 = 37.121, p < 0.001). and an interaction between Extinction * Value (F1, 16 = 51.665, p < 0.001). Post hoc analyses comparing V and DV states across Pre and Post-Extinction tests revealed a significant difference in Pre RR and Pre RI Extinction testing (assuming based on the graph), that was not apparent Post-Extinction. Stressed mice demonstrated initial goal-directed action, indicated by greater amounts of lever pressing during the valued day both before and after extinction (Fig. 7). However, goal-directed lever-pressing in the Stressed group did not reach the same levels observed pre-extinction, indicating that extinction may have influenced outcome devaluation. No statistical comparisons were made between devaluation index values in the Stressed animals compared to the Non-stressed group post-extinction (Fig. 8).

Figure 7. Devaluation Test Lever Pressing Behavior Before and After Extinction Sessions. (n=8/g)
Figure 8. Devaluation Index Before and After Extinction Sessions in Stressed and Non-Stressed Animals. (n=8/g)

            Stress reactivity differs between the sexes; women have an increased prevalence of stress-related psychiatric disorders compared to men
[12]. In animals, interactions between stress and gonadal hormones, such estrogens and glucocorticoids, are thought to underlie sex differences on stress vulnerability [13]. Due to the greater vulnerability to stress in females, we delineated our data to probe the possibility of sex differences in goal-directed behavior as a result of extinction or restraint stress. Lever pressing behavior during extinction was similar between males and females after the first session (Fig 9) and returned to levels observed during the first day of RR/RI schedule training (Fig 10).

Figure 9. Sex-Differentiated Lever Pressing Behavior Throughout Extinction Sessions. (Males=5/g, Females = 11/g)

Figure 10. Sex-Differentiated Lever Pressing Behavior Throughout Extinction Sessions Using First 60 Min Schedule Training Day as Baseline. (Males=5/g, Female = 11/g)

            In addition, the pre-extinction devaluation test showed that goal-directed lever-pressing levels were similar between males and females in both training schedule contexts (Fig 11; Fig 12). Pre-planned comparisons within the pre- and post-extinction devaluation testing sessions in the Female cohort revealed, via a repeated measures ANOVA, a main effect of Extinction (F1, 10 = 73.919, p < 0.001), a main effect of Value (F1, 10 = 27.676, p < 0.001), a trend towards a main effect of Stress (F1, 10 = 3.936, p = 0.055), and an interaction between Extinction * Value (F1, 10 = 21.216, p < 0.001). While no sound lever-pressing comparisons could be made between Non-stressed male and female mice after extinction due to a lack of animals, Stressed female mice had more goal-directed behavior than Stressed male mice in both training schedule contexts (Fig 11; Fig 12). Furthermore, Stressed female mice had greater goal-directed lever-pressing compared to Non-Stressed female mice after the extinction sessions (Fig 12). These findings were supported when lever-pressing behavior was quantified using the devaluation index, where values closer to 1 are indicative of goal-directed action. Independent of training schedule context, Stressed male mice had reduced goal-directed behavior after extinction, whereas Stressed female mice maintained similar levels of goal-directed behavior that were observed before extinction (Fig 13). In addition, Stressed female mice had greater goal-directed behavior compared to the Non-Stressed female mice, although this may have been due to the difference observed between these two groups before extinction (Fig 14).

Figure 11. Female Lever Pressing Behavior Before and After Extinction. (Pre-Extinction = 11/g, Post-Extinction Non-Stressed = 4/g, Post-Extinction Stressed = 4/g)
Figure 12. Male Lever Pressing Behavior Before and After Extinction. (Pre-Extinction = 5/g, Post-Extinction Non-Stressed = 41/g, Post-Extinction Stressed = 4/g)
Figure 13. Sex-Differentiated Devaluation Index Before and After Extinction in Stress Group. (n = 4/g)
Figure 14. Female Devaluation Index Before and After Extinction Sessions. (Pre-Extinction Non-Stressed = 7/g, Post-Extinction Non-Stressed = 7/g, Pre-Extinction Stressed = 4/g, Post-Extinction Stressed = 4/g)

Discussion
            Overall, our preliminary results show that lever-pressing behavior is reduced after a short extinction period. In addition, outcome devaluation may be reinstated in females following exposure to an acute stressor, although future experiments are warranted.
            For example, a sex-counterbalanced, larger cohort of animals would provide a better comparison between non-stressed animals following extinction. In addition, extending the restraint period to an hour could optimize the stress-response. However, this may not be optimal for the timeline of the pre-feeding and devaluation test, therefore an alternative stressor, such as exposure to fox-urine for 15 minutes, could expedite the time until the devaluation test while still strengthening the stress-response [14]. Furthermore, confirmation of the stress-response should take place before the devaluation test, either behaviorally through the open field test, or through corticosterone level measurements from blood extractions. To confirm whether reinstatement is actually occurring, the results of an initial devaluation testing session after extinction should be compared to a stress-exposed, follow-up devaluation test session. Another limitation to our experiment was a failure to replicate training schedule lever-pressing bias. It would be interesting to assess whether any of our results would hold if the pre-extinction devaluation testing session showed a difference in lever-pressing as a consequence of training schedule contexts.
            The relationship between stress and habits remains to be probed. Contrary to assumptions based on addiction behavior, at least in females, stress did not bias towards habitual behavior following extinction. However, this may have been largely due to our use of food rewards rather than an addictive drug. Of note, outcome devaluation was observed in stressed mice following extinction, bringing into question whether stress led to the recall of the association between lever pressing and reward. Finally, establishing a link between stress and outcome devaluation would do much to increase the repertoire of tools available to assess the action-strategy deficits seen in stress-induced illnesses.

References
1.            Dickinson, A., Actions and Habits: The Development of Behavioural Autonomy. Philosophical Transactions of the Royal Society of London. B, Biological Sciences, 1985. 308(1135): p. 67-78.
2.            Everitt, B.J., et al., Review. Neural mechanisms underlying the vulnerability to develop compulsive drug-seeking habits and addiction. Philos Trans R Soc Lond B Biol Sci, 2008. 363(1507): p. 3125-35.
3.            Voon, V., et al., Disorders of compulsivity: a common bias towards learning habits. Mol Psychiatry, 2015. 20(3): p. 345-52.
4.            Briand, L.A. and J.A. Blendy, Molecular and genetic substrates linking stress and addiction. Brain Res, 2010. 1314: p. 219-34.
5.            Soares, J.M., et al., Stress-induced changes in human decision-making are reversible. Transl Psychiatry, 2012. 2: p. e131.
6.            Cerqueira, J.J., et al., The prefrontal cortex as a key target of the maladaptive response to stress. J Neurosci, 2007. 27(11): p. 2781-7.
7.            Dias-Ferreira, E., et al., Chronic stress causes frontostriatal reorganization and affects decision-making. Science, 2009. 325(5940): p. 621-5.
8.            Braun, S. and W. Hauber, Acute stressor effects on goal-directed action in rats. Learn Mem, 2013. 20(12): p. 700-9.
9.            Derusso, A.L., et al., Instrumental uncertainty as a determinant of behavior under interval schedules of reinforcement. Front Integr Neurosci, 2010. 4.
10.          Zimprich, A., et al., A robust and reliable non-invasive test for stress responsivity in mice. Front Behav Neurosci, 2014. 8: p. 125.
11.          Gremel, C.M. and R.M. Costa, Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat Commun, 2013. 4: p. 2264.
12.          Sandanger, I., et al., Is women's mental health more susceptible than men's to the influence of surrounding stress? Soc Psychiatry Psychiatr Epidemiol, 2004. 39(3): p. 177-84.
13.          Bangasser, D.A. and R.J. Valentino, Sex differences in molecular and cellular substrates of stress. Cell Mol Neurobiol, 2012. 32(5): p. 709-23.
14.          Kapelewski, C.H., et al., Application of a naturalistic psychogenic stressor in periadolescent mice: effect on serum corticosterone levels differs by strain but not sex. BMC Res Notes, 2010. 3: p. 170.

Spring 2016

Memory Dysfunction and Inter-Ictal Spike Activity in Epilepsy Patients

Epilepsy is a common neurological disorder characterized by uncontrollable and atypical brain cell activity. While the majority of epileptic seizures can be attenuated by medicine, a substantial subpopulation of epileptic patients must undergo surgery to remove epileptogenic tissue.

In order to remove epileptogenic tissue, surgeons use intracranial electroencephalography (iEEG) to pinpoint seizure onset zones in the cortex. The signal recorded from iEEG is composed of local field potentials, which represent the potential caused by the summed local currents on the surface of the electrode.

Before this procedure, but after electrode implantation, patients volunteered to undergo sessions testing their short term memory recall.

Example of electrode implants. Credit to Ameya Nanivadekar.
Memory Task. Credit to Ameya Nanivadekar.

Our goal right now is to use that behavioral data and complement it with the simultaneously recorded iEEG to ask whether there is a relationship between the two.

There is imaging evidence that contralateral, homologous, and extratemporal areas of the brain responsed to inter-ictal spikes, indicating that these spikes can affect neuronal activity at a distance through synaptic connections. With the idea that inter-ictal spiking affects functional connections to and from the temporal lobe, where hippocampal disruption may affect memory formation and recall, we asked the following.

Hypothesis: Global Inter-ictal spike activity in treatment resistant epilepsy patients interferes with memory encoding and recall in a short term memory task

Using an inter-ictal spike detection algorithm, we will incorporate a generalized mixed effects logistic regression model to assess the predictive relationship between the detected spikes and patient's memory task performance.

Summary of main findings:

  • Older age associated with decrease in mean recall rates, but not average spike rates during encoding.
  • No gender difference in mean recall and spike rates.
  • Increased number of inter-ictal spikes across all electrodes during the encoding period of a word that failed recall. 
  • Inter-ictal spikes during the encoding period within the seizure onset zone did not have an effect on recall, this effect was seen only when inter-ictal spikes outside of the seizure onset zone were included in our analysis.
  • Inter-ictal spiking during the encoding period of memory formation had an effect on subsequent memory recall most notably in the Fusiform gyrus, otherwise known as the "visual word form area".

Our results suggest that epileptiform spikes disrupt short term verbal memory encoding when they occur in functionally task-relevant tissue, regardless of a patient's seizure onset zone.

Establishing increased inter-ictal spiking as an electrophysiological bio-marker for memory disruption due to epileptiform activity will provide clinicians a key, quantifiable measure to improve epileptic patient neuropsychological evaluation.

Future examination will assess the relationship between our findings and the potential detrimental effects of inter-ictal spike activity during the recall period of the task. In addition, it would be worth assessing whether this effect persists through the patient's clinical evaluation period. For example, will inter-ictal spike activity during memory encoding decrease after removal of the epileptogenic tissue?

Stay tuned.

In the meantime, here's my poster on this topic.

Past Research

Generic Inhibition of the Selected Movement and Constrained Inhibition of Nonselected Movements during Response Preparation

Download Publication.

Dissociating the Influence of Response Selection and Task Anticipation on Corticospinal Suppression During Response Preparation

Download Publication.

Modulation of Motor Excitability through Cognitive Reappraisal: A TMS Study

Download Publication.

Post Archive

Powered by Blogger.