How the Brain Learns to Predict What to Fear Based on Past Experiences

A new Rutgers-led study offers potential ways to treat anxiety and PTSD

Rutgers researchers are revealing how the brain decides which sensory experiences previously associated with an adverse experience should trigger a fear response.

When brain mechanisms for fear‑related decision-making break down, the learned fear response can drive maladaptive behaviors even when experiencing situations that don’t predict a threat. Individuals with anxiety and post-traumatic stress disorders (PTSD) often exhibit these maladaptive behaviors. 

The Rutgers-led study, published in Nature Communications, was led by researchers in the laboratory of John McGann, a professor in the Department of Psychology at Rutgers University–New Brunswick.

A single sensory stimulus, whether an odor, a sound or an image, activates many neurons throughout the brain, which often overlap for stimuli that are similar. When the brain has to decide which stimuli to avoid after a frightening experience, it must determine how much overlap in neural response is enough to treat not only the original threatening stimulus as dangerous, but also different yet similar stimuli. 

These findings suggest that the brain begins determining where to draw the boundary between dangerous and safe stimuli much earlier in sensory processing than previously expected.

John McGann

Professor, Department of Psychology, Rutgers University–New Brunswick

To distinguish the original versus related sensory stimuli (“sensory discrimination”), the brain must create boundaries (“fear boundaries”) to limit the neurons that are activated.

To understand how smell-associated fear responses to an original and related stimuli are processed in the brain, McGann and fellow researchers trained mice to associate a specific odor with a mild threat and then measured neural activity as the mice responded both to the conditioned odor and to a gradient of new, similar and dissimilar smells. 

Natalia Efimova, a first‑year doctoral degree student in McGann’s lab, contributed to the study. The researchers measured the neural activity in the olfactory bulb, a brain region involved in the initial processing of the sense of smell. By delivering precise pharmacological micro-infusions of neuromodulatory drugsdirectly into the olfactory bulb, researchers manipulated local inhibitory circuits and tested whether early sensory processing in the olfactory bulb directly controls fear boundaries and sensory discrimination.

“The study showed that, for smells, sensory discrimination and establishment of fear boundaries occur in the early sensory processing part of the brain,” Efimova said. 

By manipulating inhibitory circuits in the olfactory bulbs of mice, the researchers could artificially change how readily the animals generalized their fear to new but related odors. Fear learning naturally changes this inhibitory circuit throughout a set of neurons activated by the threat-predictive odor and also in some neurons physically adjacent to them. 

As a result, when the animal encounters a new but similar smell that activates an overlapping or neighboring set of neurons, some of those neurons already have been modified by the experience of danger. Depending on the degree of overlap and how neural representations are organized spatially, this can cause the new odor to be interpreted as threatening and helps explain why the animal exhibits a fear response to a stimuli that was never directly paired with the threat.

“These findings suggest that the brain begins determining where to draw the boundary between dangerous and safe stimuli much earlier in sensory processing than previously expected,” McGann said. 

He added that sensory training or sensory exposure therapy coupled with targeted therapeutic interventions using drugs that modulate neurotransmission might be clinically beneficial in treating fear generalization in anxiety and post-traumatic stress disorders.

Inhibitory processing in the olfactory bulb isn’t simply passive transmission of sensory information. It actively modulates how far the sensory stimuli-induced neuronal activation spreads following a fearful experience, extending the influence of the experience across different stimuli. The sensory processing region itself helps decide whether a new smell is similar enough to a previously threatening odor to trigger fear, or different enough to be treated as safe.

“The results of the study suggest that overly generalized fears, where the normal fear‑response gradient collapses and fear becomes broadly generalized to all stimuli equally, may result from neuroplasticity not only in higher brain regions known for emotion and memory, but also from early changes in how the brain represents the stimulus itself,” McGann said.

This research was funded by the National Institute of Mental Health and National Institute on Deafness and Other Communication Disorders, grant number R01MH101293. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.