Summary: Have you ever wondered why a memory can feel as vivid as a photograph? A groundbreaking study has finally cracked the “neural code” behind visual imagination. By recording the electrical activity of individual neurons in patients with implanted electrodes, researchers discovered that imagining an object reactivates the exact same brain cells used to see it in the first place.
This shared biological mechanism explains why our mental theater feels so real and provides a new roadmap for treating disorders like PTSD and OCD, where intrusive, vivid imagery can become a source of distress.
Key Facts
- The Shared Blueprint: The study found that 40% of the neurons activated when looking at a face or object “fired” in the exact same pattern when the participant later imagined that same image from memory.
- The Fusiform Gyrus: Researchers focused on this high-level visual processing area. By using AI to decode the “language” of these neurons, they could predict exactly what a person was imagining based solely on brain activity.
- Generative AI Verification: The team used AI to create entirely new, “never-before-seen” images based on the neural code they discovered, then verified that the brain responded to these AI-generated images as predicted.
- Clinical Potential: Understanding this code is vital for treating conditions marked by “uncontrolled vivid imagery.” By knowing how the brain “re-creates” images, doctors may eventually find ways to dial down the intensity of traumatic flashbacks.
- Cross-Species Confirmation: The work confirms that the neural code for object recognition previously found in primates is also present in humans, serving as the biological basis for our imagination.
Source: Cedas Sinai
Why can images of things we have seen seem so real when we later recall them from memory?
A new study led by Cedars-Sinai Health Sciences University investigators sheds light on the answer.
The study shows that the same brain neurons are activated when we imagine something and when we perceive something. The research, led by Cedars-Sinai, is the first to provide a detailed understanding of the shared mechanism that underlies visual perception and creation of mental images in the human brain.
It was published in the journal Science.
“We generate a mental image of an object that we have seen before by reactivating the brain cells we used to see it in the first place,” said Ueli Rutishauser, PhD, director of the Center for Neural Science and Medicine and professor of Neurosurgery, Neurology and Biomedical Sciences at Cedars-Sinai Health Sciences University, and the study’s joint senior author.
“Our study revealed the code that we use to re-create the images.”
The findings provide a biological basis for visual imagination, a process that is also critical for creative arts.
“Further insight into this neural process has the potential to open pathways toward developing new therapies for post-traumatic stress disorder, obsessive-compulsive disorder, and other mental conditions that involve uncontrolled vivid imagery,” said Adam Mamelak, MD, director of the Functional Neurosurgery Program and professor of Neurosurgery at Cedars-Sinai, and co-author of the study.
To conduct the study, investigators asked 16 adults with epilepsy, who had electrodes temporarily implanted in their brains for diagnosing their seizures, to view a series of images of faces and objects.
After viewing them, a subset of the participants were asked to imagine those same images from memory. Meanwhile, researchers recorded the electrical activity of hundreds of individual neurons in each participant’s brain.
When the patients viewed the images, neurons were activated in their fusiform gyrus, an area of the brain essential for high-level visual processing, particularly for faces. For 80% of the visually responsive neurons recorded in the study, the researchers uncovered the aspects of the images they reacted to, thereby revealing their neural code.
When the patients later imagined the images, about 40% of these neurons reactivated using the same code, thereby recreating the pattern of activity that occurred during the initial viewing of the images.
“Advanced artificial intelligence tools were critical to our investigation at all stages,” said Varun Wadia, PhD, a postdoctoral scientist in Rutishauser’s laboratory and first author of the study.
“We used deep visual neural networks to create numerical descriptions of objects so that we could understand the neurons’ code. We then verified the code by using generative AI to create never-before-seen images and correctly predict the brain’s responses to these images.”
The research builds on the work of Doris Y. Tsao, PhD, of the University of California, Berkeley, who is co-senior author on the study. She identifed the neural code for object recognition in nonhuman primates. The current study reveals that the same neural code is present in humans and that it explains visual imagination.
“These findings support the idea that imagining and seeing share a common neural code and may have important implications for understanding psychiatric disorders marked by disruptions in mental imagery and reality discrimination,” said Hermon Gebrehiwet, DrPH, program officer at the National Institutes of Health.
Still to be determined are what triggers the neural reactivation the investigators found, and how memories lead to reactivation of just the right subset of neurons needed, the investigators said.
Other Cedars-Sinai authors include: C. M. Reed, J. M. Chung, and L. M. Bateman
Funding: The work was supported by the National Institutes of Health’s Brain Research Through Advancing Innovative Neurotechnologies® Initiative, or The BRAIN Initiative® (U01NS117839 to UR), the Howard Hughes Medical Institute (DYT), the Simons Foundation Collaboration on the Global Brain (UR and DYT) and the Chen Center for Systems Neuroscience at Caltech (DYT).
Key Questions Answered:
A: While the code is the same, the study found that only about 40% of the neurons reactivate during imagination compared to 100% during actual sight. This “lower volume” helps your brain distinguish between the external world (reality) and internal thoughts (imagination). Disorders like schizophrenia may involve a breakdown in this “volume control.”
A: We are getting closer. By using deep visual neural networks, researchers could translate electrical signals back into numerical descriptions of objects. While we can’t record a “movie” of your thoughts yet, we can now identify the specific categories of faces or objects you are visualizing with high accuracy.
A: It suggests a biological basis for it. People with “hyperphantasia” (extremely vivid mental imagery) likely have a more efficient reactivation process, where a higher percentage of their visual neurons “fire” during imagination, making their internal world feel nearly identical to the physical one.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- Journal paper reviewed in full.
- Additional context added by our staff.
About this hyperphantasia and visual neuroscience research news
Author: Cara Martinez
Source: Cedars-Sinai Medical Center
Contact: Cara Martinez – Cedars-Sinai Medical Center
Image: The image is credited to Neuroscience News
Original Research: Closed access.
“A shared code for perceiving and imagining objects in human ventral temporal cortex” by V. S. Wadia, C. M. Reed, J. M. Chung, L. M. Bateman, A. N. Mamelak, U. Rutishauser, and D. Y. Tsao. Science
DOI:10.1126/science.adt8343
Abstract
A shared code for perceiving and imagining objects in human ventral temporal cortex
INTRODUCTION
Mental imagery refers to our brains’ capacity to generate percepts, emotions, and thoughts in the absence of external stimuli. This ability allows us to generate art, simulate actions and outcomes, remember previous experiences, and imagine new ones. Uncontrolled mental imagery can contribute to psychological disorders, including anxiety, schizophrenia, and posttraumatic stress disorder.
Despite its importance in our lives, little is known about the single-neuron mechanisms of mental imagery. Neuroimaging results support a long-standing theory that imagery of a given sense is subserved by the reactivation of that specific sensory cortex. However, these studies lack the resolution to discern whether it is the same neurons or separate circuitry roughly located in the same regions that reactivates.
RATIONALE
We investigated the single-neuron mechanisms of visual imagery by recording single neurons in human patients implanted with electrodes to localize their focal epilepsy as they viewed and subsequently imagined objects. We focused our investigations on the ventral temporal cortex (VTC), a part of the temporal lobe dedicated to representing visual objects.
We first determined the code for visual objects. We found that as in macaques, neurons in human VTC represent objects by using a distributed axis code. This code emphasizes the geometric picture that neurons project incoming stimuli—formatted as points in feature space—onto specific preferred axes and respond proportionally to the projection value. We then examined whether this code is reactivated during imagery.
RESULTS
We recorded 714 neurons in the human VTC across 16 patients as they viewed visual objects. A majority of neurons (456 of 714) were visually selective for one of the five object categories used (faces, plants, text, animals, and objects). To represent general objects with arbitrary features, we built a low-dimensional object space using the unit activations of deep networks trained to perform object classification.
Nearly ~80% (367 of 456) of all visually responsive single neurons were significantly axis tuned. We used this axis code to reconstruct objects and generate maximally effective synthetic stimuli. Last, we recorded the responses of the same neurons in a subset of patients (6 of 16) as they imagined the same objects.
Mean responses to perceived and imagined objects were comparable, with some neurons active only during perception, some only during imagery, and some during both. In particular, ~40% (43 of 107) of axis-tuned VTC neurons recorded during the imagery task reactivated, and the responses during imagery of individual neurons were proportional to the projection value of those objects onto the neurons’ viewing axes. We used this observation to reconstruct imagined objects while still easily distinguishing whether those objects were viewed or imagined.
CONCLUSION
We leveraged the opportunity to record from the same population of VTC neurons in humans as they viewed and imagined objects. Neurons use an axis code to represent visual objects, and neural activity during imagination reactivates this code. These findings provide single-neuron evidence for a generative model in the human brain.
