Serotonin receptor signaling insights may pave way for next-gen mental health drugs

In a discovery that could guide the development of next-generation antidepressants and antipsychotic medications, researchers at the Icahn School of Medicine at Mount Sinai have developed new insights into how a critical brain receptor works at the molecular level and why that matters for mental health treatments.

The study, published in the online issue of Science Advances, focuses on the 5-HT1A serotonin receptor, a major player in regulating mood and a common target of both traditional antidepressants and newer therapies such as psychedelics. The paper is titled “Structural determinants of G protein subtype selectivity at the serotonin receptor 5-HT1A.”

Despite its clinical importance, this receptor has remained poorly understood, with many of its molecular and pharmacological properties largely understudied—until now.

“This receptor is like a control panel that helps manage how brain cells respond to serotonin, a key chemical involved in mood, emotion, and cognition,” says senior author Daniel Wacker, Ph.D., Assistant Professor of Pharmacological Sciences and Neuroscience, at the Icahn School of Medicine at Mount Sinai.

“Our findings shed light on how that control panel operates—what switches it flips, how it fine-tunes signals, and where its limits lie. This deeper understanding could help us design better therapies for mental health conditions like depression, anxiety, and schizophrenia.”

Using innovative lab techniques, the research team discovered that the 5-HT1A receptor is inherently wired to favor certain cellular signaling pathways over others—regardless of the drug used to target it.

However, drugs can still influence the strength with which those pathways are activated. For example, the antipsychotic asenapine (brand name Saphris) was found to selectively engage a specific signaling route due to its relatively weak activity at the receptor.

To explore these mechanisms in greater detail, the researchers combined experiments in lab-grown cells with high-resolution cryo-electron microscopy—a cutting-edge imaging technology that reveals molecular structures at near-atomic resolution. Their work focused on how various drugs activate the 5-HT1A receptor and how the receptor interacts with internal signaling proteins known as G proteins.

Different signaling pathways controlled by the 5-HT1A receptor are linked to different aspects of mood, perception, and even pain. As scientists better understand which pathways are activated, and how, they can more precisely design drugs that treat specific symptoms or conditions without unwanted side effects.

“Our work provides a molecular map of how different drugs ‘push buttons’ on this receptor—activating or silencing specific pathways that influence brain function,” says study first author Audrey L. Warren, Ph.D., a former student at Dr. Wacker’s lab, who is now a postdoctoral researcher at Columbia University.

“By understanding exactly how these drugs interact with the receptor, we can start to predict which approaches might lead to more effective or targeted treatments and which ones are unlikely to work. It’s a step toward designing next-generation therapies with greater precision and fewer side effects.”

In a particularly surprising finding, the researchers discovered that a phospholipid—a type of fat molecule found in cell membranes—plays a major role in steering the receptor’s activity, almost like a hidden co-pilot. This is the first time such a role has been observed among the more than 700 known receptors of this type in the human body.

While current antidepressants often take weeks to work, scientists hope this new understanding of 5-HT1A signaling could help explain those delays and lead to faster-acting alternatives.

“This receptor may help explain why standard antidepressants take long to work,” says Dr. Wacker. “By understanding how it functions at a molecular level, we have a clearer path to designing faster, more effective treatments, not just for depression, but also for conditions like psychosis and chronic pain. It’s a key piece of the puzzle.”

Next, the research team plans to dig deeper into the role of the phospholipid “co-factor” and to test how their lab-based findings hold up in more complex experiments. They’re also working on turning these discoveries into real-world compounds that could become future psychiatric medications, building on their earlier success with drug candidates derived from psychedelics.