DETI mapping results from the brain of a person viewing one of the stimuli used in the experiment (far left). The central column shows a flattened topographical map of the electrodes over the back of the head, illustrating the variation of DETI maps at each electrode across that scalp region. On the right-hand side, each column shows a close up of the spatiotemporal evolution of the visual code for different electrodes (each row corresponds to a different point in time in milliseconds). Each color represents one of seven different neural population responses that were mapped to each image location, thereby revealing which neural population best coded image regions at different points in time.
Colgate University Professor of Psychology and Neuroscience Bruce C. Hansen is driven to unravel the intricate inner workings of the human brain, and it is more than just scientific curiosity.
“It’s this almost torturous thing that haunts me,” Hansen says. “But it fuels my inspiration.”
The human brain is perhaps the most complex structure in the known universe. It contains about 150 billion nerve cells called neurons, which use electrical and chemical signals to transmit information throughout the body and within the brain. Every thought, action, emotion, and experience is a product of the brain’s ultra-sophisticated operations.
Hansen focuses on how the brain codes visual information in our environments. In collaboration with Michelle R. Greene of Bates College and David J. Field of Cornell University, he introduced a brain-mapping technique that offers new and important insights into how the neural code of images of real-world environments undergoes dramatic transformations — in a fraction of a second. The project, titled “Dynamic Electrode-to-Image (DETI) mapping reveals the human brain’s spatiotemporal code of visual information,” was recently published in PLOS Computational Biology.
The journal article describes DETI mapping as “an analytical technique that capitalizes on the high-temporal resolution of electroencephalography (EEG) to render maps of visual features that are associated with different neural signals over time.”
Hansen says this is similar to passing an image through the brain and then projecting the neural code back onto the image. Using DETI, the scientists mapped neural signals to each pixel of an image, revealing when, where, and how the visual code was modified within the brain.
DETI mapping showed that real-world scenes underwent a series of unscripted modifications within isolated concentrations of neurons scattered multi-dimensionally within the brain. A movie version of this process used color codes to represent how different pockets of neurons reacted and transformed an example scene, resembling abstract paintings being frenetically altered by the brush strokes of lightning-quick artists.
“Upon viewing a new scene, the brain takes about 300 milliseconds to transform the array of light that enters the eye into meaningful content that enables intelligent behavior. But the scene is more than just a static picture in the mind,” Hansen says. “The neural codes used to identify and define the scene are constantly evolving over an enormous, multi-dimensional population of neurons.”
The EEG sensor nets worn by subjects each had 128 channels. In any given electrode, Hensen and his team were able to observe how the visual code changed over time. Each color in the DETI maps represented a different neural population responding to different types of features in the image.
“This gave us a way to not only visualize what was happening in the brain, but to analyze what was happening by examining the data we accumulated,” Hansen says. “Our mapping technique is an important first step that offers a potential avenue for future studies to explore and refine high-level representations of our visual world.”
The rapid activity of neurons within the first 300 milliseconds of viewing something in our environment occurs without conscious awareness. We take for granted what we consider to be simple things: hiking a mountain trail, grocery shopping, or even just wearily heaving our bodies off a couch and shuffling into the kitchen to get a glass of water. For most of us, the visual cues in our environments are what enable us to move from one place to another while successfully navigating a dynamic environment that the brain renders into something that can seem quite simple to us. The brain does more than just create a mental picture. With incredible speed and remarkable efficiency, it compiles a complex, interactive representation that informs the brain how to match our physical and emotional goals.
“The decisions we make. The actions we perform. Our judgments. Our prejudices. All these things are informed by these surprisingly complex codes, which the brain processes in ways that are yet to be fully understood,” Hansen says. “We focused on what happens in the first third of a second. What happens after that is yet to be defined by this technique. But deepening our understanding of how these systems operate should enable us to better comprehend the various ways in which we react and behave in a societal setting. We’ve made a lot of progress. But there is still so much to learn.”
Two dozen undergraduate students at Colgate took part in the research as summer undergraduate research students, research assistants, or as honors thesis students in the Department of Psychological and Brain Sciences.
“Being able to offer undergraduates the opportunity to engage in cutting-edge research is one of the highlights of my job,” Hansen says. “I’m excited by how many more opportunities the lab can offer our students as it becomes one of the cornerstone labs of the Robert H.N. Ho Mind, Brain, and Behavior Initiative at Colgate.”
— Jim Melvin