This is a technology problem. It will remain so for a long time and certainly matches the problem of creating the Holodec. Recall that what we do observe in the brain is a derivative of a vastly complex neuro - network we are anable to map or grasp, let alone understand.
Now grasp my conjecture which says that memory relies on addressing a physical point in space and time outside our present. Thus the brain actually operates not only in this page of time, but conveniently across many past pages of time as well. Thus any such investigation must be four dimensional at the least.
In the meantime we are watching a lot of ducks on the pond while speculating on the nature of the pond's surface.
“Hyperscans” Show How Brains Sync as People Interact
But something has been largely missing from these studies: other people. We humans are innately social, yet even social neuroscience, a field explicitly created to explore the neurobiology of human interaction, has not been as social as you would think. Just one example: no one has yet captured the rich complexity of two people’s brain activity as they talk together. “We spend our lives having conversation with each other and forging these bonds,” neuroscientist Thalia Wheatley of Dartmouth College says. “[Yet] we have very little understanding of how it is people actually connect. We know almost nothing about how minds couple.”
Group BrainprintsThat is beginning to change. A growing cadre of neuroscientists is using sophisticated technology—and some very complicated math—to capture what happens in one brain, two brains, or even 12 or 15 at a time when their owners are engaged in eye contact, storytelling, joint attention focused on a topic or object, or any other activity that requires social give and take. Although the field of interactive social neuroscience is in its infancy, the hope remains that identifying the neural underpinnings of real social exchange will change our basic understanding of communication and ultimately improve education or inform treatment of the many psychiatric disorders that involve social impairments.
Previous limits on technology were a major obstacle to studying real human interaction. Brain imaging requires stillness, and scientific rigor demands a level of experimental control that is anything but natural. As a result, it is hard to generate high-quality data about one brain. Doing so for two brains is “more than twice as hard,” neuroscientist David Poeppel of New York University says. “You have to synchronize the machinery, the data and the data acquisition.”
Initially, Montague’s lead was followed mostly by other neuroeconomists rather than social neuroscientists. But the term hyperscanning is now applied to any brain imaging research that involves more than one person. Today the techniques that fit the bill include electroencephalography (EEG), magnetoencephalography and functional near-infrared spectroscopy. Use of these varied techniques, many of them quite new, has broadened the range of possible experiments and made hyperscanning less cumbersome and, as a consequence, much more popular.
Engagement MattersBeyond the practical challenges of interactive neuroscience, a more philosophical question has circulated as to whether the neural information obtained from measuring people during social interaction is significantly different from scans taken when people are alone or acting simply as observers. Does it matter if the person you look at looks back? Is there a difference between speaking a sentence and speaking it to someone who is listening?
Yes, apparently there is. The evidence is growing, says psychiatrist and social neuroscientist Leonhard Schilbach of the Max Planck Institute of Psychiatry in Munich, that “social cognition is fundamentally different when you’re directly engaged with another person as opposed to observing another person.”
Demonstrating those differences does not necessarily require studies of more than one brain at a time, but it does require relatively naturalistic experiments that are challenging to design within the constraints imposed on standard laboratory protocols. Psychologist Elizabeth Redcay of the University of Maryland studies social interaction in autism, with a focus on middle childhood. Back in 2010, when she was a postdoctoral fellow working with Rebecca Saxe at the Massachusetts Institute of Technology, Redcay set up a pioneering experiment featuring one participant inside the scanner and another (actually a researcher) outside it interacting live through a video feed. Recorded videos of another interlocutor served as a control. In the live versus the recorded interactions, Redcay saw greater activation in brain areas involved in social cognition and reward.
Schilbach has been one of the foremost proponents of what he calls second-person neuroscience. His studies have included virtual characters who seem to respond to a participant’s gaze. In such situations, “the so-called mentalizing network and the action-observation network seem to be much more closely connected [than we knew],” Schilbach says. “They influence each other, sometimes in a complementary and sometimes in an inhibitory fashion.” Schilbach has also found that even very simple acts such as gazing at another individual and believing they are gazing back—an interaction in which you sense that your own behavior has an effect on another person—spurs activity in the brain’s reward circuitry, particularly the ventral striatum. And the more rewarding we find a behavior, the more likely we are to repeat it.