Different patterns of brain activity help to distinguish between images recalled from memory and the sight of physical objects

How the ‘mind’s eye’ calls up visual memories from the brain

Different patterns of brain activity help to distinguish between images recalled from memory and the sight of physical objects

Picture a strawberry. Most people can easily distinguish between that image in their mind’s eye and an actual strawberry. Now researchers say that they’ve worked out how the brain draws this distinction and where in the brain the process happens.

According to a study¹ in monkeys, the key part of the brain is the primary visual cortex, which is also involved in vision. The authors found that neurons in this region displayed a different activity pattern for images conjured up from memory compared with that for real-time visual input. They conclude that the primary visual cortex is crucial for recalling images stored in memory.

“It’s an intriguing study that goes beyond what we know in several ways,” says cognitive neuroscientist Floris de Lange at Radboud University in Nijmegen, the Netherlands, who was not involved in the work. But others in the field, such as Julio Martinez-Trujillo, a cognitive neurophysiologist at Western University in London, Ontario, Canada, says that another area of the brain, the prefrontal cortex, is more likely than the visual cortex to be key for recalling images.

The study was published today in Science Advances.

Total recall

Vivid memories of objects or scenes that flash into the mind’s eye as if onto a screen are called visual memories. Scientists have long wondered where in the brain those memories are made. One candidate area is the primary visual cortex, which transforms raw sensory input from the eyes into perceptions that it passes on to other areas of the brain.

Images of people’s brains “have suggested that the primary visual cortex is the primary location of visual memory”, says cognitive neuroscientist Dajun Xing at Beijing Normal University, who led the latest study. But those images don’t measure neuronal activity directly. As a result, there was little direct neuronal evidence for the role of the primary visual cortex, Xing says.

For the current study, the authors trained two macaque monkeys (one Macaca fascicularis and one Macaca rhesus) to perform various memory tasks, and recorded data from an array of microelectrodes implanted into each monkey’s primary visual cortex.

The array measured the activity of 60–70 neurons, far more than the single neurons measured during most previous studies, says Xing. Moreover, the team kept the measurements running for months, whereas previous studies generally recorded for just a few hours.

When the monkeys were remembering images, the spatial pattern of their neuronal activity differed from that recorded when they were actually looking at something. The researchers were able to determine whether the monkeys were remembering colours, faces or the orientations of shapes just from the neuronal firing pattern.

Monkey see

The cleverest part of the work was the association task, says de Lange. Over dozens of training sessions, the monkeys learnt to associate a colour — either red or blue — with an image of black and white stripes that were either vertical or sloping. The monkeys were shown one of the striped images and tasked with remembering the associated colour. Neuronal activity showed that each monkey’s primary visual cortex represented a colour during the task. They weren’t looking at any colour, which rules out the “more boring interpretations” of the results, says de Lange, “that [they were] seeing an echo of the sensory stimulus”. The monkeys must have been seeing the colour in their mind’s eye.

Xing interprets the results to mean that the primary visual cortex is essential for visual memory. Although that’s possible, it’s not convincingly supported by these results, say both de Lange and Martinez-Trujillo.

“There’s a possibility that the actual memory encoding is happening elsewhere, and that what you’re seeing in the primary visual cortex is the downstream consequences,” says de Lange. To test this, he says, researchers would need to interfere with the primary visual cortex and see whether the animals could still recall visual memories.

Martinez-Trujillo maintains that a different brain area, the prefrontal cortex, is essential for visual memory. Observational studies² dating back almost a century found that when people had lesions in their prefrontal cortex their memory was disrupted, he says.

doi: https://doi.org/10.1038/d41586-024-01757-3

This story originally appeared on: Nature - Author:Julian Nowogrodzki