Overview: Study examines how the brain uses inference learning by generating cognitive maps.
Source: UC Davis
Both humans and other animals are good at inference learning, using information we do have to figure out things we can’t directly observe. New research from the Center for Mind and Brain at the University of California, Davis shows how our brains achieve this by constructing cognitive maps.
“The work suggests a new framework for learning in structured environments beyond incremental, experiential learning of associations,” said Erie Boorman, assistant professor in the UC Davis Department of Psychology and Center for Mind and Brain and senior author of the paper. .
In structured environments, individual elements are systematically related to each other, as is often the case in the real world. The insights from the study could be used to improve educational strategies that promote the use of a cognitive map for accelerated inference learning, and potentially approaches to accelerate the transfer of machine learning learning in artificial intelligence, Boorman said.
Learning by inference vs association
Most studies on learning have focused on learning by association – how animals learn to associate one thing with another, through trial and error. The difference between what was expected and what actually happened stimulates learning in such cases.
If there is some hidden structure behind those associations, you can use direct observations to derive indirect, invisible results, leading the chain of direct association.
For example, if you know that the quality of seasonal foods is determined by weather changes, you can deduce what’s best to eat based on what foods are ripe in the same season, Boorman said. By observing ripe apples, we can conclude that pears must also be ripe, but not strawberries. This kind of structure is important to know when making decisions.
Another example is an investor concluding that the decline in Facebook stocks can be attributed to a technology bubble, suggesting that Microsoft stocks are also likely to fall soon.
“Knowing this hidden relationship can help you learn much faster,” Boorman said.
Learning to test in a structured system
To investigate how people can use a cognitive map to learn information, graduate student Phillip Witkowski, project scientist Seongmin Park and Boorman created a task. In a series of trials, volunteers were asked to choose between two of four abstract shapes that would lead to one of two different gift cards (for example, Starbucks or iTunes).
The volunteers made their choices based on two pieces of information: their estimate of the probability that each shape would lead to a particular gift card, and a randomly awarded payout for each gift card.
The shapes were divided into two pairs. In each pair, the probability that one shape would lead to a particular outcome was the inverse of the other shape.
For example, if there was a 70% chance that Form A would lead to outcome 1, there was a 30% chance that Form B would lead to the same outcome, and vice versa for outcome 2. Thus, the subjects could get information about the probability of one outcome by inference from another, such as Microsoft shares of Facebook shares.
The pairs of shapes were not connected, so the subjects could learn nothing about the results of choosing shapes C or D from the results of choosing A or B.
The researchers tracked how the subjects learned about the system by observing their progress over a series of trials. After analyzing the results, they found that the volunteers used inferential learning to make decisions about which shapes to choose.
Some volunteers were invited to the second part of the experiment, where they performed the same task while their brain activity was measured with functional magnetic resonance imaging.
Learning is reflected in the brain by a burst of activity, a “faith update” when there is a difference between your previous and newly acquired knowledge. Activity linked to inferential learning has been found in the prefrontal cortex and the region of the midbrain where the neurotransmitter dopamine is released.
At the same time, the researchers found a representation of the hidden (or latent) chance-controlling associations for A and B in the prefrontal cortex.
The fMRI results show that the brains represent different outcomes in relation to each other, Boorman said. This representation makes for those “aha” moments.
Conventional thinking holds that incremental learning about rewards from direct experience is enhanced by the release of dopamine in the brain. The new study also implicates dopamine, but for inferential learning.
“Our work suggests a more general role for dopamine signals in the updating of beliefs through inference,” Boorman said.
About this cognition research news
Original research: Closed access.
†Neural Mechanisms of Credit Allocation for Derivative Relationships in a Structured Worldby Phillip P. Witkowski et al. neuron
Neural Mechanisms of Credit Allocation for Derivative Relationships in a Structured World
- Dopaminergic midbrain and frontal areas update experienced and inferred associations
- mPFC uses a common code for perceived and inferred credit allocation
- mPFC follows the evolving estimated “position” in the association space
Animal abstract compact representations of the structure of a task, supporting accelerated learning and flexible behavior. Whether and how such abstracted representations can be used to assign credit to derived, but unobserved, relationships in structured environments is unknown.
We develop a hierarchical reversal learning task and a Bayesian learning model to assess the computational and neural mechanisms underlying how humans infer specific associations of choice outcomes through structured knowledge.
We find that the medial prefrontal cortex (mPFC) efficiently displays hierarchically related associations of choice outcomes governed by the same latent cause, using a generalized code to assign credit to both perceived and inferred outcomes.
In addition, the mPFC and lateral orbitofrontal cortex track current “position” within a latent association space that generalizes across stimuli.
Collectively, these findings demonstrate the importance of both current position tracking in an abstracted task space and efficient, generalizable representations in the prefrontal cortex for supporting flexible learning and inference in structured environments.