Reading Notes: Neuroscience-Inspired Artificial Intelligence

Neuroscience-Inspired Artificial Intelligence

Better understanding biological brains could play a vital role in building intelligent machines

More recently, the interaction has become much less common- place, as both subjects have grown enormously in complexity and disciplinary boundaries have solidified


Building human-level general AI (or ‘‘Turing-powerful’’ intelligent systems; Turing, 1936) is a daunting task, because the search space of possible solutions is vast and likely only very sparsely populated.

Therefore, two ways:

1 scrutinizing the inner workings of the human brain
2 Studying animal cognition and its neural implementation


(1) neuroscience provides a rich source of inspiration for new types of algorithms and architec- tures, independent of and complementary to the mathematical and logic-based methods and ideas that have largely dominated traditional approaches to AI

(2) neuroscience can provide validation of AI techniques that already exist.

Practically, we should not slavishly enforce adherence to biological plausibility in building AI system. From an engineering perspective, what works is ultimately all that matters.

Biological plausibility is a guide, not a strict requirement

What required to understand any complex biological system:
algorithms, architectures, functions, and representations


(1) When we say neuroscience, we mean to include all fields that are involved with the study of the brain, the behaviors that it generates, and the mechanisms by which it does so, including cognitive neuroscience, systems neuroscience and psychology.

(2) When we say AI, we mean work in machine learning, statistics, and AI research that aims to build intelligent machines

Two pivotal fields of current AI reesarch:
– DL and RL
– both took root in ideas from neuroscience

Three aspects:

  • (1) The past: DL, RL
  • (2) The present: Attention, Episodic memory, working memory, continual learning.
  • (3) The future: Intuitive Understanding of the Physical World, Efficient Learning, Transfer Learning, Imagination and Planning, Virtual Brain Analytics


purely symbolic approaches might be too brittle and inflexible to solve complex real-world problems

parallel distributed processing (PDP)

the PDP move- ment proposed that human cognition and behavior emerge from dynamic, distributed interactions within networks of simple neuron-like processing units, interactions tuned by learning pro- cedures that adjust system parameters in order to minimize error or maximize reward.

PDP showed striking success.

neuroscience has provided initial guidance toward architectural and algorithmic constraints that lead to successful neural network applications for AI.


neuroscience was also instrumental in erecting a second pillar of contemporary AI, stimulating the emergence of the field of reinforcement learning (RL)

RL methods address the problem of how to maximize future reward by mapping states in the environ- ment to actions

it is not widely appreciated among AI researchers

temporal-difference (TD). TD methods are real-time models that learn from differences between temporally successive predictions, rather than having to wait until the actual reward is delivered.

TD methods continue to supply core tech for robotic control, expert play in backgammon and Go.


Up until quite lately, most CNN models worked directly on entire images or video frames, with equal priority given to all image pixels at the earliest stage of processing. The primate visual system works differently. Rather than processing all input in parallel, visual attention shifts strategically among locations and objects, centering processing resources and representational coordi- nates on a series of regions in turn

attentional mechanisms have been a source of inspiration for AI architec- tures that take ‘‘glimpses’’ of the input image at each step, update internal state representations, and then select the next location to sample

One further area of AI where attention mechanisms have recently proven useful focuses on generative models, systems that learn to synthesize or ‘‘imagine’’ images (or other kinds of data) that mimic the structure of examples presented during training.

Episodic Memory

intelligent behavior relies on multiple memory systems

These will include not only reinforcement-based mechanisms, which allow the value of stimuli and actions to be learned incrementally and through repeated experience, but also instance- based mechanisms, which allow experiences to be encoded rapidly (in ‘‘one shot’’) in a content-addressable store. The latter form of memory, known as episodic memory.

One recent breakthrough in AI has been the successful inte- gration of RL with deep learning

the deep Q-network (DQN) exhibits expert play on Atari 2600 video games by learning to transform a vector of image pixels into a policy for selecting actions (e.g., joystick movements). One key ingredient in DQN is ‘‘experience replay,’’ whereby the network stores a subset of the training data in an instance-based way, and then ‘‘replays’’ it offline, learning anew from successes or failures that occurred in the past. Expe- rience replay is critical to maximizing data efficiency, avoids the destabilizing effects of learning from consecutive correlated ex- periences, and allows the network to learn a viable value function even in complex, highly structured sequential environments such as video games.

According to a prominent view, animal learning is supported by parallel or ‘‘complemen- tary’’ learning systems in the hippocampus and neocortex

artificial agents employing episodic control show striking gains in performance over deep RL networks, particularly early on during learning. Further, they are able to achieve success on tasks that depend heavily on one-shot learning, where typical deep RL architectures fail.

In the future, it will be interesting to harness the benefits of rapid episodic-like memory and more traditional incremental learning in architectures that incorporate both of these components within an interacting framework that mirrors the complementary learning systems in mammalian brain.

Working memory

AI research has drawn inspiration from these models, by building architec- tures that explicitly maintain information over time.

one can see close parallels between the learning dynamics in these early, neuroscience-inspired networks and those in long-short-term memory (LSTM) networks, which subsequently achieved state of the art performance across a variety of domains.

LTSMs allow information to be gated into a fixed activity state and maintained until an appropriate output is required (Hochreiter and Schmid- huber, 1997). Variants of this type of network have shown some striking behaviors in challenging domains, such as learning to respond to queries about the latent state of variables after training on computer code

Continual Learning

Intelligent agents must be able to learn and remember many different tasks that are encountered over multiple timescales. Both biological and artificial agents must thus have a capacity for continual learning, that is, an ability to master new tasks without forgetting how to perform prior tasks.

While animals appear relatively adept at continual learning, neural networks suffer from the problem of catastrophic forgetting.

The future

Intuitive Understanding of the Physical World

Something that are well developed in human infants but lacking in most AI systems:
– knowledge of core concepts about physical world:
— space
— number
— objectness

Efficient learning

Human cognition is distinguished by:
– one-shot concept learning
– repidly learn new concepts from few examples
– leverage prior knowledge for flexible inductive inferences

AI research developed networks that “learn to learn”

Transfer learning

– zero-shot inferences based on compositional representations
– progressive networks

Imagination and planning

Properties of deep RL such as DQN:
– operate mostly in a reactive way, learning mapping from perceptual inputs to actions that maximize future value
– “model-free” RL
– computationally inexpensive
– two major drawbacks:
— data inefficient, requiring large amounts of experience to derive accurate estimates
— inflexible, being insensitive to changes in value of outcomes.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s