Psychology of Artificial Neural Networks

6 min readDec 6, 2020

Image by Author

This is a neuron. It’s a nerve cell. Neurons are the fundamental units of our brain, and our brain has about 90 billion of them. Neurons are responsible for all connections to, from, and within the central nervous system (brain & spinal cord). Neurons communicate with other cells by sending and receiving electrical signals through synapses. Today, we are going to focus on the synapse.

Now the interesting thing here is that synapses do change based on the adjacent neurons’ activities. If there are more repeated and persistent signals that get transmitted between specific neurons, this synaptic connection strengthens (by means like opening more receptive channels). This is called Long-Term Potentiation (LTP), and these changes can last for a lifetime. This process is thought to be the basis of our memory

Hebbian Learning gets its name from the psychologist, Donald Hebb, who first posited this idea of stronger associations between neurons that are persistently and firing simultaneously in the 1940s. This is the fundamental concept of artificial neural networks (ANN): while neurons maintain the overall structural integrity of connections, individual connections between neurons are updated based on the co-occurrences.

So take a look at the diagram below. What’s changing is the synapse, which alters the weight of output of a pre-synaptic neuron. This weighted output is fed as the input to the post-synaptic neuron. Neurons may send the same output to multiple other neurons, but distinctive post-synaptic neurons may receive different input based on the values of the synapse between neurons. Each synaptic connection is unique to each pair of neurons.

Image by Author

This is a simple mechanism of the feedforward network, where activated input weighted by the connection-specific weight is fed as an input to the output neuron. Activation is like a neuron’s internal system that decides whether to fire an action potential given value and at what intensity.

For artificial neural networks, backpropagation is a process of updating the weights based on the prediction error between the output value and the expected value. It is a simple algorithm that computes the derivative of the activation function to divide the error term across different weights, but the actual step involved in using prediction error to update at a neural level is not a straightforward process in our brain.

site:

So how do neurons take errors into account? Hebbian theory largely works for unsupervised learning where we are simply looking at the frequency of co-occurrence to establish associative strength between neurons. But what about supervised learning where we have the expected outcome? Now we are getting into more of a developing area of the research.

Predictive coding is an overarching theory that argues that the primary mechanism of our brain is to generate predictions and update the models based on prediction errors. The cognitive process is not always (or arguably never) one-way. Our brains receive inputs from the outer world, but our brain also cues how we may intake these inputs even before we receive them, so we can focus on what matters more efficiently. Brains may already know what to expect. This notion of predicting the input before the reception is a quite robust idea that can explain many cognitive phenomena. (If you are interested, here is a great article on this topic: LINK). We don’t have conclusive evidence to say this happens at the neural level yet, but it is quite a promising and fascinating concept.

Here, we briefly discussed the relationship between a simple artificial neural network and the human neural network. Of course, the interdisciplinary implication in the field of AI and cognitive science does not end here. In the next post, I will talk about how human vision compares to the convolutional neural network (CNN). There is a lot of research going on in both the AI and cognitive science realm that effectively cross-overs and I hope to continue to discuss some of these advancements going forward.

Psychology of Artificial Neural Networks

Written by KD Pathak