What is a Neural Network?

Neural Network is a method of the deep learning process that uses interconnected nodes and neurons to process the data in a layered structure resembling the human brain.

Is this sound terrific? Do you want to learn about the basics of neural networks and their functioning without using any heavy mathematics? Then you are at the right place.

Let’s begin our learning.

We will start by picking up a group of students who have not seen koalas in their life. By the way, Koala is an animal from Australia.

Now our job is to train these students such they will be able to detect Koala images.
As trainers, we believe in teamwork so we instruct the students to delegate the tasks among themselves.

Each person will be trained especially on some part of Koala like say for eg:

• Mike will be trained on koala eye’s

• Mohan will be trained on koala’s nose

• Jyoti on koala legs and Chen on koala’s body

Once our students are trained we expect them to give their predictions in numbers as below.

Here,
0 → Indicates definitely not Koala’s eyes
0.5 → May be Koala's eyes (50: 50 chance)
1 → Indicates Koala's eyes

Simply output will be a probability ranging from 0 to 1.

Hope the expectation from students is pretty much clear now. In short, we will be training each student on a specific part and making them an expert in matching those parts with Koala’s body parts

Mike now has become an expert Koala’s eyes detector.	Mohan will be an expert Koala’s nose detector

So when we give them an image to predict, let’s say we have given an image of Lion

Here Mike will check for only eyes and give out the least score, as eyes are not matching with Koala’s. Similarly, Mohan will check for the nose in the given image and gives out the corresponding score based on his learning.

We can extend this kind of learning among different students and let's say we have one team leader for this group who will figure out the image given is Koala’s are not by taking input from all the students.

Let’s assume Serena is the team leader for this group, She uses an equation like the below for giving a prediction score of whether the image contains a Koala’s face or not.

If you observe she is attributing different weights to each input feature based on her experience. She is giving more weightage to the nose as it could be a prominent feature in detecting Koala’s face.

Here Serena uses something like the below:

She takes out individual scores from the group and put them in the equation to get the final score. Anything above 0.5 (threshold) indicates the image given is Koala.

Similarly, for the Lions image, Serena is giving out less probability score for the image to be Koala using her team’s input and equation as below.

We can now extend this to the rest of the group by dividing students into subgroups and training on a specific feature.

Here comes the final decision maker who will take input from our team leaders and gives out the final decision on whether the image is Koala’s or not.

Let’s say Sergey is our final decision maker and he takes input from Serena and Nidhi (our team leaders) to give out the final decision.

Even Sergey uses some kind of equation attributing different weights to each type of input given. Generally, we also give more weightage to the face than to the body when we are doing some task of animal detection. Isn’t it so?

So our students did excellent teamwork by delegating tasks among themselves and coming up with a final decision.

This is what a neural network is.

Here our students have used three layers of groups:

Students (Input Layer)

Team leaders ( Hidden layers)

Final decision maker (Output layer)

Each person is a single neuron. Point to note, it’s not like we can have only a single hidden layer as such, we can have multiple hidden layers based on the complexity of the problem.

But how do we actually train our neural network?

When something is unknown we actually go with a random guess. Yes, you are right.

Initially, all three layers of students don’t know anything about koalas. They will start with a random guess and send the scores to the next layer.

If no one knows the correct answer how our group will finish the task? Good question. That’s why we have allotted a supervisor to the group from the trainers team who knows all the answers.

So at each iteration, our decision maker (Sergey) goes to the supervisor to check if their group's prediction is correct or not.

After getting feedback from the supervisor, Sergey takes it back to the previous layers. If the prediction is wrong, previous layers will change their initial random guess values (adjusting the weights)

Now team comes up with new prediction scores, and Sergey takes the final score to the supervisor for feedback, and the process repeats.

This process is exactly called Backward Error Propagation. Neurons will come up with a random guess, then they take the feedback based on the ground truth label (correct answer which supervisor knows). Propagates the feedback to earlier layers and they adjust the weights accordingly.

Training repeats with multiple images. Based on their mistakes and the experience group will adjust their weights and finally land up giving the right predictions.

This is how the neural network trains. After looking at this are you thinking, is it similar to Distributed computing? Where each task will be divided into multiple sub-tasks and given to the systems for parallel processing and finally aggregates. Yes, you are right in a way.

However, an interesting and unique part of neural networks is training itself. Here neurons will divide the tasks among themselves and get specialized in that particular task.

They learn from their mistakes and get better through each iteration using the Backward error propagation technique.

So, when using neural networks only thing they expect is data. Each neuron is intelligent enough to figure out which task it has to specialize in and they are even smart enough to decide which feature is important and which is not.

Isn’t it interesting? Hope you enjoyed learning about Neural network architectures