Perceptron: Building Block Of Neural Networks

Introduction

Perceptrons are the simplest type of artificial neuron, and they are the building blocks of neural networks. A perceptron takes in a set of inputs, applies weights to each input, sums them up, and passes the result through an activation function to produce an output.

Perceptrons can be used to solve a variety of problems, including classification, regression, and clustering. They are relatively simple to understand and implement, making them a good choice for beginners to neural networks. However, perceptrons are limited in their capabilities, and they cannot be used to solve all problems.

In this blog post, we will explore the basics of perceptrons, including their architecture, how they work, and how they can be used to solve problems. We will also discuss the limitations of perceptrons and how they can be overcome.

What is it?

Perceptron can be considered an algorithm or a mathematical model, which is used for supervised machine learning. It can be used for both classification and regression. Its design is such that it automatically became a building block of neural networks (and deep learning). It takes in a set of inputs, applies weights to each input, sums them up, and passes the result through an activation function to produce an output.

Design of a Perceptron

• Inputs: Generally represented by x1, x2, ...xn is the set of features we provide as input

• Weights: Indicate the contribution of an input feature toward the output. For Eg: If We want to predict if a person will pass or fail an exam, the IQ of that person will have more impact on the result as compared to the height of that person.

• Bias: The bias is a constant value that is added to the weighted sum of the inputs.

• Summation function(Σ): It is a function that takes in a set of inputs and weights, and returns the weighted sum of the inputs(denoted by z).
  where, z = w1x1 + w2x2.....+wnxn + b (b represents bias)
  The weighted sum is then passed through an activation function to produce an output.

• Activation Function: It is a mathematical function That brings the output in a particular range. It determines how the weighted sum of the inputs is converted into an output. The most common activation function for perceptrons is the step function, which outputs 1 if the weighted sum is greater than or equal to a threshold, and 0 otherwise.

How perceptron works

While training a perceptron, our main goal is to calculate the weights. We do this by training the perceptron on a dataset.

The working of a perceptron is relatively simple. It consists of the following procedure:

The perceptron works by first multiplying each input by its corresponding weight. The results are then summed together to form a weighted sum. The weighted sum is then passed through the activation function to produce an output.

The weights and bias of the perceptron are typically learned through a process called supervised learning. In supervised learning, the perceptron is presented with a set of training examples, each of which has a known output. The perceptron then adjusts its weights and bias to minimize the error between its predicted output and the known output.

Once the perceptron has been trained, it can be used to classify new data. To classify a new data point, the perceptron simply takes in the data point's inputs and outputs a prediction. The prediction is made by passing the weighted sum of the inputs through the activation function.

Perceptron and neuron

Deep learning is inspired by how our human mind works and aims to mimic that. But we should keep in mind that although there is some inspiration, the working is not the same.
Our neural networks use neurons for taking decisions and deep learning uses perceptrons, and there are many differences b/w the two.
The most basic difference is that neurons are much more complex than perception(they process a larger input much more quickly). Our brain supports neuroplasticity whereas perception or artificial neural networks don't.

Other comparisons are stated in the following table.

Limitations

Perceptrons are limited in their capabilities, and they cannot be used to solve all problems. Here are some of the limitations of perceptrons:

• Can only solve linearly separable problems: Perceptrons can only learn to separate data points that are linearly separable. This means that the data points can be divided into two groups by a straight line or plane. Many real-world problems are not linearly separable, so perceptrons cannot be used to solve them.

• Sensitive to noise: Perceptrons are sensitive to noise in the data. This means that small changes in the data can cause the perceptron to make incorrect predictions.

• Not able to learn complex relationships: Perceptrons can only learn simple relationships between data points. They cannot learn complex relationships, such as those that are found in natural language processing or image recognition.

Conclusion

Despite their limitations, perceptrons are a valuable tool for machine learning. They are relatively simple to understand and implement, making them a good choice for beginners in machine learning. They are also relatively efficient, making them a good choice for problems where speed is important.

As a machine learning practitioner, it is important to understand perceptrons. They are simple but powerful tools that can be used to solve a variety of problems.