AnalyticBridge

A Data Science Central Community

Understanding the foundations of Deep Learning through Linear Regression

This article was written by Ajit Joakar.

In this longish post, I have tried to explain Deep Learning starting from familiar ideas like machine learning. This approach forms a part of my forthcoming book. I have used this approach in my teaching. It is based on ‘learning by exception,' i.e. understanding one concept and it’s limitations and then understanding how the subsequent concept overcomes that limitation.

The roadmap we follow is:

Linear Regression
Multiple Linear Regression
Polynomial Regression
General Linear Model
Perceptron Learning
Multi-Layer Perceptron

We thus develop a chain of thought that starts with linear regression and extends to multilayer perceptron (Deep Learning). Also, for simplification, I have excluded other forms of Deep Learning such as CNN and LSTM, i.e. we confine ourselves to the multilayer Perceptron when it comes to Deep Learning. Why start with Linear Regression? Because it is an idea familiar to many even at high school levels.