You must have heard about the term Gradient Descent in your journey to Data Science. It is one of the most important and mandatory term to understand for Machine Learning and as well as Deep Learning.  Basically in simple terms, Gradient Descent is algorithm, an optimization algorithm which repeatedly changes the variables value to have the best solution. To understand more clearly let's take and example of linear regression. In linear regression we have to find m(slope) and b(intercept). There is a typical way to solve it mathematically by using Ordinary Least Square (OLS) method. But in Gradient Descent approach, we first assume some value of m and b and we iteratively change the value of m and b with the help of training data to get the right pair of m and b finally. Gradient Descent works on a function which is differentiable at any point and find it's minimum point. In linear regression, that function is Loss function and our aim to is to minimize it. So we use gradient de...