Understanding the Mathematics behind Gradient Descent | Machine Learning - Medium

Parul Pandey, Data Science + Community + Evangelism @H2O.ai. Also curating stuff at parulpandey.com explains, A simple mathematical intuition behind one of the commonly used optimisation algorithms in Machine Learning. 

Agile is a pretty well-known term in the software development process. The basic idea behind it is simple: build something quickly ➡️ get it out there ➡️ get some feedback ➡️ make changes depending upon the feedback ➡️ repeat the process. The goal is to get the product near the user and let the user guide you with the feedback to obtain the best possible product with the least error. Also, the steps taken for improvement need to be small and should constantly involve the user. In a way, an Agile software development process involves rapid iterations. The idea of — start with a solution as soon as possible, measure and iterate as frequently as possible, is basically Gradient descent under the hood.

Gradient descent algorithm is an iterative process that takes us to the minimum of a function(barring some caveats). The formula below sums up the entire Gradient Descent algorithm in a single line.

https://www.coursehero.com/file/27927651/Gradient-Descentpdf/

But how do we arrive at this formula? Well, It is actually very simple and just includes some high school maths. Through this article, we shall try to understand as well as recreate this formula in the context of a Linear Regression model. This article is an adaption of the video titled Mathematics of Gradient Descent — Intelligence and Learning by The Coding Train. If you do not like reading, you can watch the video too...

Conclusion
The point of this article was to demonstrate the concept of gradient descent. We used gradient descent as our optimization strategy for linear regression. by drawing the line of best fit to measure the relationship between student heights and weights. However, it is important to note here that the linear regression example has been chosen for simplicity but can be used with other Machine Learning techniques too. 

Read more...

Source: Medium