Cost Function

Learning Function

+−⟲

Link to original

In the above illustration, for any given function to predict the correct outputs, we have two main parameters/weights/coefficients, that are used to improve our prediction as we move ahead in the training.

After the initial model prediction, we take all the outputs/target from the Training Data Set and compare it to the predicted value from our model, and we try to calculate the error and try minimizing the total error to improve our prediction

$$J_{(w,b)}=\frac{1}{2m}.\sum _{i=1}^m{({\hat{y}}^{(i)}-y^{(i)})}^2$$

The equation is also called Squared Error Cost Function and is mostly very popular with most Regression Models, here $J_{(w,b)}$ is the Cost Function

as $\hat{y}$ is the output of the function the above equation can also be re-written as

$$J_{(w,b)}=\frac{1}{2m}.\sum _{i=1}^m{(f_{(w,b)}(x^{(i)})-y^{(i)})}^2$$

Essentially what we try to do is try different values of $w$ and $b$ and calculate the cost function, i.e the error between the prediction and our actual value, so we try to minimize the cost function as much as possible as this will allow our model to align with the training data to provide accurate predictions.