Cost function(J) of Linear Regression is the Mean Squared Error (MSE) between predicted h(x) value and true value y.
J=1/2m Sum[ (h(x))-y ]^2 ….(1)
So, how can one interpret the Logistic Regression Cost function as above?
Logistic Regression Cost Function
Cost(h(x),y))= 1/m Sum[Cost(h(x),y) ]
Cost(h(x),y))= -1/m Sum[ y*log(h(x)) + (1-y)log(1-h(x)) ] …(2)
In equation (1) the term in the Square bracket is simple the difference, but in equation (2) the term in square brackets is confusing. Is it also the error term?
For logistic regression its basically a measure of all the cases it gets right/wrong. Logistic regression is used for a binary output of 1 or 0 so the true value y_true is 1 or 0 and the predicted value y_pred is a prediction, a value between 1 or 0 depending on the confidence of the result.
lets look at one term in the sum: y_true * log(y_pred) + (1-y_true)*log(1-y_pred)
if y_true = 1 and y_pred = 0.9 then y_pred is close to y_true so its a good approximation so we'd want a small cost value. Plugging these values in we get
1*log(0.9) + 0*log(0.1) = log(0.9)
Now lets look at a worse case where y_true = 0 and y_pred = 0.9 then we have 0*log(0.9) + 1*log(0.1) = log(0.1).
Because log(0.1)
A significant question from Syed Abuzar Bacha and a brilliant answer from Jamie Wallis.
- Badges
- Science topic
- Similar topics
- Philosophy
- Esthetics
- Interpretation