The average negative log of the probability the model gave to the true class: −(1/N) Σ [yᵢ log pᵢ + (1 − yᵢ) log(1 − pᵢ)] for binary labels. It rewards confident correct predictions and punishes confident wrong ones heavily.
Card 1 of 259. Answer: The average negative log of the probability the model gave to the true class: −(1/N) Σ [yᵢ log pᵢ + (1 − yᵢ) log(1 − pᵢ)] for binary labels. It rewards confident correct predictions and punishes confident wrong ones heavily.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Log loss is binary cross-entropy; for more than two classes it is −log of the probability assigned to the correct class, averaged over examples. It is the negative log-likelihood of a Bernoulli or categorical model, so minimizing it is maximum likelihood estimation, which is why logistic regression and neural network classifiers are trained on it. Like the Brier score, it is a proper scoring rule.
Example: for an example that is truly positive, predicting 0.9 costs −ln 0.9 ≈ 0.105, predicting 0.5 costs 0.693, and predicting 0.01 costs 4.6. One confidently wrong prediction can outweigh dozens of good ones, and a predicted 0 for a real positive costs infinity, so libraries clip probabilities slightly away from 0 and 1.
For a baseline, predicting the base rate for every example gives a log loss equal to the entropy of the labels: ln 2 ≈ 0.693 for a 50/50 split, much less for rare positives, so always compare with that number. Interviewers ask why we train on log loss but report accuracy: log loss is smooth and uses the probabilities, while accuracy is a step function of a threshold. Focal loss is a variant that down-weights easy examples.
