A regression loss that is squared for small errors and linear for large ones, with a threshold δ between the two. It behaves like squared error near zero and like absolute error for outliers, so a few extreme points can't dominate the fit.
Card 1 of 259. Answer: A regression loss that is squared for small errors and linear for large ones, with a threshold δ between the two. It behaves like squared error near zero and like absolute error for outliers, so a few extreme points can't dominate the fit.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
For an error e, Huber loss is ½e² when |e| ≤ δ and δ(|e| − ½δ) otherwise. The two pieces meet smoothly, so the loss is differentiable everywhere, and its gradient never exceeds δ in size. Peter Huber introduced it in 1964 as a robust way to estimate a location parameter.
Example with δ = 1: an error of 0.5 costs 0.125, the same as half the squared error. An error of 10 costs 1 × (10 − 0.5) = 9.5, where half the squared error would be 50. The outlier pulls on the fit about five times less, while small errors are still treated smoothly, unlike absolute error, which has a kink at zero.
δ sets where "normal noise" ends, so pick it from the scale of typical residuals or by cross-validation. The same idea appears elsewhere: deep Q-networks clip the error in a way equivalent to Huber loss, and Fast R-CNN's "smooth L1" box regression loss is Huber loss with δ = 1. If outliers should be ignored entirely rather than down-weighted, losses such as Tukey's biweight go further.
