A version of cross-entropy that down-weights easy examples: FL = −(1 − p_t)^γ log(p_t), where p_t is the probability given to the true class. With γ > 0, well-classified examples contribute little, so training concentrates on the hard ones; γ = 0 gives back cross-entropy.
Card 1 of 259. Answer: A version of cross-entropy that down-weights easy examples: FL = −(1 − p_t)^γ log(p_t), where p_t is the probability given to the true class. With γ > 0, well-classified examples contribute little, so training concentrates on the hard ones; γ = 0 gives back cross-entropy.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Lin et al. introduced focal loss in 2017 with the RetinaNet detector. One-stage detectors score huge numbers of candidate boxes per image, almost all easy background, and with plain cross-entropy those easy negatives swamp the gradient. The factor (1 − p_t)^γ shrinks the loss of anything the model already gets right; the paper used γ = 2, together with a class weight α.
Example with γ = 2: an easy example with p_t = 0.9 has cross-entropy 0.105 but focal loss 0.1² × 0.105 ≈ 0.001, a hundred times smaller. A hard example with p_t = 0.1 has cross-entropy 2.30 and focal loss 0.9² × 2.30 ≈ 1.87, barely reduced. The hard examples now dominate training.
Use it when imbalance comes with many easy negatives, as in detection, dense segmentation or rare-event classification; the alternatives are class weights, resampling and hard negative mining. It adds a hyperparameter, and on balanced data there is little to gain. Mukhoti et al. (2020) found it also tends to make networks less overconfident than cross-entropy does.
