Provides probabilities for each possible class in a multi-class classification model. The probabilities add up to exactly 1.0
Card 1 of 259. Answer: Provides probabilities for each possible class in a multi-class classification model. The probabilities add up to exactly 1.0
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
The softmax function turns a vector of raw scores, called logits, into a probability distribution: each output lies between 0 and 1 and the outputs sum to exactly 1. It is the usual output layer of a multi-class classifier: softmax(z)ᵢ = e^(zᵢ) / Σⱼ e^(zⱼ).
Example: logits [2, 1, 0.1] exponentiate to about 7.39, 2.72 and 1.11, which sum to about 11.21, giving probabilities of about 0.66, 0.24 and 0.10. The exponential keeps every output positive and stretches the gaps, so the largest logit takes most of the probability, a smooth version of argmax. Adding the same constant to every logit changes nothing, so implementations subtract the maximum first to avoid overflow.
Softmax is usually paired with cross-entropy loss, and the gradient of the pair with respect to the logits is simply p − y, the predicted probabilities minus the one-hot label. Interviewers also ask when not to use it: for multi-label problems, use one sigmoid per class. With two classes, softmax reduces to the sigmoid of the logit difference.
