0 and 1, which is what makes it the right output layer for a binary probability. Inside a deep network it is a poor choice: gradients vanish once inputs saturate at either tail, and its output is not zero-centred, so ReLU-family activations replaced it in hidden layers.
Card 1 of 259. Answer: 0 and 1, which is what makes it the right output layer for a binary probability. Inside a deep network it is a poor choice: gradients vanish once inputs saturate at either tail, and its output is not zero-centred, so ReLU-family activations replaced it in hidden layers.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Between 0 and 1. The sigmoid, or logistic, function σ(x) = 1 / (1 + e^(−x)) maps any real number into the open interval (0, 1), approaching 1 for large positive inputs and 0 for large negative ones without ever reaching either. That is why its output can be read as a probability in a binary classifier's output layer.
Some values: σ(0) = 0.5, σ(2) ≈ 0.88, σ(−2) ≈ 0.12 and σ(5) ≈ 0.993. The curve is symmetric, σ(−x) = 1 − σ(x), and its inverse is the logit, log(p / (1 − p)). Its relative tanh(x) = 2σ(2x) − 1 has the same S-shape but maps to (−1, 1).
In practice the limits do arrive, through rounding: in 32-bit floats σ(x) evaluates to exactly 1.0 once x passes about 17, and a confident wrong prediction then puts log 0 into the binary cross-entropy. That is why frameworks offer losses that take the raw score, such as PyTorch's BCEWithLogitsLoss. In hidden layers, saturation shrinks gradients, so ReLU replaced sigmoid there.
