False. The fire-or-not idea comes from early step-function neurons. An activation function's essential role is to add nonlinearity, without which stacked layers collapse into one linear map.
Proposed by @svpino
Card 1 of 259. Answer: False. The fire-or-not idea comes from early step-function neurons. An activation function's essential role is to add nonlinearity, without which stacked layers collapse into one linear map.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
False. Firing is a loose biological metaphor. The essential role of an activation function is to add nonlinearity: without it, any stack of layers collapses into a single linear function. Most modern activations make no yes-or-no decision; they output graded values whose size carries information.
The metaphor comes from early models. The 1943 neuron of Warren McCulloch and Walter Pitts and Frank Rosenblatt's 1958 perceptron used a step function: 1 if the weighted sum passes a threshold, otherwise 0. That is a true fire-or-not decision, but the step's derivative is 0 everywhere except at the threshold, so gradient descent gets no signal. Backpropagation, popularized in 1986, needed smooth activations such as the sigmoid.
Counterexamples: tanh outputs negative values, GELU dips below zero, and an identity output layer for regression decides nothing. ReLU does switch negative inputs off, but positive ones pass as 0.1 or 50, not just "on". Switch language fits best in LSTM gates, where sigmoids act as soft valves. In interviews, name nonlinearity first, differentiability second.
