True with conditions: by the universal approximation theorem, one hidden layer with enough units and a non-polynomial activation can approximate any continuous function on a closed, bounded domain. With no hidden layer, a network only draws linear boundaries and can't even fit XOR.
Proposed by @svpino
Card 1 of 259. Answer: True with conditions: by the universal approximation theorem, one hidden layer with enough units and a non-polynomial activation can approximate any continuous function on a closed, bounded domain. With no hidden layer, a network only draws linear boundaries and can't even fit XOR.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
True, with conditions. The universal approximation theorem says a network with one hidden layer, enough units and a non-polynomial activation such as sigmoid or ReLU can approximate any continuous function on a closed, bounded region to any accuracy. With no hidden layer, it draws only linear boundaries and fails even on XOR.
Example: f(x) = x² on [0, 1]. Ten ReLU units, 0.1·ReLU(x) plus 0.2·ReLU(x − k/10) for k = 1 to 9, join the curve's points at 0, 0.1, …, 1 with straight lines, with maximum error 0.0025; a hundred units give 0.000025. Cybenko, and separately Hornik, Stinchcombe and White, proved versions in 1989; Leshno and colleagues showed in 1993 that essentially any non-polynomial activation works.
Past x = 1 the network continues as a straight line, predicting 2.9 at x = 2 instead of 4. The theorem says good weights exist, not that gradient descent finds them or that they generalize, and the units needed can grow exponentially with the input dimension, one reason practitioners add depth.
