Concept
A neural network with one hidden layer can approximate any continuous function.
Understand it in one breath
"A single hidden-layer neural network can approximate any continuous function to arbitrary precision." Cybenko's theorem (1989). The first mathematical guarantee for why deep learning works. Given enough width, expressiveness is unbounded — but how wide you actually need is still open (one of deep learning theory's biggest gaps).
At a glance
Key formula
Key moments
Cybenko — the theorem for sigmoidal functions
George Cybenko proved that a neural network with one hidden layer and a sigmoidal activation can approximate any continuous function on a compact domain arbitrarily well.
Hornik — broader activation functions
Kurt Hornik showed that universal approximation depends less on a particular sigmoid and holds for broad classes of activation functions.
Why depth helps
A growing body of theory demonstrated that some functions can be represented exponentially more efficiently by deep networks than by shallow ones.
Modern applications
A theoretical foundation for neural networks, general function approximation, value-function estimation in reinforcement learning, and physics-informed neural networks for differential equations.
Beyond MathVoyage
Loading…