October 16th, 2019, 11:30 pm
I see often cited K Hornik, Multilayer feedforward networks are universal approximators, Neural Networks 2 (1989)
Pinkus is commonly cited too.
Science is not exact, it's always an approximation. That's why tge universal approximation theorem had a profound meaning. It showed that universal approximators, namely families of functions which ca approximate anything, exist. It addressed over two centuries of mathematical studies, which in turn emerged from centuries old practical problems - e.g. drawing a map, solving Navier-Stokes eqn, ... Euler had this problem when he pored over the map of Russia to approximate its territory; where do you think Chebyshev polynomials came from? etc. I think it's fair to say that 18th and 19th century mathematical analysis focused on the theory of approximation, asking if universal approximators existed (Chebyshev asking if his polynomials are such approximators). UAT answered that: for polynomials (Weierstrass), feed-forward NNs (Cybenko) and decision trees [??].
Apart from the above, in the perceptron sitting in my skull the synapses between "mathematical", "precision" and "of a theorem" are inhibitory. Sorry! Try again, pls.