- October 21st, 2019, 8:17 pm
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

Correct. Too bad, maybe a little bit of good will could help ? For me the topic is good, if I understood it well : "dig in Cybenko Theorem to turn it into a practical tool". This could really help the IA community to understand their tools. I agree. This was always -and still is- the goal. There's ...

- October 21st, 2019, 7:44 pm
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

People trying to communicate but not speaking the same language .... This thread is going nowhere. Correct. Too bad, maybe a little bit of good will could help ? For me the topic is good, if I understood it well : "dig in Cybenko Theorem to turn it into a practical tool". This could really help the...

- October 21st, 2019, 2:20 pm
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

There is a difference between numerical optimisation and statistical learning algorithms, which I think you and Cuch are missing. The goal of the first is, given f(x), find x0 = argmin x f(x). The goal of the latter is given a training set, find model parameters which lead to the expected error rat...

- October 21st, 2019, 2:06 pm
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

Here it is (can't insert an image anymore at Wilmott or did I misunderstood ?)

- October 21st, 2019, 1:53 pm
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

There is a difference between numerical optimisation and statistical learning algorithms, which I think you and Cuch are missing. The goal of the first is, given f(x), find x0 = argmin x f(x). The goal of the latter is given a training set, find model parameters which lead to the expected error rat...

- October 21st, 2019, 8:01 am
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

I am also pretty much convinced that the problem is not understood, while re-reading @outrun answer. Maybe should the AI community start by rewriting the Cybenko theorem in a more precise manner ? 1) First step clarify the discrete functional space and what is the exact role of the activation functi...

- October 20th, 2019, 1:22 pm
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

I think this is a Freudian typo : the role of theta is not clear in cybenko theorem. It seems that theta is a constant used to match the mass center.

- October 19th, 2019, 11:49 pm
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

I wish I were. Sorry, I had a hard evening !Is JohnLeM outrun? They give me the same psychoneuroimmunological reaction.

- October 19th, 2019, 11:30 pm
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

Well. If you are frightened by a constant function, consider [$]\varphi(x) = \inf (exp(1-|x|), 1)[$]. It is continuous, and fulfill Cybenko assumptions : one point is enough to get infinite accuracy on Cuchullain example. Anyhow, the set of functions [$]\sum_{i=1}^{N}v_i\varphi \left(w_i^T x+b_i\ri...

- October 19th, 2019, 12:55 pm
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

I think you did not get my point. it is today not a good idea to train an exponential function with a NN. That's not what I said at all. It was to show you constant AF doesn't work (in maths, they call it a counterexample). Other AFs do work. I think you did not get my point too : I know how to ha...

- October 19th, 2019, 12:41 pm
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

I think you did not get my point. it is today not a good idea to train an exponential function with a NN. That's not what I said at all. It was to show you constant AF doesn't work (in maths, they call it a counterexample). Other AFs do work. I think you did not get my point too : I know how to ha...

- October 19th, 2019, 11:49 am
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

Counter-example: I have just built a small C++ module for NN using UAT, activation function, etc. etc. Training [$]exp(-x)[$] using a constant AF is bad. A sigmoid gives good results as expected. https://i.imgur.com/9zIUamA.jpg I agree with you : it is today not a good idea to train an exponential ...

- October 19th, 2019, 10:56 am
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

@Cuchullain concerning the equivalence of approaches between kernels and AI methods, a consequence is this one : give me an activation function, and I can compute today precisely the dependence [$]\epsilon(N,D,\varphi)[$] in Cybenko Theorem, if you give me any activation function [$]\varphi[$]. Here...

- October 19th, 2019, 10:25 am
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

The contrary is false, and I use tons of others kernels that do not fit Cybenko theorem. For instance φ ( | x − y | ) φ(|x−y|) , AKA a radial basis function, is a kernel that does not fit Cybenko theorem. [ltr] I think you are getting confused. We are not talking about rbf. Just stay on topic pl...

- October 19th, 2019, 10:19 am
- Forum: Numerical Methods Forum
- Topic: Universal Approximation theorem
- Replies:
**251** - Views:
**42172**

Well. If you are frightened by a constant function, consider [$]\varphi(x) = \inf (exp(1-|x|), 1)[$]. It is continuous, and fulfill Cybenko assumptions : one point is enough to get infinite accuracy on Cuchullain example. Anyhow, the set of functions [$]\sum_{i=1}^{N}v_i\varphi \left(w_i^T x+b_i\ri...

GZIP: On