SERVING THE QUANTITATIVE FINANCE COMMUNITY

Search found 380 matches

  • 1
  • 2
  • 3
  • 4
  • 5
  • 26
by JohnLeM
October 21st, 2019, 8:17 pm
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

Correct. Too bad, maybe a little bit of good will could help ? For me the topic is good, if I understood it well : "dig in Cybenko Theorem to turn it into a practical tool". This could really help the IA community to understand their tools. I agree. This was always -and still is- the goal. There's ...
by JohnLeM
October 21st, 2019, 7:44 pm
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

People trying to communicate but not speaking the same language .... This thread is going nowhere. Correct. Too bad, maybe a little bit of good will could help ? For me the topic is good, if I understood it well : "dig in Cybenko Theorem to turn it into a practical tool". This could really help the...
by JohnLeM
October 21st, 2019, 2:20 pm
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

There is a difference between numerical optimisation and statistical learning algorithms, which I think you and Cuch are missing. The goal of the first is, given f(x), find x0 = argmin x f(x). The goal of the latter is given a training set, find model parameters which lead to the expected error rat...
by JohnLeM
October 21st, 2019, 2:06 pm
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

Here it is (can't insert an image anymore at Wilmott or did I misunderstood ?)
by JohnLeM
October 21st, 2019, 1:53 pm
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

There is a difference between numerical optimisation and statistical learning algorithms, which I think you and Cuch are missing. The goal of the first is, given f(x), find x0 = argmin x f(x). The goal of the latter is given a training set, find model parameters which lead to the expected error rat...
by JohnLeM
October 21st, 2019, 8:01 am
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

I am also pretty much convinced that the problem is not understood, while re-reading @outrun answer. Maybe should the AI community start by rewriting the Cybenko theorem in a more precise manner ? 1) First step clarify the discrete functional space and what is the exact role of the activation functi...
by JohnLeM
October 20th, 2019, 1:22 pm
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

I think this is a Freudian typo : the role of theta is not clear in cybenko theorem. It seems that theta is a constant used to match the mass center.
by JohnLeM
October 19th, 2019, 11:49 pm
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

Is JohnLeM outrun? They give me the same psychoneuroimmunological reaction.

Image
:) I wish I were. Sorry, I had a hard evening !
by JohnLeM
October 19th, 2019, 11:30 pm
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

Well. If you are frightened by a constant function, consider [$]\varphi(x) = \inf (exp(1-|x|), 1)[$]. It is continuous, and fulfill Cybenko assumptions : one point is enough to get infinite accuracy on Cuchullain example. Anyhow, the set of functions [$]\sum_{i=1}^{N}v_i\varphi \left(w_i^T x+b_i\ri...
by JohnLeM
October 19th, 2019, 12:55 pm
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

I think you did not get my point.  it is today not a good idea to train an exponential function with a NN. That's not what I said at all. It was to show you constant AF doesn't work (in maths, they call it a counterexample). Other AFs do work. I think you did not get my point too : I know how to ha...
by JohnLeM
October 19th, 2019, 12:41 pm
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

I think you did not get my point.  it is today not a good idea to train an exponential function with a NN. That's not what I said at all. It was to show you constant AF doesn't work (in maths, they call it a counterexample). Other AFs do work. I think you did not get my point too : I know how to ha...
by JohnLeM
October 19th, 2019, 11:49 am
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

Counter-example: I have just built a small C++ module for NN using UAT, activation function, etc. etc. Training [$]exp(-x)[$] using a constant AF is bad. A sigmoid gives good results as expected. https://i.imgur.com/9zIUamA.jpg I agree with you : it is today not a good idea to train an exponential ...
by JohnLeM
October 19th, 2019, 10:56 am
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

@Cuchullain concerning the equivalence of approaches between kernels and AI methods, a consequence is this one : give me an activation function, and I can compute today precisely the dependence [$]\epsilon(N,D,\varphi)[$] in Cybenko Theorem, if you give me any activation function [$]\varphi[$]. Here...
by JohnLeM
October 19th, 2019, 10:25 am
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

The contrary is false, and I use tons of others kernels that do not fit Cybenko theorem. For instance  φ ( | x − y | ) φ(|x−y|) , AKA a radial basis function, is a kernel that does not fit Cybenko theorem. [ltr] I think you are getting confused.   We are not talking about rbf. Just stay on topic pl...
by JohnLeM
October 19th, 2019, 10:19 am
Forum: Numerical Methods Forum
Topic: Universal Approximation theorem
Replies: 251
Views: 42172

Re: Universal Approximation theorem

Well. If you are frightened by a constant function, consider [$]\varphi(x) = \inf (exp(1-|x|), 1)[$]. It is continuous, and fulfill Cybenko assumptions : one point is enough to get infinite accuracy on Cuchullain example. Anyhow, the set of functions [$]\sum_{i=1}^{N}v_i\varphi \left(w_i^T x+b_i\ri...
  • 1
  • 2
  • 3
  • 4
  • 5
  • 26
GZIP: On