Serving the Quantitative Finance Community

 
User avatar
Cuchulainn
Topic Author
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Universal Approximation theorem

October 21st, 2019, 3:46 pm

If you're talking about UAT, then what does GD have to do with it?
Oh dear. 
UAT talks about activation functon
AF used in GD

Ergo 
UAT related to GD.
AF is a confounder here.
Another less than useful 1-liner. 
Achtung: I'm writing the requirements. Don't change the problem.

The properties of AF determine how goof GD will be.
Last edited by Cuchulainn on October 21st, 2019, 4:01 pm, edited 2 times in total.
 
User avatar
Cuchulainn
Topic Author
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Universal Approximation theorem

October 21st, 2019, 3:47 pm

There is a difference between numerical optimisation and statistical learning algorithms, which I think you and Cuch are missing. The goal of the first is, given f(x), find x0 = argmin x f(x). The goal of the latter is given a training set, find model parameters which lead to the expected error rate being "low enough" by minimising the empirical error rate. See e.g. the textbook by Shalev-Shwartz and Ben-David.
Bayesian optimisation is which?
From Wiki

Bayesian optimization is a sequential design strategy for global optimization of black-box functions[1] that doesn't require derivatives
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Universal Approximation theorem

October 21st, 2019, 5:34 pm

That was "less than useful", love.
 
User avatar
FaridMoussaoui
Posts: 327
Joined: June 20th, 2008, 10:05 am
Location: Genève, Genf, Ginevra, Geneva

Re: Universal Approximation theorem

October 21st, 2019, 6:32 pm

People trying to communicate but not speaking the same language .... This thread is going nowhere.
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Universal Approximation theorem

October 21st, 2019, 6:46 pm

Що?
 
User avatar
Cuchulainn
Topic Author
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Universal Approximation theorem

October 21st, 2019, 7:35 pm

People trying to communicate but not speaking the same language .... This thread is going nowhere.
Correct.
 
User avatar
JohnLeM
Posts: 379
Joined: September 16th, 2008, 7:15 pm

Re: Universal Approximation theorem

October 21st, 2019, 7:44 pm

People trying to communicate but not speaking the same language .... This thread is going nowhere.
Correct.
Too bad, maybe a little bit of good will could help ? For me the topic is good, if I understood it well : "dig in Cybenko Theorem to turn it into a practical tool". This could really help the IA community to understand their tools.
 
User avatar
Cuchulainn
Topic Author
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Universal Approximation theorem

October 21st, 2019, 7:56 pm

People trying to communicate but not speaking the same language .... This thread is going nowhere.
Correct.
Too bad, maybe a little bit of good will could help ? For me the topic is good, if I understood it well : "dig in Cybenko Theorem to turn it into a practical tool". This could really help the IA community to understand their tools.
I agree. This was always -and still is- the goal. There's a whole bunch of relevant use cases around this topic.
 
User avatar
Cuchulainn
Topic Author
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Universal Approximation theorem

October 21st, 2019, 7:59 pm

That was "less than useful", love.
It's what Wiki says, so it must be right..
Or was it a rhetorical question on your part?  :shock:
Last edited by Cuchulainn on October 21st, 2019, 8:43 pm, edited 1 time in total.
 
User avatar
JohnLeM
Posts: 379
Joined: September 16th, 2008, 7:15 pm

Re: Universal Approximation theorem

October 21st, 2019, 8:17 pm



Correct.
Too bad, maybe a little bit of good will could help ? For me the topic is good, if I understood it well : "dig in Cybenko Theorem to turn it into a practical tool". This could really help the IA community to understand their tools.
I agree. This was always -and still is- the goal. There's a whole bunch of relevant use cases around this topic.
maybe should we do this work once for all. It is not that hard I think, since I could reuse most of my work. I am sure also that there are a lot of use case / business cases. However is there any sponsor out there for such a study ?
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Universal Approximation theorem

October 22nd, 2019, 2:21 am

Image
Last edited by katastrofa on October 24th, 2019, 3:12 am, edited 1 time in total.
 
User avatar
JohnLeM
Posts: 379
Joined: September 16th, 2008, 7:15 pm

Re: Universal Approximation theorem

October 22nd, 2019, 6:34 am

I see. Meantime, I hear more and more voices warning that billions of public and private money are wasted in a technology without any foundation, thus inefficient. The last time that this legitimacy problem popped up, the artificial intelligence community crossed a 15 years desert as nice as your picture...
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Universal Approximation theorem

October 22nd, 2019, 10:27 am

People trying to communicate but not speaking the same language .... This thread is going nowhere.
Correct.
Too bad, maybe a little bit of good will could help ? For me the topic is good, if I understood it well : "dig in Cybenko Theorem to turn it into a practical tool". This could really help the IA community to understand their tools.
Or, equally fairly, it could help the numerical optimisation community understand machine learning theory ;-)
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: Universal Approximation theorem

October 22nd, 2019, 10:29 am


Oh dear. 
UAT talks about activation functon
AF used in GD

Ergo 
UAT related to GD.
AF is a confounder here.
Another less than useful 1-liner. 
I'm sorry. I will follow your example and start composing my posts from multiple 1-liners. Like the one I responded to above.
 
User avatar
JohnLeM
Posts: 379
Joined: September 16th, 2008, 7:15 pm

Re: Universal Approximation theorem

October 22nd, 2019, 11:28 am



Correct.
Too bad, maybe a little bit of good will could help ? For me the topic is good, if I understood it well : "dig in Cybenko Theorem to turn it into a practical tool". This could really help the IA community to understand their tools.
Or, equally fairly, it could help the numerical optimisation community understand machine learning theory ;-)
re-reading this thread, I think that the real problem is not a language problem, but an ego one. Mixing egos and science is usually a poor idea, as is to bite a tendered hand. Since we all gave a poor image of our discipline up to now, could we start to be constructive ?