Serving the Quantitative Finance Community

 
User avatar
Cuchulainn
Topic Author
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Universal Approximation theorem

May 13th, 2021, 5:44 pm

Cuchulainn, perhaps something for you: https://arxiv.org/abs/1711.10561
modulo the physics part! (-:
There's also a Part II article.
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Universal Approximation theorem

May 13th, 2021, 6:25 pm

I anticipated you'd guess it yourself.
 
User avatar
Cuchulainn
Topic Author
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Universal Approximation theorem

May 13th, 2021, 7:06 pm

I anticipated you'd guess it yourself.
Well, there's not much in I (read it), so II looks likes a fix.
I'll get back with a report, My kind of area.

Pity they used Physics example to camoflage the lack mathematical/numerical foundations.

Part III?
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: Universal Approximation theorem

May 13th, 2021, 8:40 pm

Who knows where the truth foundations lie? Maybe they are accidentally right.
 
User avatar
Cuchulainn
Topic Author
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Universal Approximation theorem

January 6th, 2023, 8:05 pm

Was it all "much ado about nothing"??
So, Moore's Law brought AI back to life in 2012?
Is this the end? hear what Erik Larson has to say.
 
User avatar
Cuchulainn
Topic Author
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Universal Approximation theorem

June 26th, 2023, 11:30 am

Introducing non-linearity via an activation function allows us to approximate any function. It’s quite simple, really. — Elon Musk
 
User avatar
Cuchulainn
Topic Author
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Universal Approximation theorem

June 26th, 2023, 12:03 pm

Learnability can be undecidable 
Shai Ben-David1 , Pavel Hrubeš2 , Shay Moran3 , Amir Shpilka4 and Amir Yehudayoff  5 * 

The mathematical foundations of machine learning play a key role in the development of the field. They improve our understanding and provide tools for designing new learning paradigms. The advantages of mathematics, however, sometimes come with a cost. Gödel and Cohen showed, in a nutshell, that not everything is provable. Here we show that machine learning shares this fate. We describe simple scenarios where learnability cannot be proved nor refuted using the standard axioms of mathematics. Our proof is based on the fact the continuum hypothesis cannot be proved nor refuted. We show that, in some cases, a solution to the ‘estimating the maximum’ problem is equivalent to the continuum hypothesis. The main idea is to prove an equivalence between learnability and compression.

The main result of this work is that the learnability of the family of sets F * over the class of probability distributions P* is undecidable. While learning F * over P* may not be directly related to practical machine learning applications, the result demonstrates that the notion of learnability is vulnerable. In some general yet simple learning frameworks there is no effective characterization of learnability. In other words, when trying to understand learnability, it is important to pay close attention to the mathematical formalism we choose to use. How come learnability can neither be proved nor refuted? A closer look reveals that the source of the problem is in defining learnability as the existence of a learning function rather than the existence of a learning algorithm. In contrast with the existence of algorithms, the existence of functions over infinite domains is a (logically) subtle issue. The advantages of the current standard definitions (that use the language of functions) is that they separate the statistical or information-theoretic issues from any computational considerations. This choice plays a role in the fundamental characterization of PAC learnability by the VC dimension. Our work shows that this settheoretic view of learnability has a high cost when it comes to more general types of learning. Data availability The data that support the findings of this study are av
Attachments
1687370290312.pdf
(1.59 MiB) Downloaded 56 times