SERVING THE QUANTITATIVE FINANCE COMMUNITY

  • 1
  • 2
  • 3
  • 4
  • 5
  • 22
 
User avatar
JohnLeM
Topic Author
Posts: 335
Joined: September 16th, 2008, 7:15 pm

Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 7th, 2019, 8:33 am

Since we started some years ago a related topic at Wilmott site, I am proposing to discuss here too the topic above, coming from this post
My thesis is that Artificial Intelligence (aka neural networks) applications to the numerical analysis of Partial Differential Equations are about to rediscover meshfree methods. Any comments welcome !
 
User avatar
Cuchulainn
Posts: 59949
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 7th, 2019, 8:54 am

There is a related DL-PDE thread

viewtopic.php?f=4&t=100975


I think that your thesis should be fleshed-out by a compelling example A-Z by comparing PDE and NN side-by-side. 
You could be right.

Question: I saw a article in which the authors solved (or so they claim) 100-dimensional PDE. Really?
 
User avatar
JohnLeM
Topic Author
Posts: 335
Joined: September 16th, 2008, 7:15 pm

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 7th, 2019, 9:05 am

Thanks for pointing me out the link. It is indeed a quite related topic, particularly this link, which I started to discuss with the authors. Here is a quite similar reference, but where the link between Neural Networks and meshfree method is starting to be explicit.
Last edited by JohnLeM on February 7th, 2019, 10:51 am, edited 1 time in total.
 
User avatar
JohnLeM
Topic Author
Posts: 335
Joined: September 16th, 2008, 7:15 pm

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 7th, 2019, 9:10 am

@Cuchullain, concerning your remark, we are already solving Kolmogorov equations (aka black and scholes) with hundreds of dimensions with meshfree methods. We've done it already 5 years ago, see for instance this post for an equity derivative application. Going back to NNs, the AI community is also starting to claim solving them. Thus there is now two potential numerical methods to tackle these problems. My thesis, that is the topic of this forum, is that the AI community is reinventing the wheel: they will end showing off meshfree methods in some years.
Last edited by JohnLeM on February 7th, 2019, 11:32 am, edited 3 times in total.
 
User avatar
FaridMoussaoui
Posts: 438
Joined: June 20th, 2008, 10:05 am
Location: Genève, Genf, Ginevra, Geneva

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 7th, 2019, 9:52 am

Jean-Marc, are you using Monte Carlo/PDE à la Pironneau?
 
User avatar
Cuchulainn
Posts: 59949
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 7th, 2019, 10:05 am

Thanks for the pointing me out the link. It is indeed a quite related topic, particularly this link, which I started to discuss with the authors. Here is a quite similar reference, but where the link between Neural Networks and meshfree method is starting to be explicit.
You're welcome, Jean-Marc.
Glad to hear you are getting feedback! They ignore me. Maybe  start a charm offensive.
 
User avatar
JohnLeM
Topic Author
Posts: 335
Joined: September 16th, 2008, 7:15 pm

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 7th, 2019, 10:28 am

You're welcome, Jean-Marc.
Glad to hear you are getting feedback! They ignore me. Maybe  start a charm offensive.
I just started some days ago to take contacts with them, inviting them to discuss these topics together, or being invited. Some kind of workshop would be a good idea to clarify things. We are waiting for their answers.
Last edited by JohnLeM on February 7th, 2019, 10:55 am, edited 2 times in total.
 
User avatar
JohnLeM
Topic Author
Posts: 335
Joined: September 16th, 2008, 7:15 pm

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 7th, 2019, 10:33 am

Jean-Marc, are you using Monte Carlo/PDE à la Pironneau?
Hi Farid. No, these are originals methods, described in this paper (I think that Pironneau was the main reviewer of this paper, even if unsure). 
 
User avatar
Cuchulainn
Posts: 59949
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 10th, 2019, 7:46 pm

Here is an article on NN for ODEs. Can someone explain - apart from the fact that it is the wrong solution for the wrong problem - what is so new and special.

The examples are trivial and the article gives ODE solvers short shrift which is bordering on recklessness (or leaps of faith that is anathema to mathematicians).
Equation (2) is a yuge ansatz. And very, very wrong. The ODE system is solved by BFGS which is probably very top-heavy, e.g. compute approximate Hessian AFAIR.

http://ijm2c.iauctb.ac.ir/article_52183 ... d810ad.pdf

JM.

(9) is just collocation with activation function as basis function? I'm getting flashbacks to good old FEM days..

LSFEM? (Farid?)

https://cfwebprod.sandia.gov/cfdocs/Com ... burger.pdf

Just because you use a sigmoid and invoke Hornick's magic wand does not make  it a NN.
 
User avatar
FaridMoussaoui
Posts: 438
Joined: June 20th, 2008, 10:05 am
Location: Genève, Genf, Ginevra, Geneva

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 10th, 2019, 11:24 pm

I read the paper on diagonals:

1) The form (2) to enforce the initial condition....
2) As their program is in Matlab, they used the minimizer "fminunc", quasi-Newton is one of the "popular" solvers there.
3) Yes, they are using the hyperbolic secant (not secont) as a basis functions.

Back to the interesting paper.  I am studing Jean-Marc's paper (CRAS) to implement the methodology for my own pleasure.
I am going to read Villani's book: Optimal Transport. Fortunately, as a Geneva resident, I can borrow the book from the university mathematics library.
 
User avatar
Cuchulainn
Posts: 59949
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 11th, 2019, 8:41 am

Thank you for the précis, Farid. It is clear.My intrerest is in knowing 'why'. e.g. the compelling reason to use this method. (the typo 'secont' is indeed).

Another paper on ODE gives less accurate results  for the 101 case dy/dx = y (Table 5) than the humble RK method. The assumption of using higher degree polynomials is not necessarily correct in all cases. If the method is << RK for this linear scalar problem what' the point in generalisation?

https://www.researchgate.net/publicatio ... _Equations

More generally, ODEs have been solved a long time ago b y ODE solvers. What's new?
 
User avatar
Cuchulainn
Posts: 59949
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 11th, 2019, 9:11 am

I am studing Jean-Marc's paper (CRAS) to implement the methodology

Does the article flesh out the algorithm to aid implementation? Or is a PhD in FEM needed? I would personally have difficulty in mapping the high-level maths to an implementation. (I did meshfree but ages ago.. AFAIR it was not as good as FDM)

I would even reduce the scope even more by taking a 1-factor problem and working it out in excruciating detail,  I prefer this approach to a top-down one.
 
User avatar
FaridMoussaoui
Posts: 438
Joined: June 20th, 2008, 10:05 am
Location: Genève, Genf, Ginevra, Geneva

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 11th, 2019, 11:19 am

The article alone is not enough to implement the algorithm (in my point of view). As usual, you have to look to the references.
The method is based on a change of variable from [$]R^d[$] to [$] [0~1]^d [$] (thus the OT theory) and the use of sampling set of the underlying process as a "grid" to solve the adjoint PDE.
It is said that the PDE is solved by a meshfree method (Fasshauer notes) but it is not explicited which one is used.

PS1: The goal of a meshfree method and quantization is to solve the curse of dimentionality.....
PS2: Have a look to Jameson or Löhner papers on meshfree methods. They are more accurate.
Last edited by FaridMoussaoui on February 11th, 2019, 11:31 am, edited 1 time in total.
 
User avatar
Cuchulainn
Posts: 59949
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 11th, 2019, 11:30 am

This article does mention the steps in DL-PDE.  It's a good start

https://arxiv.org/pdf/1708.07469.pdf

But..
they use one-sided finite differences to approximate derivatives, which is sensitive to catastrophic cancellation.
 
User avatar
Cuchulainn
Posts: 59949
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

February 11th, 2019, 11:34 am

The article alone is not enough to implement the algorithm (in my point of view). As usual, you have to look to the references.
The method is based on a change of variable from [$]R^d[$] to [$] [0~1]^d [$] (thus the OT theory) and the use of sampling set of the underlying process as a "grid" to solve the adjoint PDE.
It is said that the PDE is solved by a meshfree method (Fasshauer notes) but it is not explicited which one is used.

PS1: The goal of a meshfree method and quantization is to solve the curse of dimentionality.....
PS2: Have a look to Jameson or Löhner papers on meshfree methods. They are more accurate.
So, the advantages only kick in when [$]d \geq 4[$]?
ABOUT WILMOTT

PW by JB

Wilmott.com has been "Serving the Quantitative Finance Community" since 2001. Continued...


Twitter LinkedIn Instagram

JOBS BOARD

JOBS BOARD

Looking for a quant job, risk, algo trading,...? Browse jobs here...


GZIP: On