SERVING THE QUANTITATIVE FINANCE COMMUNITY

• 1
• 2
• 3
• 4
• 5
• 23

JohnLeM
Topic Author
Posts: 380
Joined: September 16th, 2008, 7:15 pm

Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

Since we started some years ago a related topic at Wilmott site, I am proposing to discuss here too the topic above, coming from this post
My thesis is that Artificial Intelligence (aka neural networks) applications to the numerical analysis of Partial Differential Equations are about to rediscover meshfree methods. Any comments welcome !

Cuchulainn
Posts: 62626
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

There is a related DL-PDE thread

viewtopic.php?f=4&t=100975

I think that your thesis should be fleshed-out by a compelling example A-Z by comparing PDE and NN side-by-side.
You could be right.

Question: I saw a article in which the authors solved (or so they claim) 100-dimensional PDE. Really?
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

JohnLeM
Topic Author
Posts: 380
Joined: September 16th, 2008, 7:15 pm

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

Thanks for pointing me out the link. It is indeed a quite related topic, particularly this link, which I started to discuss with the authors. Here is a quite similar reference, but where the link between Neural Networks and meshfree method is starting to be explicit.
Last edited by JohnLeM on February 7th, 2019, 10:51 am, edited 1 time in total.

JohnLeM
Topic Author
Posts: 380
Joined: September 16th, 2008, 7:15 pm

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

@Cuchullain, concerning your remark, we are already solving Kolmogorov equations (aka black and scholes) with hundreds of dimensions with meshfree methods. We've done it already 5 years ago, see for instance this post for an equity derivative application. Going back to NNs, the AI community is also starting to claim solving them. Thus there is now two potential numerical methods to tackle these problems. My thesis, that is the topic of this forum, is that the AI community is reinventing the wheel: they will end showing off meshfree methods in some years.
Last edited by JohnLeM on February 7th, 2019, 11:32 am, edited 3 times in total.

FaridMoussaoui
Posts: 507
Joined: June 20th, 2008, 10:05 am
Location: Genève, Genf, Ginevra, Geneva

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

Jean-Marc, are you using Monte Carlo/PDE à la Pironneau?

Cuchulainn
Posts: 62626
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

Thanks for the pointing me out the link. It is indeed a quite related topic, particularly this link, which I started to discuss with the authors. Here is a quite similar reference, but where the link between Neural Networks and meshfree method is starting to be explicit.
You're welcome, Jean-Marc.
Glad to hear you are getting feedback! They ignore me. Maybe  start a charm offensive.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

JohnLeM
Topic Author
Posts: 380
Joined: September 16th, 2008, 7:15 pm

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

You're welcome, Jean-Marc.
Glad to hear you are getting feedback! They ignore me. Maybe  start a charm offensive.
I just started some days ago to take contacts with them, inviting them to discuss these topics together, or being invited. Some kind of workshop would be a good idea to clarify things. We are waiting for their answers.
Last edited by JohnLeM on February 7th, 2019, 10:55 am, edited 2 times in total.

JohnLeM
Topic Author
Posts: 380
Joined: September 16th, 2008, 7:15 pm

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

Jean-Marc, are you using Monte Carlo/PDE à la Pironneau?
Hi Farid. No, these are originals methods, described in this paper (I think that Pironneau was the main reviewer of this paper, even if unsure).

Cuchulainn
Posts: 62626
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

Here is an article on NN for ODEs. Can someone explain - apart from the fact that it is the wrong solution for the wrong problem - what is so new and special.

The examples are trivial and the article gives ODE solvers short shrift which is bordering on recklessness (or leaps of faith that is anathema to mathematicians).
Equation (2) is a yuge ansatz. And very, very wrong. The ODE system is solved by BFGS which is probably very top-heavy, e.g. compute approximate Hessian AFAIR.

JM.

(9) is just collocation with activation function as basis function? I'm getting flashbacks to good old FEM days..

LSFEM? (Farid?)

https://cfwebprod.sandia.gov/cfdocs/Com ... burger.pdf

Just because you use a sigmoid and invoke Hornick's magic wand does not make  it a NN.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

FaridMoussaoui
Posts: 507
Joined: June 20th, 2008, 10:05 am
Location: Genève, Genf, Ginevra, Geneva

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

I read the paper on diagonals:

1) The form (2) to enforce the initial condition....
2) As their program is in Matlab, they used the minimizer "fminunc", quasi-Newton is one of the "popular" solvers there.
3) Yes, they are using the hyperbolic secant (not secont) as a basis functions.

Back to the interesting paper.  I am studing Jean-Marc's paper (CRAS) to implement the methodology for my own pleasure.
I am going to read Villani's book: Optimal Transport. Fortunately, as a Geneva resident, I can borrow the book from the university mathematics library.

Cuchulainn
Posts: 62626
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

Thank you for the précis, Farid. It is clear.My intrerest is in knowing 'why'. e.g. the compelling reason to use this method. (the typo 'secont' is indeed).

Another paper on ODE gives less accurate results  for the 101 case dy/dx = y (Table 5) than the humble RK method. The assumption of using higher degree polynomials is not necessarily correct in all cases. If the method is << RK for this linear scalar problem what' the point in generalisation?

https://www.researchgate.net/publicatio ... _Equations

More generally, ODEs have been solved a long time ago b y ODE solvers. What's new?
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

Cuchulainn
Posts: 62626
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

I am studing Jean-Marc's paper (CRAS) to implement the methodology

Does the article flesh out the algorithm to aid implementation? Or is a PhD in FEM needed? I would personally have difficulty in mapping the high-level maths to an implementation. (I did meshfree but ages ago.. AFAIR it was not as good as FDM)

I would even reduce the scope even more by taking a 1-factor problem and working it out in excruciating detail,  I prefer this approach to a top-down one.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

FaridMoussaoui
Posts: 507
Joined: June 20th, 2008, 10:05 am
Location: Genève, Genf, Ginevra, Geneva

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

The article alone is not enough to implement the algorithm (in my point of view). As usual, you have to look to the references.
The method is based on a change of variable from $R^d$ to $[0~1]^d$ (thus the OT theory) and the use of sampling set of the underlying process as a "grid" to solve the adjoint PDE.
It is said that the PDE is solved by a meshfree method (Fasshauer notes) but it is not explicited which one is used.

PS1: The goal of a meshfree method and quantization is to solve the curse of dimentionality.....
PS2: Have a look to Jameson or Löhner papers on meshfree methods. They are more accurate.
Last edited by FaridMoussaoui on February 11th, 2019, 11:31 am, edited 1 time in total.

Cuchulainn
Posts: 62626
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

This article does mention the steps in DL-PDE.  It's a good start

https://arxiv.org/pdf/1708.07469.pdf

But..
they use one-sided finite differences to approximate derivatives, which is sensitive to catastrophic cancellation.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl

Cuchulainn
Posts: 62626
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Are Artificial Intelligence methods (AKA Neural Networks) for PDEs about to rediscover the wheel ?

The article alone is not enough to implement the algorithm (in my point of view). As usual, you have to look to the references.
The method is based on a change of variable from $R^d$ to $[0~1]^d$ (thus the OT theory) and the use of sampling set of the underlying process as a "grid" to solve the adjoint PDE.
It is said that the PDE is solved by a meshfree method (Fasshauer notes) but it is not explicited which one is used.

PS1: The goal of a meshfree method and quantization is to solve the curse of dimentionality.....
PS2: Have a look to Jameson or Löhner papers on meshfree methods. They are more accurate.
So, the advantages only kick in when $d \geq 4$?
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl