Serving the Quantitative Finance Community

 
User avatar
Paul
Posts: 6604
Joined: July 20th, 2001, 3:28 pm

Re: DL and PDEs

August 29th, 2018, 8:42 pm

My thinking was this:

If you want a NN to solve the diffusion equation then it's going to have to learn differentiation. So why not train it to differentiate first? Any problems (lack of rigour, pathologies, strange behaviour,...) might become apparent in this simpler problem. And we can be mathematically brutal in trying to find issues, as Devil's Advocates.

Doing something with a normal distribution is a bit irrelevant. You need to train on as many functions as possible.
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: DL and PDEs

August 30th, 2018, 1:04 pm

If you want a NN to solve the diffusion equation then it's going to have to learn differentiation. So why not train it to differentiate first? 
OK, let's take this up. I'm having difficulty.
[$]\frac {\partial T}{\partial t} = a^2\frac{\partial^2 T}{\partial x^2} [$] 
How to solve? what is meant by 'solve'?
What kind of 'differentitation'? in x?
 
User avatar
Paul
Posts: 6604
Joined: July 20th, 2001, 3:28 pm

Re: DL and PDEs

August 30th, 2018, 2:56 pm

Solve means specify an initial condition and boundary conditions with a=1. But you need to train on thousands of conditions and their corresponding solutions.

But easier to start with just getting a NN to figure out d/dx.
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: DL and PDEs

September 22nd, 2018, 3:08 pm

Solve means specify an initial condition and boundary conditions with a=1. But you need to train on thousands of conditions and their corresponding solutions.

But easier to start with just getting a NN to figure out d/dx.
Thinking out loud ..
There are many solutions to a PDE so we can define a canonical solution using a combination of elementary functions and parameters. We could then use Hidden Markov Model to determine the nature of the input signal given the output?
For example, we can write the general solution of a system of ODEs in terms of eigen{values, vectors}, a particular integral and arbitrary constants. We use HMM to compute the latter.
An unfounded remark is that HMM is intuitively more appealing than NN backpropagation for this class of problems.And more robust and mathematically grounded..
What do you think,Paul?
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: DL and PDEs

February 3rd, 2019, 11:30 pm

Interesting post, I think the author is on Wilmott.
https://www.linkedin.com/pulse/artifici ... jean-marc/
 
User avatar
Paul
Posts: 6604
Joined: July 20th, 2001, 3:28 pm

Re: DL and PDEs

February 4th, 2019, 6:38 am

I always tell clients to throw in some Machine Learning for marketing purposes. There is no higher purpose than to sell stuff.
 
User avatar
katastrofa
Posts: 7440
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

Re: DL and PDEs

February 4th, 2019, 3:38 pm

IMHO, ML + domain experts can work wonders. Too bad the second component is almost never used. It's usually ML + a bunch of ignorants who treat the results like the truth received from gods.
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: DL and PDEs

February 4th, 2019, 11:18 pm

Interesting post, I think the author is on Wilmott.
https://www.linkedin.com/pulse/artifici ... jean-marc/
He's quoting only his own papers. Not a good sign.

People saying "PDE is a solved problem" should have a chat with weather forecasters.
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: DL and PDEs

February 5th, 2019, 8:00 am

Interesting post, I think the author is on Wilmott.
https://www.linkedin.com/pulse/artifici ... jean-marc/
He's quoting only his own papers. Not a good sign.

You need to lo beyond that. Address the problems and avoid ad hominem. It doesn't advance discourse.

The author is a PDE/FDM practiitioner and regular contributor here so it is allowed to quote his own work. Most articles on DL-PDE to date have been howlerrs and written mainly by computer scientists who are clever folk but have less affinity with hard maths that is needed here. Correct me if I am wrong.

The jury is out on DL-PDE it seems. For the record, authors tend not to respond to questions on their papers. Strange, don't you think?

My favourite one-liner I read was "The Galerkin method can easily be solved by NN".
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: DL and PDEs

February 5th, 2019, 8:31 am

And let us not forget that is in no small part a reaction to hype.
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: DL and PDEs

May 17th, 2019, 5:01 pm

The role of AAD.
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: DL and PDEs

May 24th, 2019, 2:47 pm

Instead of applying NNs to ODEs, apply ODEs to NNs: https://arxiv.org/abs/1806.07366
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: DL and PDEs

May 25th, 2019, 12:21 pm

Instead of applying NNs to ODEs, apply ODEs to NNs: https://arxiv.org/abs/1806.07366
These questions are not even wrong.

But ... what's the problem, i.e. the question that is trying to get out?

What's reality? Is GD closer to reality than an ODE?
 
User avatar
ISayMoo
Posts: 2332
Joined: September 30th, 2015, 8:30 pm

Re: DL and PDEs

May 25th, 2019, 4:28 pm

Instead of applying NNs to ODEs, apply ODEs to NNs: https://arxiv.org/abs/1806.07366
These questions are not even wrong.
That's your standard answer ;-)
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: DL and PDEs

May 25th, 2019, 7:01 pm

Instead of applying NNs to ODEs, apply ODEs to NNs: https://arxiv.org/abs/1806.07366
These questions are not even wrong.
That's your standard answer ;-)
Not true. What do you expect from such an inocuous one-liner.
I wasn't trying to be smart. Of course you can apply A to B and B to A, but it might be a stupid waste of time. I was trying to elicit a deeper answer beyond the trite one-liner. Do your own homework.
Even when I do take the effort to give an answer I am usually greeted with silence or your usual one-liner. You never explain yourself.

I read that article months ago. I keep my comments to myself. I already have a better solution using a gradient system solver (AD part is a side-show here).

See 
viewtopic.php?f=34&t=101662

Not once did you respond there, so a) ODEs don't interest you or b) you don't know ODEs.

Instead of applying NNs to ODEs, apply ODEs to NNs
OK, I give up, what's the answer? Go on, have a go.

Buona serata.
Last edited by Cuchulainn on May 25th, 2019, 7:43 pm, edited 6 times in total.