SERVING THE QUANTITATIVE FINANCE COMMUNITY

Paul
Posts: 10429
Joined: July 20th, 2001, 3:28 pm

### Re: DL and PDEs

If you want a NN to solve the diffusion equation then it's going to have to learn differentiation. So why not train it to differentiate first? Any problems (lack of rigour, pathologies, strange behaviour,...) might become apparent in this simpler problem. And we can be mathematically brutal in trying to find issues, as Devil's Advocates.

Doing something with a normal distribution is a bit irrelevant. You need to train on as many functions as possible.

Cuchulainn
Posts: 61529
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: DL and PDEs

If you want a NN to solve the diffusion equation then it's going to have to learn differentiation. So why not train it to differentiate first?
OK, let's take this up. I'm having difficulty.
$\frac {\partial T}{\partial t} = a^2\frac{\partial^2 T}{\partial x^2}$
How to solve? what is meant by 'solve'?
What kind of 'differentitation'? in x?
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

Paul
Posts: 10429
Joined: July 20th, 2001, 3:28 pm

### Re: DL and PDEs

Solve means specify an initial condition and boundary conditions with a=1. But you need to train on thousands of conditions and their corresponding solutions.

But easier to start with just getting a NN to figure out d/dx.

Cuchulainn
Posts: 61529
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: DL and PDEs

Solve means specify an initial condition and boundary conditions with a=1. But you need to train on thousands of conditions and their corresponding solutions.

But easier to start with just getting a NN to figure out d/dx.
Thinking out loud ..
There are many solutions to a PDE so we can define a canonical solution using a combination of elementary functions and parameters. We could then use Hidden Markov Model to determine the nature of the input signal given the output?
For example, we can write the general solution of a system of ODEs in terms of eigen{values, vectors}, a particular integral and arbitrary constants. We use HMM to compute the latter.
An unfounded remark is that HMM is intuitively more appealing than NN backpropagation for this class of problems.And more robust and mathematically grounded..
What do you think,Paul?
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

Cuchulainn
Posts: 61529
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: DL and PDEs

Interesting post, I think the author is on Wilmott.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

Paul
Posts: 10429
Joined: July 20th, 2001, 3:28 pm

### Re: DL and PDEs

I always tell clients to throw in some Machine Learning for marketing purposes. There is no higher purpose than to sell stuff.

katastrofa
Posts: 8953
Joined: August 16th, 2007, 5:36 am
Location: Alpha Centauri

### Re: DL and PDEs

IMHO, ML + domain experts can work wonders. Too bad the second component is almost never used. It's usually ML + a bunch of ignorants who treat the results like the truth received from gods.

ISayMoo
Posts: 2292
Joined: September 30th, 2015, 8:30 pm

### Re: DL and PDEs

Interesting post, I think the author is on Wilmott.
He's quoting only his own papers. Not a good sign.

People saying "PDE is a solved problem" should have a chat with weather forecasters.

Cuchulainn
Posts: 61529
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: DL and PDEs

Interesting post, I think the author is on Wilmott.
He's quoting only his own papers. Not a good sign.

You need to lo beyond that. Address the problems and avoid ad hominem. It doesn't advance discourse.

The author is a PDE/FDM practiitioner and regular contributor here so it is allowed to quote his own work. Most articles on DL-PDE to date have been howlerrs and written mainly by computer scientists who are clever folk but have less affinity with hard maths that is needed here. Correct me if I am wrong.

The jury is out on DL-PDE it seems. For the record, authors tend not to respond to questions on their papers. Strange, don't you think?

My favourite one-liner I read was "The Galerkin method can easily be solved by NN".
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

Cuchulainn
Posts: 61529
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: DL and PDEs

And let us not forget that is in no small part a reaction to hype.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

Cuchulainn
Posts: 61529
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: DL and PDEs

http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

ISayMoo
Posts: 2292
Joined: September 30th, 2015, 8:30 pm

### Re: DL and PDEs

Instead of applying NNs to ODEs, apply ODEs to NNs: https://arxiv.org/abs/1806.07366

Cuchulainn
Posts: 61529
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: DL and PDEs

Instead of applying NNs to ODEs, apply ODEs to NNs: https://arxiv.org/abs/1806.07366
These questions are not even wrong.

But ... what's the problem, i.e. the question that is trying to get out?

What's reality? Is GD closer to reality than an ODE?
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

ISayMoo
Posts: 2292
Joined: September 30th, 2015, 8:30 pm

### Re: DL and PDEs

Instead of applying NNs to ODEs, apply ODEs to NNs: https://arxiv.org/abs/1806.07366
These questions are not even wrong.

Cuchulainn
Posts: 61529
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: DL and PDEs

Instead of applying NNs to ODEs, apply ODEs to NNs: https://arxiv.org/abs/1806.07366
These questions are not even wrong.
Not true. What do you expect from such an inocuous one-liner.
I wasn't trying to be smart. Of course you can apply A to B and B to A, but it might be a stupid waste of time. I was trying to elicit a deeper answer beyond the trite one-liner. Do your own homework.
Even when I do take the effort to give an answer I am usually greeted with silence or your usual one-liner. You never explain yourself.

I read that article months ago. I keep my comments to myself. I already have a better solution using a gradient system solver (AD part is a side-show here).

See
viewtopic.php?f=34&t=101662

Not once did you respond there, so a) ODEs don't interest you or b) you don't know ODEs.

Instead of applying NNs to ODEs, apply ODEs to NNs
OK, I give up, what's the answer? Go on, have a go.

Buona serata.
Last edited by Cuchulainn on May 25th, 2019, 7:43 pm, edited 6 times in total.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget