If you want a NN to solve the diffusion equation then it's going to have to learn differentiation. So why not train it to differentiate first? Any problems (lack of rigour, pathologies, strange behaviour,...) might become apparent in this simpler problem. And we can be mathematically brutal in trying to find issues, as Devil's Advocates.

Doing something with a normal distribution is a bit irrelevant. You need to train on as many functions as possible.

Re: DL and PDEs

Posted: August 30th, 2018, 1:04 pm

by Cuchulainn

If you want a NN to solve the diffusion equation then it's going to have to learn differentiation. So why not train it to differentiate first?

OK, let's take this up. I'm having difficulty. [$]\frac {\partial T}{\partial t} = a^2\frac{\partial^2 T}{\partial x^2} [$] How to solve? what is meant by 'solve'?
What kind of 'differentitation'? in x?

Re: DL and PDEs

Posted: August 30th, 2018, 2:56 pm

by Paul

Solve means specify an initial condition and boundary conditions with a=1. But you need to train on thousands of conditions and their corresponding solutions.

But easier to start with just getting a NN to figure out d/dx.

Re: DL and PDEs

Posted: September 22nd, 2018, 3:08 pm

by Cuchulainn

Solve means specify an initial condition and boundary conditions with a=1. But you need to train on thousands of conditions and their corresponding solutions.

But easier to start with just getting a NN to figure out d/dx.

Thinking out loud ..
There are many solutions to a PDE so we can define a canonical solution using a combination of elementary functions and parameters. We could then use Hidden Markov Model to determine the nature of the input signal given the output?
For example, we can write the general solution of a system of ODEs in terms of eigen{values, vectors}, a particular integral and arbitrary constants. We use HMM to compute the latter.
An unfounded remark is that HMM is intuitively more appealing than NN backpropagation for this class of problems.And more robust and mathematically grounded..
What do you think,Paul?

I always tell clients to throw in some Machine Learning for marketing purposes. There is no higher purpose than to sell stuff.

Re: DL and PDEs

Posted: February 4th, 2019, 3:38 pm

by katastrofa

IMHO, ML + domain experts can work wonders. Too bad the second component is almost never used. It's usually ML + a bunch of ignorants who treat the results like the truth received from gods.

He's quoting only his own papers. Not a good sign.

You need to lo beyond that. Address the problems and avoid ad hominem. It doesn't advance discourse.

The author is a PDE/FDM practiitioner and regular contributor here so it is allowed to quote his own work. Most articles on DL-PDE to date have been howlerrs and written mainly by computer scientists who are clever folk but have less affinity with hard maths that is needed here. Correct me if I am wrong.

The jury is out on DL-PDE it seems. For the record, authors tend not to respond to questions on their papers. Strange, don't you think?

My favourite one-liner I read was "The Galerkin method can easily be solved by NN".

Re: DL and PDEs

Posted: February 5th, 2019, 8:31 am

by Cuchulainn

And let us not forget that is in no small part a reaction to hype.

Not true. What do you expect from such an inocuous one-liner. I wasn't trying to be smart. Of course you can apply A to B and B to A, but it might be a stupid waste of time. I was trying to elicit a deeper answer beyond the trite one-liner. Do your own homework.
Even when I do take the effort to give an answer I am usually greeted with silence or your usual one-liner. You never explain yourself.

I read that article months ago. I keep my comments to myself. I already have a better solution using a gradient system solver (AD part is a side-show here).