Page 1 of 2

Machine Learning and the physical sciences

Posted: May 20th, 2019, 12:07 pm
by ISayMoo
https://arxiv.org/abs/1903.10563

A review. Tishby is a known ML researcher, I don't know the rest of the crew.

Re: Machine Learning and the physical sciences

Posted: May 20th, 2019, 1:43 pm
by Cuchulainn
A lot of text, pictures and references. Who is the target reader group?
Not much maths to chew on.. They use the well-known equation (!).Interesting that they mention Langevin dynamics but they don't delve into it. It would be nice to show how the two are related with the exception of a 3-line summary.

I like this style applied to a simple problem with a beginning, middle and end.

https://www.researchgate.net/publicatio ... l_equation


(Something completely different! Looking at the equations and the style of the lemmas, I just realise now that  I proposed the problems and lemmas way back in 1976..Blast from the past. Documented in my C++ 2004 book pages 132-138.Based on this simple ODE allowed to discover exponential fitting for PDEs and Black Scholes.)

Re: Machine Learning and the physical sciences

Posted: May 21st, 2019, 6:44 pm
by Cuchulainn
Here's a question.: what's the major difference between a Bayesian Netwiork and a Neural Network?

Re: Machine Learning and the physical sciences

Posted: May 21st, 2019, 7:46 pm
by katastrofa
Bayesian networks represent causal reasoning (see Pearl's DAGs), hence a much more complex problem than what's done with NNs. And no, they weren't developed by the ML community (the methods and dedicated libraries are much older - Metropolis–Hastings algorithm was what year?).

Re: Machine Learning and the physical sciences

Posted: May 21st, 2019, 8:30 pm
by mtsm
Looks like a very biased review by a bunch of people who assimilate the physical sciences as being condensed matter physics... Pretty much. There is a large number of fields forming part of physical sciences where I can see this sh!t being useful, too.

Did you mean a Bayesian network or a Bayesian neural network? They are not the same. 

Re: Machine Learning and the physical sciences

Posted: May 21st, 2019, 9:01 pm
by katastrofa
Can you see "neural" between "Bayesian" and "network" in his question?

Re: Machine Learning and the physical sciences

Posted: May 21st, 2019, 9:18 pm
by Cuchulainn

Did you mean a Bayesian network or a Bayesian neural network? They are not the same. 
No, And if they were the same one would be redundant.
I am a mathematician and I try to avoid redundant words. I'm sure someone has tried to glue NNs with a posterior but my hunch is that it is a nasty fix and has no theoretical foundations.

It's similar to called an initial value problem for an ODE a Cauchy problem (only for ODEs). ODEs always have initial or boundary conditions.I call it "title inflation"
A Cauchy problem in mathematics asks for the solution of a partial differential equation that satisfies certain conditions that are given on a hypersurface in the domain.

The big difference IMO is BNs use DAGs while NNs are flat as pancake. They can't hold paths. You can't fit much structure into GD. See Cauchy's original article.

https://www.math.uni-bielefeld.de/docum ... claude.pdf

Re: Machine Learning and the physical sciences

Posted: May 21st, 2019, 9:36 pm
by Cuchulainn
I would say that "Bayesian neural network" was coined by some anonymous blogger. People googled and started using the term? Then it starts to lead a life of its own.

Re: Machine Learning and the physical sciences

Posted: May 22nd, 2019, 12:50 am
by bearish
Bayesian networks represent causal reasoning (see Pearl's DAGs), hence a much more complex problem than what's done with NNs. And no, they weren't developed by the ML community (the methods and dedicated libraries are much older - Metropolis–Hastings algorithm was what year?).
Metropolis 1953, Hastings 1970.

Re: Machine Learning and the physical sciences

Posted: May 22nd, 2019, 7:17 am
by ISayMoo

Re: Machine Learning and the physical sciences

Posted: May 22nd, 2019, 3:14 pm
by katastrofa
Bayesian networks represent causal reasoning (see Pearl's DAGs), hence a much more complex problem than what's done with NNs. And no, they weren't developed by the ML community (the methods and dedicated libraries are much older - Metropolis–Hastings algorithm was what year?).
Metropolis 1953, Hastings 1970.
Thank you, Professar Bearish! I was being rhetorically lazy :-)

Re: Machine Learning and the physical sciences

Posted: May 22nd, 2019, 4:34 pm
by Cuchulainn
Is MH a special case of MCMC? "Constructors" for BNs?

Re: Machine Learning and the physical sciences

Posted: May 22nd, 2019, 11:46 pm
by katastrofa
It's a type of an MCMC method / an implementation of an MCMC algorithm(?). Those methods are simple and straightforward.

Re: Machine Learning and the physical sciences

Posted: May 23rd, 2019, 1:42 pm
by mtsm
You're not a mathematician cuch, you're some kind of engineer. You're the guy with the wrench and the duct tape...

You need to do some googling about Bayesian neural nets also...

Re: Machine Learning and the physical sciences

Posted: May 23rd, 2019, 3:19 pm
by Cuchulainn
Small dogs bark the loudest. No wonder everyone ignores you. Why don't you say something righteous and hopeful for a change?