https://arxiv.org/abs/1903.10563

A review. Tishby is a known ML researcher, I don't know the rest of the crew.

https://arxiv.org/abs/1903.10563

A review. Tishby is a known ML researcher, I don't know the rest of the crew.

A review. Tishby is a known ML researcher, I don't know the rest of the crew.

- Cuchulainn
**Posts:**61126**Joined:****Location:**Amsterdam-
**Contact:**

A lot of text, pictures and references. Who is the target reader group?

Not much maths to chew on.. They use the well-known equation (!).Interesting that they mention Langevin dynamics but they don't delve into it. It would be nice to show how the two are related with the exception of a 3-line summary.

I like this style applied to a simple problem with a beginning, middle and end.

https://www.researchgate.net/publicatio ... l_equation

(Something completely different! Looking at the equations and the style of the lemmas, I just realise*now* that I proposed the problems and lemmas way back in 1976..Blast from the past. Documented in my C++ 2004 book pages 132-138.Based on this simple ODE allowed to discover exponential fitting for PDEs and Black Scholes.)

Not much maths to chew on.. They use the well-known equation (!).Interesting that they mention Langevin dynamics but they don't delve into it. It would be nice to show how the two are related with the exception of a 3-line summary.

I like this style applied to a simple problem with a beginning, middle and end.

https://www.researchgate.net/publicatio ... l_equation

(Something completely different! Looking at the equations and the style of the lemmas, I just realise

http://www.datasimfinancial.com

http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself

Jean Piaget

http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself

Jean Piaget

- Cuchulainn
**Posts:**61126**Joined:****Location:**Amsterdam-
**Contact:**

Here's a question.: what's the major difference between a Bayesian Netwiork and a Neural Network?

http://www.datasimfinancial.com

http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself

Jean Piaget

http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself

Jean Piaget

- katastrofa
**Posts:**8750**Joined:****Location:**Alpha Centauri

Bayesian networks represent causal reasoning (see Pearl's DAGs), hence a much more complex problem than what's done with NNs. And no, they weren't developed by the ML community (the methods and dedicated libraries are much older - Metropolis–Hastings algorithm was what year?).

Looks like a very biased review by a bunch of people who assimilate the physical sciences as being condensed matter physics... Pretty much. There is a large number of fields forming part of physical sciences where I can see this sh!t being useful, too.

Did you mean a Bayesian network or a Bayesian neural network? They are not the same.

Did you mean a Bayesian network or a Bayesian neural network? They are not the same.

- katastrofa
**Posts:**8750**Joined:****Location:**Alpha Centauri

Can you see "neural" between "Bayesian" and "network" in his question?

- Cuchulainn
**Posts:**61126**Joined:****Location:**Amsterdam-
**Contact:**

No, And if they were the same one would be redundant.

Did you mean a Bayesian network or a Bayesian neural network? They are not the same.

I am a mathematician and I try to avoid redundant words. I'm sure someone has tried to glue NNs with a posterior but my hunch is that it is a nasty fix and has no theoretical foundations.

It's similar to called an initial value problem for an ODE a Cauchy problem (only for ODEs). ODEs always have initial or boundary conditions.I call it "title inflation"

The big difference IMO is BNs use DAGs while NNs are flat as pancake. They can't hold paths. You can't fit much structure into GD. See Cauchy's original article.

https://www.math.uni-bielefeld.de/docum ... claude.pdf

Last edited by Cuchulainn on May 21st, 2019, 9:48 pm, edited 6 times in total.

http://www.datasimfinancial.com

http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself

Jean Piaget

http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself

Jean Piaget

- Cuchulainn
**Posts:**61126**Joined:****Location:**Amsterdam-
**Contact:**

I would say that "Bayesian neural network" was coined by some anonymous blogger. People googled and started using the term? Then it starts to lead a life of its own.

http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself

Jean Piaget

Metropolis 1953, Hastings 1970.Bayesian networks represent causal reasoning (see Pearl's DAGs), hence a much more complex problem than what's done with NNs. And no, they weren't developed by the ML community (the methods and dedicated libraries are much older - Metropolis–Hastings algorithm was what year?).

- katastrofa
**Posts:**8750**Joined:****Location:**Alpha Centauri

Thank you, Professar Bearish! I was being rhetorically lazyMetropolis 1953, Hastings 1970.Bayesian networks represent causal reasoning (see Pearl's DAGs), hence a much more complex problem than what's done with NNs. And no, they weren't developed by the ML community (the methods and dedicated libraries are much older - Metropolis–Hastings algorithm was what year?).

- Cuchulainn
**Posts:**61126**Joined:****Location:**Amsterdam-
**Contact:**

Is MH a special case of MCMC? "Constructors" for BNs?

http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself

Jean Piaget

- katastrofa
**Posts:**8750**Joined:****Location:**Alpha Centauri

It's a type of an MCMC method / an implementation of an MCMC algorithm(?). Those methods are simple and straightforward.

You're not a mathematician cuch, you're some kind of engineer. You're the guy with the wrench and the duct tape...

You need to do some googling about Bayesian neural nets also...

You need to do some googling about Bayesian neural nets also...

- Cuchulainn
**Posts:**61126**Joined:****Location:**Amsterdam-
**Contact:**

Small dogs bark the loudest. No wonder everyone ignores you. Why don't you say something righteous and hopeful for a change?

http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself

Jean Piaget

GZIP: On