SERVING THE QUANTITATIVE FINANCE COMMUNITY

Cuchulainn
Topic Author
Posts: 58700
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### The Final Frontier: negative probability in Machine Learning

Machine Learning, Uncertain Information, and the Inevitability of Negative Probabilities'

http://videolectures.net/mlws04_lowe_mluii/

It was only a matter of time before someone came up with this brainwave.

At its heart I want to challenge the assumption that probabilities have to be positive. I want to give several arguments, descriptive and formal, to indicate why the use of positive probabilities is an ideal which is both overly restrictive and unrealisable. Indeed I will argue that the use of non-positive probabilities' is both inevitable and natural.

JohnLeM
Posts: 298
Joined: September 16th, 2008, 7:15 pm

### Re: The Final Frontier: negative probability in Machine Learning

Machine Learning, Uncertain Information, and the Inevitability of Negative Probabilities'

http://videolectures.net/mlws04_lowe_mluii/

It was only a matter of time before someone came up with this brainwave.

At its heart I want to challenge the assumption that probabilities have to be positive. I want to give several arguments, descriptive and formal, to indicate why the use of positive probabilities is an ideal which is both overly restrictive and unrealisable. Indeed I will argue that the use of non-positive probabilities' is both inevitable and natural.
"The core of the argument is that in modelling the universe through Machine Learning, we are obliged to make inferences based on finite and hence typically less-than-complete information."
We should definitively launch finite-difference skynet !

Cuchulainn
Topic Author
Posts: 58700
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: The Final Frontier: negative probability in Machine Learning

bearish
Posts: 4509
Joined: February 3rd, 2011, 2:19 pm

### Re: The Final Frontier: negative probability in Machine Learning

The negative probability argument video is 15 years old. Oddly coincidental with our own Collector making similar observations, and a certain quant group at Morgan Stanley trying to embed it in deep models of synthetic CDOs. Not necessarily with success.

Collector
Posts: 4141
Joined: August 21st, 2001, 12:37 pm

### Re: The Final Frontier: negative probability in Machine Learning

"Issues in Artificial Intelligence, "the OOM theory suffers from the negative probability problem *NPP)

"Learned OOM may sometimes return negative probabilities for some events"

(OOM: "Observable Operator Models are generalization of HMM models")

Issues in Artificial Intelligence, Robotics and Machine Learning

I look forward to self driving cars! They are so flexible!

JohnLeM
Posts: 298
Joined: September 16th, 2008, 7:15 pm

### Re: The Final Frontier: negative probability in Machine Learning

"Issues in Artificial Intelligence, "the OOM theory suffers from the negative probability problem *NPP)

"Learned OOM may sometimes return negative probabilities for some events"

(OOM: "Observable Operator Models are generalization of HMM models")

Issues in Artificial Intelligence, Robotics and Machine Learning

I look forward to self driving cars! They are so flexible!
Well, this is quite a curious reaction. When I end computing a negative probability, I usually think "OMG, I should fix this up". Cuchullain is quite good at removing negative probas. I can give some tricks also.

ISayMoo
Posts: 1547
Joined: September 30th, 2015, 8:30 pm

### Re: The Final Frontier: negative probability in Machine Learning

Holy smoke! The POSITIVE probabilities are what should be used to describe incomplete information. If you have complete information, you don't need probability at all.

You can define an object to describe the difference between two probability distributions if you want, but don't call it "probability".

Cuchulainn
Topic Author
Posts: 58700
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: The Final Frontier: negative probability in Machine Learning

There's a H in Nicholson

Cuchulainn
Topic Author
Posts: 58700
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: The Final Frontier: negative probability in Machine Learning

Holy smoke! The POSITIVE probabilities are what should be used to describe incomplete information. If you have complete information, you don't need probability at all.

You can define an object to describe the difference between two probability distributions (Q. what's a good name?) if you want, but don't call it "probability".
Well said!

In the same way that the difference of two natural numbers is not a natural number but an integer (very precise statement). aka equivalence class of ordered pairs of natural numbers with a binary operator ~.
Feynman's incomplete article was a step in the right direction but many latched on to it as gospel. Give 'em an inch and they'll take a mile.

I would have no problems if someone defines an order relation on probability numbers. Makes things well-defined.

JohnLeM
Posts: 298
Joined: September 16th, 2008, 7:15 pm

### Re: The Final Frontier: negative probability in Machine Learning

There's a H in Nicholson
I assume, and confess, that I have a picture of Phyllis Nicolson in my bedroom since two days !

JohnLeM
Posts: 298
Joined: September 16th, 2008, 7:15 pm

### Re: The Final Frontier: negative probability in Machine Learning

You can define an object to describe the difference between two probability distributions (Q. what's a good name?) if you want, but don't call it "probability".
Deep Wasserstein Distance ?

ISayMoo
Posts: 1547
Joined: September 30th, 2015, 8:30 pm

### Re: The Final Frontier: negative probability in Machine Learning

There's loads of them.