SERVING THE QUANTITATIVE FINANCE COMMUNITY

• 1
• 2

Cuchulainn
Topic Author
Posts: 59225
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### The Final Frontier: negative probability in Machine Learning

Machine Learning, Uncertain Information, and the Inevitability of Negative Probabilities'

http://videolectures.net/mlws04_lowe_mluii/

It was only a matter of time before someone came up with this brainwave.

At its heart I want to challenge the assumption that probabilities have to be positive. I want to give several arguments, descriptive and formal, to indicate why the use of positive probabilities is an ideal which is both overly restrictive and unrealisable. Indeed I will argue that the use of non-positive probabilities' is both inevitable and natural.

JohnLeM
Posts: 298
Joined: September 16th, 2008, 7:15 pm

### Re: The Final Frontier: negative probability in Machine Learning

Machine Learning, Uncertain Information, and the Inevitability of Negative Probabilities'

http://videolectures.net/mlws04_lowe_mluii/

It was only a matter of time before someone came up with this brainwave.

At its heart I want to challenge the assumption that probabilities have to be positive. I want to give several arguments, descriptive and formal, to indicate why the use of positive probabilities is an ideal which is both overly restrictive and unrealisable. Indeed I will argue that the use of non-positive probabilities' is both inevitable and natural.
"The core of the argument is that in modelling the universe through Machine Learning, we are obliged to make inferences based on finite and hence typically less-than-complete information."
We should definitively launch finite-difference skynet !

Cuchulainn
Topic Author
Posts: 59225
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: The Final Frontier: negative probability in Machine Learning

bearish
Posts: 4725
Joined: February 3rd, 2011, 2:19 pm

### Re: The Final Frontier: negative probability in Machine Learning

The negative probability argument video is 15 years old. Oddly coincidental with our own Collector making similar observations, and a certain quant group at Morgan Stanley trying to embed it in deep models of synthetic CDOs. Not necessarily with success.

Collector
Posts: 4196
Joined: August 21st, 2001, 12:37 pm

### Re: The Final Frontier: negative probability in Machine Learning

"Issues in Artificial Intelligence, "the OOM theory suffers from the negative probability problem *NPP)

"Learned OOM may sometimes return negative probabilities for some events"

(OOM: "Observable Operator Models are generalization of HMM models")

Issues in Artificial Intelligence, Robotics and Machine Learning

I look forward to self driving cars! They are so flexible!

JohnLeM
Posts: 298
Joined: September 16th, 2008, 7:15 pm

### Re: The Final Frontier: negative probability in Machine Learning

"Issues in Artificial Intelligence, "the OOM theory suffers from the negative probability problem *NPP)

"Learned OOM may sometimes return negative probabilities for some events"

(OOM: "Observable Operator Models are generalization of HMM models")

Issues in Artificial Intelligence, Robotics and Machine Learning

I look forward to self driving cars! They are so flexible!
Well, this is quite a curious reaction. When I end computing a negative probability, I usually think "OMG, I should fix this up". Cuchullain is quite good at removing negative probas. I can give some tricks also.

ISayMoo
Posts: 1809
Joined: September 30th, 2015, 8:30 pm

### Re: The Final Frontier: negative probability in Machine Learning

Holy smoke! The POSITIVE probabilities are what should be used to describe incomplete information. If you have complete information, you don't need probability at all.

You can define an object to describe the difference between two probability distributions if you want, but don't call it "probability".

Cuchulainn
Topic Author
Posts: 59225
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: The Final Frontier: negative probability in Machine Learning

There's a H in Nicholson

Cuchulainn
Topic Author
Posts: 59225
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: The Final Frontier: negative probability in Machine Learning

Holy smoke! The POSITIVE probabilities are what should be used to describe incomplete information. If you have complete information, you don't need probability at all.

You can define an object to describe the difference between two probability distributions (Q. what's a good name?) if you want, but don't call it "probability".
Well said!

In the same way that the difference of two natural numbers is not a natural number but an integer (very precise statement). aka equivalence class of ordered pairs of natural numbers with a binary operator ~.
Feynman's incomplete article was a step in the right direction but many latched on to it as gospel. Give 'em an inch and they'll take a mile.

I would have no problems if someone defines an order relation on probability numbers. Makes things well-defined.

JohnLeM
Posts: 298
Joined: September 16th, 2008, 7:15 pm

### Re: The Final Frontier: negative probability in Machine Learning

There's a H in Nicholson
I assume, and confess, that I have a picture of Phyllis Nicolson in my bedroom since two days !

JohnLeM
Posts: 298
Joined: September 16th, 2008, 7:15 pm

### Re: The Final Frontier: negative probability in Machine Learning

You can define an object to describe the difference between two probability distributions (Q. what's a good name?) if you want, but don't call it "probability".
Deep Wasserstein Distance ?

ISayMoo
Posts: 1809
Joined: September 30th, 2015, 8:30 pm

### Re: The Final Frontier: negative probability in Machine Learning

ikicker
Posts: 6
Joined: June 8th, 2011, 12:19 am

### Re: The Final Frontier: negative probability in Machine Learning

My personal opinion - What I would do is recompute the probability so it's not negative. Problem solved. Probably you need to do something simple with a ratio to adjust P(not) upwards. Create an error handling function.

You're probably looking at something other than a probability or using an incorrect distribution. Something went wrong. Someone once told me that they computed a correlation higher than 1. I don't doubt it happened, I just acknowledge that something unanticipated happened with your math and you need to fix it.

We have a bigger problem with AI, which is probability distributions that look like waves (multi-modal distributions). You have a missing categorical variable, but until you get it you need to either split the data or do something with it. Also, multi-modal distributions overlap, so they are difficult to bifurcate. Do you perform a custom transformation? Do you run k-means and try to impute the category? It's a complete pain in the ass.
----

Cuchulainn
Topic Author
Posts: 59225
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: The Final Frontier: negative probability in Machine Learning

NP is a thorny issue and it depends who you talk to

engineers; don't believe in it, but APL supports it
physicists believe religiously in it; just change reality if necessary
mathematicians: don't care
ML : of course

Amen

Collector
Posts: 4196
Joined: August 21st, 2001, 12:37 pm

### Re: The Final Frontier: negative probability in Machine Learning

NP is a thorny issue and it depends who you talk to

engineers; don't believe in it, but APL supports it
physicists believe religiously in it; just change reality if necessary
mathematicians: don't care
ML : of course

Amen
why u also need AI.
AI can be used to detect fake news!  and fake probabilities?

AImen