Serving the Quantitative Finance Community

 
User avatar
finanzmaster
Topic Author
Posts: 61
Joined: March 11th, 2007, 1:04 pm

Big Data and Deep Learning, a technology revolution in trading or yet another hype?

October 23rd, 2016, 9:30 am

My non-technical essay.

Summary:

*BigData and DeepLearning are popular buzz words nowadays. But the number of the genuine success stories is relatively small.
*In trading the BigData technology is mostly associated with automatic analysis of the news and sentiment in social networks. But unless you are Google or Reuters, you will never be the one who gets the news first. Additionally, a market reaction both to news and sentiment is often vague and amorph.
*Large deep neural networks closely resemble a human brain, which also has a lot of neurons, interconnected in many layers. But it doesn’t mean a breakthrough to a real artificial intelligence: all is not gold that glitters.
*A positive side: trading is only a part of the financial world. Likely, BigData + DeepLearing has a high potential in adjacent areas like risk profiling and credibility analysis.

So IMO there is more hype than opportunities. However, I would be happy to hear the opposite opinions (esp. with concrete examples of success stories).
 
User avatar
Cuchulainn
Posts: 20254
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Big Data and Deep Learning, a technology revolution in trading or yet another hype?

October 23rd, 2016, 12:00 pm

 There was a discussion on this topic here.
viewtopic.php?f=16&t=100142
 
User avatar
DominicConnor
Posts: 41
Joined: July 14th, 2002, 3:00 am

Re: Big Data and Deep Learning, a technology revolution in trading or yet another hype?

November 25th, 2016, 5:30 pm

As a rough rule, if you're reading about a given technique being sucessful and profitable, it's mined out and by the time you master it, no longer as valuable.

It's use in Risk is problematical. Traditional statistical techniques allow us to say that given past data we have a probability of a given thing happening. OK, as experience shows, those probabilities were often horribly wrong but at least we had reasoning we could defend. It's maths and therefore open to inspection, validation and assigning the blame.

DL doesn't work like that at all. It does rather look like we have a choice between provable maths that doesn't actually deliver and AI/ML/DL that produces results we trust right up until they go horribly and incomprehensibly wrong.

I'd argue that this isn't a bug as such, but an inherent property of knowledge based systems. Information tells us that 'surprise' has value in the form of entropy, so the most advanced informaion system will necessariliy be the one that consists more of 'surprises' in its stucture.
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Big Data and Deep Learning, a technology revolution in trading or yet another hype?

November 25th, 2016, 7:18 pm

Provable math that doesn't match empirical distributions and stochastic behaviors at all.

One of the most exciting recent developments in DL is modeling high dimensional real world distributions with generative adversarial networks.
 
User avatar
DominicConnor
Posts: 41
Joined: July 14th, 2002, 3:00 am

Re: Big Data and Deep Learning, a technology revolution in trading or yet another hype?

November 26th, 2016, 2:18 pm

I was using the word 'proff' in the model validation sense, rather than starting from axioms, ie 'proof' by 'we did it last time and the regulators didn't whing'