SERVING THE QUANTITATIVE FINANCE COMMUNITY

• 1
• 2

gelfand
Topic Author
Posts: 141
Joined: July 14th, 2002, 3:00 am

### machine learning and big data

Proficiency with "machine learning" and "big data" is currently valued in the financial quant job market. What credentials can you acquire to demonstrate such proficiency? For some time there have been master's degree programs in finance that focus on stochastic calculus and derivatives pricing. Are they now teaching "machine learning" as well?

Cuchulainn
Posts: 61184
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: machine learning and big data

One thing for sure: everyone's talking about it. Is it a fad? Probably not.

Any ML specialists here on Wilmott?
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

liam
Posts: 175
Joined: November 16th, 2004, 11:51 am

### Re: machine learning and big data

Proficiency with "machine learning" and "big data" is currently valued in the financial quant job market. What credentials can you acquire to demonstrate such proficiency? For some time there have been master's degree programs in finance that focus on stochastic calculus and derivatives pricing. Are they now teaching "machine learning" as well?
If you are doing a masters in finance which doesn't cover these topics then I would do a Kaggle competition that uses these methods and brag about your ranking on your CV. Even if it isn't something finance related as it shows skills to build something. I really see no need to go as far as getting some Cloudera training, which has worked in more "data science" roles for people.

Even when I was interviewing all those years ago after uni my CV mentioned extra stuff I did to learn C++ at home and which models I built, so it wouldn't be abnormal to have to add things like that, especially if you are not experienced. In fact one interviewer told me to talk down the TCD (my undergrad) courses I did in C++ and to talk up this hobby I had.

I'm not sure about how ML is used in finance though. I'm more familiar with their uses for IT and tech companies like Google and Amazon and can see it is no fad for them at al.

Tbh my expertise is more the ins and outs of recommenders and neural nets and whizzing around with R and Hadoop rather than how the labour market is. Anything in that regard is anecdotal e.g. have come across some quants at conferences doing Hadoop and/or ML projects at banks, but could not give a comprehensive guide like how Dominic would (or like some other recruiters I used would when I was in finance). Again, like Cuchulainn I think I would need someone more specialised to speak on the market specifics in finance.

Cuchulainn
Posts: 61184
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: machine learning and big data

If we look at the Wiki entry, there are lots of applications of ML, none of which is finance.
https://en.wikipedia.org/wiki/Machine_learning
IMO it will take universities to catch up before they have a ML_MFE offering. It takes 15 years before a technology becomes mainstream (not including Kagglers who are the early adopters).

A naïve summary is it's a branch/application of statistics, lots of algos and std:maps.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

outrun
Posts: 4573
Joined: April 29th, 2016, 1:40 pm

### Re: machine learning and big data

There are quite a few ML memers here, even T4A (and me) are "Kaggle masters" , it's a lot of fun to do those competitions and it was very educational, I would also suggest you do that. Also, I work at a lot of different companies and all banks I've seen use ML. Mostly at the retail side, modelling behaviour of people: credit rating, mortgage prepayment, advertising, savings accounts, etc. The people who do ML at banks are either quants who acquired skills on the job or IT/database people. The quants who do that have very good statistical skills, thy know about all the statistical test, calibration methods and their pitfalls, potential biases caused by subtle censoring biasses, how to prevent overfitting etc etc

I didn't respond earlier because I have a feeling you're maybe not to interested in ML other than career-wise? On the other hand you are young and pre-career and it's very understandable that you try to figure out what to do, and so I didn't want to be negative either.

My advice would be to follow your heart instead of a shiny ball that roll along. If your really interested in ML then you would already be experimenting with it I'd imagine? I can't think of any obstacle other than time? If you enjoy modelling empirical data then ML skill will evolve naturally and you'll have no trouble during future interview because you can talk about your passion. .. so I'd suggest you get started by yourself and don't focus on certificates just yet, focus on skills, e.g. measure them against peers during Kaggle competitions.

Sometimes you can also win $10k-$1mln during a competition, that's also handy..

Cuchulainn
Posts: 61184
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: machine learning and big data

Someone on QN that ML is used by hedge funds..
?
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

mtsm
Posts: 352
Joined: July 28th, 2010, 1:40 pm

### Re: machine learning and big data

It's a bit tricky. ML isn't a discipline per se, it's a clever repackaging of a mix of subtopics coming out fields such as CS, statistics, optimization, signal processing, AI, and a couple of other fields... It's kind of engineered. US universities are quite smart about creating such hype and it works very well, since CS and statistics programs have been soaring and the demand for experts in this 'discipline' is high. It's a bit like what happened with financial engineering. Initially, there was no financial engineering, it's an engineered discipline.

ML is used everywhere in this sense, on buy and sell-side. Traditional financial engineering has gone quite cold on the other hand, certainly here in the US it has. Masters programs are struggling to adapt. NYU and Columbia are working on it though. The collapse of the derivatives industry, over-regulation and the all-round performance limits reached by a lot of traditional quant finance tools are why among others.

People with ML profiles on the other hand abound popping out of many programs. I find it interesting personally, that a lot of these appear to have a completely different background from what you would have picked up earlier. No C++ or compiled language experience for example. Only scripted languages and packages. Absolutely zero engineering and science background, only straight up math, stats and CS skills, etc... It's a new world.

gelfand
Topic Author
Posts: 141
Joined: July 14th, 2002, 3:00 am

### Re: machine learning and big data

I didn't respond earlier because I have a feeling you're maybe not to interested in ML other than career-wise? On the other hand you are young and pre-career and it's very understandable that you try to figure out what to do, and so I didn't want to be negative either.
I am a former hedge fund quant with an equity derivatives background looking to upgrade his skills. I am wondering whether to spend time augmenting my knowledge of

(1) financial econometrics (ARMA models, vector autoregressions, GARCH, stochastic volatility, cointegration) or
(2) "machine learning" topics such as neural networks and support vector machines.

Topic (2) is "cool" right now, but I wonder if it is more useful.

outrun
Posts: 4573
Joined: April 29th, 2016, 1:40 pm

### Re: machine learning and big data

I think 1) would deepen your quant skills. Eg if you are in a quant position then this option would position you better inside the corporate 2) would mean diversification and more work outside finance in all sort of roles (but also needed for Algo trading). Both paths will have competition for you, and both will typically be applied types of roles (where the money is) and less so research.

I would say 2) career wise, other fields than finance show much more growth, the trick is to get onto one of those growth paths.

ChrisGifford
Posts: 38
Joined: February 6th, 2016, 10:27 pm

### Re: machine learning and big data

<This message has been deleted by its author>
Last edited by ChrisGifford on November 26th, 2016, 12:08 pm, edited 1 time in total.

outrun
Posts: 4573
Joined: April 29th, 2016, 1:40 pm

### Re: machine learning and big data

http://blog.deepgram.com/how-to-get-a-j ... -learning/

Cuchulainn
Posts: 61184
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: machine learning and big data

ML could be called "Applied Neural Networks/AI"?? It's not a new term (1952!) but it looks like all the AI algorithms have become feasible now that computers are more powerful. For example, ICR has been around in semi-commercial document control systems since the 1990's but it did not take off for some reason. Of course, there was quite a bit of hype about new software technologies at the time.

Anyways, it's on everyone's lips. What's in a name?

Found this overview of ML from 1983

http://www.aaai.org/ojs/index.php/aimag ... ew/406/342
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

outrun
Posts: 4573
Joined: April 29th, 2016, 1:40 pm

### Re: machine learning and big data

I studied ML in the 80s, and I do it a lot today.

I don't understand why you as an uninformed bystander try to frame things so negative? Why is that? You know the AIs are reading all these posts, and they are already in your home router, ..tolerating your presence there, for now..

Cuchulainn
Posts: 61184
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: machine learning and big data

I studied ML in the 80s, and I do it a lot today.

I don't understand why you as an uninformed bystander try to frame things so negative? Why is that? You know the AIs are reading all these posts, and they are already in your home router, ..tolerating your presence there, for now..
There you go again. I knew the people in Delft from the 80s. But they called it AI and Prolog back them.

Who can give a clear definition of ML? It should not be that difficult, even for 'bystanders' like myself. Go on , have a go.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

Cuchulainn
Posts: 61184
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: machine learning and big data

This is how Microsoft defines ML; what are they talking about?

Machine learning represents the logical extension of simple data retrieval and storage. It is about developing building blocks that make computers learn and behave more intelligently.Machine learning makes it possible to mine historical data and make predictions about future trends. Without realizing it, you are probably already using the benefits of machine learning. Search engine results, online recommendations, ad targeting, fraud detection, and spam filtering are all examples of what is possible with machine learning.
Machine learning is about making data-driven decisions. While instinct might be important, it is difficult to beat empirical data.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget