Sentiment Analysis. also known as opinion mining, is the process of computationally identifying and categorizing opinions expressed in a piece of text, especially in order to determine whether the writer?s attitude towards a particular topic, product, etc. is positive, negative, or neutral. The analysis finds significant prominence in social media, stock markets, law, policy making, sociology and even customer service. In quant trading, sentiment analysis is used greatly these days. But how do they work? Are there any pitfalls and how can we avoid them?I think we can benefit from this webinar http://www.quantinsti.com/blog/quantita ... s-webinar/ on Quantitative Trading using sentiment analysis and know more about it.

I did the analysis on the options thing about 10 year ago, it's still a valid signal but it's extremely data intensive so unless you have a good historical data set it's quite hard to backtest and ensure is implemented correctly.Also it's not quite a robust signal since it seems sometimes the option writers have the correct lookahead but the onset of HFT and strong momentum trading in the last years has made it flip on occasion.The social media based stuff is all too noisy to be used effectively. All models I've seen on it are overfitted and don't create good signals out of sample.

any sources for the historical data? i agree its quite data intensive, but then the world is gradually moving to big data. can you share some models?

I used OptionMetrics and CRSP but a equity derivatives sell side desk would most likely have better data.The data intesity is more about lack of accurate clean data rather than processing power. Note I was doing this 10 years ago, I was doing this with 2GB ram and a Core2Duo.This was the paper that got me looking at it: http://people.umass.edu/nkapadia/docs/B ... S.pdfThere have been a few papers that have expanded and refined the logic a bit so a reverse reference search might turn up a few of them.