Hi everybody!Is it viable to apply data mining techniques to the financial market context? I actually mean whether is money-mining possible by applying such stuff as genetic algorithms, neural nets and fuzzy logic to identify patterns in security prices. There were a lot of papers on applying these techniques to market prediction and trading strategy development in 1990s, but what is the current situtation with that direction? Have some new techniques been developed? If money-mining is still promising, in which markets might it work best? Any expert judgements on whether it better to focus on a particular market or conduct cross-market and cross-instrument mining? Is it reasonable to use any (semi)automated trading strategies at all?Finally, can money-mining be considered as a direction in Quantitative Finance, or is all QF just about pricing securities and identifing those which are mispriced?Thanx!

This is also something I am interesting in. I guess one possibility is to get the solution faster than others. Sometimes we make money in this way. Waiting for experts' answer too. Thanks,Well

TheTiger wroteQuoteIs it viable to apply data mining techniques to the financial market context? I actually mean whether is money-mining possible by applying such stuff as genetic algorithms, neural nets and fuzzy logic to identify patterns in security prices.Sure it's possible. GAs are fancy search algorithms, neural nets are fancy function-fitters, and fuzzy logic is a fancy decision-making technique. It's easy to use some combination of the three (along with other stuff) to find all kinds of patterns in markets. The difficult part is figuring out when (or if) you've found patterns you can trade on tomorrow with some justifiable confidence. RBH

RBH,*******The difficult part is figuring out when (or if) you've found patterns you can trade on tomorrow with some justifiable confidence. ***********Just use the Henriksson-Merton test on a decently sized out of sample piece of data.TheTiger,Just think about it for a second. If academics find some predictability it means that there is a lot more to be predicted (correctly). Also, finding correct predictions is only half of quatitative trading. The other half is to figure out what portion of your wealth/portfoli to put on any given trade (books on blakjack will help you in that respect). There's no holy grail however (in case you really find it, do not hesitate to share it with me ,) but it is definitely possible to foreacast adequately a bunch of things in the markets.Good luck, you seem to embark on a long journey,Hiboumalin

RBH,Aren't you one of the Predictors? Can lack of out-of-sample data be overcomed by generating artificial time series, e.g. using bootstrapping?Hiboumalin,******* but it is definitely possible to foreacast adequately a bunch of things in the markets. *******Could you please be more specific? Any example of the things that are included in the bunch?What about FX? On the one hand, it is considered the most efficient market. On the other hand, they say broad use of technical analysis in this market may be self-fulfilling.

Quote<hr><i>Originally posted by: <b>TheTiger</b></i>Hi everybody!Is it viable to apply data mining techniques to the financial market context? I actually mean whether is money-mining possible by applying such stuff as genetic algorithms, neural nets and fuzzy logic to identify patterns in security prices. There were a lot of papers on applying these techniques to market prediction and trading strategy development in 1990s, but what is the current situtation with that direction? Have some new techniques been developed? We must remember in these algorithms, be it GA,nnet or Simulated annealing or something else, we are ultimately doing some form of curve fitting or the other. Some of them have blatant assumptions about the underlying data, Also its an open question , when we do cross validation , to test the effectivenesshow do we choose the window length? What if there are regime switches between our in-sample and out-of-sample period !I have seen a handful of papers on trading strategies particularly in the FX markets,using machine learning, Kohonen maps, Treebased methods , with good results on paper !!.As to the fact , If there is actually a person making money today , ever since the prediction company happened, i have no idea.One of the pioneers in Machine learning applied to finance is Dr John MoodyOne of the best pages on neural nets & evolutionary algorithms applied to finance is Francos Bursetti

Quantie,***** What if there are regime switches between our in-sample and out-of-sample period *****I have heard a lot about this argument. Unfortunately, I have rather an intuitive concept of what the market regime is. Could you please provide some hints on which parameters of the market are used to identify the regime?Thanks,The Tiger

Have you had a look at Derman's paper there are others who might be able to point out simillar articles.

QuoteOriginally posted by: TheTigerQuantie,***** What if there are regime switches between our in-sample and out-of-sample period *****I have heard a lot about this argument. Unfortunately, I have rather an intuitive concept of what the market regime is. Could you please provide some hints on which parameters of the market are used to identify the regime?Thanks,The TigerIf you suspect to have regime switching in your process you should estimate it. I personally use wavelets and all the non-linear treshold autoregression models (TAR, SETAR, STAR, etc....) Warning, some of these models are non-trivailly programmed.Hope this helps,Hiboumalin.

Thanks, Quantie and Hiboumalin! I guess I will need some time to work on the hints you have provided and maybe come to you later with some questions.