RenTech: using artificial intelligence to morph one trading strategy into anotherHi all,I met with a FX high speed trading guy (not necessarily super HFT) and he told me how they use artificial intelligence in trading ... and he hinted that RenTech is using AI in similar approaches...He said they don't use AI to forecast prices directly; instead, they use AI to morph one trading strategy into another according to market conditions, for example, morphing momentum into mean-reverting according to market regimes.I am guessing he's referring to using Genetic Programming... and I am guessing that the FX quant trading guys have the tendency of using GP or AI to morph strategies...Anybody knows how to do it? How might be the performance? Please shed some lights on me! Thanks!

I suspect that there are many ways of doing this.By AI, I wouldn't get too hung up on Genetic Programming. Perhaps take a look at Neural Nets, SVMs etc.One way you may find success is to implement a very simple trading strategy, for example a Bollinger Band mean-reversion strategy. Then choose to take this strategy's trades or not based on the output of an AI model. The inputs to the AI model may be other indicators, volatility estimates or whatever floats your boat etc. Just a few thoughts. Like I said, I'm not sure there is a cookbook solution for this.Stewart

I think it can also be related to dynamic weighting. Run both momentum and mean-reverting strategies in a paper trading mode and invest real money in the one that is making money at the moment. This is also a kind of trade filtering that Stew is talking about.I am testing a number of FX strategies myself and have to say their performance really depends on how you manage exits, not just entries. Anyone interested in sharing some thoughts?

Have you considered the possibility that this guy is blowing smoke up your ass?First of all the strategies and signals RenTech uses are way more sophisticated than momentum/mean reversion. I'm sure if you run look at small sub-periods a given strategy might look very similar to mean-reversion/momentum. However the difference is that it will look like its magically switching from one to the other when the profitability switches. To someone just looking at their stuff like a black box this could very much look like some sort of regime switching model (maybe using some sort of GP algorithm). However I very much doubt this is what's going on under the hood. Any firm as sophisticated as RenTech are much more interested in using microstructure indicators to generate signals ex-ante, rather than using pnl analysis to try to catch trends ex-post. The former's a hell of a lot more profitable than the latter.Furthermore RenTech's models are so far ahead of 95%+ of other market participants that it's far simpler for them to re-fit their models using all available market data. A setup that you describe would only be using their own trades to re-weight parameters. All available market data will be much richer than just looking at their own trades (which is a subset of the former anyway). Why would one re-fit a model using a more sparse data set?

Several details are here:
Forex-tsd about spectra
https://www.forex-tsd.com/forum/debates ... ion/page67
1). RenTec is not predicting the price. Price is too random for their algos. They predict the prediction of price. They trade futures. The futures price is the result of numerous decisions made by a number of humans, using their "visions" AND usual trading tools: Moving Averages etc.2). Visions of all traders and money managers on Wall Street are "analogous". They are smooth like a sound of music. Therefore the futures are more smooth than regular direct (random) prices.3). Trading tools on Wall Street are deep "analogous" too, they are too much like digital filters.4). Therefore, while RenTec cannot predict direct random price, they CAN predict the futures (semi-analogous "signal") - by using very special spectral DSP methods. Additionally, futures allow them to leverage.5). I suggest nobody in RenTec holds the whole picture of algos' chain, except Simons.5). The two main problems with spectral DSP are:(a) modern methods can not exctract spectrum from a "signal" (the price) correctly,(b) modern DSP methods can not predict the dance (the change) of price spectrum correctly,and(c) advanced methods by RenTec are unusual and VERY computing-power hungry for the real mode operations.6). For the solution of (5b) RenTec hired specialists on speech recognition from IBM.7). For the solution of (5a) they use very exotic DSP methods from VERY distant (from Wall Street) area of science. I think they started from the Simons' job at crypto, because they had to CHECK crypto long ago by some de-crypto methods. Simons does not use crypto, he started from the use of de-crypto technique, developed earlier for the avaluation of crypto power. He first has "decrypted" the price timeseries. Now they use algos from physics, and then if physics algos fail they use old de-crypto algos.8). For the solution of (5c) they built the best in industry IT dept.9). All above is almost useless without very special trick needed to DSP-ing processing of price timeseries. It is not described in books. I dont know how they do it. Maybe they dont use it at all - that is why we can see drawdowns in their returns, when the markets gone totally random."Bad news (4) I have no intention and I will not dig deeply into it here. You have to have Simons' scientists level to implement "cycle extraction" correctly. Do not take it personally.Bad news (5). Forex's signals are the most RANDOM among all type of price quotings.That is why the "cycle extraction" in forex prices' stream is the most difficult. That is why retail forex brokers are beleiving that most of novice forex traders will fail by ANY trading method or combination of trading methods, otherwise profitable in other trading areas: futures, options etc."...................."...Let me repeat by other words: you can obtain the "spectrum" of the "signal" by incorrect Fourier method or other way. If the signal is stable within the longest period in spectrum, then you can easily extrapolate the same spectrum into the future. That is the way the "Extrapolator" from MQL codebase works.But if your "signal" has a spectrum which is changing over time - then you need a "dance" of the spectrum - this is "spectrum of spectrum" or somewhere called "Cepstrum" - a dynamic representation of spectra's changes over time. Once you have got a cepstrum, you can extrapolate it to get future spectrum, then you can extrapolate spectrum for the future time, and then you can extrapolate the "signal" - the price.The faster the spectrum changes the more difficult to get the "cepstrum".That is what usually the speech recognition engineers are dealing with. This is much more difficult task for engineers, than simple spectra extraction. That is the main difficulty in what the Forex is a champion.Even if you got a cepstrum, and a spectrum, you cannot be sure that this your so-called "spectrum estimation" - a "probable suggestion" in simple words - is correct. Therefore you must periodically return to cepstrums, obtained and refused earlier by your program (by some reasons) - in order to correct your trading system. As I understand Simons and RenTec solved this problem by using VERY exotic approach - they hired as a chief scientist one on the main specialists in so-called "returning functions". And they always optimize obtained cepstums ans spectrums as to the new turns of the market. They never sure that the spectra they use at the moment for predicting the futue - is correct.You see? No place for amateurs. This is a very serious math job. "---------------------------------------------------------------------------------------------------------------------------------------------Besides, I was swiftly *banned* on stupid "atomic hvinancy" web-site for this detailed explanation in their RenTec thread. Probably, RenTec & Simons gave them a huge bribe to do that. RenTec secret  

QuoteOriginally posted by: AlexEroBesides, I was swiftly *banned* on stupid "atomic hvinancy" web-site for this detailed explanation in their RenTec thread. Probably, RenTec & Simons gave them a huge bribe to do that. RenTec secretPW is not keen on banning anyone.

QuoteOriginally posted by: outrunOh man, that thread is not promising much good!AlexEro, are you going to spam this forum and try to sell your software with cheap TellSell tricks?Nope. Here I am explaining the RenTec "secret". If you have more detailed and/or reasonable explanation of their "secret" than mine, you can write it here.

QuoteOriginally posted by: AlexEroQuoteOriginally posted by: outrunOh man, that thread is not promising much good!AlexEro, are you going to spam this forum and try to sell your software with cheap TellSell tricks?Nope. Here I am explaining the RenTec "secret". If you have more detailed and/or reasonable explanation of their "secret" than mine, you can write it here.you are not explaining anything. you are simply speculating. show how your ideas explain RenTech's claimed returns.

QuoteOriginally posted by: daveangelQuoteOriginally posted by: AlexEroQuoteOriginally posted by: outrunOh man, that thread is not promising much good!AlexEro, are you going to spam this forum and try to sell your software with cheap TellSell tricks?Nope. Here I am explaining the RenTec "secret". If you have more detailed and/or reasonable explanation of their "secret" than mine, you can write it here.you are not explaining anything. you are simply speculating. show how your ideas explain RenTech's claimed returns.Sorry, I dont get your idea. I think it is pretty clear explanation. Or do I have to upload the full code of RenTec trading algos here simply to satisfy your interest?You are probably not familiar with DSP methods. I mean with PROPER spectral methods.Here to start with:P.J. Kootsookos "A review of the frequency estimation and tracking problems."KootsookosKootsookos Review PDF

QuoteOriginally posted by: AlexEroQuoteOriginally posted by: daveangelQuoteOriginally posted by: AlexEroQuoteOriginally posted by: outrunOh man, that thread is not promising much good!AlexEro, are you going to spam this forum and try to sell your software with cheap TellSell tricks?Nope. Here I am explaining the RenTec "secret". If you have more detailed and/or reasonable explanation of their "secret" than mine, you can write it here.you are not explaining anything. you are simply speculating. show how your ideas explain RenTech's claimed returns.Sorry, I dont get your idea. I think it is pretty clear explanation. Or do I have to upload the full code of RenTec trading algos here simply to satisfy your interest?You are probably not familiar with DSP methods. I mean with PROPER spectral methods.Here to start with:P.J. Kootsookos "A review of the frequency estimation and tracking problems."KootsookosKootsookos Review PDFyou are right I am not familiar with DSP nor with RenTech's returns. I think you should explain how RenTech's returns can be replicated

QuoteOriginally posted by: daveangelQuoteOriginally posted by: AlexEroQuoteOriginally posted by: daveangelQuoteOriginally posted by: AlexEroQuoteOriginally posted by: outrunOh man, that thread is not promising much good!AlexEro, are you going to spam this forum and try to sell your software with cheap TellSell tricks?Nope. Here I am explaining the RenTec "secret". If you have more detailed and/or reasonable explanation of their "secret" than mine, you can write it here.you are not explaining anything. you are simply speculating. show how your ideas explain RenTech's claimed returns.Sorry, I dont get your idea. I think it is pretty clear explanation. Or do I have to upload the full code of RenTec trading algos here simply to satisfy your interest?You are probably not familiar with DSP methods. I mean with PROPER spectral methods.Here to start with:P.J. Kootsookos "A review of the frequency estimation and tracking problems."KootsookosKootsookos Review PDFyou are right I am not familiar with DSP nor with RenTech's returns. I think you should explain how RenTech's returns can be replicatedO-o-ps. I have already just explained it here above:= build algorithmic spectral-scope (a couple of PhDs);= build algorithmic cepstral-scope (a couple of PhDs); = buld spectral predictor;= build price extrapolator;= build medium-period trend detector (a couple of PhDs);= add here long-period trend detector - for robust results (a couple of PhDs);= build open-close trading system based on signals, generated by price extrapolator+ medium-term-detector.Put all this staff into CUDA GPU software, because this is much more comfortable than managing 350 high-end noisy servers.Put all this staff into a hedge fund.Make money.

QuoteOriginally posted by: AlexEroQuoteOriginally posted by: daveangelQuoteOriginally posted by: AlexEroQuoteOriginally posted by: daveangelQuoteOriginally posted by: AlexEroQuoteOriginally posted by: outrunOh man, that thread is not promising much good!AlexEro, are you going to spam this forum and try to sell your software with cheap TellSell tricks?Nope. Here I am explaining the RenTec "secret". If you have more detailed and/or reasonable explanation of their "secret" than mine, you can write it here.you are not explaining anything. you are simply speculating. show how your ideas explain RenTech's claimed returns.Sorry, I dont get your idea. I think it is pretty clear explanation. Or do I have to upload the full code of RenTec trading algos here simply to satisfy your interest?You are probably not familiar with DSP methods. I mean with PROPER spectral methods.Here to start with:P.J. Kootsookos "A review of the frequency estimation and tracking problems."KootsookosKootsookos Review PDFyou are right I am not familiar with DSP nor with RenTech's returns. I think you should explain how RenTech's returns can be replicatedO-o-ps. I have already just explained it here above:= build algorithmic spectral-scope (a couple of PhDs);= build algorithmic cepstral-scope (a couple of PhDs); = buld spectral predictor;= build price extrapolator;= build medium-period trend detector (a couple of PhDs);= add here long-period trend detector - for robust results (a couple of PhDs);= build open-close trading system based on signals, generated by price extrapolator+ medium-term-detector.Put all this staff into CUDA GPU software, because this is much more comfortable than managing 350 high-end noisy servers.Put all this staff into a hedge fund.Make money.this is the secret sauce ?

QuoteOriginally posted by: daveangelQuoteOriginally posted by: AlexEroQuoteOriginally posted by: daveangelQuoteOriginally posted by: AlexEroQuoteOriginally posted by: daveangelQuoteOriginally posted by: AlexEroQuoteOriginally posted by: outrunOh man, that thread is not promising much good!AlexEro, are you going to spam this forum and try to sell your software with cheap TellSell tricks?Nope. Here I am explaining the RenTec "secret". If you have more detailed and/or reasonable explanation of their "secret" than mine, you can write it here.you are not explaining anything. you are simply speculating. show how your ideas explain RenTech's claimed returns.Sorry, I dont get your idea. I think it is pretty clear explanation. Or do I have to upload the full code of RenTec trading algos here simply to satisfy your interest?You are probably not familiar with DSP methods. I mean with PROPER spectral methods.Here to start with:P.J. Kootsookos "A review of the frequency estimation and tracking problems."KootsookosKootsookos Review PDFyou are right I am not familiar with DSP nor with RenTech's returns. I think you should explain how RenTech's returns can be replicatedO-o-ps. I have already just explained it here above:= build algorithmic spectral-scope (a couple of PhDs);= build algorithmic cepstral-scope (a couple of PhDs); = buld spectral predictor;= build price extrapolator;= build medium-period trend detector (a couple of PhDs);= add here long-period trend detector - for robust results (a couple of PhDs);= build open-close trading system based on signals, generated by price extrapolator+ medium-term-detector.Put all this staff into CUDA GPU software, because this is much more comfortable than managing 350 high-end noisy servers.Put all this staff into a hedge fund.Make money.this is the secret sauce ?The secret sauce is numeric methods involved. The core is spectral method. If you will try to do it by Fourier methods (CD and MP3 music), you will fail.