Page 2 of 2

HFT Algorithms

Posted: December 9th, 2013, 3:31 pm
by AnalyticalVega
QuoteOriginally posted by: chocolatemoneyQuoteOriginally posted by: AnalyticalVegaQuoteOriginally posted by: chocolatemoneyQuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: AnalyticalVegaQuoteOriginally posted by: chocolatemoneyQuoteOriginally posted by: farmerQuoteOriginally posted by: AnalyticalVegaForward Testing with Monte Carlo?Easy there, fella. Find someone your own size to pick on. The people who run Monte Carlo need to eat, also.I have never seen anyone "forward testing" a strategy "with Monte Carlo". What the heck does it add to your trading system design?Are you serious? How do you predict your future P&L?Indeed! And how do you know if your strategy development process doesn't have a leak (accidentally using future data to trade the past) or has overfitted the backtest data?@AnalyticalVeg@Traden4Alpha: Literature is rich on the topic, isn't it?Many people use Monte Carlo. Do you have something that works better?If Monte Carlo suits your need, sure, go ahead.My point is: pay attention! You have still to make an assumption on the process and that same assumption may be already embedded in your model. Therefore, forward testing may not help you stretch your assumption and identify those edge cases where things may get very ugly. In other words, over the years, I spent significant time improving on the schoolbook forward testing Monte Carlo, building a richer framework. My all point is not about Monte Carlo, but schoolbook implementation of forward testing with Monte Carlo.Second thing. I see that you have your checklist, which is good. However, you can do all you say and still lose money and maybe do something completely different and bring home your profit.On the same line: You say "Yes. You would need to reconstruct the most likely prices in realtime, so your sample space for your empirical PDF would constantly be changing. The rate of change of the sample space would depend on market conditions which would need to be measured by a bunch of different metrics." If I understand correctly, your algos are real time adaptive and the process for adaptation is adaptive as well and parametric. I can not say it is good or not, but my modeling work in the last years has been focused on simplifying, so my question was: can't you make it simpler?Not sure if I can make it simpler, HFT is a complex field. Yes you can still lose money following my checklist. I did not enumerate everything in detail. I just gave a few basic pointers. Yes if you don't have a good backtested model to begin with monte carlo probably won't help. There are always assumptions in models, some may be bad but as always the models that are the closest to reality will be the best.

HFT Algorithms

Posted: December 10th, 2013, 7:38 am
by chocolatemoney
QuoteOriginally posted by: AnalyticalVegaQuoteOriginally posted by: chocolatemoneyQuoteOriginally posted by: AnalyticalVegaQuoteOriginally posted by: chocolatemoneyQuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: AnalyticalVegaQuoteOriginally posted by: chocolatemoneyQuoteOriginally posted by: farmerQuoteOriginally posted by: AnalyticalVegaForward Testing with Monte Carlo?Easy there, fella. Find someone your own size to pick on. The people who run Monte Carlo need to eat, also.I have never seen anyone "forward testing" a strategy "with Monte Carlo". What the heck does it add to your trading system design?Are you serious? How do you predict your future P&L?Indeed! And how do you know if your strategy development process doesn't have a leak (accidentally using future data to trade the past) or has overfitted the backtest data?@AnalyticalVeg@Traden4Alpha: Literature is rich on the topic, isn't it?Many people use Monte Carlo. Do you have something that works better?If Monte Carlo suits your need, sure, go ahead.My point is: pay attention! You have still to make an assumption on the process and that same assumption may be already embedded in your model. Therefore, forward testing may not help you stretch your assumption and identify those edge cases where things may get very ugly. In other words, over the years, I spent significant time improving on the schoolbook forward testing Monte Carlo, building a richer framework. My all point is not about Monte Carlo, but schoolbook implementation of forward testing with Monte Carlo.Second thing. I see that you have your checklist, which is good. However, you can do all you say and still lose money and maybe do something completely different and bring home your profit.On the same line: You say "Yes. You would need to reconstruct the most likely prices in realtime, so your sample space for your empirical PDF would constantly be changing. The rate of change of the sample space would depend on market conditions which would need to be measured by a bunch of different metrics." If I understand correctly, your algos are real time adaptive and the process for adaptation is adaptive as well and parametric. I can not say it is good or not, but my modeling work in the last years has been focused on simplifying, so my question was: can't you make it simpler?Not sure if I can make it simpler, HFT is a complex field. Yes you can still lose money following my checklist. I did not enumerate everything in detail. I just gave a few basic pointers. Yes if you don't have a good backtested model to begin with monte carlo probably won't help. There are always assumptions in models, some may be bad but as always the models that are the closest to reality will be the best.Well, this is all very obvious. I am not sure how much firepower you firm has, but most of my employers/clients have been willing to trade-off a lot of accuracy for simplicity, robustness and speed.P(you get dragged in a closed for a beat-up by the team of infrastructure engineers | you have a mathematical juggernaut model) = 1

HFT Algorithms

Posted: December 10th, 2013, 1:49 pm
by AnalyticalVega
QuoteOriginally posted by: chocolatemoneyQuoteOriginally posted by: AnalyticalVegaQuoteOriginally posted by: chocolatemoneyQuoteOriginally posted by: AnalyticalVegaQuoteOriginally posted by: chocolatemoneyQuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: AnalyticalVegaQuoteOriginally posted by: chocolatemoneyQuoteOriginally posted by: farmerQuoteOriginally posted by: AnalyticalVegaForward Testing with Monte Carlo?Easy there, fella. Find someone your own size to pick on. The people who run Monte Carlo need to eat, also.I have never seen anyone "forward testing" a strategy "with Monte Carlo". What the heck does it add to your trading system design?Are you serious? How do you predict your future P&L?Indeed! And how do you know if your strategy development process doesn't have a leak (accidentally using future data to trade the past) or has overfitted the backtest data?@AnalyticalVeg@Traden4Alpha: Literature is rich on the topic, isn't it?Many people use Monte Carlo. Do you have something that works better?If Monte Carlo suits your need, sure, go ahead.My point is: pay attention! You have still to make an assumption on the process and that same assumption may be already embedded in your model. Therefore, forward testing may not help you stretch your assumption and identify those edge cases where things may get very ugly. In other words, over the years, I spent significant time improving on the schoolbook forward testing Monte Carlo, building a richer framework. My all point is not about Monte Carlo, but schoolbook implementation of forward testing with Monte Carlo.Second thing. I see that you have your checklist, which is good. However, you can do all you say and still lose money and maybe do something completely different and bring home your profit.On the same line: You say "Yes. You would need to reconstruct the most likely prices in realtime, so your sample space for your empirical PDF would constantly be changing. The rate of change of the sample space would depend on market conditions which would need to be measured by a bunch of different metrics." If I understand correctly, your algos are real time adaptive and the process for adaptation is adaptive as well and parametric. I can not say it is good or not, but my modeling work in the last years has been focused on simplifying, so my question was: can't you make it simpler?Not sure if I can make it simpler, HFT is a complex field. Yes you can still lose money following my checklist. I did not enumerate everything in detail. I just gave a few basic pointers. Yes if you don't have a good backtested model to begin with monte carlo probably won't help. There are always assumptions in models, some may be bad but as always the models that are the closest to reality will be the best.Well, this is all very obvious. I am not sure how much firepower you firm has, but most of my employers/clients have been willing to trade-off a lot of accuracy for simplicity, robustness and speed.P(you get dragged in a closed for a beat-up by the team of infrastructure engineers | you have a mathematical juggernaut model) = 1If you don't have great developers who can optimize their code and great infrastructure engineers who can optimize your platform, then you have to simplify your system more, but it may cost you some $$$

HFT Algorithms

Posted: December 13th, 2013, 11:58 pm
by maxw
This might sound like a stupid question. But where do I find a mentor? Look up hft firms and see if someone will talk to me?I tried contacting someone who did it at elite traders. He told me to give up before I even started seeing I didn't have the infrastructure to compete with the big boys. I'm not looking to compete with them, I don't mind the small portion. There are really no books that teach you how to do it well and the people I've read who do it don't want to talk to you. So I'm stuck.

HFT Algorithms

Posted: December 14th, 2013, 10:37 pm
by AnalyticalVega
QuoteOriginally posted by: maxwThis might sound like a stupid question. But where do I find a mentor? Look up hft firms and see if someone will talk to me?I tried contacting someone who did it at elite traders. He told me to give up before I even started seeing I didn't have the infrastructure to compete with the big boys. I'm not looking to compete with them, I don't mind the small portion. There are really no books that teach you how to do it well and the people I've read who do it don't want to talk to you. So I'm stuck.connect on linkedin, twitter, etc.. get creative, go to events where HFT traders are and make new friends. HFT is hard but certainly not impossible. No one will give you all the answers, you have to collect info from different people. It's sort of like putting together a huge puzzle. A mentor will help you filter/sort out the info you have collected.

HFT Algorithms

Posted: December 16th, 2013, 7:31 am
by chocolatemoney
QuoteOriginally posted by: maxwThis might sound like a stupid question. But where do I find a mentor? Look up hft firms and see if someone will talk to me?I tried contacting someone who did it at elite traders. He told me to give up before I even started seeing I didn't have the infrastructure to compete with the big boys. I'm not looking to compete with them, I don't mind the small portion. There are really no books that teach you how to do it well and the people I've read who do it don't want to talk to you. So I'm stuck.It may be off-topic, anyhow: in the industry it is common to assign a penalty function to your backtest results and pnl expectations which is inversely proportional to your trading freq. This is considered best-practice at most of the companies I had the fortune to cross-paths with. I am talking about multi $$bn companies capable of deploying $$mn in infrastructure. Hope this sheds a light on industry perspective on hft.