Hi mates, I'm looking at some market making algos for a market completely dependent on another market (e.g. Oil Market driven by S&P 500 market). so I'd like to find out the relationship between the two series (assuming there is one).Any idea about robust time-series methodology to work out this dependency?I have historical intraday ticks for both markets, but the issue is that frequency of my market is not regular (say Oil...), as opposed to S&P500, for which I can have ticks every second...Any idea is more than welcome. Thanks!

Here's my 2 ideas, but bear in mind this is all speculation and not based on actually having done this. 1) Ok, the simple method may be something like using synchronized quotes at various time scales. Eg daily. to fit a relationship between oil price and sp500. Also there's been alot of work done on this sort of stuff so maybe search for macroeconomic risk indicators. Oil is usually in that list. May not be accurate enough for intraday market making, and you may have to use quite wide uncompetitive quotes via this method. 2) The more complicated method may involve some sort of bivariate process fit, using max log likelihood or something. Essentially you have 2 hidden processes of which you see an update every so often, more frequent for sp500 and less frequently with oil. So fit a model and then use realtime quotes to update the model by bayesian methods or something.

Lookup literature on the Epp's effect.

Thanks guys for your ideas.

You are lucky your boss would even allow you to do that, I told him in the past that I would like to do something like this and he told me to go f**k myself. I can do it immediately since i have the infrastructure and ability in place, but you are assuming implicitly a correlation between the two. These relationships breakdown during an oil supply forecast interruption (like shocks of instability in the middle east, or other events that drive oil supply). Yes oil demand should be related to SP500 in some sense, but you'll get destroyed on news of supply shocks.

Caesaria bring up an important point, it's not too hard to play with lagged operators and get correlations across markets and if you do enough, some will look profitable.However, it's harder to work out when your algo should look at the market and say "I don't work in these conditions", also some trading strategies show a bimodal behaviour of steady gains with occasional horribles losses (and vice versa).Although the official report around the Flash Crash seems to be more oriented towards protecting vested interests than illustrating the truth, it is always clear that strategies need to be able to survive significant collapses in liquidity. The FC was a rare and large events, but suddenly widening spreads can rip your guts out.Onr technical issue that will make your backtesting more tricky is getting timestamps across markets and relative to your infrastructure.

My gut reaction is that you'll get picked off bigtime if you try and do something like this. Also, oil & S&P have slightly overlapping different markets. The biggest question is whether or not both markets are liquid or will you experience periods of illiquidity?

Whilst I agree with your points that possibly commods and equities don't mix, I've seen far too much correlation in extreme mkt conditions between say gold and certain gold mining companies, or silver and some silver mining companies to rule out that no one is trading one against the other at high frequencies. Same goes for sugar etc. In these markets it seems sometimes just following the herd seems to be profitable, even if not fundamentally sound. Oil is a bit of funny one as there are so many varieties, and certainly West Texas I think is far from indicative of the true price of oil, as I think given it's basis as a US economic indicator it sometimes will be "adjusted" by the powers that be to make inflation numbers etc fit. (I watched Conspiracy theory the other day :0D )As DominicConnor and WinstonTJ say though, it is probably a fickle relationship, so if you do trade on this, keep the risk managed and expect it could blow up at any point. Especially as I think the CFTC aims to bring in a bunch of regulations on commods trading soon and everyone seems to be getting with smashed with margin calls these days.

Like the saying "cash is king" goes. With metals and materials you usually see the same thing - the juniors and explorers will over-extend but not always tend to lead or lag. I think that ETFs have driven correlation of the overall markets very close but not perfectly. One of the problems you are also going to have is the global aspect of this and the exchange open/close times, etc. For example, many Jr. gold & silvers are not US-Based and you would want to trade the main stock I'd assume not an ADR, GDR or a us-listed portion. This will bring into play currency exchange as well as the different exchanges operating times. Or you could buy the US or UK products and then buy/sell the currency to try and synthetically create owning the local shares. Either way - not a bad idea however I don't know that I'd want to be putting on that much risk. Even if you think that Gold or Silver Spot should be moving in a direction or priced at X (because of the equities) or vice-a-versa, there is no control over where the futures actually trade and/or depending on how close to a roll you are, etc. it can mess things up. My guess is that guys will see your quotes and pick you off very quickly. Its like ETF arbitrage vs. its basket. Just because the basket is trading at $100 doesn't mean that the ETF will trade at $100 as well. If you make a market in one security based on another market you best know that you have an edge or you'll give away a lot of money very quickly.

Definitely seen studies using upstream vs downstream industries on commods like oil. As you say depending on prices the upstream guys like explorers drillers etc may do well vs downstream guys like the refiners. There are potential lead/lag, equilibrium relationships to find and exploit according to the studies. Usually these studies are longer term but maybe there is some stuff to play intraday. Again I question how accurate this strategy is at high frequency, but doesn't mean it couldn't work. My study of the metals effect has been mainly empirical, but every time I see a big move in silver or gold, I'd check the movers on the FTSE 100 and see Fresnillo or Randgold resources in the tails of the movers, which is why I was fairly sure someone was using a relationship between the two. Agreed that you could run into problems with exchange times, currency and a bunch of other stuff and I've not done these studies myself, but as WinstonTJ said there may be something in it and as I said the moves seem too coincidental for there not to be someone doing something in this space. For the S&P future you can do a little better in that it's traded on NYMEX so you could potentially trade oil futures vs S&P futures but again don't know how stable the relationship is. Probably move more in line when S&P moves on Exxon or Chevron but not on tech. Some sort of regime based relationship. Tbh, from what I hear it seems everyone seems to play the VXX ETF vs the S&P using the VXX for downside protection. I think the VXX is the 2nd largest ETF now ? If so, that scares the crap out of me because when that perceived relationship breaks it will be ugly.