Question 1. Is it possible to estimate historic profitability of the collective set of liquidity providers in a specific futures market? (I realize "liquidity providers" is a loose term but let us define it as all traders who don't carry positions overnight.)Suppose the know information is: (A) Quote level data from the exchange(B) Trades (traded prices w/volume, also from the exchange) (C) Open interest(D) tick size and dollar valueQuestion 2. From looking at order book data can one estimate the volume at each price level due to liquidity providers? (Can you figure this out from the order arrival rates, sizes, and distance from the best bid/ask and likewise for order cancellations?)I realize both of these could be very involved - any direction, academic references, industry references, etc... would be appreciated.Thanks!

I think your definition of liquidity providers, as it stands, is going to make answering your question impossible. Certainly, many of the biggest liquidity providers often maintain sometimes very large positions overnight (big banks/brokerages). Also, it's not clear at all who is a "liquidity-provider" and who is a principal trader: all sorts of hedge-funds are massively involved in market making in electronic markets. Certainly, any half intelligent automated execution algorithm is going to at least think about executing passively rather than just sweeping when it wants to trade so, even if I am a pension fund executing a large trade via a bank, quite a lot of the "liquidity-provider" profit is going to go to me in the form of reduced execution costs.I think you can get some measure of the gross profits due to "liquidity-provision" in the whole market, though. If you have a quote-driven market and you can see trades and the prices they were executed at, then you can do this: for each trade, calculate the difference between the trade-price and the then current market mid price. That should be almost exactly what you want, so long as trades' timestamps are contemporaneous with the order-book timestamps that you see.

Thanks. That's very helpful! When you say mid price are you referring to a volume weighted midprice from the best bid/ask or an average bid/ask using only best bid/ask prices?

My intuition would say just use average best bid/ask, not volume weighted. Here's why: suppose I execute a small buy order by hitting the best offer. I don't care what the bid size is on the other side of the market. I just care what half my theoretical round-trip costs would be. If you like, too, the volume weighting is already included in the execution price of the trade, which you know for sure.By the way, there may well be a large literature on this which I don't know about (and would be interested in hearing about). There certainly is a large literature on market price impact and trading costs - try stuff at Google Scholar by Almgren et al, Obizhaeva & Wang, Gatheral. Their bibliographies may lead you elsewhere.

Ok, interesting. Let me play devils advocate or a moment assuming a FIFO (first in first out) market:QuoteMy intuition would say just use average best bid/ask, not volume weighted. Here's why: suppose I execute a small buy order by hitting the best offer. I don't care what the bid size is on the other side of the market. I just care what half my theoretical round-trip costs would be.Suppose I'm a short term trader who took the other side of your market order to buy (I was on the offer and first in the queue). If the market is very thin on the bid side I have a bigger advantage of now getting out with a profit of 1 tick than if the bid side has a lot of volume already. So if you take the difference between the price and the weighted mid price the number is bigger than if you use the mid price. This seem more inline with the how a market maker would make a profit. btw I'm thinking of weighting like this: (bidprice * askvolume + askprice * bidvolume)/(bidvolume + askvolume) For some reason I really want Open Interest to be included in this total calculation somehow. It seems like profit from liquidity provision should increase with an increase in Open Interest. I am familiar with your references and others like Farmer, Lillo, Bouchaud, Mantegna but I don' t think any of them have ever explicitly computed profits due to liquidity provision. I'll keep working on this. Thanks for your help.

Open interest is one of those quantities which one always feels ought to mean something, but often turns out not to be as useful as one would like unless you have a lot more information. Certainly, the relationship between OI and MM profits is going to be extremely complex: with very low liquidity, OI may be high is dealers are forced to hold a lot of inventory, but low-liquidity markets can be enormously profitable for dealers; with very high liquidity, OI may be low because dealers never have to keep positions for longer than a few seconds, but this can also be stunningly profitable (particularly on a risk-adjusted basis). So, definitely not monotonic! Also prob means very different things in different markets, and the meaning of it will also be massively effected by OTC derivative hedging if that's significant.As to weighted/unweighted, I agree: you can make a case for the weighted measure, too. But consider this:1) If buy/sell activity is fairly balanced, where you put the mid-point will all come out in the wash because what's added to the buy order is taken away from the sell order 100ms later.2) By doing the weighting, you may be assuming the current order-book tells you more than it actually does. Even ignoring the possibility of hidden bids/offers in some markets, the order-book changes in response to orders, so you have to think hard about how much a snapshot really tells you about execution. My main reason for liking simple mid is not really that it's simple: you know the actual price the order trade at - that gives you far better information about transactions costs than the order-book snapshot; I'm not sure you add much value at the margin by using a more complicated measure.So, it shouldn't matter, I suspect. If you find that it does matter, that should possibly make you worry a little more about how meaningful either measure really is! I would be interested to see what you find out, so long as it's not proprietary.

This is an actual and interesting question.In a liquid market the "liquidity providers" as you define them will mainly be market makers, read the paper of avellaneda & stoikov on that and you will have a quant view of the famous bid-ask-spread vs volatility paradigm. It will provide you some tools, add statistics and optimal control (I had a paper on that in the Wilmott mag issue of last November: "Rigorous Optimisation of Intraday Trading") and you will be able to go further by yourself.Of course you will discover that it is not as simple: intra day / high-freq knowledge / tactics can be leveraged when included into more classic stat arb strategies (see my paper in the last issue of the journal of trading: "Rigorous Strategic Trading: Balanced Portfolio and Mean-Reversion") so the limit of the profitability cannot only be defined in terms of microstructure (I mean opportunities can come from other strategies you already implement).Your question 2 is not clear as by defintion, only orders coming from liquidity providers are in the orderbook. I imagine that you mean "what is the fraction of the liquidity that does not come from linear/regular investors". This is really difficult to estimate: one year ago (in the Institutional Investor's liquidity guide) I proposed to estimate that looking at cross exchange flows ("The Impact of Liquidity Fragmentation on Optimal Trading").Globally, the usual number you will find in the news papers are 40% in Europe and 70% in the US.

Ok crmorcom,I used ESU9 from 8:30am to 3pm on 7/30/2009. The total volume during that time was 1,809,085 contracts on 126,985 trades.(A) I took the abs of the difference between average best bid/ask and the traded price. A = 231,756(B) I took the abs of the difference between the weighted mid price and the traded price. B = 149,194If you take these numbers and multiply by the tick size of $12.5 you get:$A = $2,896,961$B = $1,864,928QuoteSo, it shouldn't matter, I suspect. If you find that it does matter, that should possibly make you worry a little more about how meaningful either measure really is! I would be interested to see what you find outThese vales seem significantly different to me. I'm not yet convinced I've said anything. But let's take a look for a sec... If we look at it in terms of total number of contracts traded we get that A is 12% and B is 7%. In terms of dollars this would mean $1.60 per contract for A and $1.03 per contract for B. Those numbers seem reasonable to me - well, at least the orders of magnitude seem plausible. What do you think? Maybe we can find a "liquidity provider" who traded 2% of the total volume and they can tell us their P/L was around $47,000 before paying transaction costs and we'll know we should average the two methods.

That is the Sep 09 E-mini SP500, right?Equities have never been my market, but your numbers surprise me a lot. The difference between simple average and weighted average mid price must be less than the bid/ask spread. If the difference between A and B is $.57 per contract, then I am missing something: I would have expected bid/ask spreads of more like pennies for a contract like that.How do the trades split between bid side and ask side? What is the average best inside market bid/ask spread? What is the average difference between your A mid price and your B mid price?

QuoteThat is the Sep 09 E-mini SP500, right?Indeed it is front month S&P500 e-mini futures contract traded on CME.QuoteIf the difference between A and B is $.57 per contract, then I am missing something: I would have expected bid/ask spreads of more like pennies for a contract like that.The tick size is .25 which is worth $12.50. So if you buy a 1 lot on the bid and sell it at the offer you just made $12.50.http://www.cmegroup.com/trading/equity- ... mlQuoteHow do the trades split between bid side and ask side?I don't understand what you're asking. (Do you mean a count of how many traded on the bid and how many traded on the offer?)Quote What is the average best inside market bid/ask spread? It turns out to be just slightly bigger than the tick size: 0.2506395. Which makes sense since it's a tight book.QuoteWhat is the average difference between your A mid price and your B mid price?A mid price - B weighted mid price is 0.003236 BTW thank you, lehalle, for the references. I'm looking at Avellaneda and Stoikov paper now.

So, if the A price and B price differ by .003236, which is $0.1618, how is it that the aggregate difference is $0.57 per contract? The aggregate difference should be _less_ than the mean price difference because, if the mean price is closer to the bid trades, it is correspondingly further away from the ask trades, so many of the differences should cancel as you sum everything.

I follow your reasoning but I don't follow your math. How exactly does .003236 become $0.1618?

If 0.25 is $12.50, then the contract notional must be $50; $50*.003236=$0.1618.

crmorcom,Discovery: I have some data problems.Knowing what the inside market is when a trade happened turns out to be an issue. My results are untrustworthy. Sorry. Your first comment is biting me:QuoteThat should be almost exactly what you want, so long as trades' timestamps are contemporaneous with the order-book timestamps that you see. I'll try and clean it up and post back in the near future.