It is a common experience to confront severe difficulties in obtaining significant variance reduction (or even convergence to the right value) with straightforward application of LDS sequences to realistic problems (=other than asian option pricing and little more ).Often a lot of manual tweaking is needed at best, making them unsuitable for transparent usage in risk/pricing packages. The reasons for this are various and clearly relate to the complexity of such sampling methods. Unfortunately these "negative" results also in practical usage are rarely communicated, but they would be really helpful in better directing research.I would like to collect your experiences with QMC, be they successful or not - the latter being the most interesting. Would you be willing to share some of them?We're trying to build a map of approaches and difficulties encountered to identify the main sources of problems *in a practical environment*.Thanks in advance!PS PMs are also welcome if you prefer not to answer here

I really haven't had much trouble using QMC effectively. Most of my applications have been to the LMM. First issue to be aware of is that QMC gives precisely the same variance estimates. This is OK becausevariance does not express the speed of convergence.

Hi quartz,From what I have seen in the industry, most places have some semi-sensible implementation of qmc methods but when they are applied to highly structured payoffs then the results are not significantly better than antithetic variates (if at all) and nowhere that I have worked have they used qmc as their default setting for booked trades. As you say, for the simple payoffs that are considered in the academic literature (eg Asian options) there can be a significant improvement by using qmc. However, if you have a more complex payoff then the performance may be similar to Monte Carlo or significantly worse if you are not careful in how you apply the low-discrepancy sequences. In general, I believe that the idea is over-sold.If you are interested in why qmc works in an academic sense, some time ago (whilst a PhD student) I managed to prove some rigorous results on why some of the problems in finance will be effectively low-dimensional. The fundamental insight of the importance of effective low-dimensionality and its relation to the efficacy of quasi-Monte Carlo goes back to the work of Prof Art Owen at Stanford. If you have a high boredom threshold I can post my (non-copyrighted) thesis.However, life is short and the punchline is that: if you have a smooth(ish) functional (eg an Asian option payoff) of a low-dimensional diffusion (such as the Black-Scholes SDE) and this diffusion satisfies some technical conditions (which are always satisfied in one dimension) then you can expect the integrand you end up with to be well-approximated by a linear combination of low-dimensional functions (aka effectively low-dimensional in the superposition sense), moreover, the stochastic analysis tells you which subspaces actually 'matter'. Conversely, there are some diffusions for which you would not anticipate qmc to work that well for.BTW: what is your context for looking at qmc?Ciao,Andy

QuoteOriginally posted by: mjI really haven't had much trouble using QMC effectively. Most of my applications have been to the LMM. First issue to be aware of is that QMC gives precisely the same variance estimates. This is OK becausevariance does not express the speed of convergence.Thanks for breaking ice Good to know, it's a start for the map...Most of the "complaints" I get actually come from hybrid products... but already a barrier requires some tweaking.Besides that the sobol' direction numbers that have been used for a few years had some flows...What problems have you treated successfully with QMC? (and with how many samples? which sequences? things are interesting e.g. with 4000 Sobol' samples).Could you please also detail more on your comment on variance? Are you meaning calculating "variance" as if samples were random i.i.d., or randomized QMC?

QuoteFrom what I have seen in the industry, most places have some semi-sensible implementation of qmc methods but when they are applied to highly structured payoffs then the results are not significantly better than antithetic variates (if at all) and nowhere that I have worked have they used qmc as their default setting for booked trades. ....Conversely, there are some diffusions for which you would not anticipate qmc to work that well for....BTW: what is your context for looking at qmc?Dear Andy, it's interesting to hear this. The map is expanding a bit.Have you ever seen something beyond the usual Sobol'+BB/LT or is this already what you mean with semi-sensible implementation?What do you mean exactly, that the idea is over-sold? Are you referring to actual mathematical limitations or practical inappropriateness of usage in a standard production environment?I do understand this point of view, but e.g. would also say this is much more the case in GPU computing than qMC.What diffusions in particular are you finding that give the biggest problems?It'd be nice seeing your work, please go ahead. Are you staying up to date on the subject? There's a lot to say about measures of effective dimensionality :-)As for our context, better go pm, it'd be almost spam here. However the focus is simulation in general, not just pricing. Which means that one cannot rely on good convergence properties of certain quadrature rules, but must achieve really uniform distribution, and sequences are thus harder to construct.

One small comment: it is also important to generate the "quasi random numbers" using algorithms that are enhanced with scrambling characteristics

Hi Quartz,Sorry for the very long delay - have come back to the UK from Sweden to rejoin the rat race and have had my computer being shipped for quite some time.Re my thesis, I've attached a copy here.The two chapters you may be interested in are 2 and 4 (forgive me, when I wrote this I was a PhD student and didn't know anything about finance - although I was probably better at maths than I am now).In Chapter 2, some results are proved which explain why some of the examples considered in the academic literature are effectively low-dimensional (and this is linked to the strong approximation of Ito diffusions).In Chapter 4, a result is proved which suggests some problems may be intrinsically high-dimensional.Chapters 3 and 5 can be safely ignored ;-)Elaborating on my comment that I thought QMC was over-sold:(1) The examples in the academice literature are trivial: usually a relatively simple payoff in a low-factor model (often 1 factor). Chapter 2 of the thesis explains why you should expect QMC to 'work' for such examples, namely, the integrand is effectively low-dimensional in the superposition sense (in the sense of Art Owen et al).(2) I am yet to be convinced for complex payoffs (say a highly-structured interest rate derivative in a high-factor rates model) that quasi-Monte Carlo methods consistently and significantly outperform antithetic variates. For the simple problems considered in the academic literature, QMC does significantly outperform antithetic variates.Having said this, I haven't looked at QMC for a number of years and perhaps there has been some advancements that I am unaware of.Regards,Andy

QuoteOriginally posted by: quartzQuoteBesides that the sobol' direction numbers that have been used for a few years had some flows...Indeed, efficient direction numbers is the key to the perfomance of Sobol' sequence generators. For a comparison of various Sobol' sequence generators used in the industry seeI. Sobol', D. Asotsky, A. Kreinin, S. Kucherenko. Construction and Comparison of High-Dimensional Sobol' Generators, 2011, Wilmott Journal, Nov, pp. 64-79 which you can read here