I've gone through a number of papers looking for some "rule of thumb" regarding the "acceptable" number of runs for MC simulation with regards to a large portfolio of instruments (these include options). Most MC papers I've seen are very general or discuss methods which are faster/reduce the Standard Error by a certain factor.Does anyone know how one determines the "acceptable" number of runs?Thanks very much.

The term "acceptable" is a bit sloppy. The number of runs depends on your purpose.When pricing long-dated TARN structures, the number of runs may be >> 50.000 in order to arrive at price and sensitivities with sufficient accuracy. For Bermudan swaption books with a large number of trades, 50.000 runs may be in excess, if you are to have risk reports on your entire book (depending on the IT architecture, whether you aim for "real-time" reports, whether you need a full vega report etc.) On the other hand, I've seen applications involving nightly calculations of a large number of sensitivities for a universe of 1.200 callable and non-callable mortgage bonds with maturities up to 30 years. As all bonds were listed, pricing was not an issue, and various noise-reduction techniques could be applied to risk figure calculations, but time constraints still meant that only 500 runs were used in MC evaluations

Hi Skphang,You ask: Does anyone know how one determines the "acceptable" number of runs? As caroe mentions, the number of runs depends on the product and the available time before the risk report has to be produced. It also depends on the computing architecture. In particular, if parallel processing is used, the number of MC runs can be increased. The rest of this post has been transferred to a new thread, Parallel MC Simulation Poll, in this forum.---------------

QuoteOriginally posted by: caroe The term "acceptable" is a bit sloppy. The number of runs depends on your purpose.When pricing long-dated TARN structures, the number of runs may be >> 50.000 in order to arrive at price and sensitivities with sufficient accuracy. For Bermudan swaption books with a large number of trades, 50.000 runs may be in excess, if you are to have risk reports on your entire book (depending on the IT architecture, whether you aim for "real-time" reports, whether you need a full vega report etc.) On the other hand, I've seen applications involving nightly calculations of a large number of sensitivities for a universe of 1.200 callable and non-callable mortgage bonds with maturities up to 30 years. As all bonds were listed, pricing was not an issue, and various noise-reduction techniques could be applied to risk figure calculations, but time constraints still meant that only 500 runs were used in MC evaluationsCaroe, you are right that "acceptable" is a bit sloppy. I made my question vague in the hopes of getting a wider response. In my mind, "acceptable" means getting a low enough Standard Error. But, what is "low enough"? In your last example, time constraints meant that only 500 runs were used. I understand the difficulty of time constraints, but how did you justify the accuracy of the 500 runs? How does one define "sufficient accuracy"? Are there any articles on this subject?Again, many thanks.

Given that it's a tradeoff between speed and accuracy it depends of course on what you're doing with the output.You may need to be careful about using error as a measure. Some instruments are highly sensitive to values in the tails of the distribution, and a short run may be unlucky enough to not have one of these (or too many).

QuoteOriginally posted by: DCFCGiven that it's a tradeoff between speed and accuracy it depends of course on what you're doing with the output.You may need to be careful about using error as a measure. Some instruments are highly sensitive to values in the tails of the distribution, and a short run may be unlucky enough to not have one of these (or too many).Thanks for the tip.You are right that it's a tradeoff between speed and accuracy. So, assuming one has only x amount of time to complete the simulation, one can only squeeze y runs before the results are due. However, there has to be some measure to justify the results from the y runs - i.e. some reason to give a measure of comfort for the results.What measures of accuracy are there besides the error? Most papers seem to quote reductions in error to emphasize the superiority of a technique.Thanks again.

Well typically you'd plot the RMS error (if you knew an exact value) against the compute time.If you don't know the exact value you could simply take the Efficiency = [ 1/ (std error * compute time) ] Ofcourse this is presuming that your estimator is not upward/downward biased in which case you have anall together different problem. You could get a highly accurate estimate that is always biased- high from the true value

However, there has to be some measure to justify the results from the y runs - i.e. some reason to give a measure of comfort for the results.I guess a good "comfort measure" which I now define as the Feather, is the probability that a given run will be more than X away from the "true" value.So perhaps a good recipie for a time constrained MC is:1: Check that in the limit you will get the right value for some ranges of the input values.2: Make sure the harness in which your MC system works will reject values outside this range. Better a routine that says "I don't know that I'm right here", then your code bullshitting to you3: Run your MC to the limit for steps within and including these boundaries to determine the right values across the various combinations of the variables. Because of the high number of combinations this will run for a long time, but it's easy to write.3: Now for each combination of input values run your MC repeatedly until you can tell what % of times it goes out of an acceptable range. For most calcs you will actually find that the number of iterations you need varies with the input values. This data shold be preserved because it forms the basis for a very useful optimisation. It may even point out ranges that the convergence takes some huge number of iterations or even fails to converge at all. Thus the first phase of your MC will become more complex, but very much more efficient, and vastly more trustworthy; Basically it says if ( rate > 0.1 && vol <0.2 ) then IterCount = 6000; if ( rate <0.01 ) then IterCount = IterCount /2;Step (3) is at least a couple of orders of magnitude more CPU than step 2, possibly 3 or even 4. But you can leave it going, and just collect the output. Not all that hard to write code to turn the output directly into C++.4: Never turn it off. Your initial mesh will be quite sparse. Fact is that you have to deliver something soon, and the perfect is the enemy of the good. So, you now have an idle time task that gradually populates the N dimensional matrix gradually (ok very gradually) reducing the granularity, and maybe extending it beyond your initial ranges, so that when the market does something weird, you can still trust the system.In short you use Monte Carlo to build your Monte Carlo. The C++ program you write generates C++ that actually does the calculation. 4:

Here is a trick we used for MC calculations of pricing index amortizing swaps. Do a “run” R1 with N1 paths, a “run” R2 with N2 paths, a “run” R3 with N3 paths, etc (naturally with different initial seeds of the random number generator). Get statistics on the results from these runs. Now look at the results of a 2nd order composite run R12 with N1+N2 paths, a 3rd order composite run R123 with N1+N2+N3 paths, etc. You don’t actually have to do the composite runs, just save the results from the individual runs. When the composite runs looks like they are “settling down” with respect to the variations of the individual runs as a function of the order of the composite run, stop.For example, suppose you are calculating a quantity C. Do some runs {Rm} of 5K paths each, e.g. m = 1,…,6 such runs. Get the average CAvgRuns and standard deviation CSigRuns for the set {C(Rm)}. Now compare the successive results of runs with 10K, 15K, …,30K paths. If the result for some C(composite run), e.g. C(30K) is within the envelope defined by CAvgRuns and CSigRuns, and if the difference between C(30K) and C(25K) is small compared to CSigRuns, stop. Otherwise, do some more runs and continue the process. To be specific, you can define the composite runs in the sequential order specified by the random number generator and use the last random number for run Rm as the starting seed for run R(m+1).Sometimes you will not have that luxury. For a large portfolio of some mortgage products, notably CMOs, you may be restricted to even fewer than the 500 paths mentioned by caroe because of time constraints. In that case, all you can do is to try to choose the paths “wisely”. I knew some people in a mortgage group who spent time doing exactly that.---------

Also it is good to use one set of replications to compute the estimate as it reduces bias if any in your PRNG.Getting back to the original questoin you pick a efficiency ratio simillar to the one below and compare different techniques against it, maybe there is a IS estimator which is very good in reducing variance but if it means that your compute time per replication is much higher then there is no point in using Imp sampling. As you can simply crank you MC up and get the same estimate with fewer flops..Most academic papers (good ones) report a type of "efficiency ratio" . I think Glasserman uses it in his book ?

DCFC, thanks for elaborating on the "Feather". Unfortunately, for the current problem, I do not know what the true value is, but I will definitely keep your method in mind.Jan Dash, thanks very much for your trick. I will try it.Quantie, "Getting back to the original questoin you pick a efficiency ratio simillar to the one below"... I guess you are referring to the formula in your earlier post. I will probably use that ratio as well Thanks. By the way, are any of your favorite and "good" academic papers on-line? I don't have access to academic journals, so I would be grateful for references to Web documents.

I think Glasserman uses it in his book ?Do you mean the one where he says to minimise Var per replication * cost of each replication ?Also, one should of course point out with quanties point about bias in you PRNG that you should compare results from different PRNG.Of course it's arguable whether you shold really be using a PRNG at all.

QuoteOriginally posted by: DCFCI think Glasserman uses it in his book ?Do you mean the one where he says to minimise Var per replication * cost of each replication ?Also, one should of course point out with quanties point about bias in you PRNG that you should compare results from different PRNG.Of course it's arguable whether you shold really be using a PRNG at all.But yes the idea is simply to look at both the std error and the "cost" of replication for that standard error, insteadof just looking at the std error. I will have to get hold of my copy[glasserman] to confirm.[edited]On PRNG, did you get the hardware generator going?

I don't know if this helps you at all, because I will just state a standard result regarding monte-carlo techniques: In order to get one more reliable digit of you number, you have to use 100 times the number of Monte-Carlo simulations you had in the last run. This is due to the Monte-Carlo error of ( 1/sqrt(N) )*error of the estimator, where N is the number of Monte-Carlo pathes (or what ever you are simulating).This might not really solve your problem, however, it tells you, that just increasing the number of simulations by a factor 10, does not necessarily lead to a notable increase in accuracy, but the computing time might already explode exponentially.

On PRNG, did you get the hardware generator going?Yep, though got a slight glitch that the best RN's come out a bit slower than I want, development got stalled when I left my last firm.