Serving the Quantitative Finance Community

 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

The ultimate Monte Carlo framework

January 17th, 2012, 2:38 pm

QuoteOriginally posted by: outrunHee T4A, I differ on both points. I suspect that we do agree, in general, and are really discussing crucial nuances in which a general rule breaks down.QuoteOriginally posted by: outrunMy view for multiple payoffs per scenario's is that it's a must for valuations of portfolios. You need to price all elements in the portfolio (all payoffs) against various scenario's in order to build a set of scenario's of the total portfolio value.Yes, I do see your point. In some applications, reusing the paths for different instruments will reduce some types of estimation error (e.g., the estimated price difference between two call-like instruments). And, in other applications, reusing the paths for different instruments will increase some types of estimation error (e.g., the price difference between a call-like and put-like instrument or the correlation between prices for two call-like instruments). It's a matter of thinking through how errors in reused samples (versus independent samples) propagate to the final aggregate statistical estimates to see if reused samples (versus independent samples) produces a more accurate estimate.QuoteOriginally posted by: outrunFor the rnd: I think it's bad to create asymmetry in parallel algorithms (have a central CPU doing something different, having a scaling bottleneck), it will also create a lot of data copying and moving and I expect that to be much than to generate it at various locations locally.In general, I agree entirely that asymmetry is bad in parallel algorithms. But it's not always avoidable. If one must adapt a serial or single-threaded algorithm (e.g., an RNG) to parallel processing, then some kind of asymmetry seems inevitable. It's then a matter of what kind of asymmetry does one use: pre-calculation of seed intervals or a running buffered production of RNs for dispatch.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

The ultimate Monte Carlo framework

January 17th, 2012, 3:45 pm

And I agree, again. One of the trickiest parts of parallel processing is in designing the interconnections between single-threaded parts of the problem and potentially multi-threaded parts.The other issue is in picking a performance metric (which changes what might be considered the bottleneck): does one want an MC platform with the highest possible throughput (in billions of sample paths per CPU hour) or does one want a platform with the lowest possible latency (in milliseconds to a final answer from a 1 million path MC run)? And if latency is the goal, is there a rationale for getting some kind of MC estimate within X msec, even if it's not as accurate as the final result produced after N*X msec?Requirements for "what do you want" and "when do you want it" can change the architecture of the algorithm and change the advantages (or disadvantages) for reusing sample paths.
 
User avatar
Cuchulainn
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

The ultimate Monte Carlo framework

January 17th, 2012, 6:01 pm

There has been a good discussion here QED?I am doing MC103 sequentially (get it working). later I will get it optimised by parallel libraries. That's the easy part. A flexible design is much more difficult but it leads to concurrent tasks that you can/should implement in you favourite Parallel lib.QuoteIn general, I agree entirely that asymmetry is bad in parallel algorithms. But it's not always avoidable. If one must adapt a serial or single-threaded algorithm (e.g., an RNG) to parallel processing, then some kind of asymmetry seems inevitable. It's then a matter of what kind of asymmetry does one use: pre-calculation of seed intervals or a running buffered production of RNs for dispatch. It's unavoidable. e.g. readers-writers problem, PC especially if event-driven.Useful:little book of semaphores
Last edited by Cuchulainn on January 16th, 2012, 11:00 pm, edited 1 time in total.
 
User avatar
Cuchulainn
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

The ultimate Monte Carlo framework

January 17th, 2012, 9:03 pm

QuoteOriginally posted by: outrunThe way I see it, it that we can have multiple threads/machines doing individual MC (without communication / accessing shared memory), but then at some point we need to aggregate results to a single answer. Ideally you can locally pre-aggregate results -e.g. when calculating a mean- and minimize communication. Sometimes we need more difficult forms of aggregation -e.g. when calculating a 99% VAR-, a naive solution would be to merge all sample results, sort them, and estimate the 99% level, however, there are also specialized aggregation routines for specific problems that solve that much better. The main point is that we'll have a couple of phases: setup, run simulations, aggregate results.my 2ctsKnowing this is an example of a Divide and Conquer pattern, we can look it up in Mattson and apply/flesh out the steps. No point reinventing the wheel. So if we start with a good data/task decomposition these patterns fall into place in a natural way.
Last edited by Cuchulainn on January 16th, 2012, 11:00 pm, edited 1 time in total.
 
User avatar
Cuchulainn
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

The ultimate Monte Carlo framework

January 18th, 2012, 6:34 am

The payoff and related informatio in MC102 is spread across several classes; in particular it does not belong in the path evolver but in PostProcess (an interface). This is a big improvement. It means that each contract can be encapsulated in its own black box.
Last edited by Cuchulainn on January 17th, 2012, 11:00 pm, edited 1 time in total.
 
User avatar
Cuchulainn
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

The ultimate Monte Carlo framework

January 18th, 2012, 8:45 am

QuoteOriginally posted by: outrunGood! You feed the path to the black box? Can it be a function or functor? And does the result of that black box get fed as input to some sort of result aggregator ?Yes, on all counts. It is just a function signature (no OOP class hierarchy needed). And existing OO code can be non-intrusively subsumed by Boost bind.
Last edited by Cuchulainn on January 17th, 2012, 11:00 pm, edited 1 time in total.
 
User avatar
Cuchulainn
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

The ultimate Monte Carlo framework

January 18th, 2012, 8:46 am

QuoteMy view for multiple payoffs per scenario's is that it's a must for valuations of portfolios. You need to price all elements in the portfolio (all payoffs) against various scenario's in order to build a set of scenario's of the total portfolio value.QuoteYes, I do see your point. In some applications, reusing the paths for different instruments will reduce some types of estimation error (e.g., the estimated price difference between two call-like instruments). And, in other applications, reusing the paths for different instruments will increase some types of estimation error (e.g., the price difference between a call-like and put-like instrument or the correlation between prices for two call-like instruments). It's a matter of thinking through how errors in reused samples (versus independent samples) propagate to the final aggregate statistical estimates to see if reused samples (versus independent samples) produces a more accurate estimate.At this higher level we can have N:N:N (SDE, fdm, contract) configuration relationships and coarse-grained parallelism. I would to extend the design after MC103 has completed to accommodate this (it will be another MisAgent as in MC102). Again, communication is via generic functors.
Last edited by Cuchulainn on January 17th, 2012, 11:00 pm, edited 1 time in total.