Serving the Quantitative Finance Community

 
User avatar
quartz
Topic Author
Posts: 3
Joined: June 28th, 2005, 12:33 pm

Architecture and Design: data & dependencies, genericity, components, signals etc...

December 1st, 2011, 2:32 pm

(moved here from the MC thread)QuoteTo take an example in the case of 1 factor, the 3 levels are:1. SDE part (manufacturing model)2. FDM/numerics part (tracking model)3. High-level MIS info (e.g. statistics, post-processing convergence results, VAR)This process can be generalised.Design each subsystem until we get to patterns and code. This approach is used by many software designers. AFAIK this approach is needed if you want a parallel solution as bottom-up OO approach is not so easy to parallelise. ...Thus, I would avoid GOF Observer and deploy design 'parallel' from day 1. Use signals-based Propagator pattern is more flexible....The main effort is the assembly of all the bespoke 'interoperable' building blocks (which you already completed?) into the main design.Thanks Cuch!(sorry but what is MIS again?) Anyway the setting seems now clearer, and I agree especially on the parallel issue, at least in theory. In practice one problem often encountered is that part of the design constraints&inconsistencies become visible only while crafting actual code, kind of chicken&egg problem. Futile exercise until an alternative is offered, I reckon, and top-down has important advantages. So, if I got the setting correctly, it's clear that the "steps/levels" need not be fixed and can be a lot more (see e.g. the 10 step Prayer from Meucci, and there's not yet credit in there), not only in the sense of further refinement, right?And there shall be no hierarchy/directional dependency imposed a-priori; e.g. at a finer granularity one might want to get both ways:- invCDF <- CDF <- PDF <- characteristic- PDF -> characteristic -> whatever...And the levels/components can also mix/feedback sometimes (e.g. optimizing netted positions iirc).This might look like absence of structure and loss of a top-down view, but isn't really, it's mainly about loose coupling and "cross sectional" concepts reused at different stages (metaconcepts? can anyone but me understand this sentence?)... Too challenging?I feel that propagator here can be really useful! Or is this flexibility something else and playing against what you meant? But (in my ignorance) I am worried of propagator's overhead and complexity, so it would be great to get clearly defined when it's really needed versus where can lighter alternatives be enough.
Last edited by quartz on November 30th, 2011, 11:00 pm, edited 1 time in total.
 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Architecture and Design: data & dependencies, genericity, components, signals etc...

December 1st, 2011, 3:51 pm

QuoteThis might look like absence of structure and loss of a top-down view, but isn't really, it's mainly about loose coupling and "cross sectional" concepts reused at different stages (metaconcepts? can anyone but me understand this sentence?).Yes.Quote.. Too challenging?No.QuoteI feel that propagator here can be really useful! I think so. Compare with GOF Observer spinning out of control.
Last edited by Cuchulainn on November 30th, 2011, 11:00 pm, edited 1 time in total.
 
User avatar
Polter
Posts: 1
Joined: April 29th, 2008, 4:55 pm

Architecture and Design: data & dependencies, genericity, components, signals etc...

December 1st, 2011, 5:28 pm

Regarding the Design and Data with applications to Monte Carlo and parallelization (both SIMD as well as GPGPU) thereof -- I think one approach worth consideration here is data-oriented design (well-known in the game industry, for instance, hence the following links -- but transferable to many apps involving, e.g., vector processing).First things first:"I'd like to clear up potential confusion and stress that data-oriented design does not imply that something is data-driven. A data-driven game is usually a game that exposes a large amount of functionality outside of code and lets the data determine the behavior of the game. That is an orthogonal concept to data-oriented design, and can be used with any type of programming approach."So, to emphasize: "data-driven programming", while it sounds similar, is not to be confused with data-oriented design, those are orthogonal (independent) approaches.The bottom line is illustrated in Mike Acton's "Typical C++ B.S.":An example I can imagine in one of our applications, Monte Carlo pricing, would be NOT to have a, say, EuropeanCallOptionPayOff function as a function of scalar UnderlyingPrice, but rather to have a EuropeanCallOptionPayOffs function as a function of vector of UnderlyingPrices.Take vectors in, process vectors, return vectors out. Generalizable to tensors."DOD is like a factory production line. You have very simple functions working on data in a very cache friendly manner, resulting in simple code which is highly maintainable and optimisable."As a side-effect, this approach simplifies further parallelization of EuropeanCallOptionPayOffs both using SIMD (use SIMD registers and packed instructions in the for loop therein) and GPGPU (make its guts a kernel).A great place to start for an intro to data-oriented design is the following presentation:http://seven-degrees-of-freedom.blogspo ... design[PDF] http://research.scee.net/files/presenta ... _09.pdfand the discussions here:http://solid-angle.blogspot.com/2010/02 ... -designand those slides offer a pretty good summary:http://www.slideshare.net/DICEStudio/in ... designNext, these articles and slides might be of use:http://stackoverflow.com/questions/1641 ... mlThoughts?
Last edited by Polter on November 30th, 2011, 11:00 pm, edited 1 time in total.
 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Architecture and Design: data & dependencies, genericity, components, signals etc...

December 1st, 2011, 7:17 pm

All these 'discoveries' have been known for at least 40 years, aka functional decomposition, module interface language (MIL), in the works of Parnas, Yourdon, Michael Jackson (not the singer).We have tried to demonstrate by these examples that it is almost always incorrect to begin the decomposition of a system into modules on the basis of a flowchart. We propose instead that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then designed to hide such a decision from the others. Since, in most cases, design decisions transcend time of execution, modules will not correspond to steps in the processing. To achieve an efficient implementation we must abandon the assumption that a module is one or more subroutines, and instead allow subroutines and programs to be assembled collections of code from various modules.David ParnasIt was becoming clear some years ago that OOP did not scale well. I am talking about OO as a tool to *analyse* systems, devoid of implementation. IMO a data-driven approach will not be correct (at least initially) for this MC problem. Something on the lines of Domain Architectures or Jackson's Problem Frames are needed. These suggest parallel task decomposition from which data emerges. I do not see any other alternmative to design this complex system.Reading @quartz's post it is plain to see that it is a complex system and the usual methods are not going the resolve its essential difficulties.We are at a crossroads.
Last edited by Cuchulainn on December 1st, 2011, 11:00 pm, edited 1 time in total.
 
User avatar
Cuchulainn
Posts: 20255
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Architecture and Design: data & dependencies, genericity, components, signals etc...

December 1st, 2011, 8:56 pm

QuoteIn computer programming, data-driven programming is a programming paradigm in which the program statements describe the data to be matched and the processing required rather than defining a sequence of steps to be taken.[1] Adapting abstract data type design methods to object-oriented programming results in a data-driven design.[2] This type of design is used in object-oriented programming to define classes during the conception of a piece of software. Data-driven programming has been argued to lead to bad object-oriented design, with responsibility-driven design promoted as a better approachAgree. Now generalise the Wirfs-Brock method (object focused) to system level and we are done.Ms Wirfs Brock knows the essence well. What we now have is that each box is a component interacting with other components via signals and slots. We are almost there. I will deliver a POC MC simulator based on this.
Last edited by Cuchulainn on December 1st, 2011, 11:00 pm, edited 1 time in total.