Serving the Quantitative Finance Community

 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Quantlib design and usage

February 12th, 2019, 5:58 pm

Thinking about it, a possible solution to develop next generation library is to do it in C# and C++ as side kick in the form of wrapped libraries (C++/CLI))..

Native C++ interoperability is an issue in general.

Productivity rates in C# are 3-4 times higher than in C++ from what I have seen.
 
User avatar
mtsm
Topic Author
Posts: 78
Joined: July 28th, 2010, 1:40 pm

Re: Quantlib design and usage

February 27th, 2019, 8:35 pm

So, a couple of observations... FWIW...

At a high level, what strikes me as odd is that the library mixes up the 'what' of financial engineering with the 'how' at the very deepest level. There is way too much run time logic based into the basic class structure, much more than is desirable. Even 15 years ago, that would have seemed like a bad idea to most engineers working in large T1 IBs. BTW, there also too much business logic baked into the template structure, but that's tangentially off topic. In a modern library, with various language bindings, various distributed computing approaches, I would expect a lot more layering to be more natural.

To be specific, I think was a very bad idea to bake in the observer pattern so pervasively across the library. How could that ever be a good idea. There is a place for push systems or dependency graph based systems for sure, but more often than not this is a distraction and actually quite undesirable. There is a huge overhead to engineer almost anything in client applications due to this. It would be much more natural to have client application direct the dependencies and then build a push framework on top as opposed to forcing everybody to mess around with handles. The amusing thing is that the lazy object approach seems to be aimed at partially undoing the potential run time overhead caused by excessive use of observer pattern updates...


I also notice excessive, almost wild, usage of shared_ptr. This is used at a significantly too granular level. Not a good idea in my opinion. I have a definite preference for using heap memory management at a slightly more macro level and use containers to handle collections of smaller objects but not through shared_ptr. Again this causes a lot of overhead. 

Excessive usage of template programming techniques. Not my cup of tea. I may not be a sufficiently proficient C++ programmer, but in my opinion template are suitable for generic programming, not so much for financial engineering. It's debatable, but when I see how templates have been applied to introduce the various TermStructure classes, I don't actually think that this affords that much flexibility in an area where the library is lagging 10 years or so behind.

It's also weird to me that so much dynamics is built into the instruments. Pricers or pricing engine deserve to be more first class citizen. 

I observed and begin to understand that a lot of people I talk to in interviews are in the business of ripping the library for all the goodness it has. On the other hand there appear to be several consulting groups out there who have learned to put up with the nonsensical overhead and just use the library as is. 

I wonder if it would be worth starting to build a library based on QuantLib that focuses more on laying out a clean collection of objects and methods. Something lower level, that handles only a subset of functionality maybe. 
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Quantlib design and usage

March 1st, 2019, 12:41 pm

Seeing that no one has responded, here is my feedback as C++/Design patterns veteran since 1990 (caveat: I trained > 6K C++ developers and have designed apps with clients in finance, process, CAD/CAM, optical tech). Ideally, a short history of OOP/patterns should be included to put things into perspective but it might come across as a session with the psychiatrist looking into the developer's head.

OOP and GOF were very hot in the late 90s. Most of my projects were fixed-price which tends to keep one focused, thus avoiding hobbyismus and creeping featuritis.

So, I will respond to your points.
Last edited by Cuchulainn on March 1st, 2019, 2:12 pm, edited 4 times in total.
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Quantlib design and usage

March 1st, 2019, 12:42 pm

So, a couple of observations... FWIW...

DD Thanks, I for one found them constructive. In general, does the software industry carry on as usual?

At a high level, what strikes me as odd is that the library mixes up the 'what' of financial engineering with the 'how' at the very deepest level. There is way too much run time logic based into the basic class structure, much more than is desirable. Even 15 years ago, that would have seemed like a bad idea to most engineers working in large T1 IBs. BTW, there also too much business logic baked into the template structure, but that's tangentially off topic. In a modern library, with various language bindings, various distributed computing approaches, I would expect a lot more layering to be more natural.

DD Some developers in general want to have a solution ASAP while not fully understanding the problem or the consequences of their design decisions (or lack thereof). Most developers are from the OOP era (everything is an object). They missed on the 60s/70s system design. And hey might miss the FP boat if they 
are not careful.

To be specific, I think was a very bad idea to bake in the observer pattern so pervasively across the library. How could that ever be a good idea. There is a place for push systems or dependency graph based systems for sure, but more often than not this is a distraction and actually quite undesirable. There is a huge overhead to engineer almost anything in client applications due to this. It would be much more natural to have client application direct the dependencies and then build a push framework on top as opposed to forcing everybody to mess around with handles. The amusing thing is that the lazy object approach seems to be aimed at partially undoing the potential run time overhead caused by excessive use of observer pattern updates...

DD Observer pattern is a HORROR story. I could write a book just why it is wrong. You are correct! Signals and delegates is the correct approach.
Follow on: GOF pattern are outdated BIG TIME (but Ioved them in CAD/CAM).

I also notice excessive, almost wild, usage of shared_ptr. This is used at a significantly too granular level. Not a good idea in my opinion. I have a definite preference for using heap memory management at a slightly more macro level and use containers to handle collections of smaller objects but not through shared_p tr. Again this causes a lot of overhead. 

DD shared ptr is a blunt instrument, indeed, when used incorrectly. Even short-lived factory objects, which are scoped_ptr/unique_ptr.

Excessive usage of template programming techniques. Not my cup of tea. I may not be a sufficiently proficient C++ programmer, but in my opinion template are suitable for generic programming, not so much for financial engineering. It's debatable, but when I see how templates have been applied to introduce the various TermStructure classes, I don't actually think that this affords that much flexibility in an area where the library is lagging 10 years or so behind.

DD TMP is gold-plating/over-engineering. Michael Jackson (not the singer) said:

1. Don't optimise 2. DDon't optimise yet (and then for experts). TMP sounds hip.

As a project manager said: TMP is fine, but do it your own time. TMP destroys creativity and it ensures that the quant project will never get completed,

It's also weird to me that so much dynamics is built into the instruments. Pricers or pricing engine deserve to be more first class citizen. 

DD In agree 100%. I think OOP is obsessed with objects, losing sight of the bigger picture.

I observed and begin to understand that a lot of people I talk to in interviews are in the business of ripping the library for all the goodness it has. On the other hand there appear to be several consulting groups out there who have learned to put up with the nonsensical overhead and just use the library as is. 

DD What is 'ripping'?

I wonder if it would be worth starting to build a library based on QuantLib that focuses more on laying out a clean collection of objects and methods. Something lower level, that handles only a subset of functionality maybe. 

DD I once bought a quaint old farmhouse. There was  a choice between renovating it or starting from scratch. I made a choice.
QL is based on GOF/OOP and this is not the next gen paradigm. I am just the messenger. I would start again. Talk is cheap and you need a budget and 2-3 experienced team members.
I think some kind of top-down architectural design and bottom-up implementation keep 'em sharp.. 

// attachments on how I try to solve problems.
// Also a chapter on MC using delegates instead of that pernicious Observer. No inheriting a SDE from Observer ;)

https://www.datasim.nl/blogs/17/sample-chapter
Attachments
MC Application.pdf
(736.97 KiB) Downloaded 302 times
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Quantlib design and usage

March 1st, 2019, 2:02 pm

Here is a note on delegate-based design patters (in this case C# but C++ is just as easy).

AFAIK I have written several  articles in Wilmott.
Attachments
Chapter XX System Configuration and Creational Patterns in .NET.pdf
(475.01 KiB) Downloaded 294 times
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Quantlib design and usage

March 4th, 2019, 10:16 am

Hindsight is 20 20 but when the OOP revolution began some things were thrown overboard and new work practices were introduced:

A. You don't need to create a system decomposition, context diagram nor data flow diagram because all you need is objects.
https://en.wikipedia.org/wiki/System_context_diagram
I was spared this infliction because clients used a combination of Stuctured Analysis and OP. It offers many advantages.
B. Classes and inheritance model real life things, not really (bad news; they are context-sensitive).
C. Bottom-up OOP leads to ball of mud.

This article discusses the hybid model
http://citeseerx.ist.psu.edu/viewdoc/do ... 1&type=pdf
 
User avatar
mtsm
Topic Author
Posts: 78
Joined: July 28th, 2010, 1:40 pm

Re: Quantlib design and usage

March 21st, 2019, 3:29 pm

Thanks, these are great comments. 

I'll try and read some of your stuff on the topic. Where can I find it?
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Quantlib design and usage

March 21st, 2019, 5:18 pm

You'e welcome., glad it was of use. Some resources

1. Many publicly available articles in Wilmott last years.
viewtopic.php?f=11&t=100869&start=30

2. My recent 2nd edition of "Financial Instrument Pricing using C++" 2018, including code for all this stuff.
3. My online students at www..quantnet.com who learn the bespoke Domain Achitecture and apply to MC, PDE etc. (> 40 at least).
4. I used to be requirements  analyst (very enjoyable) for large complex s/w systems in a range of industries. Long story short: I distilled all applications into 5 categories. The link to my 2004 DA book "Domain Architectures", Wiley 2004. To be honest it was 20 years ahead of its time but it works.
5 DA design is so flexible that it can be

a. language-independent (C++, C#)
b. OOP, FP
c single-threading, parallel (tasks, actor model).

6. And training courses.

A new QL style based on this approach wouldn't do any harm. Batti il ferro finché è caldo.
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Quantlib design and usage

March 26th, 2019, 12:12 pm

Some alternatives to "GOF hegemony"

http://www.datasimfinancial.com/forum/v ... =32&t=1159
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Quantlib design and usage

March 31st, 2019, 5:45 pm

This is a summary of the appropriate application  of design patterns (a while back 1997 for a bio reactor). The code had to be portable and extendable. You get some hints from this (they are not called patterns for nothing!)

// It's a docx file but I had to zip for this forum.

The Visitor pattern is very useful. These days you see it in Boost.
Attachments
Boiler.zip
(84.41 KiB) Downloaded 254 times
 
User avatar
agnoatto
Posts: 12
Joined: November 23rd, 2011, 8:12 am

Re: Quantlib design and usage

February 20th, 2020, 10:14 am

My 2 cents: Quantlib is overengineered. I remember taking at look at it in the bank. For a period of time I was amazed by the structure, but I soon started to realize that the structure was more a limitation rather than a resource. There are more modern open source projects that can help you.

If your are into flow/plain vanilla instrument, the best project in my opinion is OpenGamma Strata: https://github.com/OpenGamma/Strata Stephen Colebourne is one of the best Java programmers out there (He is the designer of the modern date class (LocalDate) of the Java platform).

If you are looking for a solid Monte Carlo Engine then take a look at Finmath https://github.com/finmath/finmath-lib by Christian Fries. If you combine these two projects then you have a pretty nice (and modern!) software framework to start from. One nice thing about Strata is that you can purchase support I think.
Prof Alessandro Gnoatto, PhD
Department of Economics
University of Verona
Via Cantarane 24 - 37129 Verona - Italy
37129, Verona, Italy
 
User avatar
Cuchulainn
Posts: 20253
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Quantlib design and usage

February 20th, 2020, 11:18 am

Historically (from the 1990s) the Gamma Patterns were considered the best practice as well as the naive belief that everything is an object and class inheritance was the way to go.
This OOP approach is not good at modelling algorithms.

From 30000 feet, it is just part of the evolution of software 'engineering' which is in its infancy.