SERVING THE QUANTITATIVE FINANCE COMMUNITY

 
User avatar
Cuchulainn
Topic Author
Posts: 59986
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Using Quantlib

December 20th, 2017, 12:16 pm

..and since the MT engines are identical (apart from speed) you'll be testing for statistical bias of functions (paths, payoffs) of the boost ziggurat normal distribution vs de unknown VS std version.

Right?
YEP. It's running now. I'm using the same payoff and Euler for each choice. So any differences can only be attributed to Ziggy stardust vs Irwin-Hall?

//
Variants
Boost MT with C++ Normal
C++ MT with Boost Normal

in addition to (interface OK?)
Boost MT with Boost Normal
C++ MT with C++ Normal
Last edited by Cuchulainn on December 20th, 2017, 12:26 pm, edited 2 times in total.
 
User avatar
outrun
Posts: 4573
Joined: April 29th, 2016, 1:40 pm

Re: Using Quantlib

December 20th, 2017, 12:19 pm

Great! 
Irwin-Hall was of course a joke :-) A fancy name for something horrible. Paul used it in PWOQFV1. 

Still, it would be fun to reverse engineer what VS is doing. Or maybe there *is* VS source code on your disk, since the code is probably template code that can't be distributed as binaries?
 
User avatar
Cuchulainn
Topic Author
Posts: 59986
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Using Quantlib

December 20th, 2017, 12:52 pm

Boost MT with Boost Normal
C++ MT with C++ Normal

I get this on my SDE/FDM stuff (no QL yet): conclusions?

1 factor MC with explicit Euler

EXACT is 5.8462822
 
Price Boost: 5.85409,
Price C++11: 5.84447,
 
Price Boost: 5.82869,
Price C++11: 5.84913,
 
Price Boost: 5.84515,
Price C++11: 5.8488,
 
Price Boost: 5.83798,
Price C++11: 5.84617,
 
Price Boost: 5.85198,
Price C++11: 5.83416,
 
Price Boost: 5.84473,
Price C++11: 5.84468,
 
Price Boost: 5.85285,
Price C++11: 5.83517,
 
Price Boost: 5.8521,
Price C++11: 5.85048,
 
Price Boost: 5.86105,
Price C++11: 5.85371,
 
Price Boost: 5.8468,
Price C++11: 5.84469,
 
Price Boost: 5.8412,
Price C++11: 5.84183,
 
Price Boost: 5.84521,
Price C++11: 5.84638,
 
Price Boost: 5.8568,
Price C++11: 5.84458,
 
Price Boost: 5.85084,
Price C++11: 5.83251,
 
Price Boost: 5.84797,
Price C++11: 5.84186,
 
Price Boost: 5.8383,
Price C++11: 5.83792,
 
Price Boost: 5.84131,
Price C++11: 5.84605,
 
Price Boost: 5.83899,
Price C++11: 5.84621,
 
Price Boost: 5.84719,
Price C++11: 5.8517,
 
Price Boost: 5.83802,
Price C++11: 5.84004,
0: 14514284786278117030 ==? 14514284786278117030
1: 4620546740167642908 ==? 4620546740167642908
2: 13109570281517897720 ==? 13109570281517897720
3: 17462938647148434322 ==? 17462938647148434322
 
User avatar
Cuchulainn
Topic Author
Posts: 59986
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Using Quantlib

December 20th, 2017, 1:05 pm

Great! 
Irwin-Hall was of course a joke :-) A fancy name for something horrible. 
Jaysus :D 
Bernstein polynomials  of degree 15..
 
User avatar
outrun
Posts: 4573
Joined: April 29th, 2016, 1:40 pm

Re: Using Quantlib

December 20th, 2017, 3:05 pm

Very good.

I'm on my phone now, so I can't do it right now, but I would estimate the mean and stdev of the two sets of 20 prices (sample stdev, divide by n-1) this gives two means and two stdevs, m1,m2,s1,s2. The compute means of the 20 prices with have a standard deviation of s1/sqrt(20), is and we can then compute how much these were off compared to the analytical price p,expressed as number of standard deviation.

How big is (p-m1) / (s1 / sqrt(20))? We expect it to be between -2 and +2 in 95% of the times we did this experiment.. *if* there is no bias. If it's +8 then a bias is very likely, if it's between -2 and 2 then thats a (statistical) insignificant deviation.
 
User avatar
Cuchulainn
Topic Author
Posts: 59986
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Using Quantlib

December 20th, 2017, 4:14 pm

I have now a test with NSIM = 2 10^6; maybe it gives some insights between:

A. Boost MT with Boost Normal
B. C++ MT with C++ Normal

B is "better"? It looks like it but not always.

Price Boost: 5.84622,
Price C++11: 5.83643,
 
Price Boost: 5.84572,
Price C++11: 5.85144,
 
Price Boost: 5.8369,
Price C++11: 5.84463,
 
Price Boost: 5.8528,
Price C++11: 5.84164,
 
Price Boost: 5.8394,
Price C++11: 5.84871,
 
Price Boost: 5.84025,
Price C++11: 5.85415,
 
Price Boost: 5.84594,
Price C++11: 5.84004,
 
Price Boost: 5.85373,
Price C++11: 5.84629,
 
Price Boost: 5.8543,
Price C++11: 5.8464,
 
Price Boost: 5.85098,
Price C++11: 5.84119,
 
Price Boost: 5.84864,
Price C++11: 5.84072,
 
Price Boost: 5.83862,
Price C++11: 5.84437,
 
Price Boost: 5.85463,
Price C++11: 5.85057,
 
Price Boost: 5.8523,
Price C++11: 5.8446,
 
Price Boost: 5.84239,
Price C++11: 5.83955,
 
Price Boost: 5.84221,
Price C++11: 5.84237,
 
Price Boost: 5.85203,
Price C++11: 5.83589,
 
Price Boost: 5.8505,
Price C++11: 5.8457,
 
Price Boost: 5.83623,
Price C++11: 5.84911,
 
Price Boost: 5.84338,
Price C++11: 5.83508,
0: 14514284786278117030 ==? 14514284786278117030
1: 4620546740167642908 ==? 4620546740167642908
2: 13109570281517897720 ==? 13109570281517897720
3: 17462938647148434322 ==? 17462938647148434322
 
User avatar
outrun
Posts: 4573
Joined: April 29th, 2016, 1:40 pm

Re: Using Quantlib

December 20th, 2017, 6:44 pm

Ok, so I get these statistics:
 Boost C++11
1 5,8541 5,8445
2 5,8287 5,8491
3 5,8452 5,8488
4 5,8380 5,8462
5 5,8520 5,8342
6 5,8447 5,8447
7 5,8529 5,8352
8 5,8521 5,8505
9 5,8611 5,8537
10 5,8468 5,8447
11 5,8412 5,8418
12 5,8452 5,8464
13 5,8568 5,8446
14 5,8508 5,8325
15 5,8480 5,8419
16 5,8383 5,8379
17 5,8413 5,8461
18 5,8390 5,8462
19 5,8472 5,8517
20 5,8380 5,8400
----------------------
ave 5,8461 5,8440
stdev 0,0076 0,0056
 
true value 5,8463 5,8463
deviation -0,0002 -0,0023
deviation stdevs -0,130034688 -1,790442442
.. so boost is 0.13 sample noise stdev away from the mean, c++11 1.79

You can't conclude either being biased. E.g. you might thing that boost is very good, but the variation in the 20 runs tells us we can expect it to be +1 or -1 next time you try it again. In this experiment the sample noise is much bigger than the average deviation we see.
 
User avatar
Cuchulainn
Topic Author
Posts: 59986
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Using Quantlib

December 20th, 2017, 7:07 pm

what we expected I suppose

C. Boost MT with C++ Normal
D. C++ MT with C++ Normal

Price Boost: 5.84876,
Price C++11: 5.84876,
 
Price Boost: 5.85386,
Price C++11: 5.85386,
 
Price Boost: 5.8359,
Price C++11: 5.8359,
 
Price Boost: 5.84263,
Price C++11: 5.84263,
 
Price Boost: 5.84603,
Price C++11: 5.84603,
 
Price Boost: 5.84565,
Price C++11: 5.84565,
 
Price Boost: 5.85704,
Price C++11: 5.85704,
 
Price Boost: 5.84845,
Price C++11: 5.84845,
 
Price Boost
 
User avatar
outrun
Posts: 4573
Joined: April 29th, 2016, 1:40 pm

Re: Using Quantlib

December 20th, 2017, 7:12 pm

Indeed, identical. 

Very early on I didn't understand how it could be any different, but now I know that the normal_distribution can vary between compilers. 

I don't expect any implementation to have any relevant deficiencies, ..some will be faster though,.. and they *can* give different results (but I don't expect in the limit as nsim->infinity)
 
User avatar
Cuchulainn
Topic Author
Posts: 59986
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Using Quantlib

December 21st, 2017, 11:38 am

what we expected I suppose

E. Boost MT with Boost Normal
F. C++ MT with Boost Normal
Price Boost: 5.83682,
Price C++11: 5.83682,
 
Price Boost: 5.8531,
Price C++11: 5.8531,
 
Price Boost: 5.84405,
Price C++11: 5.84405,
 
Price Boost: 5.84927,
Price C++11: 5.84927,
 
Price Boost: 5.83648,
Price C++11: 5.83648,
 
Price Boost: 5.8455,
Price C++11: 5.8455,
 
Price Boost: 5.85291,
Price C++11: 5.85291,
 
Price Boost: 5.84386,
Price C++11: 5.84386,
 
Price Boost: 5.84282,
Price C++11: 5.84282,
 
Price Boost: 5.84378,
Price C++11: 5.84378, 
 
User avatar
outrun
Posts: 4573
Joined: April 29th, 2016, 1:40 pm

Re: Using Quantlib

December 21st, 2017, 11:53 am

Yes, that's also as expected.
 
User avatar
Cuchulainn
Topic Author
Posts: 59986
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Using Quantlib

December 21st, 2017, 12:19 pm

Yes, that's also as expected.
It would be very  shocking if the answers were different. 
Conclusion is the libraries are interoperable. BTW Boost use Ziggy stardust since 1.59. I suppose it is more intensive than MT?
 
User avatar
Billy7
Posts: 282
Joined: March 30th, 2016, 2:12 pm

Re: Using Quantlib

December 21st, 2017, 12:49 pm

BTW Boost use Ziggy stardust since 1.59. I suppose it is more intensive than MT?
This is confusing. The Ziggurat takes a uniform variate (provided by some RNG like MT) and turns it into a normal variate.
MT is an RNG. Unless of course you mean something else by MT other than Mersenne Twister?
 
User avatar
outrun
Posts: 4573
Joined: April 29th, 2016, 1:40 pm

Re: Using Quantlib

December 21st, 2017, 12:49 pm

Yes, that's also as expected.
It would be very  shocking if the answers were different. 
Conclusion is the libraries are interoperable. BTW Boost use Ziggy stardust since 1.59. I suppose it is more intensive than MT?
You're mixing up two concepts I think.
*There are "engines" like mersenne twister, lineair congruent generator etc. These generators generate random *intergers*. They try to be both uniform in the interger range as well as independent across time. To test this people try a large range of statistical test that can potentially detect violations of these two requirement.  
* Then there is "distributions" that consume one or more intergers from an engine and convert it into samples from some target distribution. Normal_distribution with Ziggurat startdust is one of those distributions. The Ziggurat algorithm for Normal variates is well known and accepted, e.g. it's also used by Matlab.

* These two are orthogonal, you can mix engines with distributions (like you just did). I really like that design..

I was talking to developer at the time of the switch from BM to Ziggurat because I was adding the "non central Chi2 distribution" to boost (because of Heston) and he had to review it. I then notices him working on Ziggurat and asked about it. He never answered... He answers approx one in 20 questions.
 
User avatar
Cuchulainn
Topic Author
Posts: 59986
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Re: Using Quantlib

December 21st, 2017, 12:56 pm

 He answers approx one in 20 questions.
That's way above average for these lads. 

BTW QL has Ziggurat in an experimental directory. Don't know why because Boost uses it in boost::notmal_distribution.
ABOUT WILMOTT

PW by JB

Wilmott.com has been "Serving the Quantitative Finance Community" since 2001. Continued...


Twitter LinkedIn Instagram

JOBS BOARD

JOBS BOARD

Looking for a quant job, risk, algo trading,...? Browse jobs here...


GZIP: On