Page 9 of 13

Re: Using Quantlib

Posted: December 20th, 2017, 12:16 pm
by Cuchulainn
..and since the MT engines are identical (apart from speed) you'll be testing for statistical bias of functions (paths, payoffs) of the boost ziggurat normal distribution vs de unknown VS std version.

Right?
YEP. It's running now. I'm using the same payoff and Euler for each choice. So any differences can only be attributed to Ziggy stardust vs Irwin-Hall?

//
Variants
Boost MT with C++ Normal
C++ MT with Boost Normal

in addition to (interface OK?)
Boost MT with Boost Normal
C++ MT with C++ Normal

Re: Using Quantlib

Posted: December 20th, 2017, 12:19 pm
by outrun
Great! 
Irwin-Hall was of course a joke :-) A fancy name for something horrible. Paul used it in PWOQFV1. 

Still, it would be fun to reverse engineer what VS is doing. Or maybe there *is* VS source code on your disk, since the code is probably template code that can't be distributed as binaries?

Re: Using Quantlib

Posted: December 20th, 2017, 12:52 pm
by Cuchulainn
Boost MT with Boost Normal
C++ MT with C++ Normal

I get this on my SDE/FDM stuff (no QL yet): conclusions?

1 factor MC with explicit Euler

EXACT is 5.8462822
 
Price Boost: 5.85409,
Price C++11: 5.84447,
 
Price Boost: 5.82869,
Price C++11: 5.84913,
 
Price Boost: 5.84515,
Price C++11: 5.8488,
 
Price Boost: 5.83798,
Price C++11: 5.84617,
 
Price Boost: 5.85198,
Price C++11: 5.83416,
 
Price Boost: 5.84473,
Price C++11: 5.84468,
 
Price Boost: 5.85285,
Price C++11: 5.83517,
 
Price Boost: 5.8521,
Price C++11: 5.85048,
 
Price Boost: 5.86105,
Price C++11: 5.85371,
 
Price Boost: 5.8468,
Price C++11: 5.84469,
 
Price Boost: 5.8412,
Price C++11: 5.84183,
 
Price Boost: 5.84521,
Price C++11: 5.84638,
 
Price Boost: 5.8568,
Price C++11: 5.84458,
 
Price Boost: 5.85084,
Price C++11: 5.83251,
 
Price Boost: 5.84797,
Price C++11: 5.84186,
 
Price Boost: 5.8383,
Price C++11: 5.83792,
 
Price Boost: 5.84131,
Price C++11: 5.84605,
 
Price Boost: 5.83899,
Price C++11: 5.84621,
 
Price Boost: 5.84719,
Price C++11: 5.8517,
 
Price Boost: 5.83802,
Price C++11: 5.84004,
0: 14514284786278117030 ==? 14514284786278117030
1: 4620546740167642908 ==? 4620546740167642908
2: 13109570281517897720 ==? 13109570281517897720
3: 17462938647148434322 ==? 17462938647148434322

Re: Using Quantlib

Posted: December 20th, 2017, 1:05 pm
by Cuchulainn
Great! 
Irwin-Hall was of course a joke :-) A fancy name for something horrible. 
Jaysus :D 
Bernstein polynomials  of degree 15..

Re: Using Quantlib

Posted: December 20th, 2017, 3:05 pm
by outrun
Very good.

I'm on my phone now, so I can't do it right now, but I would estimate the mean and stdev of the two sets of 20 prices (sample stdev, divide by n-1) this gives two means and two stdevs, m1,m2,s1,s2. The compute means of the 20 prices with have a standard deviation of s1/sqrt(20), is and we can then compute how much these were off compared to the analytical price p,expressed as number of standard deviation.

How big is (p-m1) / (s1 / sqrt(20))? We expect it to be between -2 and +2 in 95% of the times we did this experiment.. *if* there is no bias. If it's +8 then a bias is very likely, if it's between -2 and 2 then thats a (statistical) insignificant deviation.

Re: Using Quantlib

Posted: December 20th, 2017, 4:14 pm
by Cuchulainn
I have now a test with NSIM = 2 10^6; maybe it gives some insights between:

A. Boost MT with Boost Normal
B. C++ MT with C++ Normal

B is "better"? It looks like it but not always.

Price Boost: 5.84622,
Price C++11: 5.83643,
 
Price Boost: 5.84572,
Price C++11: 5.85144,
 
Price Boost: 5.8369,
Price C++11: 5.84463,
 
Price Boost: 5.8528,
Price C++11: 5.84164,
 
Price Boost: 5.8394,
Price C++11: 5.84871,
 
Price Boost: 5.84025,
Price C++11: 5.85415,
 
Price Boost: 5.84594,
Price C++11: 5.84004,
 
Price Boost: 5.85373,
Price C++11: 5.84629,
 
Price Boost: 5.8543,
Price C++11: 5.8464,
 
Price Boost: 5.85098,
Price C++11: 5.84119,
 
Price Boost: 5.84864,
Price C++11: 5.84072,
 
Price Boost: 5.83862,
Price C++11: 5.84437,
 
Price Boost: 5.85463,
Price C++11: 5.85057,
 
Price Boost: 5.8523,
Price C++11: 5.8446,
 
Price Boost: 5.84239,
Price C++11: 5.83955,
 
Price Boost: 5.84221,
Price C++11: 5.84237,
 
Price Boost: 5.85203,
Price C++11: 5.83589,
 
Price Boost: 5.8505,
Price C++11: 5.8457,
 
Price Boost: 5.83623,
Price C++11: 5.84911,
 
Price Boost: 5.84338,
Price C++11: 5.83508,
0: 14514284786278117030 ==? 14514284786278117030
1: 4620546740167642908 ==? 4620546740167642908
2: 13109570281517897720 ==? 13109570281517897720
3: 17462938647148434322 ==? 17462938647148434322

Re: Using Quantlib

Posted: December 20th, 2017, 6:44 pm
by outrun
Ok, so I get these statistics:
 Boost C++11
1 5,8541 5,8445
2 5,8287 5,8491
3 5,8452 5,8488
4 5,8380 5,8462
5 5,8520 5,8342
6 5,8447 5,8447
7 5,8529 5,8352
8 5,8521 5,8505
9 5,8611 5,8537
10 5,8468 5,8447
11 5,8412 5,8418
12 5,8452 5,8464
13 5,8568 5,8446
14 5,8508 5,8325
15 5,8480 5,8419
16 5,8383 5,8379
17 5,8413 5,8461
18 5,8390 5,8462
19 5,8472 5,8517
20 5,8380 5,8400
----------------------
ave 5,8461 5,8440
stdev 0,0076 0,0056
 
true value 5,8463 5,8463
deviation -0,0002 -0,0023
deviation stdevs -0,130034688 -1,790442442
.. so boost is 0.13 sample noise stdev away from the mean, c++11 1.79

You can't conclude either being biased. E.g. you might thing that boost is very good, but the variation in the 20 runs tells us we can expect it to be +1 or -1 next time you try it again. In this experiment the sample noise is much bigger than the average deviation we see.

Re: Using Quantlib

Posted: December 20th, 2017, 7:07 pm
by Cuchulainn
what we expected I suppose

C. Boost MT with C++ Normal
D. C++ MT with C++ Normal

Price Boost: 5.84876,
Price C++11: 5.84876,
 
Price Boost: 5.85386,
Price C++11: 5.85386,
 
Price Boost: 5.8359,
Price C++11: 5.8359,
 
Price Boost: 5.84263,
Price C++11: 5.84263,
 
Price Boost: 5.84603,
Price C++11: 5.84603,
 
Price Boost: 5.84565,
Price C++11: 5.84565,
 
Price Boost: 5.85704,
Price C++11: 5.85704,
 
Price Boost: 5.84845,
Price C++11: 5.84845,
 
Price Boost

Re: Using Quantlib

Posted: December 20th, 2017, 7:12 pm
by outrun
Indeed, identical. 

Very early on I didn't understand how it could be any different, but now I know that the normal_distribution can vary between compilers. 

I don't expect any implementation to have any relevant deficiencies, ..some will be faster though,.. and they *can* give different results (but I don't expect in the limit as nsim->infinity)

Re: Using Quantlib

Posted: December 21st, 2017, 11:38 am
by Cuchulainn
what we expected I suppose

E. Boost MT with Boost Normal
F. C++ MT with Boost Normal
Price Boost: 5.83682,
Price C++11: 5.83682,
 
Price Boost: 5.8531,
Price C++11: 5.8531,
 
Price Boost: 5.84405,
Price C++11: 5.84405,
 
Price Boost: 5.84927,
Price C++11: 5.84927,
 
Price Boost: 5.83648,
Price C++11: 5.83648,
 
Price Boost: 5.8455,
Price C++11: 5.8455,
 
Price Boost: 5.85291,
Price C++11: 5.85291,
 
Price Boost: 5.84386,
Price C++11: 5.84386,
 
Price Boost: 5.84282,
Price C++11: 5.84282,
 
Price Boost: 5.84378,
Price C++11: 5.84378, 

Re: Using Quantlib

Posted: December 21st, 2017, 11:53 am
by outrun
Yes, that's also as expected.

Re: Using Quantlib

Posted: December 21st, 2017, 12:19 pm
by Cuchulainn
Yes, that's also as expected.
It would be very  shocking if the answers were different. 
Conclusion is the libraries are interoperable. BTW Boost use Ziggy stardust since 1.59. I suppose it is more intensive than MT?

Re: Using Quantlib

Posted: December 21st, 2017, 12:49 pm
by Billy7
BTW Boost use Ziggy stardust since 1.59. I suppose it is more intensive than MT?
This is confusing. The Ziggurat takes a uniform variate (provided by some RNG like MT) and turns it into a normal variate.
MT is an RNG. Unless of course you mean something else by MT other than Mersenne Twister?

Re: Using Quantlib

Posted: December 21st, 2017, 12:49 pm
by outrun
Yes, that's also as expected.
It would be very  shocking if the answers were different. 
Conclusion is the libraries are interoperable. BTW Boost use Ziggy stardust since 1.59. I suppose it is more intensive than MT?
You're mixing up two concepts I think.
*There are "engines" like mersenne twister, lineair congruent generator etc. These generators generate random *intergers*. They try to be both uniform in the interger range as well as independent across time. To test this people try a large range of statistical test that can potentially detect violations of these two requirement.  
* Then there is "distributions" that consume one or more intergers from an engine and convert it into samples from some target distribution. Normal_distribution with Ziggurat startdust is one of those distributions. The Ziggurat algorithm for Normal variates is well known and accepted, e.g. it's also used by Matlab.

* These two are orthogonal, you can mix engines with distributions (like you just did). I really like that design..

I was talking to developer at the time of the switch from BM to Ziggurat because I was adding the "non central Chi2 distribution" to boost (because of Heston) and he had to review it. I then notices him working on Ziggurat and asked about it. He never answered... He answers approx one in 20 questions.

Re: Using Quantlib

Posted: December 21st, 2017, 12:56 pm
by Cuchulainn
 He answers approx one in 20 questions.
That's way above average for these lads. 

BTW QL has Ziggurat in an experimental directory. Don't know why because Boost uses it in boost::notmal_distribution.