Serving the Quantitative Finance Community

 
User avatar
Cuchulainn
Topic Author
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Using Quantlib

December 16th, 2017, 12:22 pm

What are your compiler settings? 64bit?

Do they generate the same sequence when given the same seed? What you are saying with "less accurate" is that it is biased and you are the first to observe that. That in turn means that it's deviating from the documented MT with has been extensively tested for it's statistical properties. That is all a bit unlikely. How did you measure "accuracy" of the boost MT? Is it eg an observed statistical significant deviation from expected sample noise?

variate_generator is an old interface, you can be C++11 compliant by not using it. When the C++11 <random> standard came out it was based in boost, but with differences. Boost implemented the new standard interface and definitions to be compatible with the C++11 standard, and also left things in for backward compatibility.
You missed the main point, i.e. performance (my point 1).

These are valid questions but are not the point just yet. You need to realise that many applications are not  C++11 compliant and will remain so for some time.
The point of the post is in the difference in performance in performance and accuracy (when we used MC).

That in turn means that it's deviating from the documented MT with has been extensively tested for it's statistical properties. 
MT in C++11 and MT in Boost. I have not seen a comparison, Where should I look then?

variate_generator is an old interface, 
Well, Boost uses it. And it looks like it is much faster than C++11. The code I posted can be put in a loop and run it. You can easily compare the relative speeds for yourself. 3 choices: Maybe there a 'better' way to use C++11 Rng?? Maybe the RNG experts can shed some light on this.

C++ > Boost
C++ < Boost
C++ ~ Boost

I think it is an attention area.
 
User avatar
Cuchulainn
Topic Author
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Using Quantlib

December 16th, 2017, 12:51 pm

 And  I am not the only one..

I want to generate pseudo-random numbers in C++, and the two likely options are the feature of C++11 and the Boost counterpart. They are used in essentially the same way, but the native one in my tests is roughly 4 times slower.

Boost mt19937 is 4.9 ns per rn.

If I knew the answer I wouldn't be asking the question. Why is C++11 slower is the question. 
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Using Quantlib

December 16th, 2017, 1:23 pm

What is your definition of accuracy?? And how did you measure accurcy, what test did you do? 

If they give the same sequence when seeded with the same value (which I expect) then they are exactly as accurate since one can't be statistically biased and the other not -if they are identical-, no?

If so then the difference in accuracy must be because of difference in seed, i.e. sample noise! This confuses people and causes trouble, ..what if someone believes this accurcy statement and then later learn the rngs are in fact identical?

I know that the C++11 version is very well documented, all the constants are there, they say what the 10.000th number must be 4123659995 http://en.cppreference.com/w/cpp/numeri ... ter_engine (from the standard)

I also know that in on of the unit test in boost the same  4123659995 for the 10000th draw pops up http://www.boost.org/doc/libs/1_37_0/li ... m_test.cpp
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Using Quantlib

December 16th, 2017, 1:25 pm

about the speed: are you compiling to 32bit or 64bit? "debug" mode in VC or "release"

edit: 
I'm asking because I remember that with my 64bit random engine VC on Windows that target 32bit would see a slow down due to VC emulating 64bit operations very slow on 32bit targets. Also, I've seen people measuring performance in debug mode, which is never good for research.
 
User avatar
Cuchulainn
Topic Author
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Using Quantlib

December 17th, 2017, 7:46 pm

I took 32 bir release mode in all cases (so,  a level playing field)!  

i don't want to talk about "accuracy" just yet,  as I already mentioned. So can we focus on performance (one task at a time).


Here is a test from someone else.(Boost is faster than C++11 and GSL)

https://dilawarnotes.wordpress.com/2016 ... e-twister/

The point is there seems to be various implementations of MT. And that is sloppy.
Last edited by Cuchulainn on December 17th, 2017, 8:11 pm, edited 2 times in total.
 
User avatar
Cuchulainn
Topic Author
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Using Quantlib

December 17th, 2017, 7:53 pm

about the speed: are you compiling to 32bit or 64bit? "debug" mode in VC or "release"

edit: 
I'm asking because I remember that with my 64bit random engine VC on Windows that target 32bit would see a slow down due to VC emulating 64bit operations very slow on 32bit targets. Also, I've seen people measuring performance in debug mode, which is never good for research.
RELEASE of course.
Default mode in VS is DEBUG. That's not the issues Besides DEBUG is ~ TEN time slower.
Anyway we can test C++11 in debug mode with Boost in debug mode. 
I'm very familiar with VS.
Last edited by Cuchulainn on December 17th, 2017, 8:19 pm, edited 1 time in total.
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Using Quantlib

December 17th, 2017, 8:11 pm

about the speed: are you compiling to 32bit or 64bit? "debug" mode in VC or "release"

edit: 
I'm asking because I remember that with my 64bit random engine VC on Windows that target 32bit would see a slow down due to VC emulating 64bit operations very slow on 32bit targets. Also, I've seen people measuring performance in debug mode, which is never good for research.
RELEASE of course.
Default mode in VS is DEBUG. That's not the issues Besides DEBUG is ~ TEN time slower.
Anyway we can test C++11 in debug mode with Boost in debug mode. 
ok, very good! And what about the the bits?  Remember that (only) you had speed issues with the parallel RNG I once wrote, and it ended up being causes by my rng using 64bit integer operations, you compiling to 32bit, and VC decided to emulate those 64bit integer operations with 32bit integers not very efficiently.

In C++11 (and hence also in boost) there are two version of MT: std::mt19937, and std::mt19937_64.

The _64 versions will be faster when you compile to 64bit.
 
User avatar
Cuchulainn
Topic Author
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Using Quantlib

December 17th, 2017, 8:22 pm

Did you read my links at all? Here is from someone else. Do you know why there are differences?  The answer is yes or no. If no, then just say it.
N = 1000000000
 Baseline (vector storage time) ends. Time 0.458415
 MOOSE start.. ends. Time 5.69154
 STL starts .. ends. Time 4.09148
 BOOST starts .. ends. Time 2.02454
 GSL starts .. ends. Time 5.85985
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Using Quantlib

December 17th, 2017, 8:29 pm

are you talking about accuracy or speed?
 
User avatar
Cuchulainn
Topic Author
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Using Quantlib

December 17th, 2017, 8:32 pm

are you talking about accuracy or speed?
What does the data suggest? Speed, of course, Vergeet nauwkeurigheid voorloopig.

Do you know why there are speed differences? 
Last edited by Cuchulainn on December 17th, 2017, 8:34 pm, edited 1 time in total.
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Using Quantlib

December 17th, 2017, 8:34 pm

yes there are many implementations of MT. The C++11 standard specifies the interface, the required O() of operators, and the exact numerical results (the 10.000th draw must be X). So all MT need to implement the exact same algorithm, but like any C++ aspect each compiler vendor is free to implement things the way they want. 

You can expect different speed for different compiler on different platforms or different STL libraries. All programs should however give the exact same output (accuracy)
 
User avatar
Cuchulainn
Topic Author
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Using Quantlib

December 17th, 2017, 8:36 pm

yes there are many implementations of MT. The C++11 standard specifies the interface, the required O() of operators, and the exact numerical results (the 10.000th draw must be X). So all MT need to implement the exact same algorithm, but like any C++ aspect each compiler vendor is free to implement things the way they want.
OK, clear. Different implementations. hehe
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Using Quantlib

December 17th, 2017, 8:41 pm

yes there are many implementations of MT. The C++11 standard specifies the interface, the required O() of operators, and the exact numerical results (the 10.000th draw must be X). So all MT need to implement the exact same algorithm, but like any C++ aspect each compiler vendor is free to implement things the way they want.
OK, clear. Different implementations. hehe
Yes,.. that goed for *anything*, even the baseline "allocating a std::vector" or sin(x) will be implemented differently by different vendors. If it wasn't there wouldn't be much point in having different compilers. 
 
User avatar
outrun
Posts: 4573
Joined: January 1st, 1970, 12:00 am

Re: Using Quantlib

December 17th, 2017, 8:42 pm

.. but the exact same numbers coming out of the rng!
 
User avatar
Cuchulainn
Topic Author
Posts: 20256
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Using Quantlib

December 18th, 2017, 1:19 pm

.. but the exact same numbers coming out of the rng!
I haven't looked too much, not really my interest nor area. I am assuming RNG stuff is accurate. It is up to RNG folk to tell the rest of us why!
A few quants found speed differences as well.