SERVING THE QUANTITATIVE FINANCE COMMUNITY

  • 1
  • 2
  • 3
  • 4
  • 5
  • 10
 
User avatar
jcrew
Topic Author
Posts: 32
Joined: February 8th, 2007, 10:30 pm

Speed testing between C++, C# and VB

December 20th, 2007, 4:28 am

I created two sets of benchmark applications to test the speed performance between the 3 languages. I would expect the order from fastest to slowest would be C++ > CSharp > VB.The first app was a discrete fourier transform implementation to test the floating point performance. Initial results was that VB was marginally faster than CSharp and both were order of magnitude faster than C++. After some investigation, the trig functions in the <cmath> lib was causing the slowdown and after replacing it with a simple trig table lookup, C++ came out as the fastest.The 2nd app was a quicksort implementation to test recursion performance. The results were that VB is a little faster than CSharp which are both an order of magnitude faster than C++. I'm rather stumped on this one. Are stack ops that much slower in C++ in comparison to VB or CSharp?
 
User avatar
rector
Posts: 97
Joined: July 14th, 2002, 3:00 am

Speed testing between C++, C# and VB

December 20th, 2007, 10:09 am

If you are using Visual C++, make sure that the "target" is "Release" and not "Debug" (default). For a proper native code the presense of symbolic information necessary for debugging tends to slow down the code by an order of magnitude. For .NET it doesn't make difference as long as you don't run the program from the debugger. If you are using C++.NET, also make sure that you compile it as native C++ code and not C++/CLI.C++ is extremely efficient at using stack but heap allocations may be slower compared to .NET depending on how you handle them.
 
User avatar
jcrew
Topic Author
Posts: 32
Joined: February 8th, 2007, 10:30 pm

Speed testing between C++, C# and VB

December 20th, 2007, 2:48 pm

All the tests were released mode and the CLI option was off for C++. Also heap allocation operations happens prior to the measured start time.
Last edited by jcrew on December 19th, 2007, 11:00 pm, edited 1 time in total.
 
User avatar
Y0da
Posts: 387
Joined: August 7th, 2007, 4:48 pm

Speed testing between C++, C# and VB

December 20th, 2007, 5:26 pm

Strange that C++ isn't faster than that. More comments on thisone would be appreciated.
 
User avatar
Cuchulainn
Posts: 59944
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Speed testing between C++, C# and VB

December 20th, 2007, 8:34 pm

QuoteOriginally posted by: jcrewAll the tests were released mode and the CLI option was off for C++. Also heap allocation operations happens prior to the measured start time.Could you post the code so we can test it? We must compare apples and apples!
 
User avatar
Cuchulainn
Posts: 59944
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Speed testing between C++, C# and VB

December 22nd, 2007, 12:51 pm

If the full code is not possible, scope it _down_ while still keeping in the essential performance bottlenecks. ONE important thing is the quality of YOUR code, by which I mean if it has been optimised. In one example, I brought the execution time from 99 sec to 16 sec in C++ just by avoiding the creation of temporary objects! So, that's another factor in the analysis.
Last edited by Cuchulainn on December 21st, 2007, 11:00 pm, edited 1 time in total.
 
User avatar
jcrew
Topic Author
Posts: 32
Joined: February 8th, 2007, 10:30 pm

Speed testing between C++, C# and VB

December 22nd, 2007, 3:11 pm

This is the C++ implementation
 
User avatar
jcrew
Topic Author
Posts: 32
Joined: February 8th, 2007, 10:30 pm

Speed testing between C++, C# and VB

December 22nd, 2007, 3:12 pm

This is the VB implementation
 
User avatar
jcrew
Topic Author
Posts: 32
Joined: February 8th, 2007, 10:30 pm

Speed testing between C++, C# and VB

December 22nd, 2007, 3:20 pm

As you can notice the dataset is randomly generated, but if run each of the code several times, you can begin to see the mean/distribution of the results of each test. During my testing, the results were far apart enough that you can make a conjecture which one is faster after a few tests. I ran the code in Release mode. Also I was trying create comparable code, so I didn't not try to optimize.
Last edited by jcrew on December 21st, 2007, 11:00 pm, edited 1 time in total.
 
User avatar
Wibble
Posts: 426
Joined: January 23rd, 2004, 3:15 pm

Speed testing between C++, C# and VB

December 24th, 2007, 7:21 am

hmm ... passing by reference and passing by value
 
User avatar
Cuchulainn
Posts: 59944
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Speed testing between C++, C# and VB

December 24th, 2007, 9:49 am

jcrew.So this test is to create a random array of size 10^7 and then sort it? There are two issues 1) memory allocation and 2) function callingQuotehmm ... passing by reference and passing by value The code look fine in this case, yes? Arrays by ref and atomic types by Val is the accepted practice.
Last edited by Cuchulainn on December 23rd, 2007, 11:00 pm, edited 1 time in total.
 
User avatar
ExSan
Posts: 4548
Joined: April 12th, 2003, 10:40 am

Speed testing between C++, C# and VB

December 24th, 2007, 12:35 pm

QuoteOriginally posted by: CuchulainnCould you post the code so we can test it? We must compare apples and apples!are we talking about the same apples?The same random array should be used with c++, c#, VB. shouldn't it?
 
User avatar
Cuchulainn
Posts: 59944
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Speed testing between C++, C# and VB

December 24th, 2007, 12:47 pm

QuoteOriginally posted by: robsan7QuoteOriginally posted by: CuchulainnCould you post the code so we can test it? We must compare apples and apples!are we talking about the same apples?The same random array should be used with c++, c#, VB. shouldn't it?You mean we should use exactly the same data for each language-specific program? VB might be faster but what about accuracy RAND() etc.But for this test I reckon the accuracy does not matter, just the performance??? So, we use Box-Muller (Granny Smith) or MT (Cox's pippins). So, choose the same scheme on each platform. Eventually, it all boils down to how good sin(), cos() and exp() are implemented. jcrew,Could you specify the problem in words? It is still not clear.
Last edited by Cuchulainn on December 23rd, 2007, 11:00 pm, edited 1 time in total.
 
User avatar
jcrew
Topic Author
Posts: 32
Joined: February 8th, 2007, 10:30 pm

Speed testing between C++, C# and VB

December 24th, 2007, 3:49 pm

I'm trying to test the speed of recursion performance between the languages using a standard Quicksort algorithm. Before the test, I naturally assume C++ would be far faster than VB or CSharp. However my results show that VB is about twice as fast as VC++. Wondered if anyone had an explanation. Does the framework of .net somehow optimize function calls? I don't know.Ideally we would want to operate on the same data set, but if you run the test several times you can see the difference between distribution of the performance quickly enough, so I don't think it's really an issue. Memory allocation happens before the start time was recorded so I think that excludes it. Maybe heap access is somehow slower? I wouldn't think so, but could be a caching issue.
Last edited by jcrew on December 23rd, 2007, 11:00 pm, edited 1 time in total.
ABOUT WILMOTT

PW by JB

Wilmott.com has been "Serving the Quantitative Finance Community" since 2001. Continued...


Twitter LinkedIn Instagram

JOBS BOARD

JOBS BOARD

Looking for a quant job, risk, algo trading,...? Browse jobs here...


GZIP: On