Serving the Quantitative Finance Community

 
User avatar
Cuchulainn
Posts: 23029
Joined: July 16th, 2004, 7:38 am

Matrix Calculations in Managed Code (eg C#)

June 3rd, 2011, 6:45 am

QuoteOriginally posted by: CurtHagenlocherQuoteOriginally posted by: CuchulainnGovert,Maybe JIT does a lot for you regarding removing bounds checking??QuoteI even changed the lookups in C# to use unsafe pointers in an attempt to remove bounds checking, and it was still slower.It's my understanding that the JIT is pretty good at hoisting or eliminating bounds checks on one-dimensional arrays. It's much less effective at doing so (or doesn't even try) for higher-dimensional arrays.I suppose - like STL vectors - that this memory is all continguous?
 
User avatar
Cuchulainn
Posts: 23029
Joined: July 16th, 2004, 7:38 am

Matrix Calculations in Managed Code (eg C#)

June 3rd, 2011, 6:46 am

QuoteOriginally posted by: shatinanalyticsI use Numerical Algorithms Group's Fortran Library which can be called from C# code.They have about 25 functions related to matrix calculations along with hundreds ofother functions that all calculate very fast.You can find more info at: http://www.nag.co.uk/numeric/FL/nagdoc_ ... ts.htmlHow does it work out? Is is a C++/CLI wrapper? Would it be possible to show (code example) how is it is used in C#?
Last edited by Cuchulainn on June 2nd, 2011, 10:00 pm, edited 1 time in total.
 
User avatar
CurtHagenlocher
Posts: 0
Joined: May 26th, 2010, 12:42 pm

Matrix Calculations in Managed Code (eg C#)

June 3rd, 2011, 12:23 pm

QuoteOriginally posted by: CuchulainnI suppose - like STL vectors - that this memory is all continguous?Yes. And in fact, if you're willing to use unsafe code, you can access it as a block by using Marshal.UnsafeAddrOfPinnedArrayElement to get the address of the first element and then performing indexing yourself. It's a C-style layout. I have no idea whether or not this is guaranteed to be stable in future versions of the CLR.
 
User avatar
shatinanalytics
Posts: 0
Joined: October 19th, 2010, 2:21 am

Matrix Calculations in Managed Code (eg C#)

June 3rd, 2011, 7:28 pm

Look here for examples calling NAG Fortran DLL from C# code:http://www.nag.co.uk/numeric/csharpinfo.asp
 
User avatar
Govert
Posts: 0
Joined: January 28th, 2006, 10:02 am

Matrix Calculations in Managed Code (eg C#)

June 3rd, 2011, 7:56 pm

Ah! That link also confirms that Fortran (and NAG) expects the matrices in column-major form:QuoteIn Fortran two dimensional arrays are stored contiguously in memory by columns while in C# two dimensional arrays of the type [,] are stored contiguously by rows.which explains the double[] representation used in Math.NET.
 
User avatar
Cuchulainn
Posts: 23029
Joined: July 16th, 2004, 7:38 am

Matrix Calculations in Managed Code (eg C#)

June 5th, 2011, 2:37 pm

QuoteOriginally posted by: shatinanalyticsLook here for examples calling NAG Fortran DLL from C# code:http://www.nag.co.uk/numeric/csharpinfo.aspGreat. Thanks.
 
User avatar
Cuchulainn
Posts: 23029
Joined: July 16th, 2004, 7:38 am

Matrix Calculations in Managed Code (eg C#)

June 5th, 2011, 2:49 pm

QuoteOriginally posted by: CurtHagenlocherQuoteOriginally posted by: CuchulainnI suppose - like STL vectors - that this memory is all continguous?Yes. And in fact, if you're willing to use unsafe code, you can access it as a block by using Marshal.UnsafeAddrOfPinnedArrayElement to get the address of the first element and then performing indexing yourself. It's a C-style layout. I have no idea whether or not this is guaranteed to be stable in future versions of the CLR.I am getting the hang of Marshal. To test it, I do:1. AllocHGlobal to allocate unsafe global heap memory.2. Iterate over the array using Marshal.UnsafeAddrOfPinnedArrayElement and Marshal.ReadInt32()So far, so good. The remaining issues are how to process arrays of doubles (use char[]??) and making the code into a black box method with all the alloc/dealoc stuff in place.
Last edited by Cuchulainn on June 4th, 2011, 10:00 pm, edited 1 time in total.
 
User avatar
Govert
Posts: 0
Joined: January 28th, 2006, 10:02 am

Matrix Calculations in Managed Code (eg C#)

June 5th, 2011, 3:26 pm

None of that is normally needed to call the unmanaged functions via P/Invoke - and you certainly need not iterate over an array and pin each element!Here is some discussion: http://stackoverflow.com/questions/2218 ... 32-doe.And some MSDN docs: http://msdn.microsoft.com/en-us/library ... 1).aspx.If the array is needed only for the duration of the call, the runtime marshaler will always pin it and pass the pointer. If you need to get an address that is stable you can pin the whole array using 'fixed' in an unsafe block, or using GCHandle.So for calling the NAG Fortran libraries, you can directly pass a double[] array with the data in column-major order. You see a few of the examples do a transpose of double[,] data to get it from the native row-major layout to coluimn-major form.The P/Invoke marshaling is very powerful - you nearly never need to do any of the conversion or memory management yourself.
 
User avatar
Cuchulainn
Posts: 23029
Joined: July 16th, 2004, 7:38 am

Matrix Calculations in Managed Code (eg C#)

June 5th, 2011, 4:59 pm

Thanks for link.My understanding at this moment:. GCHandle for managed objects. AllocHGlobal is for unmanaged memory which you then access from .NET clientMarshal is a behemoth of a class. QuoteSo for calling the NAG Fortran libraries, you can directly pass a double[] array with the data in column-major order. You see a few of the examples do a transpose of double[,] data to get it from the native row-major layout to coluimn-major form.A clever indexing model should obviate the need to transpose?? In Boost multi-array is possible to switch Rows/Columns.
Last edited by Cuchulainn on June 4th, 2011, 10:00 pm, edited 1 time in total.
 
User avatar
Govert
Posts: 0
Joined: January 28th, 2006, 10:02 am

Matrix Calculations in Managed Code (eg C#)

June 5th, 2011, 8:48 pm

QuoteOriginally posted by: Cuchulainn[...]QuoteSo for calling the NAG Fortran libraries, you can directly pass a double[] array with the data in column-major order. You see a few of the examples do a transpose of double[,] data to get it from the native row-major layout to coluimn-major form.A clever indexing model should obviate the need to transpose?? In Boost multi-array is possible to switch Rows/Columns.I don't think so.Clever indexing (in Boost/C++ or in managed code) won't change what the Fortran library finds in memory once you've passed the pointer.
 
User avatar
Cuchulainn
Posts: 23029
Joined: July 16th, 2004, 7:38 am

Matrix Calculations in Managed Code (eg C#)

June 6th, 2011, 12:01 am

QuoteOriginally posted by: GovertQuoteOriginally posted by: Cuchulainn[...]QuoteSo for calling the NAG Fortran libraries, you can directly pass a double[] array with the data in column-major order. You see a few of the examples do a transpose of double[,] data to get it from the native row-major layout to coluimn-major form.A clever indexing model should obviate the need to transpose?? In Boost multi-array is possible to switch Rows/Columns.I don't think so.Clever indexing (in Boost/C++ or in managed code) won't change what the Fortran library finds in memory once you've passed the pointer.Boost uses array wrappers so that C arrays can work with Fortran arrays so I would expect that it is some referencing that is going, and hopefully not copy. But I can't see that from the docs. Then at adapter level you switch i and j.
Last edited by Cuchulainn on June 5th, 2011, 10:00 pm, edited 1 time in total.
 
User avatar
MarcusCuda
Posts: 0
Joined: July 30th, 2004, 11:48 am

Matrix Calculations in Managed Code (eg C#)

June 7th, 2011, 8:35 am

QuoteOriginally posted by: GovertI presume they eliminate the checks in other functions within the library.But the matrix class would need to expose some accessor that allows the library user to get to the individual entries. There one probably should check, and this is the code you are seeing.Right, internally we don't use the indexer. We either use the At method or work on the array directly.