Serving the Quantitative Finance Community

 
User avatar
ask
Topic Author
Posts: 0
Joined: April 2nd, 2003, 10:37 pm

Memory issue: calling C++ DLL from VBA

April 3rd, 2003, 2:39 am

Hi All.I have a recurring problem with calling very straightforward C++ DLLs from VBA (i.e. from a module in an Excel spreadsheet). Memory allocation for anything more complex than a double -- especially if I'm using the STL -- goes completely awry and causes Excel to shut down (using older versions of Visual Studio causes some kind of access violation). The C++ code works fine when run as a standalone, and the debugger indicates everything is being passed into the DLL correctly. This happens frequently enough that it essentially is halting development of one version of a model, so I must be missing something stupid.Any help?
 
User avatar
asd
Posts: 17
Joined: August 15th, 2002, 9:50 pm

Memory issue: calling C++ DLL from VBA

April 3rd, 2003, 5:04 am

I am guessing if this is because of differences in memory management or automatic garbage collection.VBA defaults pointers as smart pointers that auto-delete when they go out of scope,while C++ needs explicit destruction for pointer objects. - Or due to name mangling/decoration when exporting dll functions?Ideally, it might be best if they could be COM wrapped...asd
 
User avatar
DominicConnor
Posts: 41
Joined: July 14th, 2002, 3:00 am

Memory issue: calling C++ DLL from VBA

April 3rd, 2003, 7:31 am

Is the code small enough to be postable here ?asd's view sounds good. However, this often leads to leaks in memory leaks, so can be quite subtle.ask Turn private messaging on...
 
User avatar
ask
Topic Author
Posts: 0
Joined: April 2nd, 2003, 10:37 pm

Memory issue: calling C++ DLL from VBA

April 3rd, 2003, 1:55 pm

I'd really rather avoid learning COM for this issue -- the time invested doesn't seem worth it. I don't think it's a name mangling issue. To create the DLL, I'm using C++ in Visual Studio with a .DEF file, and everything seems to be called OK. It literally seems that the new operator fails in rather trivial situations (e.g. defining a fixed length array of double), so I could do the error trapping, but I still wouldn't know what magic I should be doing to the heap to make this work. I can step into source code files such as xmemory, but all I learn is that my memory allocation failed. I'll try to produce a representative condensation of the code involved, so I can post it here.
 
User avatar
MarkFogg
Posts: 0
Joined: April 3rd, 2003, 1:40 pm

Memory issue: calling C++ DLL from VBA

April 3rd, 2003, 2:04 pm

COM is an ungainly solution. Memory problems like you describe come from the differences of the basic types (MS has different library implementations of several of the types, that is, different between C++, VB, and VBA). The solution is simple: 1. make a unique C++ dll for each of the types you wish to pass between C++ and VBA. 2. hook up your live data to the C++, and pass the data, and have VBA do something with it, and show you what it did in a dialog box, or text box, or whatever. 3. after verifying that you are passing the data correctly, combine the arguments. 4. if you are passing a structure, or an array of structures, do the same thing, adding a single element at a time.We have had big delays trying to interface from everything into VBA.Cheers!MR
 
User avatar
DominicConnor
Posts: 41
Joined: July 14th, 2002, 3:00 am

Memory issue: calling C++ DLL from VBA

April 3rd, 2003, 9:08 pm

Perhaps I'm misunderstanding you here but...Why don't you just run it under the debugger ? If you did all this building of DLLs, I'm not surprised you had delays.
 
User avatar
asd
Posts: 17
Joined: August 15th, 2002, 9:50 pm

Memory issue: calling C++ DLL from VBA

April 4th, 2003, 4:18 am

QuoteOriginally posted by: askI'd really rather avoid learning COM for this issue -- the time invested doesn't seem worth it. I don't think it's a name mangling issue. To create the DLL, I'm using C++ in Visual Studio with a .DEF file, and everything seems to be called OK. It literally seems that the <b>new</b> operator fails in rather trivial situations (e.g. defining a fixed length array of double), so I could do the error trapping, but I still wouldn't know what magic I should be doing to the heap to make this work. I can step into source code files such as xmemory, but all I learn is that my memory allocation failed. I'll try to produce a representative condensation of the code involved, so I can post it here.If you are in the debug mode in C++, memory management for new/delete will internally try to control/optimisation as a result of which even if you accidentally erase memory from the wrong pointer or the same pointer,I have seen that the error does not come at that point but will come at a very strange random place,but that place of failure is consistent each time you restart the application.One way to know if the error is really in the 'new' statement as you said,is to shuffle your code, or comment out chunks of code before you call this statement.I guessed that since Excel gets crashed,this might be due to access violation. Memory leak do not usually crash the application unless system resources go deplete.Access violation is caused if a pointer memory location is deleted twice,eg. if the pointer is an autoptr,as well as you delete it using delete statement and then it goes out of scope.Hope it helps,asd
 
User avatar
demingz
Posts: 0
Joined: September 26th, 2002, 12:04 pm

Memory issue: calling C++ DLL from VBA

April 4th, 2003, 4:36 am

Could people share their expeirence of COM tech for wrapping C/C++ code for Excel add-in. How hard to learn? Give efficient xll? Memory leak reducing?In gemeral, what is the most efficient way to allow C/C++ interfacing Excele?ThanksDeming
 
User avatar
asd
Posts: 17
Joined: August 15th, 2002, 9:50 pm

Memory issue: calling C++ DLL from VBA

April 4th, 2003, 5:35 am

Hi Demingz, COM could be mainly implemented by either of the methods in C++:- ATL which uses multiple inheritance- MFC which uses nested classesTo actually interface your existing code,you could create an ATL simple object generated by the ATL COM wizard,and the pop-menu allows you to simply add COM methods that get added automatically to the class and IDL file as well.You don't need to go deep into COM knowledge and can get started from day One. After creating the COM methods,let the implementation of the COM methods call your C++ code.It could be made just a pass-through.We had converted huge librarries of our existing code,into a few wrappers of COM interface,in a matter of few days.MFC also has a wizard for creating Active X dlls,but more maual work of pasting the methods in COM class/IDL is required. ATL is also more scalable.One very good thing COM provides is a discipline for memory management,and infrastructure for synchronization issues,which will pay off in the long run.There are laid out rules that who is responsible for destruction of the pointer objects - Client/Server?It is attained by reference counting.When you export out objects directly from C++, the interfacing developer will be wondering whether you destroy the object in the implementation.But with COM,he does not need to.This gives tremendous scalability if the component could simply be put on some other machine for distributed parallel processing.Apartment threading models take care of synchronization issues.Aside from all this, COM has scalablity problems across platforms as compared to CORBA,but grades better in performance.Thanks,asd
 
User avatar
jens
Posts: 0
Joined: July 14th, 2002, 3:00 am

Memory issue: calling C++ DLL from VBA

April 4th, 2003, 8:35 am

Hi Deming,writing an Excel Add-In means to consider two different technologies. a) You write a COM library to make your functions and objects available in eg. Visual Basic or VBASearch google for a good COM introduction AND get a textbook with the details. Memory management can make COM with C++ quite complicated. Another option here is to use C# (or managed C++) and use the .NET COM interop features. This requires to have the .NET framework installed. Given that future Excel versions can use your .NET objects directly, this is probably the best investment.b) You can write a "classic" XLL add-in and make C functions (not objects) available as Excel worksheet functionsHave a look at XLW or libXLL. libXLL allows you to define Worksheet functions in IDL ("COM style") that are also directly accessible in VBA. There is a step-by-step introduction for libXLL to get you going (it references the xlw directory, which must be the libXLL you can download from the above location, or a CVS version of XLW that has libXLL included).Jens.
 
User avatar
demingz
Posts: 0
Joined: September 26th, 2002, 12:04 pm

Memory issue: calling C++ DLL from VBA

April 4th, 2003, 2:04 pm

asd and jens:thank you very much for your input. I will take look at XLW and libXLL.QuoteWe had converted huge librarries of our existing code,into a few wrappers of COM interface,in a matter of few days.asd's experience of converting existing code sounds very appealing. Over the years, we have built a large number of code running on Unix/Linux using gcc/g++ and would like to be able to port them so that Excel can call these functions. Most COM books I have seen focus on building COM from stratch. Could anyone recommand some tutorials on this converting process?Deming
 
User avatar
Willowdog
Posts: 0
Joined: February 3rd, 2003, 1:17 am

Memory issue: calling C++ DLL from VBA

April 10th, 2003, 3:47 pm

For reference, you can call objects from an XLL, you just need to "Extern "C"" the xll wrapper functions.