Serving the Quantitative Finance Community

 
User avatar
Cuchulainn
Topic Author
Posts: 22931
Joined: July 16th, 2004, 7:38 am

Parallel Processing in C++

July 10th, 2006, 7:42 am

I am looking for standards for parallel and shared memory models. I know of two standards, MPI and OpenMP.In principle the software should run on a Windows desktop. Linux interoperability is an added bonus. The objective is to run parts of a program in parallel.Any ideas?Thanks
 
User avatar
blade
Posts: 0
Joined: June 18th, 2002, 3:28 pm

Parallel Processing in C++

July 10th, 2006, 10:35 am

Hi Cuch, Have a look at the MPICH libraries. They are free, work cross-platform and at the last stage I used them, had relatively low latency in the message passing stage. MPI libraries usually tend to be machine specific, and will usually run faster, but are hard to port between platforms or even sometimes the same platform but a newer OS. Blade
 
User avatar
Cuchulainn
Topic Author
Posts: 22931
Joined: July 16th, 2004, 7:38 am

Parallel Processing in C++

July 10th, 2006, 11:14 am

QuoteOriginally posted by: bladeHi Cuch, Have a look at the MPICH libraries. They are free, work cross-platform and at the last stage I used them, had relatively low latency in the message passing stage. MPI libraries usually tend to be machine specific, and will usually run faster, but are hard to port between platforms or even sometimes the same platform but a newer OS. BladeThanks Blade,At one stage I remember that MPIOO? created an OO wrapper for MPICH but I am sure if this is a blessing or not. Calling MPICH from C++ might be just as easy.
 
User avatar
Athletico
Posts: 14
Joined: January 7th, 2002, 4:17 pm

Parallel Processing in C++

July 11th, 2006, 2:16 pm

Cuch, you might be interested in a new Intel library: Intel Threading Building BlocksIt includes lock-free containers (C++ templates), obviating the need for most expensive mutex operations. Some ACE-like frameworky stuff for parallel applications too -- it's highly optimized for multi-CPU & multi-core systems. Looks pretty slick.
 
User avatar
zeta
Posts: 26
Joined: September 27th, 2005, 3:25 pm
Location: Houston, TX
Contact:

Parallel Processing in C++

July 12th, 2006, 11:56 am

QuoteIn principle the software should run on a Windows desktop. Linux interoperability is an added bonus.you want PVM. I use it for simple data decomposition and don't honestly know how it compares to MPI... PVM isn't fussed on the details of your machines/network and is very good for a patchwork quilt. I used it with a homebrew beowulf (just 4 machines) and simply 100baseT ethernet; worked great. Also has nice gui, XPVM
 
User avatar
alexv
Posts: 0
Joined: March 14th, 2005, 9:45 pm

Parallel Processing in C++

July 12th, 2006, 3:23 pm

QuoteOriginally posted by: zetaQuoteIn principle the software should run on a Windows desktop. Linux interoperability is an added bonus.you want PVM. I use it for simple data decomposition and don't honestly know how it compares to MPI... PVM isn't fussed on the details of your machines/network and is very good for a patchwork quilt. I used it with a homebrew beowulf (just 4 machines) and simply 100baseT ethernet; worked great. Also has nice gui, XPVMI second that. I used PVM to build large distributed system consisting of dozens of Windows, Linux and Solaris machines. I used C++ wrapper library called CPPvm which makes life so much easier. You can, for instance, use C++ streams to transparently transmit data across the network.
 
User avatar
stuntprogrammer
Posts: 0
Joined: July 10th, 2005, 7:23 pm

Parallel Processing in C++

July 13th, 2006, 10:56 am

If you are writing new code, you would be better off to use MPI. Its far more widely used and supported for cluster programming. When Microsoft entered the cluster computing market recently, they confirmed its popularity by creating their own version (based on mpich iirc) for use from C/C++/Fortran77/Fortran90. MPI-1 had an official non-standard C++ binding (ie it was in an appendix and not require). MPI2 has an official standard binding as part of the spec. You also get dynamic process management, paralllel io and remote memory access support. OOMPI is favored by some people.. but I don't particularly care.btw. I say this as someone who used pvm a lot in the 90s, and wrote a fault tolerant cluster programming lib for C++ based on it.Vaguely related. I'm curious as to the use of not-quite-high-performance computing. I know some banks have large clusters for monte carlo type apps. How about the next level down at the team or dept. level. Say 4 8-core opteron boxes or some such. What are people using for high-performance coding on this smaller scale?
 
User avatar
pascalroca
Posts: 0
Joined: October 19th, 2005, 6:45 pm

Parallel Processing in C++

July 19th, 2006, 6:56 am

Last edited by pascalroca on July 18th, 2006, 10:00 pm, edited 1 time in total.
 
User avatar
Galerkin
Posts: 0
Joined: February 1st, 2006, 11:32 am

Parallel Processing in C++

July 19th, 2006, 10:10 am

I back MPI without doubts ..pvm is something of the 90s when MPI standards were still being developed. C++ code written using MPI functions is very portable, I would recommend lam-mpi:http://www.lam-mpi.org/You can even run in parallel on your laptop to debug applications...if you have got a dual processor you will enjoy it a lot.However if you consider using windows platforms you might want to have a look at MPICH http://www-unix.mcs.anl.gov/mpi/mpich1/index.htmAs for those new to parallel computing using MPI, I would suggest to have a look at Peter Pacheco's guide to MPI, which is in my opinion the most effective intro to start programming in parallel.Regards,M.
 
User avatar
batesh
Posts: 0
Joined: December 9th, 2005, 12:29 pm

Parallel Processing in C++

July 19th, 2006, 11:15 am

Slightly off the specific C++ question...What to use is oftentimes driven by what level the parallelism is aimed at. At a general level there could be a distributed grid network where if someone wants to run an Excel simulation then the grid will run the job on the least used machine, even if it is already used by something else.At the task farming level so beloved of graphics ray tracing you break the task into chunks that do the same thing but return different results, like a Monte Carlo simulation. Split the jobs over 20 computers and then amalgamate the results. This could be done a grid which would require some Parallel library API coding but usually very little.At the real deep code level by splitting up a routine, loop unrolling etc.. Sometimes you can get by just using such language elements as FORALL (from my High Performance Fortran days) which would transparently split the FORALL loop onto multiple machines.Also, beware of applying parallelism to everything. In some computational fluiud dynamics code I was putting into parallel, of the 16,000 lines+ of code it was one routine covering about 10 lines performing a Pentadiagonal Matrix Elimination algorithm that was taking up 80% of the computer time in a real simulation. Spending time on the other parts of the code would have been effort for little reward.The real benefit of using something like MPI/PVM is at the deep code level or when there is no grid network setup so more grid related calls have to be made to execute jobs on other machines etc...I hope that this helps...
 
User avatar
Cuchulainn
Topic Author
Posts: 22931
Joined: July 16th, 2004, 7:38 am

Parallel Processing in C++

September 13th, 2006, 1:25 pm

QuoteI'm going to be very interested to see how people cope with the shift to multicore, wider/better vector units and the concomitant dependence on parallelism for speedups rather than waiting 18 months for a faster single threaded chip. Not only hardware but also the software design. An example is to design a 3-factor PDE model in a parallel way.
Last edited by Cuchulainn on September 12th, 2006, 10:00 pm, edited 1 time in total.
 
User avatar
lballabio
Posts: 0
Joined: January 19th, 2004, 12:34 pm

Parallel Processing in C++

September 13th, 2006, 1:42 pm

On a related note, the Boost community recently started the review of a proposed Boost.MPI library---which, if accepted into Boost, might be likely to bias the C++ community towards MPI...Luigi
 
User avatar
Cuchulainn
Topic Author
Posts: 22931
Joined: July 16th, 2004, 7:38 am

Parallel Processing in C++

September 13th, 2006, 1:57 pm

QuoteOriginally posted by: lballabioOn a related note, the Boost community recently started the review of a proposed Boost.MPI library---which, if accepted into Boost, might be likely to bias the C++ community towards MPI...LuigiSounds good. What's the main advanatage when compared to 'plain' MPI?Most C++ programmers work in single threads I suppose while HPC is well-established.
 
User avatar
lballabio
Posts: 0
Joined: January 19th, 2004, 12:34 pm

Parallel Processing in C++

September 13th, 2006, 2:12 pm

QuoteOriginally posted by: CuchulainnSounds good. What's the main advanatage when compared to 'plain' MPI?I haven't looked into it, so I'll just copy the blurb from the review announcement:QuoteAlthough there exist C++ bindings for MPI, they offer little functionality over the C bindings. The Boost.MPI library provides an alternative C++ interface to MPI that better supports modern C++ development styles, including complete support for user-defined data types and C++ Standard Library types, arbitrary function objects for collective algorithms, and the use of modern C++ library techniques to maintain maximal efficiency. Luigi
Last edited by lballabio on September 12th, 2006, 10:00 pm, edited 1 time in total.
 
User avatar
Cuchulainn
Topic Author
Posts: 22931
Joined: July 16th, 2004, 7:38 am

Parallel Processing in C++

September 13th, 2006, 2:15 pm

Luigi,Sounds good (EDIT: as well). In the past we did some MPI in C++ but indeed a lot was very close to C. Now Boost will let us use Modern C++. The data types are crucial.DD
Last edited by Cuchulainn on September 12th, 2006, 10:00 pm, edited 1 time in total.