Serving the Quantitative Finance Community

 
User avatar
MCarreira
Topic Author
Posts: 64
Joined: January 1st, 1970, 12:00 am

Choosing hardware/software for QF

March 25th, 2007, 11:22 pm

The time has come for me to upgrade my computer, and I'd like some advice on hardware/software.1. SoftwareI've been using Mathematica almost exclusively, but I keep wondering whether I really should put the effort and learn C++ to run MC and risk analysis faster.Questions are:1.1 Is it worth ?1.2 If yes, should I go and buy Visual Studio or something like that ? Is there another alternative for a professional-like use of C++ ?I've also got PCTeX, but could not make the Mathematica-TeX integration work as smoothly as it should.1.3 Is there a better alternative for a TeX editor (I've got some books about TeX/LaTex, but havent had the time yet to study them) ?Although what I've seen from Mathematica 6 (better interface for coding, the command Manipulate letting you change multiple parameters on sliding scales, and better treatment for dates) seems good, I'd also like to know:1.4 Is it worth learning and using another software to control input/output or to wrap calculations (I do have Mathematica Link for Excel, I'd like to hear about alternatives).2. HardwareAssuming that I'll be using some or all of the stuff above, should I even consider switching to Mac's ? My intended configuration is a desktop with a QuadCore processor, 4GB RAM and a laptop with DualCore and 2GB RAM.Thanks in advance for the help.
 
User avatar
kilimanjaro
Posts: 0
Joined: March 23rd, 2007, 3:23 am

Choosing hardware/software for QF

March 26th, 2007, 6:03 am

This depends completely on details you have omitted.You are obviously going to be limited on hardware (it's not like you have a grid you can farm processes off to). That alone is reason enough to learn a language like C++, because otherwise you end up cutting into your throughput. But it doesn't have to be C++. You may find that using C# or Java suits you better. At the very least, you need to consider a) what sort of support you are going to be getting (this could be as minimal as asking for help via the internet), b) the overhead of learning as well as the actual efficiency with which you can use the languge, and c) whatever constraints you have on run-time. These are all factors that only you can properly evaluate.If you are developing on Windows, then go ahead and get Visual Studio. If you are using linux, you can do professional work with g++ and whatever ad-hoc toolset you throw together. People will probably laugh at me when I say that I run Debian and use Emacs for all development. Really though, the choice of tools depends on the user. I certainly wouldn't recommend learning emacs to somebody who wants to hit the ground running in a production environment. Visual Studio is certainly more intuitive. On the other hand, Emacs does have a decent LaTeX mode....There's no real reason to buy a Mac, unless you want a unix system with out of the box support for expensive hardware. You will need to learn a whole new system, and jumping ship without first evaluating your current workflow would be silly. Go ahead and buy a commodity PC, spend the extra cash on some beer.
 
User avatar
exneratunrisk
Posts: 0
Joined: April 20th, 2004, 12:25 pm

Choosing hardware/software for QF

March 26th, 2007, 6:41 am

As mentioned in other threads, we have made good experiences with Mathematica+Java+C++ development(C++ for the numerical engine).For the organisation of this hybrid development we use the Wolfram workbench.But, the workbench, great for Mathematica and Java development, is not preferrable for C++ (it is based on Eclipse and therefore inherit this "weakness").For a quadcore computer you might want to look into Mathematica Personal Grid Edition (it let you easily exploit 4 cores in parallel).When you use Mathematica as platform, you are cross platform. The runtime plattform shall depend on the segments you want to deploy your solutions to?We like Macs and Mac OS X very much, but our products for the market are Windows based. Most of our grid based solutions in banks are implemented in racks with Dell computers (selection of our customers).We shall expect 8 core PCs soon?
 
User avatar
MCarreira
Topic Author
Posts: 64
Joined: January 1st, 1970, 12:00 am

Choosing hardware/software for QF

March 26th, 2007, 8:46 am

About the details, the development I want to do is more of a personal let's-try-to-do-it-myself rather than I-need-to-deploy-this-for-the-bank stuff; I heard about the Mathematica Personal Grid Edition, not sure yet if it's worth the price.In my experience, I like to use Mathematica for the symbolical stuff and quick-to-code numerical stuff, but for running MC simulations where I'd like to see risk profiles during the life of the portfolio it seems to take too long, that's why I was thinking about C++.So is getting Visual C++ Express Edition good enough ? Is the Microsoft Visual J# Express Edition a good choice for Java development ?Thanks for the answers.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Choosing hardware/software for QF

March 26th, 2007, 9:51 pm

The decision to move to C++ is non-trivial. Your code will be much faster in C++, but your development cycle will be slower. The differential productivity gains will depend on the ratio of runs per model versus models per year. If you tend to develop a new model only occasionally and do extensive studies with each model (i.e., lots of runs with different parameter values, data inputs, etc.), then the CPU time/run advantage of C++ will win. If you tend to try lots of models, doing only a few runs with each model before changing the code, then Mathematica will offer a better developer-time/model advantage.Have you (or anyone) looked at Mathcode C++? At $3,800, its not cheap, but then your time isn't cheap either. It might be a way to leverage your familiarity with Mathematica (and previously developed code) as part of a C++ development process.I know everyone always rolls their eyes when someone says buy tech++ coming out in t+delta, but you might consider waiting for dual quadcore PCs to be more common. An eightcore system would surely be a nice performance boost, assuming you take the time to learn the finer points of multithreaded code (or buy Mathematica Personal Grid Edition). And if you want a real challenge and don't mind spending lots of time with your fingers in the silicon, then look at running your MC on a GPU ( http://www.gpgpu.org/ )You can save money, save development time, or save run time -- pick any 2.
 
User avatar
MCarreira
Topic Author
Posts: 64
Joined: January 1st, 1970, 12:00 am

Choosing hardware/software for QF

March 27th, 2007, 1:07 am

Traden4Alpha says:The decision to move to C++ is non-trivial.Yes, that is why I put this question here ... I really don't have any good experience in C++ coding, and from what I see it takes a lot of dedication to really become a good C++ programmer; and because I'm not a professional developer, but someone who likes to simulate stuff and control what goes on within the models, I was wondering whether it would be worth dedicating the time to follow the C++ route.Have you (or anyone) looked at Mathcode C++?Not really, just checked it out ... it seems to be what I wanted: development of a model in Mathematica (with quick checking of results) and the heavy lifting done in C++ code execution; the price tag is impressive indeed, but I agree that time is also expensive.you might consider waiting for dual quadcore PCs to be more commonYes, I was just monitoring how the price of a QuadCore chip fell enough that it could be a reasonable purchase compared with a DualCore... I'm probably better upgrading my laptop now and planning my desktop with more time, and look at the possibility of using Personal Grid.look at running your MC on a GPUThat seems a good challenge ... it seems that I'll have to follow farmer's advice on the What you don't know about C++ thread and look at what goes on inside the hardware (a bit at least) in order to really be able to get more productivity.Thanks for the advice and the new information,MCarreira
 
User avatar
Cuchulainn
Posts: 22929
Joined: July 16th, 2004, 7:38 am

Choosing hardware/software for QF

March 27th, 2007, 2:33 pm

You use OpenMP for C++ and Fortran (it's free) to incrementally and non-intrusively port sequential code to parallel code. And the threading stuff is taken care of by the OS. On the other hand, s you say, you are beginner so I would not worry about all this high-falutin' stuff, just yet. Learn the fundamentals first.
Last edited by Cuchulainn on March 26th, 2007, 10:00 pm, edited 1 time in total.
 
User avatar
MCarreira
Topic Author
Posts: 64
Joined: January 1st, 1970, 12:00 am

Choosing hardware/software for QF

March 27th, 2007, 3:30 pm

Cuchulainn says:Learn the fundamentals firstIn my particular case, I wanted to:1. Take advantage of the availability of multiprocessing today to improve performance, and2. Get better execution times (Mathematica => C++)What is the path that you would recommend ? When you say fundamentals, you mean C++ fundamentals ?Thanks for the answer.
 
User avatar
Cuchulainn
Posts: 22929
Joined: July 16th, 2004, 7:38 am

Choosing hardware/software for QF

March 28th, 2007, 6:05 am

QuoteOriginally posted by: MCarreiraCuchulainn says:Learn the fundamentals firstIn my particular case, I wanted to:1. Take advantage of the availability of multiprocessing today to improve performance, and2. Get better execution times (Mathematica => C++)What is the path that you would recommend ? When you say fundamentals, you mean C++ fundamentals ?Thanks for the answer.Well, here is one 'level' of fundamentals:blogOn the other hand, for parallel work one must design a system with 'parallel in mind' in order to get optimal performance. Porting spaghetti code will be difficult IMO (because no possibility for divide and conquer techniques).I think that other skills are needed in this situation when compared to sequential programming.Again, OpenMP is a nice start, is free and the learning curve is not so steep.
Last edited by Cuchulainn on March 27th, 2007, 10:00 pm, edited 1 time in total.
 
User avatar
DominicConnor
Posts: 41
Joined: July 14th, 2002, 3:00 am

Choosing hardware/software for QF

March 28th, 2007, 8:56 am

Cuch, do you know of a good text book on s/w design for multi threading/multi core systems ?I've got various CS books on parallel programming, and some that focus on the relatively low level primitives for creating and managing threads, but no work on the design process. Also all I find are scattered web articles on debugging as well.
 
User avatar
Cuchulainn
Posts: 22929
Joined: July 16th, 2004, 7:38 am

Choosing hardware/software for QF

March 28th, 2007, 10:12 am

QuoteOriginally posted by: DCFCCuch, do you know of a good text book on s/w design for multi threading/multi core systems ?I've got various CS books on parallel programming, and some that focus on the relatively low level primitives for creating and managing threads, but no work on the design process. Also all I find are scattered web articles on debugging as well.Here are some OK ones IMO. Pthreads programming by Nichols et al (concepts well-explained and C code). Parallel prog ..... in OpenMP by Chandra et al. Patterns for parallel programming Mattson et alThese are mentioned on the OpenMP, HPC and MPI threads if my memory serves me well.These are a good mix of C/C++ and design issues.
Last edited by Cuchulainn on March 27th, 2007, 10:00 pm, edited 1 time in total.
 
User avatar
MCarreira
Topic Author
Posts: 64
Joined: January 1st, 1970, 12:00 am

Choosing hardware/software for QF

March 28th, 2007, 11:36 am

Cuchulainn,Thanks for the answers.Another question then: How the crescent availability of multicore processors will change the existing implementations ? Will institutions rewrite their code to take advantage of this on a desktop level ?
 
User avatar
Cuchulainn
Posts: 22929
Joined: July 16th, 2004, 7:38 am

Choosing hardware/software for QF

March 28th, 2007, 1:28 pm

QuoteHow the crescent availability of multicore processors will change the existing implementations ? Will institutions rewrite their code to take advantage of this on a desktop level ? MCThere are lots of scenarios I suppose. It reminds me a bit when the world moved from DOS to Windows, 16 to 32 bits. What happened? Object orientation (MFC !) became awful popular (esp. inheritance). Some supermarket kassa systems still use DOS. On an aside, it was really Microsoft that made OO popular as our organisation experienced first hand in the mid-90's, i.e. lots of courses + app migrations.Not being able to predict the fuure, but one thing is sure: you will need to 'think/design parallel' before programming. Large monolithic apps will be difficult to 'parallelize' in the future just as they are difficult to upgrade by inserting low-level design patterns. Analogously; it's like wanting to upgrade a bungalow to a 2-story or vice versa (some people actually knock down their house(really)) and build again, in the long term it's cheaper pm) I think HPC developers will be much in demand in the next 5-10 years.What do others think?
Last edited by Cuchulainn on March 27th, 2007, 10:00 pm, edited 1 time in total.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Choosing hardware/software for QF

March 28th, 2007, 2:03 pm

QuoteOriginally posted by: CuchulainnQuoteHow the crescent availability of multicore processors will change the existing implementations ? Will institutions rewrite their code to take advantage of this on a desktop level ? I think HPC developers will be much in demand in the next 5-10 years.What do others think?I agree with Cuchulainn. In the past, developers could depend on Intel et al to hide Microsoft's and developer's sins of inefficient bloated code. More MHz compensated for a multitude of sloppy design decisions. But the shift to multiple cores changes everything because the cores don't automatically deliver speed without smart HPC-like design. Every application that does real crunching (not just calls to OS-provided media processing) will need redesigning using HPC principles.I wonder if anyone has created a CASE/IDE tool that helps slice and dice a design for parallelism? A tool that lets the designer define potential objects, estimated cardinalities of instances, temporal dependencies, and use case frequencies would help the identify the top opportunities for using multiple cores.
 
User avatar
Cuchulainn
Posts: 22929
Joined: July 16th, 2004, 7:38 am

Choosing hardware/software for QF

March 28th, 2007, 2:26 pm

QuoteI wonder if anyone has created a CASE/IDE tool that helps slice and dice a design for parallelism? A tool that lets the designer define potential objects, estimated cardinalities of instances, temporal dependencies, and use case frequencies would help the identify the top opportunities for using multiple cores. I think design knowledge is in the heads of the few 100 HPC experts on the planet. Seems we are in for a new learning spiral and in a few years we will be talking about how we developed 'my first Master-Slave' application on a GPU. Then the folklore will become a method (the way GOF patterns emerged).QuoteBut the shift to multiple cores changes everything because the cores don't automatically deliver speed without smart HPC-like design.This is now like a law of physics. Bad designs will run slow.Disclaimer: the crystal balls I use were developed some time ago, so the rules may have changed.
Last edited by Cuchulainn on March 27th, 2007, 10:00 pm, edited 1 time in total.