SERVING THE QUANTITATIVE FINANCE COMMUNITY

 
User avatar
DominicConnor
Topic Author
Posts: 11684
Joined: July 14th, 2002, 3:00 am

Better Programming Environment

April 13th, 2006, 9:33 am

Someone asked me a hard question yesterday, and I'm blowed if I could think of a good answer.If you had Windows XP as a baseline, what would you have in Vista that would make it a better environment to write code ?There's obvious cheap shots like support Linux of course, but you can now run Linux in a virtual m/c so that's not really it.I couldn't actually think of anything except the virtualisation, and even that's not a very new thing.
Last edited by DominicConnor on April 12th, 2006, 10:00 pm, edited 1 time in total.
 
User avatar
nocturne2
Posts: 150
Joined: January 11th, 2006, 5:45 pm

Better Programming Environment

April 13th, 2006, 11:15 am

Without a doubt: better shell. The cmd shell is pretty lame. Adopt bash/cygwin - make it first-class extension of the kernel.
 
User avatar
RGB
Posts: 19
Joined: April 11th, 2006, 10:14 pm

Better Programming Environment

April 13th, 2006, 12:22 pm

Ok, I'll take a shot. According to the rumors, the Vista release has been pushed back due to many problems encountered and some of the executives in charge of this release have been fired. One rumor has it that Vista is not very backward compatible with many hardware and software solutions. So, I am hoping that MS gets their act together and expends the necessary resources to completely test out backward compatibility so that we don't all have to rush out and buy new hardware and software.Visual Studio .NET is MS latest platform to develop code for the Windows enviroment. I believe that this includes Vista as well as XP. A better question is probably:What changes would you make in the next version of Visual Studio .NET to make it a better environment to write code?Before I answer this, let me provide some background on how MS handles memory in .NET. Right now, there are 2 different ways of handling memory allocation. One way is to allocate the memory on the stack. This is very efficient and is used for the built-in types like int, float, double, String, DateTime, etc. and structures defined with the Stuct keyword. The other way is to allocate the memory on the heap. This is done for all objects defined with the Class keyword. A structure can be created using either the Class keyword or the Struct keyword. If the Struct keyword is used, then the structure can only be used within the function that defines it. If the structure is passed to another function, then a "boxing" operation occurs where the structure is copied to the heap and a reference to the object on the heap is then passed to the called function. It is my belief that MS implemented it this way so that it could avoid some of the pitfalls of COM's reference counting scheme. Since boxing is a very inefficient operation, most .NET houses would be well advised to avoid using Structs entirely.In C++, the developer has control over whether a class or variable is instantiated on the heap or the stack. In C++ objects declared on the stack call the object's destructor when the function exits. If a reference to this object is stored in a table or passed to another thread, then this reference becomes invalidated when the function exits. This is a major source of bugs in C++ programs. For C++ objects defined on the heap, it is the developer's responsibility to delete the object at the appropriate time. This is also another major source of bugs. Mostly because the developer forgets to delete the object which results in memory leaks that eventually force a reboot of the computer. But there are also times when the developer deletes the object too soon and when some forgotten little used function tries to access the reference, it crashes the program. So, the .NET platform uses a fairly efficient garbage collector that keeps track of all memory created on the heap and collects it when all references to that memory have gone out of scope. Java also uses a garbage collector and the garbage collector is probably the most significant improvement for developing more reliable code because it reliably releases memory when it no longer is used and all of the C++ pitfalls dealing with memory deletion are avoided.Now, the one thing that I would really like to see in the next version of Visual Studio .NET is the capability to treat the built-in types as references. Why is that? When a reference to an object is passed from function to function, the entire memory behind that object is on the heap. All that is being passed around is the address to the object on the heap. When the object is stored in a table, the address is all that is stored. So, if some other fucntion modifies that object and later down the road another function that has the reference to that object needs the latest values for the object - no problem because the object is stored on the heap and the function has the reference (address) to it. However, lets say that you want to do the same thing with a built-in type like an int. Can it be done? Well, since built-in types are always using the value and not the address - there is no way to do this gracefully. However, it can be done in another way, but its somewhat of a headache. There is no way to specify that the address of an int be used instead of its value (without using pointers and using unsafe code in C#; for VB, I don't think it can be done at all). However, the int can be wrapped in a class of its own and then accessed and set with get/set functions. But, this is an ugly kluge. Hopefully MS will take a clue from Java. Java has a duplicate list of built-in types that can be used to treat built-in types as references.
 
User avatar
Cuchulainn
Posts: 61574
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Better Programming Environment

April 13th, 2006, 2:39 pm

QuoteBefore I answer this, let me provide some background on how MS handles memory in .NET. Right now, there are 2 different ways of handling memory allocation. One way is to allocate the memory on the stack. This is very efficient and is used for the built-in types like int, float, double, String, DateTime, etc. and structures defined with the Stuct keyword. The other way is to allocate the memory on the heap. This is done for all objects defined with the Class keyword. A structure can be created using either the Class keyword or the Struct keyword. If the Struct keyword is used, then the structure can only be used within the function that defines it. If the structure is passed to another function, then a "boxing" operation occurs where the structure is copied to the heap and a reference to the object on the heap is then passed to the called function. It is my belief that MS implemented it this way so that it could avoid some of the pitfalls of COM's reference counting scheme. Since boxing is a very inefficient operation, most .NET houses would be well advised to avoid using Structs entirely.In C++, the developer has control over whether a class or variable is instantiated on the heap or the stack. In C++ objects declared on the stack call the object's destructor when the function exits. If a reference to this object is stored in a table or passed to another thread, then this reference becomes invalidated when the function exits. This is a major source of bugs in C++ programs. For C++ objects defined on the heap, it is the developer's responsibility to delete the object at the appropriate time. This is also another major source of bugs. Mostly because the developer forgets to delete the object which results in memory leaks that eventually force a reboot of the computer. But there are also times when the developer deletes the object too soon and when some forgotten little used function tries to access the reference, it crashes the program. So, the .NET platform uses a fairly efficient garbage collector that keeps track of all memory created on the heap and collects it when all references to that memory have gone out of scope. Java also uses a garbage collector and the garbage collector is probably the most significant improvement for developing more reliable code because it reliably releases memory when it no longer is used and all of the C++ pitfalls dealing with memory deletion are avoided.I find this a bit confusing. Are you talking about ISO C++ or C++/CLI? . C++ is a chain saw that can cut your leg off; we know that memory management is difficult. That is not front-page news anymore. This is not a reason to seek refuge in a language that does GC. There are 2 kinds of 'struct', C style and .NET style. They are indeed completely different.. The fact that everything is an object leads to all kinds of funny tricks like boxing and unboxing A possible better design would have been generics, too late. Anyway, we can live with it.
Last edited by Cuchulainn on April 12th, 2006, 10:00 pm, edited 1 time in total.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget
 
User avatar
Cuchulainn
Posts: 61574
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Better Programming Environment

April 13th, 2006, 7:47 pm

Quote A better question is probably:What changes would you make in the next version of Visual Studio .NET to make it a better environment to write code?Can a good programmer write good code in any hardware/software environment?It's the same question as "do I need an expensive UML CASE tool to design OO systems". Example: how to design a pen that works in outer space. One can use a pencil.For me, I use VC 6.0 a lot because I can develop algorithms (minus UI and whistles) before going to VS2005, for example. There are too many key strokes and button clicks in the new environment.
Last edited by Cuchulainn on April 12th, 2006, 10:00 pm, edited 1 time in total.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget
 
User avatar
nocturne2
Posts: 150
Joined: January 11th, 2006, 5:45 pm

Better Programming Environment

April 13th, 2006, 9:09 pm

On Visual Studio: VS can take a page or two from Eclipse. Right out of the box Eclipse is feature-packed, whereas VS requires a dozen plugins (unit test, resharper, etc), some of which cost $, to be functional. You can design OO using paper and pencil, but you will have to live without round-trip engineering. As you design and implement, your initial paper UML diagram will be outdated. Good code does not absolutely require good tools. Writing good code _fast_ does require good tools.Paco de Lucia still sounds awesome with a $200 Yamaha, but will he use it?
 
User avatar
Cuchulainn
Posts: 61574
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Better Programming Environment

April 13th, 2006, 9:48 pm

QuoteYou can design OO using paper and pencil, but you will have to live without round-trip engineering. As you design and implement, your initial paper UML diagram will be outdated. Sure.But I meant: some It'ers jump to the screen immediately and start messing around with boxes and lines without having the correct mental model first.In my experience, I have seen few(none?) successful round-trip RUP projects. We prefer Barry Boehm's Spiral Model (risk-driven). RUP tends to produce masses of documents.If you know Brooks' Mythical Man Month: he used a small set of critical design blueprints to drive the s/w process. That's at loogerheads with 'modern' processes which I must (sadly) say have been less than successful.The problem is not tools, but the design models that we first have to specify. Edit: I have made some extensions to my house; do I need to update the original architect's plan??
Last edited by Cuchulainn on April 12th, 2006, 10:00 pm, edited 1 time in total.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget
 
User avatar
axs
Posts: 82
Joined: February 15th, 2003, 1:14 am

Better Programming Environment

April 13th, 2006, 9:49 pm

port valgrind to vistamore debugging tools to see what memory and threads are doing
 
User avatar
RGB
Posts: 19
Joined: April 11th, 2006, 10:14 pm

Better Programming Environment

April 14th, 2006, 11:46 am

QuoteI find this a bit confusing. Are you talking about ISO C++ or C++/CLI? . C++ is a chain saw that can cut your leg off; we know that memory management is difficult. That is not front-page news anymore. This is not a reason to seek refuge in a language that does GC. There are 2 kinds of 'struct', C style and .NET style. They are indeed completely different.. The fact that everything is an object leads to all kinds of funny tricks like boxing and unboxing A possible better design would have been generics, too late. Anyway, we can live with it.I'm talking about the standard version of C++. You know, the one that Bjarne Stroustrup wrote the specification about? The point is that whoever first came up with the idea of a garbage collector was brilliant! I encourage anyone doing things in C++ to move to either Java or a .NET programming language that uses safe code and the GC. Most organizations will be much better off afterwards. By the way, the .NET framework 2.0 that was recently released for Visual Studio 2005 now supports generics.
 
User avatar
lballabio
Posts: 983
Joined: January 19th, 2004, 12:34 pm

Better Programming Environment

April 14th, 2006, 12:22 pm

QuoteOriginally posted by: RGBI encourage anyone doing things in C++ to move to either Java or a .NET programming language that uses safe code and the GC. Most organizations will be much better off afterwards.Or use smart pointers for automatic memory management, or add a garbage collector by means of a library. This, of course, assuming we're talking of finance: there are fields where garbage collection cannot be used (if you're developing for a real-time system with hard deadlines, you can't allow garbage collection to start at unpredictable times and steal precious milliseconds.)QuoteBy the way, the .NET framework 2.0 that was recently released for Visual Studio 2005 now supports generics.Yes, but C# and Java generics are just a pale imitation of C++ templates, as they cannot be specialized.Luigi
 
User avatar
RGB
Posts: 19
Joined: April 11th, 2006, 10:14 pm

Better Programming Environment

April 14th, 2006, 12:51 pm

QuoteOriginally posted by: CuchulainnQuote A better question is probably:What changes would you make in the next version of Visual Studio .NET to make it a better environment to write code?Can a good programmer write good code in any hardware/software environment?It's the same question as "do I need an expensive UML CASE tool to design OO systems". Example: how to design a pen that works in outer space. One can use a pencil.For me, I use VC 6.0 a lot because I can develop algorithms (minus UI and whistles) before going to VS2005, for example. There are too many key strokes and button clicks in the new environment.Yes, a good programmer can write code in any hardware/software environment. However, I would encourage you to download the C# Express edtion from MS website while it is still free and start playing around with C#. Get a good book on the .NET framework. I like the book by Jeffrey Richter, as well as some of his other books which provide much insight into some of the Windows system functions like threading, critical sections, mutexes, semaphores, etc.I would say no, you don't really need an expensive CASE tool to design an OO system. Its more important to have skilled developers, a customer who is very knowledgeable about what the system should do, intelligent management who knows how to divide and conquer a system and assign the pieces to small groups that won't interfere with each other, and a clearly defined methodology that includes automated regression tests along with feature tests written by QA.I have worked on an exterme programming project before (now referred to as agile technology) and would warn anyone against using all of that methodology. The automated build, automated unit tests, and QA feature tests that all need to pass (or the tests that were passing before your changes were added must pass before you can check your code in) are all very good and sound principles to help manage a software project. However, the following agile methods are not good:1. Requiring developers to check code out at the start of the day and do a build on their maching. This is a waste of time.2. Requiring developers to re-check out code changes at the end of the day in order to do a merge and build on their machine, followed by checking in their code is another very big waste of time.3. Checking code in and out this frequently just eats up too much developer time. Its even worse if developers are encouraged to check code in and out several times a day, as was the case on the project I worked on.4. No written requirements and little if any time allocated to design.5. Requiring developers to work in pairs is another waste of manpower. Let the developers decide when they need help and let them decide when they need to work together to solve a difficult problem.6. Assigning tasks at random with little thought as to how multiple developers chekcing in code to the same module at the same time will impact each other. For me, this was an utter nightmare.I would work on a series of tasks. Merge my new code with new code that had been checked in since my last checkout, do a build, and run all the tests. The time to do a build and run all the unit and feature tests went from around 15 minutes when I first joined the project to over an hour midway through. Most of the time, before I can check my code in, someone else has checked in on top of me. Now I have to pull their code, do another merge, and run the tests again. There were times when I would go through this procedure 4 or 5 times before I could check in my code! I usually had to do at least 2 merges before checking my code in. After going through this, I finally understood the wisdom of working in small groups and assigning larger pieces of the system (preferably different exes or dlls) to each developer so that they can work relatively indepently from each other. Most of the projects I have worked on in the past were like this, and most of them went quite well.See the following link for more info on why much of the original agile technology methodology is faulty:Case Against XP
 
User avatar
RGB
Posts: 19
Joined: April 11th, 2006, 10:14 pm

Better Programming Environment

April 14th, 2006, 1:14 pm

QuoteOr use smart pointers for automatic memory management, or add a garbage collector by means of a library. This, of course, assuming we're talking of finance: there are fields where garbage collection cannot be used (if you're developing for a real-time system with hard deadlines, you can't allow garbage collection to start at unpredictable times and steal precious milliseconds.)According to some books I've read, the MS GC is very fast and efficient. MS goal was for the GC to take no longer than it takes for a page fault in memory. However, you make a good point about whether even that would interfere with the real-time operations of an order routing system for example. Since the GC stops all threads while performing its duties, I have often wondered if a real-time socket connection would lose data or the connection if the GC happened to kick in hard. I have never used sockets in .NET, but have used them with VC++ 6.0 for an order routing system some time ago.Can someone with actual experience using the Socket class under .NET shed some light on this subject?
Last edited by RGB on April 13th, 2006, 10:00 pm, edited 1 time in total.
 
User avatar
Cuchulainn
Posts: 61574
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Better Programming Environment

April 14th, 2006, 1:39 pm

QuoteOriginally posted by: RGBQuoteI find this a bit confusing. Are you talking about ISO C++ or C++/CLI? . C++ is a chain saw that can cut your leg off; we know that memory management is difficult. That is not front-page news anymore. This is not a reason to seek refuge in a language that does GC. There are 2 kinds of 'struct', C style and .NET style. They are indeed completely different.. The fact that everything is an object leads to all kinds of funny tricks like boxing and unboxing A possible better design would have been generics, too late. Anyway, we can live with it.I'm talking about the standard version of C++. You know, the one that Bjarne Stroustrup wrote the specification about? The point is that whoever first came up with the idea of a garbage collector was brilliant! I encourage anyone doing things in C++ to move to either Java or a .NET programming language that uses safe code and the GC. Most organizations will be much better off afterwards. By the way, the .NET framework 2.0 that was recently released for Visual Studio 2005 now supports generics.RGBI think you are losing me. It's like a discussion on which kind of coke I should drink. Of course, knowing quite a few ITers I know how these discussions can lead to language wars (BTW happens every few months here on Wilmott). A language is a tool and a means to an end. Most of the people around here are QF quant developers etc. and involved with good solutions in this field. At this moment C++ is the most popular language. Of course C# is nice but it is based on C++ and, as Luigi points outs, it is not as good (yet).QuoteYes, a good programmer can write code in any hardware/software environment. However, I would encourage you to download the C# Express edtion from MS website while it is still free and start playing around with C#. Get a good book on the .NET framework.Unfortunately, most IT books I don't like because 1) not deep enough, 2) sometimes they are 800-page code dumps 3) they are theoretical, no relevance to QF. That's why I decided to write my own books.C++/ISO is a .NET language and supports the holy grail of programming the GC.
Last edited by Cuchulainn on April 13th, 2006, 10:00 pm, edited 1 time in total.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget
 
User avatar
RGB
Posts: 19
Joined: April 11th, 2006, 10:14 pm

Better Programming Environment

April 14th, 2006, 2:49 pm

QuoteOriginally posted by: CuchulainnRGB I think you are losing me. It's like a discussion on which kind of coke I should drink. Of course, knowing quite a few ITers I know how these discussions can lead to language wars (BTW happens every few months here on Wilmott).A language is a tool and a means to an end. Most of the people around here are QF quant developers etc. and involved with good solutions in this field. At this moment C++ is the most popular language. Of course C# is nice but it is based on C++ and, as Luigi points outs, it is not as good (yet).QuoteYes, a good programmer can write code in any hardware/software environment. However, I would encourage you to download the C# Express edtion from MS website while it is still free and start playing around with C#. Get a good book on the .NET framework.Unfortunately, most IT books I don't like because 1) not deep enough, 2) sometimes they are 800-page code dumps 3) they are theoretical, no relevance to QF. That's why I decided to write my own books.C++/ISO is a .NET language and supports the holy grail of programming the GC.Your implication is probably correct - that Quants can and should be free to choose the language of their choice for analyzing models and other work. I am also aware that the QuantLib project is in C++. As you have already surmised, I'm not a quant, but a software developer interested in finance with many years developing software in C++ and C#. At some point in time though, the industry (and employers) will choose the software development tools that yield the quickest development time, highest reliability, and the fastest execute time. C++ is superior when it comes to execution time. But, it lags badly in the other 2 areas. At one point in time, I imagine that most Quants were using Fortran (before C or C++ was developed). Maybe some still are. But, how employable are they? Right now, I think you are right that C++ is the #1 language that Quants should know. But, Hedge funds, large investment banks, and other financial institutions are businesses as well and they will have their way with you some day (hopefully far off in the future...).
 
User avatar
Cuchulainn
Posts: 61574
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Better Programming Environment

April 14th, 2006, 3:24 pm

QuoteBut, Hedge funds, large investment banks, and other financial institutions are businesses as well and they will have their way with you some day (hopefully far off in the future...). Switching from C++ to C# is not difficult. So, the business drives technology, no problem.What I do notice is thet development time in C# is really fast. However, it is not a ISO standard and it could change.Fortran is a great language for number crunching.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget
ABOUT WILMOTT

PW by JB

Wilmott.com has been "Serving the Quantitative Finance Community" since 2001. Continued...


Twitter LinkedIn Instagram

JOBS BOARD

JOBS BOARD

Looking for a quant job, risk, algo trading,...? Browse jobs here...


GZIP: On