SERVING THE QUANTITATIVE FINANCE COMMUNITY

 
User avatar
DevonFangs
Posts: 3004
Joined: November 9th, 2009, 1:49 pm

Quant work after teaching

March 27th, 2012, 2:13 pm

QuoteOriginally posted by: barnyQuoteOriginally posted by: MattFSounds like a great opportunity to rewrite it then. I presume it's some sort of numerical analysis package? Port it to <modern language of choice>, add the ability to distribute calculations via "the cloud", add the missing customisation or interfaces that it needs, and you'll look like a superstar.The code is already lightning fast, and scales to hundreds of thousands of processors (using MPI) very well. Re-writing 40,000 LOC would be pointless (it wouldn't be as fast) and infeasible since it would take one man 5 years or more probably. The point is in the search for speed and parallelisation usability and maintainability have been massively sacrificed. It's probably bearable to use if you're part of the development team at this other university, but from where I'm sat, all alone 1000's of miles away, it's impossible. I'm completely screwed, and the PhD was essentially sold to me on a lie. I now have a significant amount of pressure from my out-of-touch supervisor about results, and as I say, I can't imagine I'll EVER get results with this beast. I'm am proverbially pissing in the wind, and my plan was to cut my losses and move on sharpish rather than just "suck it up".Listen, not sure I get the drama here. I've known tons of PhD students using an existing giant code black-box, so you can be sure you're not the first, like condensed matter physicist or chemists using density functional code, theoretical physicist with libraries to calculate scattering amplitudes, QCD on lattice and crap like that. If your supervisor is a manager rather than a researcher, maybe he's really good at managing people and distributing tasks or something, so I'm pretty sure he'll give you a project or you'll find your own eventually.Also, if you have this 40k LOC beast with a low level memory management so freaking awesome to run at the speed of light on *hundreds of thousands* of processors, maybe you can learn something from it.EDIT speaking of which, MPI on a MILLION processors
Last edited by DevonFangs on March 26th, 2012, 10:00 pm, edited 1 time in total.
 
User avatar
CrashedMint
Posts: 2591
Joined: January 25th, 2008, 9:12 pm

Quant work after teaching

March 27th, 2012, 2:40 pm

QuoteOriginally posted by: barny[My research] It's a piece of shit. The code has even been used as an example of bad code design and management by some other programmers in my group.lol, it's never a good sign if one's work is used by one's peers to demonstrate how not to do it. So, if you suck so much, why did they accept you into the Phd program in the first place. My school tends to be really, really selective and it's considered quite an accomplishment to even become a doctoral student. Anyways, it seems that you really want to quit and expect some sort of "Go ahead!" from "the finance world", Wilmott's board here being the latter. That's not really a good way to go. You will want to make that decision for yourself. From my perspective i) quitting is never a good thing and ii) anything worthwhile is always at the fringe of your skills, and iii) a doctorate is the highest degree you can get so obviously it's not gonna be a walk in the park.But, here's a more practical solution: Cut your work into manageable bits and try to complete one bit with a set amount of time. e.g. try to clean up the code until july 1st so that by the time you get a clear understanding if there is a chance that the whole thing works out, or not. If by July 1st, you realize that it's all going south, then quit. Obviously you will want to communicate a lot with your advisor in the next months.If you quit, then quitting sooner than later is of course better. Also, you will want to avoid blaming people. Nobody likes a quitter, but everybody hates a "blamer". So if you do elaborate rants on how it's all your stupid advisers fault, this will make it sound so much worse. Instead you will want to say that you tried PhD research, but quickly realized that you didn't like it which is why you cut your losses. You are an active, hands-one guy that thrives in a quick environment, but sitting in the library doing theoretical ivory-tower research that is of purely hypothetical value is just not something you will want to waste your time with. Your school was great, but it's really just not something that you would want to spend time doing. You had to try to it, to know it, but immediately you realized that it wasn't working out.
Last edited by CrashedMint on March 26th, 2012, 10:00 pm, edited 1 time in total.
 
User avatar
croot
Posts: 104
Joined: July 23rd, 2006, 8:30 pm

Quant work after teaching

March 27th, 2012, 2:46 pm

My positive message being drowned in a sea of webclutter, I will reiterate:skin cream and handcuffs , saying "yes" (practice to self with mirror)are the keys to success.Every minute you spend working on technology, a salesperson tells a rude joke, gets a lapdance and saves towards his new mansion in the islands.Optimal transportation, C++ design standards etc... -> Red herrings designed to distract you from the money.I have already told you about some thinker I like: "what do you care what other people think". At the risk of coming across as snooty I will add to his phrase: "take the blue pill".
 
User avatar
Cuchulainn
Posts: 62626
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Quant work after teaching

March 27th, 2012, 2:47 pm

QuoteOriginally posted by: quantmehQuoteOriginally posted by: frenchXYou have 3 years, to rewrite 40 000 line of code is really enough. once i re-wrote 30K LOC of C++ junk in 2 months.No lasting side-effects?
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
DevonFangs
Posts: 3004
Joined: November 9th, 2009, 1:49 pm

Quant work after teaching

March 27th, 2012, 2:49 pm

QuoteOriginally posted by: CuchulainnQuoteOriginally posted by: quantmehQuoteOriginally posted by: frenchXYou have 3 years, to rewrite 40 000 line of code is really enough. once i re-wrote 30K LOC of C++ junk in 2 months.No lasting side-effects?Apart from the *new* 30K LOC of C++ junk he wrote?
 
User avatar
Cuchulainn
Posts: 62626
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Quant work after teaching

March 27th, 2012, 2:57 pm

QuoteOriginally posted by: DevonFangsQuoteOriginally posted by: CuchulainnQuoteOriginally posted by: quantmehQuoteOriginally posted by: frenchXYou have 3 years, to rewrite 40 000 line of code is really enough. once i re-wrote 30K LOC of C++ junk in 2 months.No lasting side-effects?Apart from the *new* 30K LOC of C++ junk he wrote? He never liked C++ anyways.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
Cuchulainn
Posts: 62626
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

Quant work after teaching

March 31st, 2012, 6:54 pm

QuoteOriginally posted by: quantmehQuoteOriginally posted by: frenchXYou have 3 years, to rewrite 40 000 line of code is really enough. once i re-wrote 30K LOC of C++ junk in 2 months.Quality, not quantity. In general, 20 lines of real code (e.g. clever algo) per day is a lot. So, I reckon that (30 - 20*5*4*2)K LOC was padding or generated. 30K per 2 m is 15K p m is 4K per week is 900 LOC per day ==> BS.
Last edited by Cuchulainn on March 30th, 2012, 10:00 pm, edited 1 time in total.
Step over the gap, not into it. Watch the space between platform and train.
http://www.datasimfinancial.com
http://www.datasim.nl
 
User avatar
spv205
Posts: 478
Joined: July 14th, 2002, 3:00 am

Quant work after teaching

March 31st, 2012, 8:09 pm

so you're saying the developers are on the other side of the world....why don't you move there for a year...
 
User avatar
Polter
Posts: 2526
Joined: April 29th, 2008, 4:55 pm

Quant work after teaching

March 31st, 2012, 8:37 pm

QuoteOriginally posted by: CuchulainnQuoteOriginally posted by: quantmehQuoteOriginally posted by: frenchXYou have 3 years, to rewrite 40 000 line of code is really enough. once i re-wrote 30K LOC of C++ junk in 2 months.Quality, not quantity. In general, 20 lines of real code (e.g. clever algo) per day is a lot. So, I reckon that (30 - 20*5*4*2)K LOC was padding or generated. 30K per 2 m is 15K p m is 4K per week is 900 LOC per day ==> BS.QuoteDecremental Development* Reduce software size TO 10%o while keeping required functionalityo while improving its qualityo while improving its design> measure productivity by Lines of Code removed (LoCR)
Last edited by Polter on March 30th, 2012, 10:00 pm, edited 1 time in total.
 
User avatar
quantmeh
Posts: 5974
Joined: April 6th, 2007, 1:39 pm

Quant work after teaching

March 31st, 2012, 8:42 pm

QuoteOriginally posted by: CuchulainnNo lasting side-effects?i had an enormous test set: the real input data. so i ran it through the C++ code and got the outputs, intermediate and final. the intermediate outputs are from within the components, like interest rate paths etc. once i ported the code to Java, the same input was ran through the new code. next, we compared the outputs, and tracked the errors. it may sound like impossible work, but in reality it took us a only few iterations to find all bugs, and by that i mean producing the exact match output. if we had just a few test cases, we wouldn't succeed. the key was in having a test set, which touches almost entire code base. in Java i could get the stats on code coverage, and it was close to 100%.the way it works is funny. at fist you get pretty much 100% of output not matching. then you go track the bug in components, and suddenly you see how interest rate paths are getting closer and closer, until they match exactly. then you move to a next component. then at some point components are more or less Ok, and some of the final outputs are matching, i.e. end-to-end the code seems to work for many test cases. then you isolate those cases, which don't produce right outputs. then you see how the set of not matching cases is shrinking exponentially, until finally you get all outputs match. it was a lot of fun. after i got the code ported correctly, i got into re-factoring, i.e. re-writing the components to optimize performance. this was again aided by the test case. so if i break something, i can see it immediately. i was also logging all my performance metrics, so i could experiment with different approaches for caching, pooling, pre-fetching etc.
 
User avatar
traderjoe1976
Posts: 1544
Joined: May 19th, 2006, 9:50 am

Quant work after teaching

March 31st, 2012, 10:55 pm

I knew one software developer who used to put all his code in the header files. Everyone thought that he was a genius because he could develop very sophisticated systems with very little code. He also had all the source code of every company which he had ever worked with in his entire career (which is very common among consultants). If there was common functionality required in a new project, he would simply copy and paste the code from the other project, tweak it a bit and put it in a header file. People were constantly amazed at how fast and efficiently he worked and he would have a merry old time while billing $1,200 per day.
 
User avatar
quantmeh
Posts: 5974
Joined: April 6th, 2007, 1:39 pm

Quant work after teaching

April 1st, 2012, 12:39 am

QuoteOriginally posted by: traderjoe1976I knew one software developer who used to put all his code in the header files. Everyone thought that he was a genius because he could develop very sophisticated systems with very little code. He also had all the source code of every company which he had ever worked with in his entire career (which is very common among consultants). If there was common functionality required in a new project, he would simply copy and paste the code from the other project, tweak it a bit and put it in a header file. People were constantly amazed at how fast and efficiently he worked and he would have a merry old time while billing $1,200 per day.he's in jail now, right?
 
User avatar
capafan2
Posts: 924
Joined: June 20th, 2009, 11:26 am

Quant work after teaching

April 1st, 2012, 1:17 am

Quotehe's in jail now, right? Like TJ says, this is very common among consultants. It must have stopped now because lot of places put strong network policies around use of USB's. Before that this was just so easy to do. Most of the code is not really like trading algo's. They are just standard routines like String Manipulations, Array Manipulations, etc. But it saves a heck of a lot of time when you have to go to each project and redo it. Over time you just build a library. To most Java, .NET or Python or Ruby guys this is not useful as almost everything they need is in the core library or free in the open source world. I know a SAS guy who does this and I looked at his library and it made sense why he did what he did. There would be too much boiler plate code to rewrite from place to place. How can we Spring Users sympathize with his plight? We are so spoiled!!
 
User avatar
quantmeh
Posts: 5974
Joined: April 6th, 2007, 1:39 pm

Quant work after teaching

April 1st, 2012, 1:51 am

QuoteOriginally posted by: capafan2There would be too much boiler plate code to rewrite from place to place.i get it. if you're hungry it's ok to steal food
 
User avatar
capafan2
Posts: 924
Joined: June 20th, 2009, 11:26 am

Quant work after teaching

April 1st, 2012, 2:13 am

QuoteOriginally posted by: quantmehQuoteOriginally posted by: capafan2There would be too much boiler plate code to rewrite from place to place.i get it. if you're hungry it's ok to steal foodDepends on how you define stealing. Not that I have stolen string manipulation routines but there have been times when I wished I had. Outside of ideas which have competitive advantage I think the whole "stealing code" is defined too tightly to protect companies. You set the bar so high that you will prevent true ideas from being stolen. It makes complete practical sense but I doubt if anyone goes after their low level coders for stealing string manipulation routines. Much like very few IB's would pull you up if you consistently left work at 9:00 instead of 8:30 say so you can take the car service home. There is a difference between legal and moral and ethical. I am sure given where you work you know it. Buying homes you could not afford always banking on foreclosure is legal in the United States but not so moral or ethical in my opinion. Such an attitude will land you in Jail in UAE where the law is against you in this context.
Last edited by capafan2 on March 31st, 2012, 10:00 pm, edited 1 time in total.
ABOUT WILMOTT

PW by JB

Wilmott.com has been "Serving the Quantitative Finance Community" since 2001. Continued...


Twitter LinkedIn Instagram

JOBS BOARD

JOBS BOARD

Looking for a quant job, risk, algo trading,...? Browse jobs here...


GZIP: On