Serving the Quantitative Finance Community

 
User avatar
Cuchulainn
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 14th, 2017, 2:46 pm

Do philosophers & mathematicians have predictable rates of production?
No.
Perhaps CS is the same and can never have the kinds of reliable production methods found in other types of engineering.
I see CS in its current form as a branch of (discrete) mathematics, for a large part the study of ADTs and operations on ADTs.


The crafting/engineering fun begins when we apply CS to problems.

//
Fred Brooks

rule of thumb for scheduling a software task:

1/3 planning

1/6 coding

1/4 component test and early system test

1/4 system test, all components in hand.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 14th, 2017, 4:01 pm

No.
Perhaps CS is the same and can never have the kinds of reliable production methods found in other types of engineering.
I see CS in its current form as a branch of (discrete) mathematics, for a large part the study of ADTs and operations on ADTs.


The crafting/engineering fun begins when we apply CS to problems.

//
Fred Brooks

rule of thumb for scheduling a software task:

1/3 planning

1/6 coding

1/4 component test and early system test

1/4 system test, all components in hand.
Makes sense. I can't help but think that constructing software system is like constructing an interlocking set of mathematical proofs -- proving by logic & construction that one can get the right outputs regardless of input. If so, estimating the resources for a given software project is like estimating the resources to construct a set of proofs. That gets us back to the predictability of productivity problem.

Brook's task allocation might make sense although I don't know if his numbers are from anecdote or true science. Yet knowing how to slice the pie does not solve the far more important problem of knowing how big the pie must be.
 
User avatar
Cuchulainn
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 15th, 2017, 1:34 pm

Many s/w project spend big % in debugging mode possibly because developers started with fuzzy, missing and incorrect requirements. We used to do fixed-price project by talking the time to estimate each activity using the triangle distribution and PERT. It worked very well and each party was kept happy. People underestimate software size.

Some areas (process control, telecom) have reference models and developers create new applications based on these models. Unfortunately, most developers tend not to use proven reference models and prefer to be creative and craft a new solution even though _it_has_all_been_done_before (one of Achilles' heels of CS?). Analogical; reasoning is not on the radar screen.

A large subcategory is all sorts of vending/gambling machine with various hardware/ security/national laws etc. (an ACL application).

In the past I used to design and set up VAX/VMS (best OS ever) networks with 1000 users in oil/gas. Security was important and we used the seminal Reference Moniitor

http://h41379.www4.hpe.com/doc/84final/ ... l#acl_part

Later I generalised it (==> the category Access Control Systems) to all kinds of systems and software architects I trained used it with great success.I actually spoke to one about this very topics

And there are architects who do not know/want to know RM and just develop their own model which in mot cases is not completely correct. 

constructing software system is like constructing an interlocking set of mathematical proofs -
CS is not there yet IMO, (you don't learn it at uni). Maybe more like Lego assembly with plugs and sockets. 
 
User avatar
Cuchulainn
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 30th, 2017, 12:32 pm

In the army, each (foot) soldier needs >= 3 support people to do his job,

How many 'managers' does the average programmer 'need'?
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 30th, 2017, 1:54 pm

In the army, each (foot) soldier needs >= 3 support people to do his job,

How many 'managers' does the average programmer 'need'?
Doesn't that depend on the programmer, the nature of the program, and the technical environment? There's programmers who can whip up a beautiful 100,000 line program in no time and others that need a half dozen support staff to get "hello world" right. Second, some programs are pretty simple -- yet another printer driver of the -B400 model -- and some involve a a complex combination of user-experience, database, network, and advanced algorithms. Finally, if the program must fit into the middle of a bunch of pre-existing/legacy back-end and front-end systems, then managers from both ends need to be involved.

At an absolute minimum, there's the IT manager and the manager representing the user population. But more complex situations may require management oversight from all the interacting subsystems that the programmer might affect.

Note: when I worked in aerospace, every engineering drawing needed six signatures in addition to that of my manager.
 
User avatar
Cuchulainn
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 30th, 2017, 8:55 pm

There's programmers who can whip up a beautiful 100,000 line program in no time 

Don't believe that.  KLOC of code? anyone can copy and paste from Google these days.

They need 100 maintainers, testers and technical writers. And FDA?
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 30th, 2017, 9:13 pm

A 100 maintainers, testers, and technical writers?!?!?! That must be for one of those cost-plus government contracts.

Your EU Tax Money At "Work", what?
 
User avatar
Cuchulainn
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 31st, 2017, 11:28 am

A 100 maintainers, testers, and technical writers?!?!?!  That must be for one of those cost-plus government contracts.

Your EU Tax Money At "Work", what?
I wrote it in the spirit of your hilarious 100KLOC comment. A good programmer writes < 20 LOC per week.
What has EU got to do with it? Money for quality services has to come from somewhere. Usually from them that pays tax.
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 31st, 2017, 1:26 pm

A 100 maintainers, testers, and technical writers?!?!?!  That must be for one of those cost-plus government contracts.

Your EU Tax Money At "Work", what?
I wrote it in the spirit of your hilarious 100KLOC comment. A good programmer writes < 20 LOC per week.
What has EU got to do with it? Money for quality services has to come from somewhere. Usually from them that pays tax.
Less than 20 per week seems far lower than the averages I've seen which run 100-200 LOC per week. And given the huge variance in programmer productivity, if the average is 150 LOC/week, the good programmers are maybe 500 LOC per week and the best may be O(1 KLOC/week). How long was your CAD program and how many weeks of labor did it it have?

I mentioned the EU because I assumed the 100 maintainers comment came from your experiences in the EU. I see a lot of high quality services in the world (government, education, healthcare) that are bloated because "quality" becomes an excuse for over-charging and then the high revenues becomes an excuse for hiring extra labor. It's certainly not isolated to the EU.
 
User avatar
Cuchulainn
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 31st, 2017, 4:15 pm

 How long was your CAD program and how many weeks of labor did it it have?


If in doubt, ask :)
It is a library in C++ and then C# for mechanical, metal and later holography. It evolved as new requirements emerged. Most of the work was relearning Cartesian geometry.

We should a distinction between market-driven and one-off single customer consultancy projects.
 
User avatar
Cuchulainn
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 31st, 2017, 4:18 pm

 How long was your CAD program and how many weeks of labor did it it have?


If in doubt, ask:)
It is a library in C++ and then C# for mechanical, metal and later holography. It evolved a new requirements emerged. Most of the work was relearning Cartesian geometry.

We should a distinction between market-driven and one-off single customer consultancy projects.

Less than 20 per week seems far lower than the averages I've seen which run 100-200 LOC per week. 
How many (modified) lines per week for pacemaker software?
 
User avatar
Traden4Alpha
Posts: 3300
Joined: September 20th, 2002, 8:30 pm

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

December 31st, 2017, 5:33 pm

 How long was your CAD program and how many weeks of labor did it it have?


If in doubt, ask:)
It is a library in C++ and then C# for mechanical, metal and later holography. It evolved a new requirements emerged. Most of the work was relearning Cartesian geometry.

We should a distinction between market-driven and one-off single customer consultancy projects.
Less than 20 per week seems far lower than the averages I've seen which run 100-200 LOC per week. 
How many (modified) lines per week for pacemaker software?
Sure!

But the point remains that some high-performance programmers working on some types of non-critical code do crank out the KLOCs even if most programmers working on most types of code are slower and some programmers working on some types of highly-critical code are glacial. Between a proverbial 10:1 programmer productivity ratio and a proverbial 10:1 program-criticality ratio, there's probably a 100:1 ratio in the rates of "finished" LOC production.
 
User avatar
Cuchulainn
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

January 12th, 2018, 12:19 pm

 How long was your CAD program and how many weeks of labor did it it have?


If in doubt, ask:)
It is a library in C++ and then C# for mechanical, metal and later holography. It evolved a new requirements emerged. Most of the work was relearning Cartesian geometry.

We should a distinction between market-driven and one-off single customer consultancy projects.
Less than 20 per week seems far lower than the averages I've seen which run 100-200 LOC per week. 
How many (modified) lines per week for pacemaker software?
Sure!  

But the point remains that some high-performance programmers working on some types of non-critical code do crank out the KLOCs even if most programmers working on most types of code are slower and some programmers working on some types of highly-critical code are glacial.  Between a proverbial 10:1 programmer productivity ratio and a proverbial 10:1 program-criticality ratio, there's probably a 100:1 ratio in the rates of "finished" LOC production.
This is anecdotal. A lot of code is 10% algorithmic and 90% on other things like acceptance testing, debugging, integration etc. So, we can't rreally compare 'programmers' in this way. It's not a p***ing contest.
 
User avatar
Cuchulainn
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

January 12th, 2018, 1:25 pm

 high-performance programmers
How do you quantify this idea? 

In the olde systems days "if you have a guy who is good in a crisis, fire him, otherwise you will always have one".

"Productivity follows quality"
 
User avatar
Cuchulainn
Posts: 20252
Joined: July 16th, 2004, 7:38 am
Location: 20, 000

Re: Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

January 12th, 2018, 1:45 pm

The problem is that programmers in O-O have been experimenting in incestuous applications and aiming low in abstraction, instead of high. For example, they have been building classes such as linked-list or set instead of classes such as user-interface or radiation beam or finite-element model. Unfortunately the selfsame strong type-checking in C++ that helps programmers to avoid errors also makes it hard to build big things out of little ones.

// we design large-grained classes that address concepts our clients are already working with, they can understand and question the
design as it grows, and they can cooperate in the design of test cases. My ophthalmology collaborators don't care about stacks; they
do care about Legendre polynomial shape descriptions of corneas.  Small encapsulations yield small benefits.


The answer is simple. It is because [O-O] has been tied to a variety of complex languages. Instead of teaching people that O-O is a type of design, and giving them design principles, people have taught  that O-O is the use of a particular tool. We can write good or bad programs with any tool. Unless we teach people how to design, the languages matter very little. The result is that people do bad designs with these languages and get very little value from them. If the value is small, it won't catch on.