SERVING THE QUANTITATIVE FINANCE COMMUNITY

Cuchulainn
Posts: 60193
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

QuoteOriginally posted by: CuchulainnWhat is a developer's mental model? For example, the customer has written this specWhat's triggered in Grammar-Pattern Programmer's head? How does it pan out?No one?This is a serious question. It's a bit like this but then for computers
Last edited by Cuchulainn on April 14th, 2016, 10:00 pm, edited 1 time in total.

Posts: 23951
Joined: September 20th, 2002, 8:30 pm

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

QuoteOriginally posted by: CuchulainnHow do committees invent? QuoteA contract research organization had eight people who were to produce a COBOL and an ALGOL compiler. After some initial estimates of difficulty and time, five people were assigned to the COBOL job and three to the ALGOL job. The resulting COBOL compiler ran in five phases, the ALG0L compiler ran in three. QuoteOne could argue that each and every programmer (and user) is using a slightly different version of the interpreter/compiler to reach different conclusions about what the contract means and what the software should do.Is this your finding? Don't think medical systems would accept this..It means the final software system fails and you should reduce the team size until quality improves.Actually, the example that prompted this came from the nuclear power industry in which the specification writer and the programmer interpreted the language around measuring coolant levels in different ways. I forget the details (something to do with reacting to measurements over time) but one could see how each side had a valid but different interpretation.Team size > 1 all but guarantees these of kinds problems in which two or more people think the interpretation of the specification is obvious and everyone says "yes, I understand it" but they fail to realize they don't actually agree on the details.

fulmerspot
Posts: 511
Joined: July 8th, 2009, 12:44 pm

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

QuoteOriginally posted by: CuchulainnWhat is a developer's mental model? For example, the customer has written this specWhat's triggered in Grammar-Pattern Programmer's head? How does it pan out?3 months rolling contract at Market Rates and an opportunity to improve my database skills, I'd recommend an OLAP cube with reports available to the trader through a pivot report so I could hone those skills too. Just setting up all that would take up the first three months.That's what it triggered in my Eye-On-The-Cash Programmers head.Do I get the Gig?Herr KartoffelKopf

Posts: 23951
Joined: September 20th, 2002, 8:30 pm

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

QuoteOriginally posted by: CuchulainnQuoteOriginally posted by: CuchulainnWhat is a developer's mental model? For example, the customer has written this specWhat's triggered in Grammar-Pattern Programmer's head? How does it pan out?No one?This is a serious question. It's a bit like this but then for computers My first reaction is that the diagram is incomplete unless everyone is typing blind commands. Where does "old price" come from and who sees it? It's also unclear what the ORMS does. Does it only hold the current price state or does it accumulate a time series on each traded instrument?If the trader submits a long spreadsheet with many price changes, the grammar-pattern programmer would probably sort by date, sort or reorganize by symbol, and then foreach the symbols or rows. If end-of-the-week prices are all that matters, then there's some clever algie vector comparison code that extracts the last price foreach symbol in the spreadsheet. If the trader is entering price deltas, then the algie would use cumulative sums and some tricks to extract to cumulative deltas. If the spec calls for continuous updating, then the algie code and the looper code may be pretty similar.

Cuchulainn
Posts: 60193
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

QuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: CuchulainnQuoteOriginally posted by: CuchulainnWhat is a developer's mental model? For example, the customer has written this specWhat's triggered in Grammar-Pattern Programmer's head? How does it pan out?No one?This is a serious question. It's a bit like this but then for computers My first reaction is that the diagram is incomplete unless everyone is typing blind commands. Where does "old price" come from and who sees it? It's also unclear what the ORMS does. Does it only hold the current price state or does it accumulate a time series on each traded instrument?If the trader submits a long spreadsheet with many price changes, the grammar-pattern programmer would probably sort by date, sort or reorganize by symbol, and then foreach the symbols or rows. If end-of-the-week prices are all that matters, then there's some clever algie vector comparison code that extracts the last price foreach symbol in the spreadsheet. If the trader is entering price deltas, then the algie would use cumulative sums and some tricks to extract to cumulative deltas. If the spec calls for continuous updating, then the algie code and the looper code may be pretty similar.This diagram is incomplete (of course). But the added value is that all stakeholders are looking at the same page. The answers should help elicit precise requirements.The danger I think is the high-level architecture will be immediately lost using a grammarian approach based on low-level data.
Last edited by Cuchulainn on April 15th, 2016, 10:00 pm, edited 1 time in total.

Cuchulainn
Posts: 60193
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

QuoteOriginally posted by: fulmerspotQuoteOriginally posted by: CuchulainnWhat is a developer's mental model? For example, the customer has written this specWhat's triggered in Grammar-Pattern Programmer's head? How does it pan out?3 months rolling contract at Market Rates and an opportunity to improve my database skills, I'd recommend an OLAP cube with reports available to the trader through a pivot report so I could hone those skills too. Just setting up all that would take up the first three months.That's what it triggered in my Eye-On-The-Cash Programmers head.Do I get the Gig?Herr KartoffelKopfI like this initial feedback. The right mix IMHO. The part I liked most is that MFC was not named. In your proposal will you mention ADO.NET and C#?
Last edited by Cuchulainn on April 15th, 2016, 10:00 pm, edited 1 time in total.

Posts: 23951
Joined: September 20th, 2002, 8:30 pm

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

Cuchulainn
Posts: 60193
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

QuoteIn this diagram, it's not clear what must be kept true in this system.Hold yer horses,In fairness, the inkblot was quickly scribbled up on a beer mat, so there's a lot of lacunae. Having said that (and initial feedback from fulmerspot) the system system is probably MIS whose DFD is something like:If we reach agreement on this we can reduce project risk and get use cases for free (MIS is WELL_KNOWN). Then we can design a prototype in 3 months. Of course, more precise interfaces must be discovered.
Last edited by Cuchulainn on April 15th, 2016, 10:00 pm, edited 1 time in total.

Cuchulainn
Posts: 60193
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

And if we agree on the kind of reference model you can detail design in your favourite lingo and idiom:

Cuchulainn
Posts: 60193
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

QuoteOriginally posted by: CuchulainnWhat is a developer's mental model? For example, the customer has written this specWhat's triggered in Grammar-Pattern Programmer's head? How does it pan out?We missed out on ORMSXL == Excel?Crucial.
Last edited by Cuchulainn on April 15th, 2016, 10:00 pm, edited 1 time in total.

Cuchulainn
Posts: 60193
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

Are we in agreement? Have we left something out?

Posts: 23951
Joined: September 20th, 2002, 8:30 pm

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

QuoteOriginally posted by: CuchulainnQuoteIn this diagram, it's not clear what must be kept true in this system.Hold yer horses,In fairness, the inkblot was quickly scribbled up on a beer mat, so there's a lot of lacunae. Having said that (and initial feedback from fulmerspot) the system system is probably MIS whose DFD is something like:If we reach agreement on this we can reduce project risk and get use cases for free (MIS is WELL_KNOWN). Then we can design a prototype in 3 months. Of course, more precise interfaces must be discovered.All of my comments reflect discussions that would help refine the beer mat diagram rather than being criticisms -- all specs have to start somewhere.From a DFD standpoint, I'm always deeply suspicious of one-way arrows which seem to imply uncontrolled data sources and deadend data sinks. Even that MIS DFD has some. Apparently "Source" can't see any of the transaction data or existing data that their input will affect. The ad hoc report system has no inputs except the merged data (does "source" define an ad hoc report even though they can't see into the system?). At the very least, I'd expect some kind of corrective/editor element that can directly view and change erroneous transactions or existing data.As of the original diagram, the overall one-way loop seems to imply that all the price changes and updates could take place as a once-a-week batch job. That's a very different architecture than if the trader and/or the data team expect to see the most recent "old price" (versus last week's old price) before entering in a "new price". The instant the spec changes from slow batch loops to double-ended arrows, the system potentially starts to have response time issues and resource contention issues.

Cuchulainn
Posts: 60193
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

QuoteAll of my comments reflect discussions that would help refine the beer mat diagram rather than being criticisms -- all specs have to start somewhere.True. But we don't want premature opimisation. Of course, this is an ORMS system which means many of the business processes are known and will be taken care of. Of course, we must read the docs. Avoids discovering requirements in the code (costs $$).QuoteFrom a DFD standpoint, I'm always deeply suspicious of one-way arrows which seem to imply uncontrolled data sources and deadend data sinks. Even that MIS DFD has some. Apparently "Source" can't see any of the transaction data or existing data that their input will affect. The ad hoc report system has no inputs except the merged data (does "source" define an ad hoc report even though they can't see into the system?). At the very least, I'd expect some kind of corrective/editor element that can directly view and change erroneous transactions or existing data.DFD is the data view. We need to define a control view. Data sources/sinks are autonomous external systems.DFDs are only an initial train-spotting mechanism. We need more, later.QuoteAs of the original diagram, the overall one-way loop seems to imply that all the price changes and updates could take place as a once-a-week batch job. That's a very different architecture than if the trader and/or the data team expect to see the most recent "old price" (versus last week's old price) before entering in a "new price". The instant the spec changes from slow batch loops to double-ended arrows, the system potentially starts to have response time issues and resource contention issues. I would not draw many conclusions from the inkblot diagram. it needs to be fleshed out. It was an initial sketch. Last edited by Cuchulainn on April 17th, 2016, 10:00 pm, edited 1 time in total. Traden4Alpha Posts: 23951 Joined: September 20th, 2002, 8:30 pm ### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower? QuoteOriginally posted by: CuchulainnQuoteAll of my comments reflect discussions that would help refine the beer mat diagram rather than being criticisms -- all specs have to start somewhere.True. But we don't want premature opimisation. Of course, this is an ORMS system which means many of the business processes are known and will be taken care of. Of course, we must read the docs. Avoids discovering requirements in the code (costs$$).Whether a system can be a batch process or must have continuous updating seems like a fundamental architectural decision, not an optimization. We know a lot of companies that are trying to improve visibility into their operations but are discovering that their batch-process MIS systems simply cannot provide it. It's possible that the current ORMS cannot support the needs of this new system if the ORMS is a batch system and the traders now want a realtime system.QuoteQuoteFrom a DFD standpoint, I'm always deeply suspicious of one-way arrows which seem to imply uncontrolled data sources and deadend data sinks. Even that MIS DFD has some. Apparently "Source" can't see any of the transaction data or existing data that their input will affect. The ad hoc report system has no inputs except the merged data (does "source" define an ad hoc report even though they can't see into the system?). At the very least, I'd expect some kind of corrective/editor element that can directly view and change erroneous transactions or existing data.DFD is the data view. We need to define a control view. Data sources/sinks are autonomous external systems.DFDs are only an initial train-spotting mechanism. We need more, later.Agreed! And by that definition, the beer mat diagram is probably missing most of the data flow volume. I'd wager that traders are greater consumers of data than they are producers of data. They probably review historical trades/positions/prices N>>1 times for each new trade they create making the thickest arrow a missing one from the database to the trader.QuoteQuoteAs of the original diagram, the overall one-way loop seems to imply that all the price changes and updates could take place as a once-a-week batch job. That's a very different architecture than if the trader and/or the data team expect to see the most recent "old price" (versus last week's old price) before entering in a "new price". The instant the spec changes from slow batch loops to double-ended arrows, the system potentially starts to have response time issues and resource contention issues. I would not draw many conclusions from the inkblot diagram. it needs to be fleshed out. It was an initial sketch.Agreed! Yet if one cannot draw any conclusions from the inkblot diagram because it is missing more that 50% of the bare bones of the system, what can one do?Maybe that's the broader point -- any sketch of a system will be missing most of the information needed to define a good architecture.

Cuchulainn
Posts: 60193
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Grammar-Pattern Programmers: Is Over-Abstraction Delivering Projects Faster Or Slower?

QuoteWhether a system can be a batch process or must have continuous updating seems like a fundamental architectural decision, not an optimizationI can't remember any one saying that it was an optimization.I cannot see anywhere that the inkblot diagram is or is not a batch system. It's just that way, maybe. Requirements Elicitation resolves this issue. QuoteAgreed! Yet if one cannot draw any conclusions from the inkblot diagram because it is missing more that 50% of the bare bones of the system, what can one do?Maybe that's the broader point -- any sketch of a system will be missing most of the information needed to define a good architecture. The inkblot is past tense.. enter the system architects.. who speak statecharts, concurrency and so on. Quotebatch-process MIS systems simply cannot provide it. It's possible that the current ORMS cannot support the needs of this new system if the ORMS is a batch system and the traders now want a realtime system.What then?
Last edited by Cuchulainn on April 17th, 2016, 10:00 pm, edited 1 time in total.