wilmott.com

Stub topic for testing strategy ideas.In general code should only be included in a release if:1. There are no compiler warnings2. Code meets coding standards + is commented3. Code has been tested (how platform independent do we wish to be) and tests documented4. Documentation is up-to-date

Yes - I like the linux model of:Submit what you are going to work onSubmit designSubmit code mid-way through (optional)Submit completed code+doco+unit testsMerge into trunktestupdateitterate until stablerelease

What about units, dimensional analysis?We don't want system 1 to use feet and system 2 meters.

QuoteOriginally posted by: CuchulainnWhat about units, dimensional analysis?We don't want system 1 to use feet and system 2 meters.That could never happen!QuoteThe MCO MIB has determined that the root cause for the loss of the MCO spacecraft was the failure to use metric units in the coding of a ground software file, ?Small Forces,? used in trajectory models. Specifically, thruster performance data in English units instead of metric units was used in the software application code titled SM_FORCES (small forces). The output from the SM_FORCES application code as required by a MSOP Project Software Interface Specification (SIS) was to be in metric units of Newtonseconds (N-s). Instead, the data was reported in English units of pound-seconds (lbf-s). The Angular Momentum Desaturation (AMD) file contained the output data from the SM_FORCES software. The SIS, which was not followed, defines both the format and units of the AMD file generated by ground-based computers. Subsequent processing of the data from AMD file by the navigation software algorithm therefore, underestimated the effect on the spacecraft trajectory by a factor of 4.45, which is the required conversion factor from force in pounds to Newtons. An erroneous trajectory was computed using this incorrect data.? Mars Climate Orbiter Mishap Investigation Phase I Report

Here is Ariane 5 report by J.L. Lions, father of the French Numerical Analysis. The extensive reviews and tests carried out during the Ariane 5 Development Programme did not include adequate analysis and testing of the inertial reference system or of the complete flight control system, which could have detected the potential failure.For some critical applications, UML Boost State Machines have mant advantages. But it might be too technical. They allow us to 1) discover requirements 2) implement complete systems in C++ 3) check all paths in code (tautologies, unreachable code).

QuoteOriginally posted by: CuchulainnWhat about units, dimensional analysis?We don't want system 1 to use feet and system 2 meters.Also, there the issues of range and precision. Implicit 16-bit or 32-bit counters can overflow or wrap. Algorithms can output different results if run with 32-bit vs. 64-bit floats. Input parameters might have untested corner cases or untested ranges. EDIT: Ariane 5 failure was due to an overflow in 16-bit int cast from a float.

Testing would seem to imply at least three requirements for the library project:1. A test database that stores each test as a tuple with the code version, platform, inputs, and outputs (i.e., data sufficient to recreate and compare the test on a new version or platform)2. Documentation to justify the "correct" answer for a given algorithm and input. This implies a traceable/auditable process for creating "correct" answers with some other process (e.g., some other trusted library, Matlab code, analytic result, published paper, etc.)3. A process for revoking test results (and associated code) if someone finds a flaw that invalidates a stored "correct" answer.

GBM is indeed a strictly positive process.

On testing, will it be white box or black box or grey box?