STATISTICAL SQA implies an approach in which the SPECIFICITY of quality requirements drives the quality assurance process, and in which quality is MEASURED rather than asserted. THE FOUR STEPS (Pressman): 1. INFORMATION ABOUT SOFTWARE DEFECTS IS COLLECTED AND CATEGORISED. 2. AN ATTEMPT IS MADE TO TRACE EACH DEFECT TO ITS UNDERLYING CAUSE — a specification error, a design flaw, a violation of standards, poor communication with the customer, and so on. 3. USING THE PARETO PRINCIPLE (80% of the defects can be traced to 20% of all possible causes), ISOLATE THE 20% — the "vital few". 4. ONCE THE VITAL FEW CAUSES HAVE BEEN IDENTIFIED, MOVE TO CORRECT THE PROBLEMS THAT CAUSED THE DEFECTS. THE CONCEPTUAL SHIFT: the aim is not to fix these defects but to STOP PRODUCING THEM. Step 2 — root cause, not symptom — is where the leverage is, and it is the step most often skipped. TYPICAL DEFECT CAUSE CATEGORIES (Pressman's list, used for classification): IES incomplete or erroneous specification MCC misinterpretation of customer communication IDS intentional deviation from specification VPS violation of programming standards EDR error in data representation IMI inconsistent module interface EDL error in design logic IET incomplete or erroneous testing IID inaccurate or incomplete documentation PLT error in programming language translation of design HCI ambiguous or inconsistent human-computer interface MIS miscellaneous WORKED — a full Pareto analysis. One year of defect data, classified by cause, and split into SERIOUS / MODERATE / MINOR: cause total serious moderate minor ──────────────────────────────────────── IES 205 34 68 103 MCC 156 12 68 76 IDS 48 1 24 23 VPS 25 0 15 10 EDR 130 26 68 36 IMI 58 9 18 31 EDL 45 14 12 19 IET 95 12 35 48 IID 36 2 20 14 PLT 60 15 19 26 HCI 28 3 17 8 MIS 56 0 15 41 ──────────────────────────────────────── TOTAL 942 128 379 435 STEP 1 — RANK BY TOTAL COUNT and compute cumulative percentage: IES 205 21.8% cumulative 21.8% MCC 156 16.6% 38.3% EDR 130 13.8% 52.1% IET 95 10.1% 62.2% PLT 60 6.4% 68.6% IMI 58 6.2% 74.7% MIS 56 5.9% 80.7% ← the 80% line IDS 48 5.1% 85.8% EDL 45 4.8% 90.6% IID 36 3.8% 94.4% HCI 28 3.0% 97.3% VPS 25 2.7% 100.0% SEVEN of twelve causes account for 80.7% of all defects. Not quite the textbook 20%, which is normal — the Pareto principle is a heuristic, not a law. But the top THREE causes alone (IES, MCC, EDR) account for 52.1%, and that is genuinely actionable. STEP 2 — BUT RANK BY SERIOUSNESS AND THE ANSWER CHANGES: IES 34 serious 26.6% of all serious EDR 26 20.3% cum 46.9% PLT 15 11.7% cum 58.6% EDL 14 10.9% cum 69.5% MCC 12 9.4% cum 78.9% IET 12 9.4% cum 88.3% IMI 9 7.0% cum 95.3% HCI 3, IID 2, IDS 1, VPS 0, MIS 0 LOOK AT WHAT MOVED. MIS was 3rd-largest by TOTAL count (56 defects) but contributes ZERO serious defects. PLT and EDL are mid-ranked by count but 3rd and 4th by seriousness. RANKING BY RAW COUNT WOULD SEND EFFORT TO THE WRONG PLACE. THE WEIGHTED APPROACH — a defect index. Weight each severity: serious = 10, moderate = 3, minor = 1 index(IES) = 34×10 + 68×3 + 103×1 = 647 index(EDR) = 26×10 + 68×3 + 36×1 = 500 index(MCC) = 12×10 + 68×3 + 76×1 = 400 index(IET) = 12×10 + 35×3 + 48×1 = 273 index(PLT) = 15×10 + 19×3 + 26×1 = 233 index(EDL) = 14×10 + 12×3 + 19×1 = 195 index(IMI) = 9×10 + 18×3 + 31×1 = 175 index(IDS) = 1×10 + 24×3 + 23×1 = 105 index(MIS) = 0×10 + 15×3 + 41×1 = 86 index(HCI) = 3×10 + 17×3 + 8×1 = 89 index(IID) = 2×10 + 20×3 + 14×1 = 94 index(VPS) = 0×10 + 15×3 + 10×1 = 55 TOTAL = 2,852 top three by index: IES 647 + EDR 500 + MCC 400 = 1,547 = 54.2% of the weighted total STEP 3 — ACT ON THE VITAL FEW. And note that the actions are PROCESS changes, not code fixes: IES (incomplete/erroneous specification, 647) → requirements reviews with a completeness checklist; formal acceptance-criteria sign-off. This is the requirements-validation activity from ACtE0801, and the data says it is the single highest-value investment. EDR (error in data representation, 500) → a reviewed data dictionary; explicit units and types in interface specifications — precisely the interface specification discipline. MCC (misinterpretation of customer communication, 400) → prototypes and scenario walkthroughs with the customer; written confirmation of decisions. THREE PROCESS CHANGES ADDRESSING 54% OF WEIGHTED DEFECTS. That is what statistical SQA is for, and it is why the method ends in process improvement rather than in a bug list.

SIX SIGMA — the most widely used strategy for statistical quality assurance in industry, originating at Motorola. The term derives from six standard deviations: at most 3.4 defects per million opportunities. THE CORE METHODOLOGY, DMAIC, for existing processes: DEFINE customer requirements, deliverables and project goals via well-defined methods of customer communication MEASURE the existing process and its output to determine current quality performance (collect defect metrics) ANALYSE defect metrics and determine the vital few causes IMPROVE the process by eliminating the root causes of defects CONTROL the process to ensure that future work does not reintroduce the causes of defects DMADV, for developing NEW processes: DEFINE · MEASURE · ANALYSE · DESIGN · VERIFY Note that ANALYSE is the Pareto step above, and IMPROVE and CONTROL are what distinguish this from merely measuring. Without CONTROL, the causes come back. STATISTICAL USAGE TESTING — the Cleanroom counterpart, and a genuinely different idea from ordinary testing. THE INSIGHT: not all defects matter equally, because not all code is used equally. Testing should therefore sample the input space according to how USERS ACTUALLY USE the system, not uniformly and not according to what is easy to test. THE METHOD: 1. build a USAGE MODEL — a probability distribution (often a Markov chain) over the operations users perform 2. generate test cases by RANDOMLY SAMPLING from that distribution 3. run them, and measure the failure rate 4. from the observed failures, CERTIFY the reliability statistically — producing an MTTF estimate with a confidence interval WORKED — why usage weighting changes the priority: operation usage frequency defects found ───────────────────────────────────────────────── status lookup 62% 4 new application 21% 9 fee payment 11% 6 annual report 4% 11 bulk export 2% 14 BY RAW DEFECT COUNT, bulk export (14) looks like the worst module and the obvious place to invest. BY EXPECTED USER IMPACT — defects × usage: status lookup 4 × 0.62 = 2.48 new application 9 × 0.21 = 1.89 fee payment 6 × 0.11 = 0.66 annual report 11 × 0.04 = 0.44 bulk export 14 × 0.02 = 0.28 total expected impact = 5.75 STATUS LOOKUP IS THE TOP PRIORITY and bulk export the LOWEST, which is the exact reverse of the raw-count ranking. Four defects in a path used by 62% of requests will be met by far more users than fourteen defects in a path used by 2%. THIS IS THE WHOLE ARGUMENT for usage-based testing, and it generalises: RELIABILITY AS EXPERIENCED BY USERS is a property of the defect distribution WEIGHTED BY THE USAGE DISTRIBUTION, never of the defect count alone. RELIABILITY CERTIFICATION then follows: if random usage-model sampling produces n test cases with f failures, the observed failure rate f/n estimates the probability a real user interaction fails — which is POFOD from the metrics topic, now with a defensible statistical basis rather than a guess. THE LIMITATIONS, honestly: · the usage model must be RIGHT, and early in a product's life nobody knows the usage distribution · usage changes over time, so the model needs maintaining · rare-but-catastrophic paths are UNDER-WEIGHTED by construction. A safety-critical shutdown path may be used 0.001% of the time and still matter more than everything else — so usage weighting must be combined with a criticality analysis, never used alone. · defect classification is subjective; two engineers may assign different causes to the same defect, which introduces noise into the Pareto analysis. Agreeing the categories and reviewing classifications matters.