FOUR TERMS THAT ARE ROUTINELY CONFUSED (and examined): MEASURE provides a quantitative indication of the extent, amount, dimension, capacity or size of some attribute of a product or process. → a single data point: "this module is 340 LOC" MEASUREMENT the ACT of determining a measure. METRIC a quantitative measure of the degree to which a system, component or process possesses a given attribute. (IEEE) Relates individual measures in some way. → "average module size is 210 LOC" — derived from measures INDICATOR a metric or combination of metrics that provides insight into the software process, project or product itself, ENABLING A MANAGER OR ENGINEER TO ADJUST something. → the purpose of measuring at all: an indicator is a metric someone can act on A metric that nobody can act on is data collection, not measurement. That test — "what decision would change if this number changed?" — is the most useful filter to apply to any proposed metric. WHY MEASURE? Four reasons: · to CHARACTERISE, gaining understanding of processes, products and resources, and to establish baselines for comparison · to EVALUATE, determining status with respect to plans · to PREDICT, so we can plan — by understanding relationships between processes and products and building models of those relationships · to IMPROVE, by identifying roadblocks, root causes and inefficiencies THREE DOMAINS OF MEASUREMENT: PROCESS METRICS measure the software process — effort, calendar time, defect removal efficiency, review yield. Collected across ALL projects, over long periods. Used for strategic process improvement. PROJECT METRICS used by a project manager to assess status, track risk, adjust workflow, evaluate the team. Tactical, and used on the project being measured. PRODUCT METRICS measure the software itself — size, complexity, quality attributes. Used to assess and improve the product. PROCESS METRICS ARE COLLECTED ACROSS ALL PROJECTS AND OVER LONG PERIODS OF TIME, and the intent is to provide indicators that lead to long-term process improvement. Project metrics are used tactically. Confusing the two — using process metrics to judge a project, or project metrics to judge a process — is a common error. DIRECT vs INDIRECT MEASURES: DIRECT cost, effort, lines of code, execution speed, memory size, defects reported over a set period INDIRECT functionality, quality, complexity, efficiency, reliability, maintainability — inferred from direct measures Everything we most want to know about software is INDIRECT, which is the fundamental difficulty of the field.

SIZE-ORIENTED METRICS — normalise by lines of code: errors per KLOC · defects per KLOC · cost per LOC · pages of documentation per KLOC · errors per person-month · LOC per person-month · cost per page of documentation THE ARGUMENT AGAINST LOC: · LANGUAGE-DEPENDENT — the same functionality is 8.4 KLOC in Python and 40.0 KLOC in C (computed in the cost modelling topic). Normalising by LOC therefore penalises expressive languages, making the more productive choice look worse. · PENALISES SHORT, WELL-DESIGNED PROGRAMS · CANNOT BE MEASURED before the code exists, which is precisely when estimates are needed · counting rules vary — blank lines? comments? generated code? A 20% difference is available purely from the counting convention. THE ARGUMENT FOR LOC: it is easy to count, universally understood, and abundant historical data exists. Used with awareness of its limits, it remains a reasonable size proxy WITHIN one language and one organisation. FUNCTION-ORIENTED METRICS — normalise by function points, computed as in the cost modelling topic (UFP from EI/EO/EQ/ILF/ EIF, adjusted by VAF = 0.65 + 0.01 × TDI). errors per FP · defects per FP · cost per FP · pages of documentation per FP · FP per person-month ✔ language-independent, computable from the specification ✘ subjective weighting; "no direct physical meaning" — a function point is a made-up unit, defensible only because it correlates with effort FEATURE POINTS and the 3D FUNCTION POINT extend the idea to algorithmic and real-time systems, where the plain function point undercounts computational complexity. WORKED COMPARISON — three projects, all delivered: project KLOC FP defects defects/KLOC defects/FP ───────────────────────────────────────────────────────── A 32.4 612 91 2.81 0.149 B 18.0 540 44 2.44 0.081 C 56.2 580 88 1.57 0.152 BY defects/KLOC, project C looks BEST (1.57). BY defects/FP, project C is essentially the WORST (0.152), barely behind A, and B is clearly best (0.081). WHY THE REVERSAL: C is verbose — 56.2 KLOC for 580 function points is 97 LOC/FP, against B's 33. Dividing by a large denominator flatters it. The FUNCTIONALITY delivered per defect is what a customer experiences, and by that measure C is not good at all. THE LESSON: THE NORMALISER DETERMINES THE CONCLUSION. Always ask what a metric is divided by.

HALSTEAD'S SOFTWARE SCIENCE — the classic product metrics derived from counting operators and operands. Worth knowing because it is frequently examined and shows how far you can get from counting alone. n1 = number of DISTINCT operators n2 = number of DISTINCT operands N1 = total occurrences of operators N2 = total occurrences of operands VOCABULARY n = n1 + n2 LENGTH N = N1 + N2 ESTIMATED LENGTH N̂ = n1·log₂n1 + n2·log₂n2 VOLUME V = N × log₂ n DIFFICULTY D = (n1 / 2) × (N2 / n2) EFFORT E = V × D TIME T = E / 18 seconds WORKED — a module with n1 = 14, n2 = 12, N1 = 42, N2 = 38: n = 14 + 12 = 26 N = 42 + 38 = 80 V = 80 × log₂ 26 = 80 × 4.7004 = 376.0 D = (14/2) × (38/12) = 7 × 3.1667 = 22.17 E = 376.0 × 22.17 = 8,335 N̂ = 14·log₂14 + 12·log₂12 = 14 × 3.8074 + 12 × 3.5850 = 53.30 + 43.02 = 96.3 Note N̂ = 96.3 against the actual N = 80. Halstead's claim was that these should be close; a large divergence was taken to indicate impurity in the code. Here the estimate is 20% high, which is within the range normally observed. V is in BITS — the information content of the implementation — which is why log₂ appears. The metric's appeal is that it is computed mechanically from source with no judgement required; its weakness is that the empirical basis for D, E and T is thin, and T's divisor of 18 ("Stroud number", mental discriminations per second) is not something you should defend too vigorously. ALSO RECALL, from the design heuristics topic: CYCLOMATIC COMPLEXITY V(G) = E − N + 2P = decisions + 1 STRUCTURAL COMPLEXITY S(i) = f_out(i)² MORPHOLOGY size = n + a, arc-to-node r = a/n A structure with 23 nodes and 31 arcs has size 54 and r = 31/23 = 1.35, and comparing r across candidate designs is a cheap way to compare their coupling. SPOILAGE — a simple, useful quality metric: spoilage = effort spent fixing defects after release / total project effort A project spending 191 of 2,852 person-hours on post-release repair has spoilage = 191/2852 = 6.7%. Tracked over time, falling spoilage is direct evidence that process changes are working. THE ETIQUETTE OF METRICS — Pressman's rules, and the most important content in this topic: · USE COMMON SENSE and organisational sensitivity when interpreting metrics data · PROVIDE REGULAR FEEDBACK to the individuals and teams who collect measures · DON'T USE METRICS TO APPRAISE INDIVIDUALS · WORK WITH PRACTITIONERS AND TEAMS to set clear goals and metrics that will be used to achieve them · NEVER USE METRICS TO THREATEN individuals or teams · METRICS DATA THAT INDICATE A PROBLEM AREA SHOULD NOT BE CONSIDERED "NEGATIVE" — such data are merely an indicator for process improvement · DON'T OBSESS ON A SINGLE METRIC to the exclusion of other important metrics THE GOAL/QUESTION/METRIC (GQM) PARADIGM — the standard discipline for choosing metrics, and the answer to "which metrics should we collect?": 1. state a GOAL (what do we want to achieve, for whom, in what context) 2. derive the QUESTIONS whose answers would tell you whether the goal is met 3. choose the METRICS that answer those questions Metrics chosen top-down from a goal are used; metrics collected because a tool produces them are not. Example: GOAL reduce defects reaching customers QUESTIONS where are defects found? what fraction escape? which causes dominate? METRICS defects by phase found, DRE = E/(E+D), Pareto by defect cause Note that this produces exactly the metrics of the previous topics — which is the point. THE GOAL SHOULD GENERATE THE METRIC, never the reverse.