KNOWLEDGE ACQUISITION is the process of extracting, structuring and organising knowledge from a source — usually a human expert — so that it can be encoded in a knowledge base. IT IS CALLED THE BOTTLENECK because it dominates the cost and the schedule of every expert-system project, and because no amount of better tooling has removed it. WHY IT IS HARD — six reasons, and each has a mitigation: 1. TACIT KNOWLEDGE. Experts know far more than they can say. Skill that has become automatic is no longer introspectable — which is precisely the Moravec paradox from the AI concepts topic, met from the inside. MITIGATION: observe the expert working, and ask about specific past cases rather than general rules. 2. EXPERTS DESCRIBE WHAT THEY BELIEVE THEY DO, NOT WHAT THEY DO. The stated procedure is often a reconstruction — the textbook method rather than the shortcut actually used. MITIGATION: compare stated rules against observed decisions and investigate the disagreements. The disagreements are the real knowledge. 3. VOCABULARY MISMATCH. The knowledge engineer does not know the domain; the expert does not know what a rule base can express. Each misunderstands the other for weeks. MITIGATION: build a glossary first, exactly as the software requirements unit prescribes. 4. KNOWLEDGE IS DISTRIBUTED. Several experts hold different parts, and they DISAGREE — often for good reasons that nobody has articulated. MITIGATION: surface the disagreement rather than averaging it. A disagreement usually means a hidden condition that distinguishes the two cases. 5. THE EXPERT'S TIME IS THE SCARCEST RESOURCE. The best expert is the busiest, and knowledge acquisition asks for many hours of it. MITIGATION: prepare thoroughly, and never use expert time for anything a document could have answered. 6. KNOWLEDGE DECAYS. Domains change — new drugs, new regulations, new products — so the knowledge base needs continuous re-acquisition, not a one-off effort. THIS IS THE ONE THAT KILLED THE INDUSTRY, and it is worth seeing quantified. THE ARITHMETIC OF THE BOTTLENECK. Published experience puts productive knowledge acquisition at roughly TWO TO THREE RULES PER DAY of combined expert and engineer time, once verification and refinement are counted: MYCIN, about 450 rules at 2 rules/day → 225 person-days ≈ 10.7 person-months XCON, which grew past 3,000 rules at 2 rules/day → 1,500 person-days ≈ 71 person-months, nearly six person-years of specialist time AND THE MAINTENANCE: if 10% of a 1,000-rule base changes annually, that is 100 rules a year, or 50 person-days EVERY YEAR just to stand still — roughly a quarter of a full-time post, forever. THAT ONGOING COST, NOT THE BUILD COST, IS WHAT MADE EXPERT SYSTEMS UNECONOMIC. A system that is expensive once can be justified; a system that is expensive every year, and degrades if you stop paying, cannot.

THE ELICITATION TECHNIQUES, with what each is good for. Note how closely they parallel the requirements elicitation of ACtE0801 — they are the same problem. INTERVIEWS UNSTRUCTURED — open exploration to map the domain. Good early, poor for detail. STRUCTURED — a prepared agenda covering specific decisions. Efficient once you know what to ask. ✘ WEAK ON TACIT KNOWLEDGE, for the reasons above. CASE STUDIES AND PROTOCOL ANALYSIS (think-aloud) Give the expert a real case and ask them to THINK ALOUD while solving it. Record everything and analyse the transcript for the decisions actually made. ✔ THE MOST PRODUCTIVE SINGLE TECHNIQUE, because it captures reasoning in action rather than reconstruction after the fact ✔ reveals the order in which information is sought, which is itself knowledge — a doctor's question sequence encodes their diagnostic strategy ✘ expensive to transcribe and analyse OBSERVATION Watch the expert at work without interrupting. Reveals the workarounds and shortcuts that no interview surfaces. ✘ slow, and shows only existing practice TEACHBACK The knowledge engineer explains the domain BACK to the expert, who corrects the errors. Remarkably effective, because correcting a wrong statement is far easier than producing a right one from nothing. CARD SORTING AND LADDERING Give the expert concepts on cards and ask them to group and rank them, then ask WHY each grouping. Elicits the taxonomy and the distinguishing attributes — the isa hierarchy of the KR section, obtained directly. REPERTORY GRID (Kelly's personal construct theory) Present three items and ask: in what way are two of these alike and different from the third? The answer names an ATTRIBUTE the expert uses to discriminate, often one they had never stated. ✔ excellent at eliciting the FEATURES that matter, which is exactly what tacit knowledge consists of QUESTIONNAIRES Cheap at scale, and only answer questions you already knew to ask. DOCUMENT ANALYSIS Manuals, regulations, case records, worked examples. Much neglected, and often the cheapest source of the uncontroversial 60% of a knowledge base — leaving expert time for the hard remainder. AUTOMATED / MACHINE LEARNING INDUCE rules from a database of solved cases instead of interviewing. Decision-tree induction on a case archive is the classic form, and it is the subject of ACtE0905. ✔ scales, and captures relationships no expert articulated ✘ needs many labelled cases, and the induced rules may be accurate without being comprehensible THE PRACTICAL SEQUENCE most projects converge on: 1. document analysis, to learn the vocabulary cheaply 2. unstructured interviews, to map the territory 3. protocol analysis on real cases, for the actual reasoning 4. structured interviews, to fill specific gaps 5. teachback, to validate 6. iterate with a running prototype NOTE STEP 6: NOTHING ELICITS CORRECTIONS LIKE A WRONG ANSWER FROM A WORKING SYSTEM. An expert who cannot state a rule will readily say "no, that conclusion is wrong, because…" — and the "because" is the rule. This is the prototyping-as- validation finding from the requirements unit, and it is the single most effective technique on the list.

THE ROLE OF THE KNOWLEDGE ENGINEER — the specialist who mediates between the expert and the system: · elicits the knowledge, using the techniques above · chooses the REPRESENTATION — rules, frames, a taxonomy, or a combination, per the KR issues topic · encodes and structures it · tests the system against cases with KNOWN answers · refines the rules where the system disagrees with the expert · manages the knowledge base over its life THE SKILL PROFILE IS UNUSUAL: enough domain fluency to ask intelligent questions, enough formal training to know what the representation can express, and enough interviewing skill to get at tacit knowledge. THE SCARCITY OF THIS COMBINATION WAS ITSELF A BOTTLENECK, and is part of why expert-system projects were expensive. VERIFICATION AND VALIDATION OF A KNOWLEDGE BASE — and the vocabulary is the same as the software engineering unit's: VERIFICATION is the knowledge base internally sound? VALIDATION does it give the right answers? THE STRUCTURAL DEFECTS a verification tool looks for — these are examinable: REDUNDANT RULES two rules with the same premises and conclusion CONFLICTING RULES the same premises, contradictory conclusions SUBSUMED RULES one rule's premises are a subset of another's with the same conclusion, so the more specific one never adds anything CIRCULAR RULES A ⇒ B, B ⇒ C, C ⇒ A — an infinite loop waiting to happen UNREACHABLE RULES a rule whose premises can never all be satisfied DEAD-END RULES a conclusion no other rule uses and no user sees MISSING RULES a combination of inputs for which no rule applies — the completeness gap, and the hardest to detect VALIDATION IS DONE AGAINST CASES: · run the system on cases with known outcomes and compare · the classic standard is a BLIND EVALUATION: give a panel of human experts the system's conclusions mixed with conclusions from human specialists, unlabelled, and ask them to rate all of them. MYCIN was validated this way and scored at or above the human specialists — which is a genuinely strong result, and a good model for evaluating any decision system. THE MAINTENANCE PROBLEM, quantified, because it is the real lesson of this topic: a rule base of n rules has up to n(n−1)/2 possible pairwise interactions 100 rules → 4,950 pairs 500 rules → 124,750 pairs 1,000 rules → 499,500 pairs NOBODY CAN REASON ABOUT HALF A MILLION INTERACTIONS. Beyond a few hundred rules, adding one rule can change behaviour that seems unrelated, and there is no local reasoning available. THIS IS THE SAME n(n−1)/2 COUPLING ARGUMENT as the software design unit's — a rule base is a system with no modularity, so every rule is potentially coupled to every other. MITIGATIONS: partition the rule base into modules with defined interfaces; keep a REGRESSION SUITE OF CASES and re-run it on every change — which is exactly the automated-testing discipline of ACtE0803, applied to knowledge. THE HISTORICAL VERDICT worth stating plainly: the expert-system architecture was sound and its successes were real, but ACQUIRING AND MAINTAINING KNOWLEDGE BY HAND DOES NOT SCALE. Machine learning did not displace expert systems by reasoning better — it displaced them by extracting knowledge from data instead of from people, which removes the bottleneck this topic is about.