CONVENTIONAL SOFTWARE DEVELOPMENT assumes the requirements can be specified before construction. EXPERT SYSTEM DEVELOPMENT cannot make that assumption, for a specific reason: THE SPECIFICATION IS THE KNOWLEDGE, AND THE KNOWLEDGE IS NOT AVAILABLE UNTIL IT HAS BEEN EXTRACTED — which, from the previous topic, requires showing the expert something wrong and recording their objection. YOU CANNOT SPECIFY FIRST BECAUSE ELICITATION IS ITSELF THE HARD PART. THE CONSEQUENCES: · development is INCREMENTAL and ITERATIVE by necessity, not by preference · a PROTOTYPE is required early, because it is the elicitation instrument · the KNOWLEDGE ENGINEER is a permanent role, not a phase · "complete" is not a meaningful state — the knowledge base grows and decays continuously · TESTING is against CASES with known outcomes rather than against a specification THE COMPARISON WITH CONVENTIONAL DEVELOPMENT: CONVENTIONAL EXPERT SYSTEM ───────────────────────────────────────────────────────────── specify, then build build to elicit the spec requirements from users knowledge from an expert correctness against a spec performance against cases testing finds coding errors testing finds MISSING KNOWLEDGE delivered, then maintained never finished; continuously re-acquired programmers build it knowledge engineer + expert build it together ───────────────────────────────────────────────────────────── THE MOST IMPORTANT ROW IS THE FOURTH: in conventional software a failing test usually means the code is wrong; in an expert system it usually means A RULE IS MISSING. That changes what testing is for — it is knowledge discovery, not defect detection.

THE STANDARD LIFECYCLE (Buchanan and colleagues, as followed by most texts). FIVE PHASES, and note that they cycle rather than complete. 1. IDENTIFICATION Establish WHAT PROBLEM is being solved and whether an expert system is the right tool. · define the problem and its scope precisely · identify the PARTICIPANTS: the domain expert(s), the knowledge engineer, the eventual users · identify the RESOURCES — expert availability, funding, reference material, existing case data · define the GOALS and how success will be measured THE FEASIBILITY QUESTIONS TO ASK FIRST, because most failures are projects that should not have started: · does genuine EXPERTISE exist? If nobody is demonstrably better than average at this task, there is nothing to capture. · is the task performed by REASONING rather than by perception or physical skill? Expert systems cannot capture what a radiologist's eye does. · does it take an expert MINUTES TO HOURS, not seconds and not months? Seconds suggests perception; months suggests it is not one task. · is the domain NARROW and STABLE? · can the expert EXPLAIN their reasoning at all? · is the knowledge mostly HEURISTIC rather than algorithmic? If an algorithm exists, write the algorithm. THESE ARE THE SAME KIND OF SUITABILITY CHECKS the AI applications topic applied to machine learning, and they are just as often skipped. 2. CONCEPTUALISATION Make explicit the KEY CONCEPTS and the RELATIONS between them, and the information flow of the reasoning. · what are the objects, attributes and relationships? · what kinds of data are available, and what is the expert asking for and when? · what are the subtasks, strategies and constraints? OUTPUT: a conceptual model — a vocabulary and a rough structure, not yet a representation. This is the glossary and domain model of the requirements unit, under another name. 3. FORMALISATION Map the concepts onto a REPRESENTATION and choose the tools. · rules, frames, a taxonomy, probabilities, or a combination — the choice guided by the four KR properties · select a SHELL or build an engine · decide the INFERENCE STRATEGY: forward chaining suits data-driven monitoring and design tasks; backward chaining suits goal-driven diagnosis, where you have a hypothesis and want to test it · decide the UNCERTAINTY treatment: certainty factors, probabilities, or none 4. IMPLEMENTATION Encode the formalised knowledge into a working prototype. THE PROTOTYPE IS NOT A DRAFT PRODUCT — IT IS THE ELICITATION TOOL. Its purpose is to produce wrong answers in front of the expert. 5. TESTING / EVALUATION Run cases, compare with the expert, and refine. · cases with KNOWN outcomes, held back from development · the expert reviews both the conclusions AND the explanations, because a right answer for a wrong reason will fail on the next case · BLIND EVALUATION for final validation, as MYCIN was assessed AND THEN RETURN TO AN EARLIER PHASE. Testing reveals missing knowledge (back to conceptualisation), a bad representation choice (back to formalisation), or simply more rules to add (implementation). THE PROTOTYPE PROGRESSION most projects follow: DEMONSTRATION PROTOTYPE ~50–100 rules, one narrow subproblem. Proves the approach and secures funding. RESEARCH PROTOTYPE ~200–500 rules, credible on most of the domain, still fragile FIELD PROTOTYPE refined against real use, adequate reliability PRODUCTION MODEL rewritten for performance and integration; the knowledge base may be ported to a faster engine COMMERCIAL SYSTEM maintained, documented, supported NOTE THAT REWRITING IS EXPECTED between field prototype and production. The prototype optimised for elicitation, not for speed or integration — and pretending otherwise produces a demonstration that cannot be deployed.

A WORKED FEASIBILITY ASSESSMENT — three candidate projects against the criteria: CANDIDATE A: diagnosing faults in a district's water pumps genuine expertise? YES — one technician is markedly better than the others reasoning, not perception? MOSTLY — though some diagnosis is by sound, which is a warning sign minutes to hours? YES — typically 20 minutes narrow and stable? YES — a fixed catalogue of pump models can explain? YES heuristic? YES — no algorithm exists VERDICT: GOOD CANDIDATE, with the sound-based diagnosis explicitly excluded from scope and left to the human. CANDIDATE B: recognising crop disease from leaf photographs genuine expertise? YES reasoning? NO — this is PERCEPTION, done in under a second VERDICT: WRONG TOOL. This is a machine-learning problem, and the AI applications topic's checklist applies instead. A rule base asking "is the lesion irregular?" merely moves the perception to the user. CANDIDATE C: deciding a student's next revision topic genuine expertise? WEAK — no recognised specialist narrow and stable? YES heuristic? YES VERDICT: MARGINAL as an expert system. There is no expert whose knowledge is worth capturing, but there IS data on what works — so it is a learning problem, or a small set of hand-written rules that need no engine at all. THE THIRD OPTION IS OFTEN RIGHT: if fifteen rules suffice, an expert system is over-engineering, which is question 6 of the AI applications checklist. WHY EXPERT SYSTEM PROJECTS FAIL — the recorded reasons, and each maps to a phase: IDENTIFICATION FAILURES · the task was perceptual or algorithmic, not heuristic · no genuine expertise existed · the domain was too broad — the commonest error, and it appears as a system that works on the demonstration cases and nothing else · the expert was unavailable in practice, whatever was promised CONCEPTUALISATION AND FORMALISATION FAILURES · the wrong representation, so natural knowledge had to be contorted to fit · uncertainty ignored, producing a system that asserts conclusions it should hedge IMPLEMENTATION AND TESTING FAILURES · the rule base outgrew comprehensibility — the n(n−1)/2 interaction problem from the acquisition topic · no regression suite, so each fix broke something else · no explanation facility, so users would not trust it ORGANISATIONAL FAILURES, and these are the ones that actually killed deployed systems: · NOBODY OWNED MAINTENANCE after delivery. The knowledge decayed and the system became wrong, which is worse than absent because people still consult it. · the expert who supplied the knowledge left, and nobody could interpret the rule base · users were not involved, so a technically sound system was rejected · the system threatened the expert's status, so cooperation quietly evaporated — a POLITICAL factor, exactly as the requirements elicitation topic warned THE SINGLE MOST USEFUL LESSON: PLAN FOR MAINTENANCE FROM THE START. Name the person responsible, budget the annual cost (50 person-days per 1,000 rules per year, from the previous topic), and keep the regression suite of cases. A knowledge base without a maintainer has a shelf life, and the systems that survived the second AI winter were the ones somebody was paid to keep current.