The classification follows the structures of the earlier topic, with the learning agent as a fifth cutting across them. 1. SIMPLE REFLEX AGENTS Act only on the CURRENT PERCEPT, via condition-action rules. REQUIREMENT: a fully observable environment. EXAMPLES: a thermostat · a vacuum that sucks when it senses dirt · a mail rule that files by sender · an automatic door COMPLEXITY: the rule table grows with the number of distinguishable situations, so it is only viable when that number is small. FAILURE MODE: cannot act on anything it cannot currently see, and loops in partially observable environments unless randomised. 2. MODEL-BASED REFLEX AGENTS Maintain INTERNAL STATE using a transition model and a sensor model, then apply rules to the estimated state. REQUIREMENT: knowledge of how the world changes. EXAMPLES: a robot vacuum building a map · an object tracker · a lane-change assistant that remembers the car alongside WHAT IT BUYS: partial observability becomes workable. WHAT IT COSTS: the state estimate can be WRONG, and a confidently wrong state estimate is worse than admitted ignorance. 3. GOAL-BASED AGENTS Hold explicit goals and choose actions expected to achieve them, typically by SEARCH or PLANNING. EXAMPLES: a route planner · a puzzle solver · a warehouse robot instructed to fetch an item WHAT IT BUYS: FLEXIBILITY. The knowledge is explicit, so changing the goal changes the behaviour without rewriting rules. WHAT IT COSTS: computation. Search is expensive, and a dynamic environment may change before the plan finishes. 4. UTILITY-BASED AGENTS Use a UTILITY FUNCTION to compare outcomes, choosing the action with the highest EXPECTED UTILITY. EXAMPLES: portfolio management · a taxi choosing between a fast risky and a slow safe route · any scheduler balancing cost against time WHAT IT BUYS: it can express PREFERENCE and handle CONFLICTING or UNCERTAIN goals, which goals alone cannot. WHAT IT COSTS: someone must supply the utility numbers, and those numbers are a value judgement rather than a fact. 5. LEARNING AGENTS Any of the above, plus the four components from the structure topic — learning element, performance element, critic, problem generator. EXAMPLES: a spam filter that adapts · a recommender · a game player improving through self-play WHAT IT BUYS: operation in unknown or changing environments, and escape from hand-coding knowledge. WHAT IT COSTS: behaviour changes over time, so it is harder to validate and to certify. THE PROGRESSION IS ONE OF CAPABILITY AGAINST COST, and each step is justified only by a requirement the previous type cannot express — which is the selection principle of this topic.

FOUR PROBLEMS, and the cheapest type that actually works. The reasoning is the examinable skill. PROBLEM 1 — a room thermostat Q: does the action depend on anything not currently sensed? No — only on the current temperature and the setpoint. Q: are there few distinguishable situations? Yes. → SIMPLE REFLEX. Anything more is waste, and a thermostat that planned would be worse, not better. PROBLEM 2 — a robot vacuum in a house Q: fully observable? NO — it cannot see the whole house, and must know which rooms it has already cleaned. Q: does it need to reach a described future state? Only weakly ("everywhere clean"), and coverage can be achieved by systematic sweeping rather than by planning. → MODEL-BASED REFLEX with a map. Many commercial units are exactly this, and adding goal-directed planning improves efficiency rather than enabling anything new. PROBLEM 3 — a delivery route planner Q: does the right action depend on a desired future state? YES — which turn is correct depends entirely on the destination, and the destination is many steps away. Q: do several plans reach the goal with meaningfully different quality? If all that matters is arriving, GOAL-BASED suffices. If fuel, time and driver hours must be traded off, it does not. → GOAL-BASED if the objective is single; UTILITY-BASED as soon as the objectives conflict. Real logistics is the second case, which is why real routing systems optimise a weighted cost rather than searching for any valid route. PROBLEM 4 — a spam filter Q: fully observable? The message is fully visible, so a reflex agent is conceivable — and early filters were exactly hand-written rule sets. Q: does the environment CHANGE IN RESPONSE TO THE AGENT? YES. Spammers observe which messages get through and adapt. → LEARNING AGENT, necessarily. This is the decisive consideration and it is not about observability at all: the environment is adversarial, so any fixed rule set decays. A REFLEX AGENT WOULD BE CORRECT ON THE DAY IT SHIPPED AND WRONG WITHIN A MONTH. THE DECISION PROCEDURE, generalised: 1. Does the correct action depend only on the current percept? → yes: SIMPLE REFLEX 2. Does it depend on things not currently observable? → yes: add INTERNAL STATE 3. Does it depend on a desired future state several steps away? → yes: GOAL-BASED, with search 4. Do outcomes differ in quality, conflict, or carry uncertainty? → yes: UTILITY-BASED 5. Is the environment unknown, changing, or adversarial, or is the knowledge infeasible to hand-code? → yes: add LEARNING ANSWER THEM IN ORDER AND STOP AT THE FIRST "no". The commonest professional error is skipping straight to step 5 because learning is fashionable, when step 1 would have answered the requirement in an afternoon.

CLASSIFICATIONS THAT CUT ACROSS THE FIVE TYPES, all of which appear in exam questions: BY MOBILITY STATIC agents remain on one host; MOBILE agents migrate across a network to where the data is. Mobile agents were heavily researched in the 1990s and are now rare, largely because the security problem — executing someone else's code on your machine — proved harder than the benefit. BY NUMBER SINGLE-AGENT systems have one decision maker. MULTI-AGENT systems have several, and then: COOPERATIVE agents share a goal — the problem is coordination and communication COMPETITIVE agents have opposing goals — the problem is game-theoretic, and this is what adversarial search in ACtE0902 addresses SELF-INTERESTED agents have their own goals which may merely conflict — the problem is negotiation and mechanism design BY REASONING STYLE REACTIVE architectures respond to percepts with no internal symbolic model. Brooks's SUBSUMPTION ARCHITECTURE is the landmark: layered behaviours, each a simple stimulus-response rule, with higher layers able to subsume lower ones. Brooks's claim — "intelligence without representation" — was that complex behaviour emerges from simple layered reactions, and his insect-like robots walked over rough terrain that carefully planned robots could not. DELIBERATIVE architectures maintain an explicit symbolic world model and reason about it, typically BDI (belief, desire, intention). HYBRID architectures combine them in LAYERS, which is what practical systems use. THE HYBRID ARGUMENT, and it is the resolution of the topic: Reactive architectures are fast and robust and cannot plan. Deliberative architectures plan and are too slow to react. Neither alone is adequate for a robot in the physical world, so real systems layer them: ┌──────────────────────────────────────┐ │ DELIBERATIVE LAYER │ seconds │ planning, goals, world model │ ├──────────────────────────────────────┤ │ EXECUTIVE / SEQUENCING LAYER │ ~100 ms │ selects and monitors behaviours │ ├──────────────────────────────────────┤ │ REACTIVE LAYER │ ~10 ms │ collision avoidance, balance │ └──────────────────────────────────────┘ faster and simpler downward; the lower layer may OVERRIDE the upper THE TIMING ARGUMENT MAKES IT CONCRETE, and it is the same calculation as the properties topic: with a 10 ms reactive loop and a 2-second replan, 200 reactive cycles pass during one deliberation. A single loop running at the deliberative rate could not avoid an obstacle; a single loop running at the reactive rate could not plan a route. THE LAYERS ARE NOT an architectural preference but an arithmetic necessity. AND THE OVERRIDE DIRECTION MATTERS: the reactive layer overrides the planner, not the other way round. A plan that says "go forward" must lose to a sensor that says "obstacle", because the cost of being wrong is asymmetric.