THE McCULLOCH-PITTS (MP) NEURON, also called the THRESHOLD LOGIC UNIT, is the simplest possible neuron model. ITS DEFINING RESTRICTIONS: · inputs are BINARY: xᵢ ∈ {0, 1} · the output is BINARY: y ∈ {0, 1} · weights are FIXED — the model has NO LEARNING · connections are either EXCITATORY (weight +1) or INHIBITORY · there is a fixed THRESHOLD θ · all inputs arrive synchronously, in one time step THE COMPUTATION: net = Σ wᵢ xᵢ y = 1 if net ≥ θ = 0 if net < θ This is the STEP or THRESHOLD activation function. INHIBITORY INPUTS in the original formulation are ABSOLUTE: if an inhibitory input is active, the neuron cannot fire regardless of the excitatory input. (The commoner modern treatment gives inhibitory connections a weight of −1 and lets them be outvoted; both appear in textbooks, so state which you are using.) WHAT IT CANNOT DO, stated up front because it defines the model: · NO LEARNING — the weights and threshold are set by the designer, so an MP network is a designed circuit, not a trained one. This is the single most important limitation, and removing it is what the perceptron did. · only BINARY inputs and outputs · all weights are ±1, so it cannot express that one input matters more than another · it can only implement LINEARLY SEPARABLE functions — the limitation that becomes decisive in the perceptron topic

THE CENTRAL DEMONSTRATION: MP neurons compute the basic logical functions. These derivations are the guaranteed exam question, so each is worked with its truth table. AND — fires only when both inputs are 1 weights w₁ = w₂ = 1, threshold θ = 2 x₁ x₂ │ net = x₁+x₂ │ net ≥ 2? │ y ────────┼───────────────┼────────────┼──── 0 0 │ 0 │ no │ 0 0 1 │ 1 │ no │ 0 1 0 │ 1 │ no │ 0 1 1 │ 2 │ YES │ 1 ✓ correct for AND OR — fires when either input is 1 weights w₁ = w₂ = 1, threshold θ = 1 x₁ x₂ │ net │ net ≥ 1? │ y ────────┼───────┼────────────┼──── 0 0 │ 0 │ no │ 0 0 1 │ 1 │ YES │ 1 1 0 │ 1 │ YES │ 1 1 1 │ 2 │ YES │ 1 ✓ correct for OR NOTE THAT AND AND OR DIFFER ONLY IN THE THRESHOLD. Same weights, θ = 2 gives AND and θ = 1 gives OR — the threshold is doing all the work, which is a neat illustration of what a threshold unit actually is. NOT — a single inhibitory input w₁ = −1, θ = 0 x₁ │ net │ net ≥ 0? │ y ────┼───────┼────────────┼──── 0 │ 0 │ YES │ 1 1 │ −1 │ no │ 0 ✓ correct for NOT NOR — with weights −1 and threshold 0 x₁ x₂ │ net │ net ≥ 0? │ y ────────┼───────┼────────────┼──── 0 0 │ 0 │ YES │ 1 0 1 │ −1 │ no │ 0 1 0 │ −1 │ no │ 0 1 1 │ −2 │ no │ 0 ✓ correct for NOR AND THE RESULT THAT MATTERS: NAND AND NOR ARE FUNCTIONALLY COMPLETE — every boolean function can be built from either alone. THEREFORE ANY BOOLEAN FUNCTION CAN BE COMPUTED BY A NETWORK OF MP NEURONS. That is the 1943 theorem, and it is why the paper mattered: it established that neural computation is computation in the full sense. XOR — the one that cannot be done with a SINGLE neuron: x₁ x₂ │ desired y ────────┼──────────── 0 0 │ 0 0 1 │ 1 1 0 │ 1 1 1 │ 0 WHY NO SINGLE MP NEURON WORKS — the proof, which is short and worth being able to reproduce: we need w₁x₁ + w₂x₂ ≥ θ exactly for the two middle rows. from (0,0) → 0 : 0 < θ, so θ > 0 from (0,1) → 1 : w₂ ≥ θ from (1,0) → 1 : w₁ ≥ θ from (1,1) → 0 : w₁ + w₂ < θ But w₁ ≥ θ and w₂ ≥ θ with θ > 0 give w₁ + w₂ ≥ 2θ > θ, CONTRADICTING the last requirement. NO VALUES OF w₁, w₂, θ SATISFY ALL FOUR. ∎ THE SOLUTION IS A NETWORK, and XOR requires an intermediate layer: XOR(x₁, x₂) = (x₁ OR x₂) AND NOT(x₁ AND x₂) so it is built from three MP neurons — an OR, a NAND, and an AND combining them: x₁ ──┬──▶ [OR, θ=1] ──┐ │ ├──▶ [AND, θ=2] ──▶ y x₂ ──┴──▶ [NAND ] ─┘ CHECK (1,1): OR gives 1, NAND gives 0, AND of (1,0) with θ=2 gives 0. ✓ CHECK (0,1): OR gives 1, NAND gives 1, AND gives 1. ✓ CHECK (0,0): OR gives 0, NAND gives 1, AND gives 0. ✓ THIS IS THE SINGLE MOST IMPORTANT FACT IN THE NEURAL NETWORK SECTION: XOR NEEDS A HIDDEN LAYER. Minsky and Papert's 1969 proof of the equivalent result for perceptrons stopped neural network research for over a decade, as the AI history topic recorded — even though this multilayer construction was already known. What was missing was not the architecture but a way to TRAIN it, which arrived with backpropagation in 1986.

THE HISTORICAL SIGNIFICANCE, stated precisely: 1. IT MADE NEURAL COMPUTATION MATHEMATICAL. Before 1943 the brain was studied biologically; afterwards it could be studied as a computing system with provable properties. 2. IT PROVED UNIVERSALITY FOR BOOLEAN FUNCTIONS. Networks of these units compute anything a logic circuit can, which established that neural mechanisms are sufficient in principle for any computation of that kind. 3. IT INTRODUCED THE ARCHITECTURE that everything since has used: weighted inputs, a summation, a nonlinear threshold. Every unit in every modern network is a McCulloch-Pitts neuron with the restrictions relaxed. 4. IT INFLUENCED VON NEUMANN'S computer architecture work, and the paper is cited in the design documents of early machines — so the model contributed to conventional computing as well as to AI. THE PATH FROM MP TO MODERN NETWORKS — each step relaxes one restriction, and this progression is the structure of the rest of this section: MP NEURON (1943) binary in/out, fixed ±1 weights, no learning ↓ add REAL-VALUED WEIGHTS and a LEARNING RULE PERCEPTRON (1958) learns from examples; still a step function, still linearly separable only ↓ replace the step with a DIFFERENTIABLE activation ADALINE / DELTA RULE gradient descent on a continuous error becomes possible ↓ add HIDDEN LAYERS MULTILAYER PERCEPTRON can represent XOR and, in principle, any continuous function ↓ add an algorithm to TRAIN the hidden layers BACKPROPAGATION (1986) the field becomes practical ↓ add depth, data, and specialised architectures DEEP LEARNING (2012–) WHAT MP CANNOT DO, as a summary: · LEARN — the fatal limitation · handle real-valued inputs or produce graded outputs · weight inputs differently, since all weights are ±1 · compute a non-linearly-separable function with one unit · represent temporal patterns (though the original paper did consider networks with delays) AND ONE POINT WORTH KEEPING IN PERSPECTIVE: the MP neuron is not a simplification of a biological neuron so much as an IDEALISATION of one aspect of it. Real neurons integrate over time, have thousands of inputs with graded strengths, and respond with spike trains. THE MODEL KEPT ONLY THE THRESHOLD-SUMMATION IDEA — which turned out to be the part that mattered for computation.