NATURAL LANGUAGE PROCESSING (NLP) is the branch of AI concerned with enabling computers to understand, interpret and generate human language. LANGUAGE IS ANALYSED AT SEVERAL LEVELS, conventionally listed from the surface inwards. Each level has its own ambiguities. 1. PHONOLOGY — the sound structure of language Relevant to speech input and output. "recognise speech" versus "wreck a nice beach" is a phonological ambiguity: the same acoustic signal, two segmentations. 2. MORPHOLOGY — the structure of words Words decompose into MORPHEMES, the smallest meaningful units: unhappiness = un- + happy + -ness running = run + -ing STEMMING reduces a word to a crude root by chopping suffixes: "running" → "run", but also "universities" → "univers" — fast and sometimes wrong. LEMMATISATION reduces it to its dictionary form using knowledge of the language: "ran" → "run", "better" → "good". Slower and correct. INFLECTIONAL morphology marks grammar (plural, tense); DERIVATIONAL morphology creates new words (happy → happiness). 3. LEXICAL — the words themselves and what they can mean TOKENISATION splits text into TOKENS. Harder than it looks: "Dr. Sharma isn't in Kathmandu—he's in New Delhi." Where do the sentences end? Is "isn't" one token or two? Is "New Delhi" one token? Is the em-dash a boundary? PART-OF-SPEECH (POS) TAGGING assigns a grammatical category to each token: noun, verb, adjective… LEXICON: the vocabulary, with each word's possible parts of speech and senses. 4. SYNTAX — how words combine into phrases and sentences PARSING builds a structure showing the grammatical relationships. Two common outputs: CONSTITUENCY (phrase-structure) TREE — nested phrases DEPENDENCY TREE — direct head-modifier links between words A GRAMMAR specifies the legal structures. CONTEXT-FREE GRAMMARS are the standard formalism: S → NP VP NP → Det N | Det N PP | ProperNoun VP → V NP | V NP PP PP → P NP 5. SEMANTICS — the meaning of the words and structures WORD SENSE DISAMBIGUATION picks which sense of a word is intended. SEMANTIC ROLE LABELLING identifies who did what to whom: the AGENT, the PATIENT, the INSTRUMENT. Output is often a LOGICAL FORM — a first-order or similar representation, connecting this section to ACtE0903. 6. PRAGMATICS — meaning in context, beyond the literal "Can you pass the salt?" is a request, not a question about ability. "It's cold in here" may be a request to close the window. Includes SPEECH ACTS, IMPLICATURE, and the fact that the intended meaning may differ from the stated one. 7. DISCOURSE — structure across sentences ANAPHORA RESOLUTION: what does a pronoun refer to? "Ram told Hari that he had passed." — who passed? COREFERENCE: which expressions denote the same entity? COHESION and the overall structure of a text. 8. WORLD KNOWLEDGE — everything else "The trophy would not fit in the suitcase because it was too big." — "it" is the trophy. "…because it was too small." — "it" is the suitcase. THE SENTENCES ARE SYNTACTICALLY IDENTICAL. Resolving the pronoun requires knowing that big things do not fit into small containers, which is not linguistic knowledge at all. THIS IS THE HARDEST LEVEL, and it is why NLP is an AI problem rather than a linguistics problem.

AMBIGUITY OCCURS AT EVERY LEVEL, and the readings MULTIPLY. LEXICAL AMBIGUITY — one word, several senses "bank" — a financial institution, a river's edge, to tilt an aircraft, a row of switches. WordNet lists 10 noun senses and 8 verb senses. "I went to the bank." — which? SYNTACTIC (STRUCTURAL) AMBIGUITY — one sentence, several parses "I saw the man with the telescope." reading 1: I used the telescope to see him (PP attaches to the verb) reading 2: the man had the telescope (PP attaches to the noun) BOTH PARSES ARE GRAMMATICALLY VALID. The grammar cannot choose; only meaning and context can. AND THE COUNT GROWS ALARMINGLY. The number of ways to attach n prepositional phrases follows the CATALAN NUMBERS: 1 PP → 1 parse 2 PPs → 2 3 PPs → 5 4 PPs → 14 5 PPs → 42 6 PPs → 132 "I saw the man on the hill with the telescope in the park near the river" has SIX attachment sites and therefore 132 syntactically valid readings, of which a human considers exactly one. SEMANTIC AMBIGUITY — one structure, several meanings "Every student read a book." — the same book, or one each? This is the quantifier scope problem from the FOPL topic: ∃b ∀s Read(s, b) versus ∀s ∃b Read(s, b) REFERENTIAL AMBIGUITY — pronouns "The city council refused the demonstrators a permit because they feared violence." — who feared? The council. "…because they advocated violence." — the demonstrators. Again, syntactically identical; resolved only by knowing how councils and demonstrators behave. PRAGMATIC AMBIGUITY "Can you open the window?" — a question or a request? THE MULTIPLICATION IS THE KILLER. A five-word sentence in which the words have 3, 4, 2, 5 and 3 senses respectively has 3 × 4 × 2 × 5 × 3 = 360 sense combinations, BEFORE syntactic ambiguity multiplies it further. A parser that enumerates possibilities and a disambiguator that scores them cannot be separated — the space is too large. HOW HUMANS RESOLVE IT, and it is worth stating because it explains the field's trajectory: · WORLD KNOWLEDGE (trophies, suitcases, councils) · CONTEXT from the surrounding discourse · FREQUENCY — the common sense of a word is usually right · PLAUSIBILITY — which reading describes a likely situation THE FIRST TWO ARE HARD TO ENGINEER; THE LAST TWO ARE STATISTICAL. That asymmetry is exactly why NLP shifted from hand-written grammars to statistical methods trained on corpora — the tractable part of human disambiguation is the part that counts frequencies.

TERMS THAT APPEAR IN EXAM QUESTIONS: CORPUS (plural CORPORA) a body of text used for training or analysis. An ANNOTATED corpus has labels — parts of speech, parses, named entities — and building one is the expensive part. TREEBANK a corpus annotated with parse trees. The Penn Treebank made statistical parsing possible. N-GRAM a contiguous sequence of n tokens. Unigram, bigram, trigram. The basis of the simplest language models. LANGUAGE MODEL a probability distribution over sequences: how likely is this sentence? Used to choose between candidate readings. STOP WORDS very frequent words (the, of, and) often removed in retrieval tasks BAG OF WORDS a representation counting words and ignoring order. Crude and surprisingly effective for classification. TF-IDF term frequency × inverse document frequency — weights a word by how often it appears here and how rare it is overall NAMED ENTITY RECOGNITION finding and classifying names of (NER) people, places, organisations, dates EMBEDDING a dense numeric vector representing a word or sentence, positioned so that similar meanings are nearby PERPLEXITY a measure of how well a language model predicts held-out text; lower is better ZIPF'S LAW, worth knowing because it shapes every practical decision in NLP: word frequency is inversely proportional to rank. The consequences: · the top ~100 words account for roughly half of all tokens in running text · a LONG TAIL of words appears once or twice, so any corpus, however large, contains words the system has never seen — the OUT-OF-VOCABULARY problem · this is why subword tokenisation exists: break rare words into familiar pieces rather than treating them as unknown THE TWO TRADITIONS IN NLP, and the field's trajectory: SYMBOLIC / RULE-BASED hand-written grammars, lexicons and rules ✔ precise, explainable, works with no data ✔ handles rare constructions the statistics never saw ✘ BRITTLE — real language violates every rule ✘ enormous hand-crafting effort, and the ambiguity explosion above means the rules must also encode preferences THE ACQUISITION BOTTLENECK AGAIN, in a new domain. STATISTICAL / MACHINE LEARNING learn from annotated or raw corpora ✔ robust to variation, and improves with more data ✔ frequency and plausibility come for free, which is exactly the part of human disambiguation that is tractable ✘ needs data; inherits its biases ✘ opaque, and confidently wrong on inputs unlike the training text THE TRAJECTORY: rules (1950s–80s) → statistical methods with hand-designed features (1990s–2000s) → neural networks learning the features (2010s) → large pre-trained models (2020s). AT EACH STEP MORE OF THE SYSTEM WAS LEARNED AND LESS WAS DESIGNED. WHAT HAS NOT CHANGED: the LEVELS in the first section still describe what must happen, and AMBIGUITY is still the central problem. A modern model resolves it by having seen enormous quantities of text rather than by reasoning about trophies and suitcases — which is why it succeeds on common cases and can fail oddly on constructed ones. THE PROBLEM WAS NOT SOLVED SO MUCH AS OVERWHELMED WITH DATA.