Testing Chomsky's Hypothesis With Small Language Models

Developmental psycholinguists are currently using small language models (SLMs) to test Noam Chomsky’s famous "poverty of the stimulus" hypothesis. This article explores how researchers train these compact AI models on transcripts of actual speech directed at children to determine if human-like grammar can be learned purely from limited linguistic input, or if innate biological knowledge is truly necessary for language acquisition.

The Poverty of the Stimulus Hypothesis

Proposed by linguist Noam Chomsky, the poverty of the stimulus hypothesis argues that the language children hear in their early years is too incomplete, noisy, and grammatically simple to explain how quickly they master complex grammar. Therefore, Chomsky suggested that humans must be born with an innate "Universal Grammar" hardwired into the brain. Historically, testing this was impossible because researchers could not ethically isolate a human child to control their linguistic input.

Why Small Language Models Are Necessary

Large language models like GPT-4 cannot be used to test this hypothesis because they are trained on trillions of words—thousands of times more language than a human hears in a lifetime. Passing a grammar test after reading the entire internet does not disprove Chomsky.

To create a fair test, psycholinguists build small language models. These models restrict the AI's training data to match the exact volume of words a human child hears by age four or five, typically between 10 million and 50 million words. Because the models are "blank slates" with no built-in grammar rules, they rely entirely on statistical learning.

Training on Child-Directed Speech

Researchers train these SLMs using datasets like the CHILDES (Child Language Data Exchange System) corpus. This database contains thousands of hours of transcribed, real-world conversations between parents and young children.

Using this data ensures the AI only learns from "child-directed speech." This type of speech is characterized by short sentences, simple vocabulary, frequent repetition, and occasional grammatical errors, accurately mirroring the exact "impoverished" stimulus a human toddler experiences.

Testing the AI on Complex Grammar

Once the SLM is trained solely on these transcripts, researchers subject it to psycholinguistic tests. A common method is the "minimal pair" test, where the AI is presented with two similar sentences—one grammatically correct and one incorrect—and must calculate which sentence is more likely to be valid.

Psycholinguists specifically test for complex linguistic phenomena that rarely appear in child-directed speech, such as:

• Subject-verb agreement across clauses: Understanding that "The keys to the cabinet are on the table" is correct, while "The keys to the cabinet is on the table" is wrong.
• Filler-gap dependencies: Tracking how words move around when forming complex questions.
• Hierarchical structure: Recognizing that language relies on nested phrases rather than just a linear sequence of words.

What the Results Reveal

Recent studies using SLMs have shown surprising success. Despite only being exposed to a few million words of simple child-directed speech, these small models consistently learn complex grammatical rules and prefer grammatically correct sentences over incorrect ones at near-human levels.

These findings directly challenge Chomsky’s hypothesis. By proving that a blank-slate AI can extract deep grammatical rules from limited, everyday toddler-directed speech, developmental psycholinguists have demonstrated that statistical learning alone can overcome the "poverty of the stimulus," reducing the theoretical need for an innate Universal Grammar.