George Box's 1976 R.A. Fisher Memorial Lecture — the paper that gave us "all models are wrong." Box argues that science advances through a motivated feedback loop between theory and practice, not by speculation or data accumulation alone. He uses Fisher's decade at Rothamsted Experimental Station as the master case study: dung data → orthogonal polynomials, a lady tasting tea → experimental design, cotton yarn → extreme value theory, potatoes → ANOVA. Every major statistical advance Fisher made was prompted by a specific practical problem. When the loop breaks, two pathologies emerge: cookbookery (forcing all problems into routine techniques) and mathematistry (theory untethered from practice). The paper is both a philosophy of science and a biography of method — and its core arguments about models, parsimony, and selective worrying are more relevant to AI engineering than most papers written in the last five years.
The article quotes: "Since all models are wrong the scientist cannot obtain a 'correct' one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena." The paper's most famous line, often misquoted as "all models are wrong, but some are useful." Box's actual point is sharper: parsimony isn't an aesthetic preference, it's a logical consequence of models being approximations. Over-elaboration doesn't fix wrongness — it compounds it. This is the statistical version of Gall's Law. The article quotes: "It is inappropriate to be concerned about mice when there are tigers abroad." Box's maxim on selective worrying. The scientist must distinguish between model inadequacies that matter and those that don't. This is the hard part — it requires domain judgment, not just statistical technique. The parallel in software: don't optimize the error handling of a component that might be deleted next sprint. The article quotes: "The statistician knows, for example, that in nature there never was a normal distribution, there never was a straight line, yet with normal and linear assumptions, known to be false, he can often derive results which match, to a useful approximation, those found in the real world." This is the pragmatic philosophy of applied mathematics in one paragraph. You make assumptions you know are false because they're useful. The test isn't truth — it's whether the consequences match reality well enough to act on. This is exactly how we use LLMs: we know they hallucinate, we know they're stochastic, yet we build systems around them because the output is useful. The article quotes: "In the inferential stage, the analyst acts as a sponsor for the model. Conditional on the assumption of its truth he selects the best statistical procedures for analysis of the data. Having completed the analysis, however, he must switch his role from sponsor to critic." The sponsor/critic switch is Fisher's great methodological insight, and it maps cleanly onto how we should use coding agents. First, be the sponsor: assume the generated code is correct and build on it. Then switch to critic: what's wrong with it, where are the gaps? The people who get the most out of AI tools are the ones who can inhabit both roles fluently. The article quotes: "Fisher says if we want to have a small variance for Q we should keep r small — a demonstration of the value of parsimony, helping to justify his use of polynomials of only fifth degree." Box is describing Fisher's 1921 analysis of crop data, where Fisher showed mathematically that lower-degree polynomials give more stable predictions. The math justified the parsimony. This is the inverse of modern machine learning's "throw more parameters at it" approach — Fisher derived parsimony from the structure of the problem, not from a regularization hyperparameter. The article quotes: "I have no hesitation in advising that such a centre as you have under discussion should plan to integrate teaching closely with project work in which practical experience can be gained by those who are capable of learning from it; in contradistinction to the ruinous process of segregating the keener minds into a completely sterile atmosphere." Fisher in 1952, quoted by Box, on how to train statisticians. The same argument applies to training AI engineers: you can't learn this from textbooks alone. You need contact with real problems, real data, real failures. The article quotes: "The statistician's job did not begin when all the work was over — it began long before it was started. The statistician was not a curator of dusty relics. His responsibility to the scientific team was that of the architect." Box describes how Fisher transformed the role of the statistician from post-hoc analyst to experimental architect. The parallel with the shift from "QA at the end" to "quality engineered in from the start" is exact. This is also the argument for involving data scientists and ML engineers in product design, not just handing them a dataset and asking for insights.
- Theory-practice iteration: Science as a feedback loop, not a linear process. The error signal — discrepancy between theory and observation — is the learning mechanism. This is cybernetics before cybernetics was cool. - Parsimony: Economical models beat elaborate ones. Not because simple is beautiful, but because complex models amplify the consequences of being wrong. "Overelaboration and overparameterization is often the mark of mediocrity." - Sponsor/critic switch: The methodological move from "assume the model is true and analyze" to "assume the model is false and diagnose." This is the intellectual engine of data analysis, and it's exactly what good code review does. - Mathematistry: Box's neologism for theory untethered from practice. The pathology of closed-loop academia: problems redefined to fit available mathematical tools rather than tools developed to solve real problems. As relevant to AI research in 2026 as it was to statistics in 1976. - Cookbookery: The opposite pathology — mindlessly applying standard techniques without understanding their assumptions or the problem's actual structure. The statistical version of "import tensorflow as tf and pray." - Randomization as the guarantor of validity: Fisher's insistence that physical randomization, not mathematical assumptions, is what makes inference valid. The distribution-free properties come from the act of randomizing, not from the choice of test statistic. - Fisher's method: Real problem → tentative model → mathematical development → confrontation with data → modified model. Every major statistical technique Fisher invented (ANOVA, experimental design, extreme value theory, distributed lags) emerged from a specific practical problem posed by a colleague at the Rothamsted tea urn.
Box's paper is a masterpiece of indirect argument. Ostensibly a memorial lecture about Fisher, it's actually a polemic against the mathematization of statistics in the 1970s — and by extension, against any field that severs its connection to practice. The structure is brilliant: rather than make abstract arguments, Box shows you Fisher's method through a series of vignettes, each one demonstrating theory emerging from practice. You can't read about Fisher deriving orthogonal polynomials from 67 years of Broadbalk manure data and still believe that pure mathematical development is the path to useful knowledge. The paper's weaknesses are mostly sins of omission. Box doesn't engage with the question of when theory-led development does work — there are genuine examples of mathematics developing tools that only later found applications (group theory in physics, for instance). He also doesn't address the institutional economics that produce mathematistry: publish-or-perish incentives, the difficulty of getting real-world data, the career advantage of clean theoretical papers over messy applied ones. The "cookbookery" critique similarly underplays why practitioners reach for standard techniques — they're often the only tools they were taught, and the alternative (developing bespoke approaches for every problem) requires statistical sophistication most practitioners don't have. But these are quibbles. The paper's core thesis — that progress requires the theory-practice loop to be closed, and that both "theory only" and "practice only" are dead ends — is evergreen. Reading it in 2026, the parallels to AI research are uncanny. We have our own mathematistry (benchmark-chasing papers that improve SOTA by 0.3% with no practical import) and our own cookbookery ("just use RAG," "just finetune," "just add more context"). Box's framework gives us the diagnostic: is the loop closed? Are theoretical advances being tested against real problems, and are real problems driving theoretical advances? The paper also offers a useful corrective to the "vibe coding" discourse. Box's Fisher is neither a pure theoretician nor a pure tinkerer — he's someone who moves fluidly between abstraction and concreteness, using each to inform the other. The modern equivalent isn't "always use AI" or "never use AI," but the sponsor/critic switch: use the tool to generate, then switch roles and diagnose what's wrong. The people who are best at this, like Fisher, are "mentally ambidextrous." One final observation: the paper's most quoted line ("all models are wrong") is usually deployed as a cheap excuse for model inaccuracy. That's not what Box meant. He meant that because models are wrong, you must be rigorous about checking how they're wrong — and you must worry selectively about the wrongness that matters. The aphorism isn't a license to be sloppy; it's a call to be disciplined.
- The Mundanity of Excellence — Excellence as qualitatively different choices, not more effort. Box's Fisher makes qualitatively different choices about engaging with practice. - A Non-Anthropomorphized View of LLMs — Halvar Flake: LLMs are functions through ℝⁿ, not proto-minds. Box's distinction between pure and applied mathematics maps onto this cleanly. - Vibe Maths and the Erdős Breakthrough — AI's superpower is innocence, not intelligence. The theory-practice iteration in a new form. - Not-Knowing (Vaughn Tan) — Four-type diagnostic framework for uncertainty. Box's "worrying selectively" is uncertainty diagnosis. - Guardrails and Feedback Loops — Linters beat prompts. The feedback loop as the engine of quality. Box's theory-practice iteration is the ur-framework for this. - Smart Models Dumb Pipes — Models own decisions, pipes own execution. Box would recognize the boundary-drawing instinct. - Estimating Pi with a Coin — Playful math as insight. Fisher's tea-tasting experiment has the same spirit. - Life at Low Reynolds Numbers — Another classic science paper that uses vivid, concrete examples to teach fundamental principles. - Software Engineering Craft — The fundamentals don't change. Box's paper is 50 years old and reads like it was written yesterday about AI engineering.