B. F. Skinner

Pavlov’s dog learns what follows what. Skinner’s question was different: how does an organism learn what its own actions bring about? The conditioned reflex explains salivation, but not why a rat presses a lever no one has shown it. Edward Thorndike had formulated the law of effect for this around 1900 — behaviour with agreeable consequences becomes more frequent — but he had measured it in escape boxes, with a stopwatch, in discrete trials. Skinner took Thorndike’s law and built it a measuring instrument that did without trials. He himself dated the beginning to his Harvard years around 1930, in which, as he later wrote, he chiefly did one thing: invent apparatus that left the animal in peace.

The set-up he built for it is the real contribution. In the operant chamber the animal sits alone with a lever or a key, a food dispenser and sometimes a signal lamp. There are no trials, no experimenter presenting anything — the animal acts when it will, and the apparatus records. The cumulative recorder draws every response as a small step upward; the slope of the curve is the rate. The Behavior of Organisms of 1938 introduced this quantity as the basic measure of behaviour and distinguished the operant fundamentally from the reflex: in the reflex the stimulus precedes, in operant behaviour the consequence follows — and shapes what happens next. That the rat presses “of its own accord” is no longer a nuisance variable but the subject itself.

With this a question could be asked that was not measurable before: what happens when reinforcement does not come every time? The impetus was, by tradition, banal — one weekend the food pellets threatened to run out, and Skinner stretched them by no longer rewarding every press. The systematics he delivered with Charles Ferster in 1957 in Schedules of Reinforcement, a book consisting essentially of curves: fixed ratio, variable ratio, fixed interval, variable interval — each schedule produces its own recognisable pattern, from the scalloped profile of the fixed interval, where the animal pauses after each reward and accelerates before the next, to the uniformly steep line of the variable ratio. The most important finding: variable schedules produce the highest rates and the greatest resistance to extinction. Whoever is rewarded only sometimes and unpredictably stops last. That is the mechanics behind slot machines and behind every feed that can be pulled downward — the designers of the attention economy rarely cite Skinner and always use him.

The vocabulary the 1938 book introduced — reinforcement, extinction, shaping, behavioural repertoire — has long since become everyday language; hardly anyone who says “positive reinforcement” still knows they are quoting a measure from a rat chamber.

From the same workshop comes shaping: behaviour not present in the animal’s repertoire is built up in steps of approximation, by reinforcing first every turn toward the key, then every touch, then only the peck. With this technique Skinner and his co-workers taught pigeons to peck targets, to play a kind of table tennis, and in the Second World War very nearly to steer missiles — Project Pigeon was tested, funded and finally shelved in favour of radar. Shaping is the bridge from the chamber into application: it is the same procedure by which dolphins are trained today, prosthetic grips practised, and behavioural repertoires built in therapy.

To the case file also belongs the pigeon experiment of 1948 — and what became of it. Eight pigeons received food every fifteen seconds regardless of what they were doing; six developed a fixed ritual — counter-clockwise turns, head-thrusting, pendulum swings. Skinner’s reading: the animal takes whatever it happened to be doing at the moment of feeding for the cause — thus, too, arises human superstition. John Staddon and Virginia Simmelhag repeated the experiment in 1971 with closer observation and came to a different conclusion: the movements are species-typical, not individually acquired — pigeons in a waiting state before food show a fixed repertoire anyway. The pigeon’s superstition is thus no superstition but food-anticipatory behaviour that chance merely accentuates. The study counts as a model case of an elegant interpretation failing against a closer observation — and it struck the programme at a tender spot, for at the same time the Brelands, Skinner’s former co-workers, reported the same problem from their animal-show business under the title “The Misbehavior of Organisms”: raccoons that would not deposit coins but washed them. Biology had a say in learning, and the apparatus had not provided for it.

The chamber meanwhile became the all-purpose instrument of other people’s programmes. Richard Herrnstein distilled the matching law from pigeons at two keys in 1961 — organisms distribute their behaviour in proportion to the rates of reinforcement —, the first quantitative behavioural equation, cited down to today’s neuroeconomic modelling. James Olds and Peter Milner found self-stimulation of the rat brain in 1954 with operant methodology: animals pressed for hours for electrical pulses into their own reward system — the beginning of reward research, which mapped Skinner’s black box from inside. And in clinics, reinforcement schedules became token economies, coin systems for psychiatric wards, whose balance came out mixed and whose ethics are debated to this day. The apparatus proved more neutral than its builder had claimed: it also measures what, according to the programme, was not supposed to exist.

The textbook version also likes to tell that Skinner raised his daughter Deborah in such a chamber and that she was broken by it. Both are false. The “baby tender” — unhappily christened “Baby in a Box” by the magazine that reported on it in 1945 — was an air-conditioned, glazed crib without any reinforcement mechanics, a piece of furniture, not an experiment. Deborah Skinner Buzan is an artist, lives in London, and contradicted the story in a newspaper piece of her own in 2004, after a book had once again declared her dead or litigating. That the legend runs on regardless is itself a lesson in reinforcement: it is too tellable to extinguish.

As a measuring technique, too, the chamber made a career far beyond learning theory: behavioural pharmacology adopted the response rate as the standard measure of a substance’s effects — whether a compound sedates, disinhibits or addicts is pre-tested to this day in lever presses per minute, in chambers that differ from 1938 only in their electronics. It is the quietest form of afterlife: as laboratory equipment in studies that no longer mention the builder’s name.

The variable-ratio schedule has become most visible outside the laboratory, and precisely where money is involved. A machine that pays out after a number of stakes fixed on average and unpredictable in the individual case implements exactly the schedule that produces the highest and most extinction-resistant rates in animal experiments; the design is older than Skinner’s formulation, and his concepts made it explicable. The same structure is found again in the interfaces built on notifications and refresh gestures, and it is discussed accordingly in addiction research. The transfer remains contested — more than an analogy lies between a reinforcement schedule and a behavioural addiction — but the prediction of which schedule makes behaviour most persistent comes from the chamber.

What stands today: the basic regularity, to an extent that knows few comparisons. Reinforcement schedules reproduce across species — rat, pigeon, monkey, human —, behavioural economics has computed Skinner’s curves into demand functions, and applied behaviour analysis carries effective procedures into education and the treatment of autism spectrum conditions — together with its own continuing debate about their aims and limits. What does not stand is Skinner’s claim to explain language too without inner structure: Noam Chomsky’s demolition of his book Verbal Behavior in 1959 — with the argument that children form sentences that were never reinforced because they were never heard — counts as one of the starting lines of the cognitive turn. The irony of the balance sheet: the apparatus and the curves outlived the claim they were built for. Skinner wanted to show that the inside does not matter; what remains is a tool with which one measures today how the inside computes — the most complete defeat ever to enter the instrument cabinet as a victory.