The Marshmallow Test Failed. So Does Hiring on One Trait

The marshmallow test is the best-known argument for hiring on a single trait, and in 2018 it mostly fell apart. A replication with more than 900 children found that the link between waiting at age four and achievement at fifteen was half the size of the original. Once family background was taken into account, two thirds of what remained disappeared. If one of the most famous traits in psychology can't carry that weight, "I can tell he's resilient" after a thirty-minute interview can't either.

How many children was the original story about?

About thirty-five. Walter Mischel ran the experiments at Stanford's campus nursery school with more than 600 preschoolers. But the claim everyone repeats, that the children who waited went on to do better in life, comes from a 1990 follow-up (Shoda, Mischel and Peake, Developmental Psychology). The researchers could reach only 185 of the original 653 children, and they had SAT scores for roughly 35 of them. In that group, seconds waited at age four correlated 0.57 with SAT math and 0.42 with SAT verbal.

A campus nursery school means parents who were mostly faculty and graduate students. The sample was narrow from the start.

The popular version dropped all of that and kept one sentence: self-control is a trait, measurable at four, predictive for life. Management books, interviewer training and parenting forums have run on that sentence for decades.

What the 2018 replication found

Most of what looked like a trait was environment. Tyler Watts, Greg Duncan and Haonan Quan published the study in Psychological Science in 2018, using a long-running US federal child development dataset. Children did the same waiting task at age four and a half, with a seven-minute ceiling. Achievement and behavior were measured at fifteen. The analytic sample was 918.

To contrast with the original's well-off families, the authors focused on 552 children whose mothers had not finished college. The results come in three steps.

With no controls, each extra minute of waiting at four went with achievement 0.24 standard deviations higher at fifteen. That is already about half the original.

Add controls for family background, cognitive ability at age four and the home environment, and it falls to 0.08 standard deviations, roughly a tenth. Two thirds of the effect is gone.

Control for the child's other abilities at the same age as well, and it drops to 0.05, no longer statistically significant.

Behavior was simpler. Waiting time had virtually no relationship with mother-reported behavior problems at fifteen, even before any controls.

The authors are careful. An intervention that only changed a child's ability to wait, without changing broader cognitive and behavioral capacities, would likely have limited effects on later outcomes, they write. They don't say the original was wrong or that self-control doesn't matter. They say the act of waiting contains far more than one trait.

Half the waiting was the room

An earlier experiment had already pulled "waiting" apart. In 2013 Celeste Kidd and colleagues published a study in Cognition with 28 children, average age four and a half. Before the marshmallow, a small setup: the adult promises better crayons or stickers. For one group the adult comes back with them. For the other, the adult doesn't.

Children in the kept-promise group waited 12 minutes 2 seconds on average, and 9 of 14 lasted the full 15 minutes. In the broken-promise group the average was 3 minutes 2 seconds, and 1 of 14 lasted.

Same four-year-olds, same marshmallow. The only difference was a lesson learned minutes earlier about whether this particular adult is worth waiting for.

Anyone who has managed people has seen the adult version. Someone who spent three years at a company that kept moving the goalposts arrives at yours and reads as "impatient." Are you measuring the person, or the last employer?

The interview version

In interviews the marshmallow goes by other names: resilience, drive, grit, ownership. The method is the same. One or two questions, or one scale on a personality test, produce a trait, and the trait is used to forecast three years of performance.

Foreign companies hiring in Japan get this wrong in a specific way. Japanese candidates tend to be restrained in interviews. They don't interrupt, don't oversell, and rarely say "I can definitely do that." Interviewers trained elsewhere read that as low drive. The fluent, enthusiastic candidate gets read as a self-starter. Both readings hand a lifetime of meaning to one marshmallow.

I look at people in four layers: core, patterns, equipment, context. The marshmallow test failed on the outer two. The original gave all the credit for waiting to the core, and ignored equipment (the cognitive ability a child already had at four) and context (whether there were books at home, whether adults kept their word). Put those two layers back and the signal left in the core is small.

Interviews work the same way. What a candidate does under pressure is partly their pattern and partly what the last boss trained into them. So I record a trait seen in an interview as a claim, never as a fact, and go looking for evidence in past behavior. That's the approach in After the resume screen: what I look for in an interview, and the replication didn't change it.

What a single trait is good for: a floor, not a ranking

The most useful detail in the 2018 paper is buried below the headline. Of the 552 children, 23% couldn't wait 20 seconds. Most of the predictive power came from that line. Before any controls, children who waited between 20 seconds and two minutes already scored about a third of a standard deviation higher at fifteen than those who didn't reach 20 seconds. But once background was controlled, children who waited two minutes and children who lasted the full seven were not significantly different from each other.

The test could identify the bottom group. It couldn't rank anyone above it.

That maps onto how I use Does this person fit? Veto, weakest layer, five levels. A single-trait check works as a veto. There are roles you shouldn't give to someone who can't wait 20 seconds. Using the same check to rank the rest, to pick the "seven-minute person" out of the "two-minute people," is exactly what the data won't support. It's also what interviewers most want to do.

The cost is not abstract. One hire in Japan costs 1.2 to 1.5 times salary. For a manager on 8 million yen, a wrong hire burns around 10 million yen, plus the six months it takes to hire again. That's what rides on ten minutes of "he seems tough."

My rule for famous experiments hasn't changed since Psychology Studies Managers Quote: Only 36% Replicated. The better the story, the sooner you should check the original sample size. Thirty-five children of Stanford academics can't hold up a hiring rule used by tens of thousands of companies.

One thing I'm not sure about. The replication's task had a seven-minute ceiling, and more than half the children hit it. The longest waiters are indistinguishable in the data, which may understate the trait's predictive power at the top. Even so, what's understated is the ranking above seven minutes. The 20-second floor doesn't move. The floor is the part worth taking into the interview room.