Why do queues get long all at once?

A queue doesn't grow in step with how busy you are. It stays short for a long time, then gets long all at once. If you've run recruiting, you've seen this. The "any update?" emails from candidates don't trickle in. They arrive in the same week, and usually it's a week when nothing obvious changed. Nobody slacked off. Applications didn't double. The team got a little busier, and replies started slipping by days.

I once scheduled interviews for 12 business units: 400 requests, all done within two weeks. Counting only weekdays, that's about 40 a day. When I try to understand what happens under that kind of load, the most useful tool I know is the simplest model in queueing theory. There's one formula below. You can skip it and still follow along.

20% more customers, nearly four times the wait

One-barista coffee stand, capacity 60 an hour. Average wait in minutes at 45 vs 55 customers an hour

• Now45 an hour
  3 min
• After promo: intuition“about the same share”
  4 min
• After promo: modelM/M/1
  11 min

Intuition scales the wait with the load. The real wait jumps where the slack runs out.

Start with a coffee stand

Picture a coffee stand with one barista. She makes a drink a minute on average, so 60 an hour. In the morning about 45 customers come in each hour. A quarter of her capacity goes unused.

Then the owner runs a promotion and traffic rises to 55 an hour. That's 22% more. Most people would guess the wait grows by about the same share. It doesn't. Run the numbers and the average wait goes from 3 minutes to 11.

Nearly four times as long.

Why it happens

Two things are going on. Customers don't arrive evenly. And the barista's idle time can't be saved for later.

The chart below shows 16 customers over 24 minutes. That's fewer than one a minute on average. Yet a few times, three people walk in almost together, and that's when the line forms. Over the same 24 minutes the counter sits empty for 8 minutes in total. None of that time can be carried over to the next rush.

There is idle time, and still a line forms

16 customers over 24 minutes at a one-barista stand, about one drink a minute

1. 01Customers don’t arrive evenlyFewer than one a minute on average, yet three people sometimes walk in almost together
2. 02That is when the line formsUp to 3 people wait at once. The longest wait is 2.1 minutes
3. 03Idle time can’t be savedThe counter sits empty for 8 minutes in total. None of it carries over to the next rush

• Idle time in 24 minuteslost capacity
  8 min

Average slack is no help in the moment of a rush. Lines come from uneven arrivals and idle time you can’t bank.

Less slack, much longer waits

The formal name for this is the M/M/1 model. Arrivals are random, service times are random, and there's one server. How busy the system is comes down to a single number, utilization ρ, which is the arrival rate divided by the service rate. The average wait is ρ/(μ−λ). As the denominator gets close to zero, the wait explodes.

The numbers say it more plainly. With capacity fixed at 60 an hour, the average wait is 1 minute at 50% utilization, 4 minutes at 80%, 9 at 90% and 19 at 95%. Going from 80% to 95% makes the wait almost five times longer. Drag the slider and watch where the red dot starts to shoot up.

Past 80% utilization, the wait turns steep

M/M/1 model, capacity fixed at 60 an hour. Average wait Wq (min) and average queue Lq by utilization

• 50%calm, queue 0.5
  1 min
• 80%the knee, queue 3.2
  4 min
• 90%steep, queue 8.1
  9 min
• 95%near collapse, queue 18.1
  19 min

Fifteen more points of utilization make the wait almost five times longer.

Real lines are messier than the average

The formula gives you averages. A real line stays quiet for a while, then jumps, then takes a long time to drain. The simulation below sends customers in at random. Set it to about 54 an hour and watch for a minute. You'll see the kind of mess an average hides. Leave it running long enough and the measured wait drifts toward the theoretical value.

An average of 9 minutes doesn’t mean everyone waits 9 minutes

M/M/1 model at 54 customers an hour (90% utilization), capacity 60 an hour. Theoretical average wait: 9 minutes

1. 01Quiet for a whileHardly any line forms
2. 02Then it jumpsA run of arrivals and the line gets long at once
3. 03It takes a long time to drainIdle time can’t be saved, so a long line shrinks slowly

• Theoretical average waitat 54 an hour
  9 min

The measured average only drifts toward theory over time. The waiting piles up in the moments the line jumps.

What this means for recruiting

Back to recruiting. Almost everything on a recruiter's desk is a queue. Scheduling interviews, replying to candidates, waiting on a hiring manager's decision, getting an offer approved. Work arrives at random and the number of people handling it is fixed. When twelve departments send requests, each on its own schedule, the arrivals are about as random as the coffee stand's.

Many recruiting teams I've seen are managed by utilization. If someone has room, they get another req. The curve shows where that goes wrong. Give one more req to a team already above 90% and replies slow down for everyone on it. Candidates remember how many days they waited.

People also say a team at 90% still has 10% to spare. On average, it does. But at that point the average queue is over 8, sixteen times what it is at 50%.

If you can't add people, work on variability. Block interview slots at the start of each week. Standardize request forms so each task takes about the same time. Neither adds capacity. Both pull the curve down. You won't get variability to zero, though. Even if every task took exactly the same time, random arrivals mean you only cut the average queue in half. Some slack has to stay.

From the outside, a team with slack can look like it isn't working hard. I think that slack is what protects the candidate experience. How much to keep probably depends on the kind of work, and I don't have a settled answer. If you pushed me for a number, I'd say around 80%, where the curve starts to bend.

Model assumptions: Poisson arrivals, exponential service times, one server, first come first served. Figures use μ = 60 per hour.