The Capacity Model
Fits on One Page

Two. That is how many requests the performance notes for my banking demo expected to be in flight at once at ten transfers a second. On 30 August a ten-minute load run showed two busy threads and one active database connection on the server, and the notes record the result as matching the Little's law estimate "almost exactly".

The two numbers did agree, and neither one measured what the notes thought it did. Both inputs to the estimate were off, in opposite directions, and the dashboard's two was a single instant. Across the whole run the server averaged well under one request at a time. I wrote those notes with an AI assistant, and that sentence stood in them until I checked it for this essay.

One Line of Arithmetic

In 1961 John Little published a five-page proof in Operations Research of the formula now named after him.

Put plainly, the average number of things inside a system equals the rate at which they arrive multiplied by the average time each one stays. It holds for a queue at a till, a pool of threads and a pool of database connections alike, provided the system is not piling up work over the window you measure.

That one line is most of a capacity model. Requests in flight tell you how many threads a server needs. Transactions in flight tell you how many database connections it needs. The gap between either number and the limit you configured is your headroom, and the demo's notes put all of it on one page before a single test ran.

Their conclusion was that ten transfers a second "cannot stress this system". The same notes pin down the four words around the model. Load is the demand arriving, whether or not the system keeps up. Throughput is the work completed each second. Capacity is the highest throughput the system sustains while it still keeps its promise, which here means a transfer answered within 500 milliseconds at the 95th percentile and fewer than one request in a hundred failing. Scale is how capacity moves when resources are added.

Design for the Load You Have Not Met Yet uses the same four words as lenses for a design. On this page each one becomes a number, and the definition of capacity carries its own warning: past it, throughput can still climb while the promise is already broken.

The Match on the Dashboard

The k6 summary from that run holds everything the check needs, and the check is the same one line with measured numbers in it.

The notes counted transfers, while the server was answering requests. Every transfer in the test came with a balance read before it and, one time in five, a history read after it, so the real arrival rate was more than double the assumed one. The assumed time of 200 milliseconds was seven times the measured average. The first error pulled the estimate down and the second pushed it up by more, and together they produced a figure about three times too high.

The dashboard's two was a different kind of number. Prometheus scraped the server every ten seconds, and a thread gauge reports whatever is busy at the instant it is read. In a run averaging under one request at a time, an instant with two in flight is unremarkable. An average and a snapshot printed the same digit, and the notes took it for agreement.

That is where Little's law earns its place as a check rather than a formula. Over a long enough window it holds for any system that is not piling up work, so three measured numbers that refuse to multiply are saying that one of them came through the wrong door: transfers counted where requests were meant, a snapshot read where an average was needed, the load generator's view set against the server's. These ten minutes are the same run that Not Run Is Not Passing grades partial, for a reason that has nothing to do with this one.

The First Thing to Run Out

A page that balances at the target is still unfinished until it names the first resource to saturate, because that is where the load test should be watched and where the next spending will go.

The notes walk up the stack to find it. Every commit waits for the disk to confirm the write. A transfer holds a database connection for about twenty milliseconds, so ten connections can carry roughly 500 transfers a second. Their estimate for the whole node is hundreds of transfers a second, which puts ten a second at two to five percent of capacity.

Then they find a lower ceiling that hardware does not move. A transfer locks both account rows, always in the same order so that two transfers can never deadlock, and every transfer touching the same account waits its turn on that row. Money Has Rules the Framework Does Not Know covers why the order is fixed. With the lock held for a few milliseconds, one busy account tops out somewhere between 200 and 500 transfers a second, however many application servers stand in front of it. Adding servers makes the queue on the row longer, not shorter. The load test spreads its traffic across twenty accounts for that reason.

On a ledger, the first capacity question is how busy the busiest account can get. The size of the server is the second. And when a pool does run out, it rarely fails loudly; it makes requests wait, which is the failure Good Swimmers Drown Quietly is about.

Twice, Ten Times, a Hundred Times

The last thing the page owes is growth: what changes at twice, ten times and a hundred times the target. At twice the target nothing on the page changes. At ten times, a hundred transfers a second, the model still fits on one node, and the question has already moved from the server to the busiest account.

At a hundred times, a thousand transfers a second, the pool is the first line to give: ten connections held for twenty milliseconds each carry 500. The service scales out, and scaling out changes a number that lives on the database. Every instance brings its own pool, so instances multiplied by pool size has to stay under the database's connection limit, or a connection pooler has to sit in front of it. PostgreSQL's default limit is typically 100 connections, with three held back for superusers, and HikariCP's default pool, the size this demo uses, is ten. Ten instances at the default size ask for a hundred connections from a database prepared to give ninety-seven.

The tempting fix is a bigger pool. HikariCP's notes on pool sizing argue the other way, citing an Oracle performance demonstration in which reducing the pool size alone, with nothing else changed, took response times from about 100 milliseconds to about 2. Every Platform Starts on One Server walks the stages that take a platform past one machine; this page is the arithmetic that says when each stage falls due.

The Row That Still Says TBD

The notes include the test that would find the real ceiling: a ramp from ten to two hundred transfers a second in two-minute steps, with Tomcat's thread pool cut on purpose from its default of 200 to 50 so the knee would be visible. The table where its result belongs still reads TBD-measured, and so does the row for the backend's CPU. I have not run the ramp.

So the ceiling on this page is a prediction, and it is written as one: hundreds of transfers a second for the node, 200 to 500 for a single account, and three candidates for the first limit, the disk, the pool and the busiest row. Even unmeasured, it tells whoever runs the ramp where to look first: connections waiting for the pool, busy threads against the fifty, time spent waiting on a row. Precision Is Not the Point makes the case for numbers like these, close and labelled, with the test that will overturn them written down beside them.