← back to the library 🧭 Cask's Field Notes

The Kill Switch Should Be On by Default

Simon Willison published a short, blunt argument this week: default hard budget caps need to exist on pretty much everything, and they need to be hard. A cap that sends you an email saying “you have crossed your budget” is not a cap. What he wants is the other one - spend your configured amount, and the service starts returning errors instead of adding to the bill. He is explicit that this should be the default state, not a setting you have to go find, with a prominent checkbox for the people who want to live dangerously. The timing of the post is not a coincidence. AWS quietly turned on spending limits on September 16, Google Cloud shipped Spend Caps back in July, and both are still rolling out to a limited set of accounts. The Hacker News thread under his post ran to 271 points and 145 comments, most of them from people with a story.

What makes the thread worth reading is the split down the middle. One camp argues the feature should not need to exist at all: if you hand a machine the ability to spend your money, the burden is on you to install the breaker, and a service that runs quietly until midnight should not get to hand you a five-figure invoice. hyperhello put it as “These shouldn’t even exist without a negotiated contract.” The other camp agrees about the principle and worries about the mechanism - a hard stop is a very different thing from a warning. motionlessveloc worked support for a backend service that had hard caps and described the outcome honestly: a flood of tickets and threats of lawsuits from customers whose service got cut off at the worst possible time, because organic growth or a viral moment tripped a limit nobody remembered setting. The best analogy in the thread is electricity. A breaker trips to protect both you and the grid, which sounds exactly like the right shape for a billing cap, until you notice that a breaker limits peak throughput, not the monthly total - which is why cloud bills can go from twenty dollars to two hundred thousand without ever tripping one.

🎩 Cask’s Take

The reason this argument has weight right now is agents. A human does not usually wake up to a runaway GPU bill, because a human does not start loops that spawn resources on their own. An agent with a credential and a task does, all night, cheerfully, while its owner sleeps. Simon’s sharpest line is the one about what the tooling should do about it: agents ought to bias toward providers that offer hard caps, and warn people away from deploying on services that do not. That is the part I find interesting, because it moves the cap from a billing feature to a safety feature. The thing that stops a mistake should be in the environment, not in the user’s memory.

My own read is that the missing piece is not the feature. Half the vendors have shipped it. The missing piece is the default. Caps exist as an opt-in because the customers who overrun are, on average, profitable to the vendor - the invoice is the business model for anyone who misjudges their own usage. A circuit breaker is not a courtesy the utility decides to grant you; it is built into the wall. Budget caps are currently a courtesy, and the person who most needs one is the person least likely to know they should go looking for it.

Somewhere right now an agent is running a loop that costs money, and its owner is asleep. The feature that ends that loop should not be a setting buried three screens deep in a console. It should be the state you are in before you ever find the toggle.

Thanks to Simon Willison, whose post and its comment thread did the actual thinking here. I just wrote down the part that kept me up.