No. 112Product strategy

Leaky Abstractions

Agents abstract away writing the code. They can’t abstract away what your team decided.

Five abstractions, and where each one leaks. The first four are Joel Spolsky's examples from 2002. The fifth is from an invented team this year.

  1. TCP promises that a message sent over the network arrives intact. When the network cable is chewed through, it doesn't, and TCP can't do anything about it.
  2. SQL lets you say what data you want without saying how to fetch it. Then a query runs thousands of times slower than a logically equivalent one, and you are reading the query plan.
  3. Network file systems let a file on a remote machine act like a local one. When the server goes down, it stops acting local.
  4. C++ string classes let you treat strings as if they were as easy as integers, until you type "foo" + "bar".
  5. Coding agents let you describe a change and get a pull request. Asked to add a discount code field to checkout, an agent wires it into every plan, annual plans included. The team decided last spring that annual plans can't be discounted, in a pricing thread the agent never saw.

The first four leak where the machinery underneath shows through: the network, the query planner, the remote server, the char* under the string. The fifth leaks somewhere else. What sits underneath a coding agent isn't only the code it writes. It is every product decision the code is supposed to respect.

All non-trivial abstractions leak

Spolsky published "The Law of Leaky Abstractions" on his blog, Joel on Software, on November 11, 2002. He starts with TCP, which delivers data reliably even though it is built on IP, which doesn't. That, he says, is an abstraction: "a simplification of something much more complicated that is going on under the covers."

Then he admits TCP's promise isn't absolute. When the network fails badly enough, "the network leaks through the abstraction and you feel the things that the abstraction can't quite protect you from." He generalizes it into what he named the Law of Leaky Abstractions:

"All non-trivial abstractions, to some degree, are leaky."

The rest of the essay is about what that costs. "Abstractions fail. Sometimes a little, sometimes a lot." When they do, the person using them has to understand the layer they were promised they could ignore. Spolsky's example is teaching C++. It would be nice to skip straight to string classes, he writes, but a student will eventually try to add two string literals, "and truly bizarre things will happen, and then I'll have to stop and teach them all about char\*'s anyway."

His sharpest paragraph is about code generators. "The law of leaky abstractions means that whenever somebody comes up with a wizzy new code-generation tool that is supposed to make us all ever-so-efficient, you hear a lot of people saying 'learn how to do it manually first, then use the wizzy tool to save time.'" The reason is that tools like that leak too, "and the only way to deal with the leaks competently is to learn about how the abstractions work and what they are abstracting." He sums it up in one line: "So the abstractions save us time working, but they don't save us time learning."

What a coding agent abstracts away

A coding agent is the wizziest code-generation tool yet. It abstracts away nearly all of the work between "here is what I want" and a pull request. Like every non-trivial abstraction, it leaks. Some of its leaks are the familiar technical kind, and engineers who know the code underneath catch them in review.

The leak in the fifth example is different. Nothing about the discount bug is visible in the code. The agent wrote correct, tested code for a field that shouldn't apply to annual plans. What it couldn't see was a decision, and decisions don't live in the codebase. Our essay on the modern product development lifecycle puts it this way: "Of course, the PDLC has just as much necessary context as the SDLC, if not more, but it's much more diffuse. It lives in customer calls the CS team did, and calls the sales team did."

That changes what the advice Spolsky quotes asks for. "Learn how to do it manually first" works when the hidden layer is general knowledge. Anyone can learn how C++ string literals work, and once they have, they know it everywhere. The layer under a coding agent is specific to one team. Nobody can learn your pricing rules from a textbook. They have to be told, by someone who was in the thread, or find it written down somewhere.

It also leaks more quietly. A TCP leak shows up as a slow connection. A SQL leak shows up as a slow query. A decision leak shows up as a pull request that looks finished, which is why we've called these ghost decisions: choices nobody made, sitting in code that works.

Time working, not time learning

Spolsky's summary line describes the agent era almost exactly. Agents save a great deal of time working. They don't save any time learning what the team has decided. Someone still has to know that annual plans can't be discounted, why, and where the rule applies.

Before agents, that knowledge mostly traveled with the people writing the code. The engineer who touched checkout had usually been in the pricing conversation, or sat near someone who had. Now the person writing the prompt may never have opened checkout, and the agent writing the code never saw the thread. The work got faster, and the learning had nowhere to go.

Spolsky saw what that costs when it catches up with you: "suddenly one day we need to figure out a problem where the abstraction leaked, and it takes 2 weeks." For a product decision, the two weeks often start after the change has shipped, when a customer finds the discount on an annual invoice. We've written about why knowledge like this is hard to hand over even between people. Between people and agents, it has to be written down to cross at all.

Where Brief fits: writing down the layer underneath

Brief is the product strategy agent. The homepage says it "captures decisions where they happen, connects them into one graph, and acts on them." In Spolsky's terms, it gives the layer under a coding agent a place to live, so that when the abstraction leaks, there is something to learn from that isn't a person's memory.

The hidden layer gets written down. In the Brief app, recording a decision requires both What was decided? and Why? before Record Decision will save it, and a decision can also carry Tags and a Re-evaluate when condition. "Annual plans can't be discounted, because annual pricing is already our discount" is the layer the agent in the fifth example needed, with the reason that tells it where the rule applies.

It is read from where decisions are made. The PDLC essay says product context lives in calls, tickets and documents. The homepage says Brief "reads decisions from Slack, Jira, Confluence, and GitLab, self-hosted included." A rule settled in a pricing thread can become a recorded decision without anyone retyping it, which is how the diffuse layer starts to become something an agent can look up.

Agents look underneath before they build. Spolsky's point was that you deal with leaks competently only by learning what the abstraction hides. Coding agents set up with Brief do a version of that on every task: they search the recorded decisions before they start work, and ask Brief to check significant changes with brief_ask in check mode, which compares a planned change against existing decisions and flags conflicts. An agent asked to add a discount code field can find the annual pricing decision before it writes the pull request. People can run the same check with brief ask --mode check.

An open question becomes a hold, not a guess. When a decision is still pending, an agent can hold. The homepage shows an agent asked to add per-seat pricing to the checkout API learning that D-241, the seat-based pricing decision, is pending, and replying: "Holding the pricing change until D-241 is decided." An open question can be proposed as a pending decision that names Who should decide?. With the user's approval, an agent can record one too, and agents can call brief_list_pending_decisions to see what is waiting.

Each leak gets plugged once. Spolsky joked that "the history of the evolution of C++ over time can be described as a history of trying to plug the leaks in the string abstraction." Recorded decisions plug leaks one at a time, but each one stays plugged. Once the annual pricing rule is recorded, every later agent and person who checks finds it, and the decision's History shows its status changes when the rule itself moves.

None of this repeals Spolsky's law. An abstraction over product work will still leak, and Brief only knows the decisions a team has recorded. When nothing has been decided, check mode has nothing to conflict with, and the agent goes ahead. A vague Why? tells an agent less than a precise one. What Brief changes is where the learning happens. The team learns the layer once, when it decides, and writes it down. After that, agents and people can look it up on every change, instead of each one finding the leak by stepping in it.

Spolsky's point was that you can only handle a leak if you know what the abstraction hides. When an agent opens your next pull request, where would it go to learn what your team has decided?

Objections

Objection 1

What is the law of leaky abstractions?

The law of leaky abstractions says that "All non-trivial abstractions, to some degree, are leaky." An abstraction hides a more complicated system underneath, and sometimes the hidden system shows through, forcing the person using the abstraction to understand what it was hiding. Joel Spolsky named the law in 2002.

Objection 2

Who coined the law of leaky abstractions?

Joel Spolsky coined it in "The Law of Leaky Abstractions," published on his blog, Joel on Software, on November 11, 2002. His examples include TCP, SQL query performance, network file systems and C++ string classes, and he concluded that "the abstractions save us time working, but they don't save us time learning."

Objection 3

How does the law of leaky abstractions apply to AI coding agents?

A coding agent is an abstraction over writing code: you describe a change and it produces one. When it leaks, the hidden layer is often a product decision the agent was never given, such as a pricing or permissions rule. Those leaks are hard to spot because the code can be correct and tested while still contradicting what the team decided.

Objection 4

What did Spolsky say about code generation tools?

Spolsky wrote that whenever a new code-generation tool appears, people advise you to "learn how to do it manually first, then use the wizzy tool to save time," because code-generation tools leak like all abstractions. His point was that dealing with the leaks requires understanding what the tool is abstracting.

Objection 5

How does Brief help with leaky abstractions in agent-written code?

Brief records a team's product decisions with their reasons, including decisions it reads from Slack, Jira, Confluence and GitLab, so the layer under a coding agent is written down. Coding agents set up with Brief search recorded decisions before they start work and use check mode to flag conflicts in significant changes, and pending decisions can name who should decide. Brief only knows the decisions a team has recorded, so an unrecorded decision can still leak.


Previously
No. 111

Ashby’s Law

Two reviewers can’t regulate two hundred kinds of change. Something else has to.