What If Your Postmortems Could Write Their Own Guardrails?

I keep coming back to what happens after something breaks.

A skills repository matters when the result of one skill changes what the next one can do. Sometimes that’s modest: a conversation about somebody’s actual routines, constraints, and voice can give a brand-design skill something better to work with than a pile of aspirational adjectives. Your city’s public data can move through statistical analysis and into a financial model, turning an interesting map into a decision with assumptions and downside cases.

Most teams know how to write a postmortem. The incident gets a timeline, a root cause, a list of things people did to contain it, and some action items. Then it becomes a document in a folder. The next release starts from the same set of untested assumptions that produced the incident in the first place.

That is not a failure of writing. It is a failure of follow-through.

Google’s SRE guidance on postmortem culture has always been clear that a postmortem is supposed to produce preventive action, not punishment. It says a blameless postmortem looks for contributing causes without indicting an individual or team, because people who expect punishment stop surfacing problems. More bluntly: you can’t fix people, but you can fix the systems and processes that left them with incomplete information.

I think an AI-enhanced postmortem can make that principle much more practical. Not by letting an AI decide who was at fault. That would be a miserable use of the technology. The useful move is narrower: use AI to help carry the learning from an incident into the systems that govern the next release.

I wrote earlier about incidents as training data, including why raw postmortems are a bad substitute for clean labels. This is a different argument. The goal is not to pour an incident folder into a model and hope it learns wisdom. The goal is to turn a verified lesson into a guardrail that survives the meeting where the lesson was learned.

The postmortem cannot be the last artifact#

Imagine an agent that calls a billing service. One day the service begins timing out. The agent retries too aggressively, burns through the request window, and sends the user a confusing result. Nobody lost money, but the system behaved badly enough that it should not happen again.

The conventional postmortem might say: cap retries and improve the alert.

That is not wrong. It is incomplete.

A stronger outcome would say:

1. An external tool failure gets no more than two retries within a bounded time window.
2. The workflow may not commit an irreversible side effect until the tool confirms success.
3. A user gets a recoverable outcome within an agreed latency threshold.
4. The failure is recorded as a redacted, structured event, not as a grab bag of customer content and raw prompts.
5. A simulated tool timeout must pass before a release reaches production.

Now the incident has changed the system. The next person doesn’t need to remember the meeting. The next implementation doesn’t need to infer what “improve retries” meant. The release process has something concrete to reject.

If that postmortem sounds familiar, the gap is follow-through. The action item exists, but nothing carries it all the way to a changed release decision.

This is where a repository of composable skills matters. The workflow I have in mind uses five complementary skills from my agent-skills repository.

None is especially magical alone. The value is that the chain keeps the lesson moving until deployed evidence decides whether the system actually changed.

1. Find the cause before writing the lesson#

The first skill, systematic debugging, has an intentionally unfashionable rule: no fixes without root-cause investigation.

That sounds obvious until an incident is happening. Under pressure, a team sees a timeout, adds another retry, changes a timeout value, pushes, and waits. They can easily convert one failure into a slower and more expensive failure.

A postmortem that begins with a guess is already compromised. Before it can create a guardrail, it needs a reproducible failure, an actual path through the system, and evidence about where the behavior diverged. Was the problem the model’s choice, the tool contract, stale data, a deployment change, a policy gap, or telemetry that hid the relevant signal? Those are different failures. They should not receive the same fix.

This first step matters because it protects the rest of the process from polished nonsense. The root-cause report becomes the input to the reliability review. If the root cause is wrong, the guardrail will be wrong with unusual confidence.

2. Turn impact into a reliability promise#

The next step belongs to site reliability engineering. A root cause explains what happened, but it doesn’t yet say what the service owes its users.

Was this a broken critical user journey? Did it consume enough of the error budget to justify halting feature work? What is an acceptable recovery time? Who owns the alert? What should happen if the same dependency fails again?

Google’s example error-budget policy makes the connection concrete. Its policy is not framed as a punishment for missing an SLO. It gives teams permission to stop shipping features and work on reliability when the data says reliability is the more important problem. It also requires a postmortem with at least one P0 action item addressing the root cause when a single incident consumes more than 20 percent of the four-week error budget.

That is the right posture for an AI-assisted process too. The resulting reliability promise tells the evaluation step which user journey, failure boundary, and recovery expectation it has to test. The point is not to make an agent look busy after an incident. The point is to decide, in advance and with evidence, when the organization owes users a reliability intervention instead of another feature.

3. Transform the incident into a safe test case#

This is the AI-specific step, and it needs the most restraint.

Agent failures aren’t always ordinary software failures. A wrong answer can come from a retrieval step, a tool call, an unexpected interaction between two tools, a policy that didn’t cover the situation, or a trajectory that wandered into a bad loop. Seeing only the final response is often not enough to distinguish them.

That doesn’t mean every trace should be retained forever. It definitely doesn’t mean postmortems should become a covert archive of prompts, customer data, or employee messages.

The agent-evaluation skill makes the distinction that matters: observability provides diagnostic context, while evaluation asks whether behavior met a declared criterion. It recommends turning verified incidents and near misses into cases only after consent, transformation, privacy controls, and contamination review. The resulting test should preserve the failure’s structure without treating a customer’s real interaction as reusable raw material.

Commercial tools are already selling a trace-to-regression loop. LangChain’s description of agent observability says production traces can reveal a recurring failure pattern that becomes a regression case. That confirms the loop is real, but it doesn’t supply the governance model. You still have to decide what gets retained, who can inspect it, what is redacted, and which failures deserve a release-blocking test.

The practical output is not “save this trace.” It is a deliberately designed scenario: the dependency is slow, the retry budget is constrained, the state is checked, and the expected recovery behavior is visible. That scenario becomes the raw material for an implementation requirement.

4. Write the guardrail where implementation has to respect it#

A good action item should not be a sentence that begins with “improve.” It should become a requirement with a threshold and a test.

That is where spec-driven development turns the lesson into a non-functional requirement: a measurable property the implementation has to satisfy. You’ve probably seen a postmortem where “improve retries” meant entirely different things to the three people who read it. A requirement closes that gap.

For the timeout case, the requirement might say that the workflow must return a recoverable result within eight seconds at the 95th percentile while a downstream dependency is failing, must attempt no more than two retries, and must not commit a payment before confirmation. The specification includes the condition and the verification method, not only the desired outcome.

This is the quiet part of the workflow that changes everything. The postmortem stops asking a future developer to remember what happened. It tells the future developer what the system must do when the same class of problem returns. The requirement and its test plan become the verifier’s checklist.

But none of it matters until someone tests the promise against the system that will actually ship.

5. Let a separate check decide whether the lesson survived#

Finally, verification needs to be independent of the person who wrote the requirement and the person who implemented it.

The verification skill is intentionally blunt: identify the criteria, collect direct evidence, and mark every one passed, failed, blocked, or not applicable. Missing evidence is not a pass.

That matters because the most common failure in postmortem follow-up is not malice. It is completion theater. Someone writes “added retries” or “improved observability,” the ticket closes, and no one confirms that the deployed system behaves differently.

A release gate makes the question precise: did the simulated tool failure produce the required recovery behavior, preserve the required state, and emit the allowed telemetry? If not, the postmortem’s action item is still open, no matter how tidy the document looks.

Blameless does not mean automated judgment#

There is a bad version of this idea waiting to happen.

An organization could feed incident chat, employee activity, and customer conversations into an AI system, ask it to identify mistakes, and call the result a blameless postmortem because the output uses neutral language. That is surveillance with better copywriting.

Blamelessness is not a tone setting. It is a decision about what the system is trying to learn.

The system should be looking for missing information, unsafe defaults, confusing interfaces, absent rollback paths, weak dependency boundaries, and requirements that were never written down. It should preserve only the smallest authorized evidence needed to answer those questions. Human beings should retain authority over accountability, remediation, and any decision that affects another person.

The compounding value is organizational memory#

A good postmortem already teaches the people in the room. The problem is that the room dissolves.

People rotate off call. The person who understood the dependency leaves. A new implementation arrives six months later with a different model, a new tool, and exactly enough confidence to recreate an old failure.

A workflow that carries an incident from root cause to reliability promise, transformed evaluation case, explicit requirement, and independent verification does something more durable. It converts a painful moment into a constraint on future behavior.

That is what a skills repository can do when its skills compose: it can turn a postmortem into follow-through, follow-through into a guardrail, and a guardrail into memory that still matters after everyone has forgotten why it was written.

The question is not whether an AI can write a prettier postmortem. It can.

The question is whether the postmortem changes what the next release is allowed to do.