Late in a long interview process, a take-home assignment landed in my inbox. I opened it, read through the prompt, and laughed. It asked me to present an initial executive point of view on helping an organization move from AI-assisted software development toward something more autonomous: how to diagnose where the organization sits today, how to sequence the next moves, what to protect, and how to account for the people and functions that would be reshaped. And underneath all of it: prove that the thinking came from hands-on practice, not management abstraction.
I laughed because the assignment was asking me to package a methodology I’d already spent years developing in the open: writing about it across this site and Groktopus, building open-source agent skills, shipping the building blocks in public. The take-home didn’t create the point of view. It gave me a reason to assemble work that was already done.
But the article you’re reading isn’t about that assignment. It’s about the method.
Start with the constraint, not the tool#
AI transformation shouldn’t begin with tools, vendors, model selection, or an organization-wide autonomy target. It begins by locating the current bottleneck in a specific workflow.
You need to know which question you’re actually answering. Not “what can we do with AI.” Not “which platform should we buy.” The right first question is: where does validated work actually stall?
Map a handful of representative flows, enough to see the shape. Where is time really going: discovery, specification, coding, review, evaluation, release, operations, or incident recovery? Where do things pile up? Where do people feel the friction but can’t name it in a sprint retro?
If coding dominates, the binding constraint might be implementation throughput. But if coding is fast and review is the bottleneck, making coding even faster will only increase the review queue, not the delivery rate. If review has been automated and the constraint is now evaluation coverage, adding more automated gates without improving the evaluations is confidence theater.
The diagnostic rule is simple: find the moving bottleneck. Don’t optimize what’s already fast enough.
Maturity is workflow-specific, not company-wide#
A company doesn’t have one AI maturity level. Different teams, different products, different risk profiles, different codebases sit at different levels of readiness and always will.
The progression I use has five levels, but the number matters less than the behaviors at each one:
- Informal assistance. Engineers use copilots or coding agents individually. No shared operating model. The bottleneck is human coding throughput, even if the tools make individuals faster.
- Generated work with human review. Whole implementations arrive as generated pull requests. Review throughput becomes the binding limit. Comprehension matters more than generation speed.
- Automated gates. Generated changes pass through automated tests and evaluations before human approval. The constraint moves to scenario and evaluation coverage: a shallow test suite makes the gate worthless.
- Bounded autonomy with escalation. Engineers primarily author specifications and scenarios. Agents implement within defined bounds and escalate novel or high-consequence conditions. Specification throughput becomes the constraint.
- Dark factory. Humans don’t routinely write or review production code. The organization specifies what must be built and continuously validates outcomes.
Level 5 is a clarifying horizon, not a required destination. For consequential systems where failure carries real-world harm, Level 4 with accountable human escalation may be the responsible ceiling. Maximizing autonomy isn’t the goal. Earning the autonomy that can be responsibly supported is.
Different work classes can safely sit at different levels. Internal documentation tooling might advance faster than financial authorization or access-control logic. The maturity model is diagnostic, not a uniform mandate.
Cross-functional readiness caps permitted autonomy#
Engineering can’t safely outrun the functions it depends on. Product has to express validated intent that agents can act on, and QA has to evaluate generated behavior, not just execute test cases. InfoSec must govern model access, data boundaries, and agent identity. Data Science needs to measure whether autonomy improves real outcomes or merely accelerates existing failure modes. Human Operations has to define non-negotiable invariants, escalation triggers, and safe degraded modes.
The weakest required capability caps the permitted autonomy for a given workflow. Strong scores in engineering can’t average away a dangerous gap in evaluation coverage or safety escalation. If you can generate code at Level 3 but your QA function still operates at Level 1, the workflow operates at Level 1. The gap controls the ceiling.
This isn’t a call to slow engineering down until everyone catches up. It’s a call to strengthen the adjacent capability before advancing. Autonomy that rests on an unready dependency isn’t autonomy: it’s unrecorded human cleanup pretending to be automation (and the cleanup rarely gets logged).
The Cone of Certainty#
The maturity ladder describes possible states. The Cone of Certainty describes how confidently to plan them.
The adjacent next step after you measure the current constraint: that can be planned concretely. You know the binding limit and what capability sits right next to it, so plan that rung with specificity and commit to it.
The rung after that is a hypothesis (a well-informed one, but still). You understand what it would likely require, but you’re making that call from the current rung, and evidence from the next one will change your understanding. Treat it as directional, not committed.
Distant maturity is speculative. It’s useful for architecture and strategy, but it isn’t a reliable delivery promise. Technology, regulation, organizational capacity, and the evidence you haven’t gathered yet may all change the appropriate destination.
The operational shorthand: plan the next rung, then learn before committing to the one after.
Choose experiments by consequence and reversibility#
Don’t announce a transformation program. Start with a small number of bounded experiments instead.
Pick workflows that are observable, reversible, and safely degradable. Define what evidence would constitute success, what safety thresholds must hold, what triggers escalation or rollback, and who retains accountable human authority before the experiment starts. If you can’t name those things, you don’t understand the workflow well enough to change how it operates.
Start small for a reason beyond risk management: the organization needs to learn what autonomy actually feels like before it can make informed decisions about expanding it. An engineer who has experienced Level 3 gates on a low-risk workflow understands what would be required to apply them to a higher-consequence system. An executive who has seen the evidence from one bounded experiment can evaluate the next proposal with calibrated judgment rather than vendor enthusiasm or fear.
Strengthen the adjacent capability#
But don’t confuse bounded experimentation with tool adoption. The most common AI transformation mistake is installing a favorite platform and calling it done. The platform is rarely the binding constraint.
If coding speed has created a review overload, strengthen comprehension and review evidence. If review has been automated with gates, invest in scenarios and evaluations. If implementation has become the cheap part, invest upstream in product discovery and specification quality. The bottleneck migrates, and the organization’s investment must follow it.
This is why the Spec Ceiling matters. AI coding agents accelerate implementation to the point where the bottleneck isn’t writing code anymore. It’s deciding what to build, specifying it well enough that an agent can act on it, and verifying that what came back is correct. The constraint migrates upstream: from coding to review, then to evaluation, then to product discovery and specification. If you only make coding faster, you create a review crisis. If you fix review with automation, you create an evaluation crisis. If you fix evaluation, the ceiling becomes your organization’s ability to discover and specify what should be built.
Each transition requires investing in the capability that sits adjacent to the current limitation, not doubling down on what’s already working.
Measure the total system#
Speed is the easiest thing to measure and the most misleading.
A generated pull request that ships quickly but creates incidents, increases support burden, and forces senior engineers to reverse-engineer its behavior isn’t faster. It transferred the work to people who weren’t consulted and didn’t consent. The cost disappears inside a velocity chart, though.
Measure transferred work: who is doing more now that wasn’t part of the automation decision? Measure cognitive load, defect rates, incident frequency, support burden, and customer or operational outcomes. Measure cost per verified outcome, not per token. Measure disparity: does autonomy amplify some people and leave others behind? Measure whether humans are cleaning up unrecorded failures that the dashboard doesn’t see.
If the organization can’t answer whether outcomes actually improved, it doesn’t know whether the autonomy was earned.
Feed learning back into durable artifacts#
The postmortem isn’t the outcome. What matters is the durable constraint that changes future behavior. A written retrospective that nobody reads again doesn’t change anything by itself.
When an incident reveals a gap in the specifications, evaluations, or guardrails, those artifacts must be revised. When a near-miss exposes a weak evaluation scenario, the scenario suite must grow. The system must behave differently afterward, and the change must be visible in the artifacts that govern it.
I’ve written about this mechanism twice: once to make the general case that the incidents are the training data, and once to show the concrete mechanism for turning a verified lesson into a deterministic guardrail that survives the meeting where the lesson was learned.
The learning loop isn’t “write a postmortem and file it.” It’s: identify contributing causes, convert the failure into an explicit requirement and regression case, add verification, validate the fix without weakening another safety property, and monitor for recurrence. The durable artifact is an improved constraint on future behavior.
Expand, hold, narrow, or reverse#
Autonomy is earned authority. It’s not a reward for model capability or executive enthusiasm, and it doesn’t only expand.
After gathering evidence from a bounded experiment, the organization must decide: expand the autonomy envelope, hold at the current boundary, narrow it because the evidence doesn’t support the current level, or reverse the experiment entirely and restore the previous operating model.
Higher consequence lowers the permitted ceiling or raises the evidence bar. If a workflow affects safety, money, or irreversible decisions, the evidence required to advance is higher, and the escalation path to a human is shorter. Humans retain accountable authority where consequences demand it, regardless of technical capability.
The operating model this implies#
The method leads toward a spec-driven, evaluated hybrid computing architecture: three layers designed to interact rather than replace each other.
Probabilistic systems (models, agents) provide capability: interpretation, generation, classification, and the handling of ambiguity. Deterministic systems enforce permissions, invariants, gates, rollout, and rollback. Humans judge ambiguity and consequence, author validated intent, and retain accountable authority where it matters.
The architecture separates what changes from what must survive the change. Models are replaceable runtime dependencies. Their capability, cost, latency, and availability shift too quickly to serve as the durable foundation. The provider-neutral harness is where organizational knowledge compounds: specifications, evaluations, policy, orchestration, observability, and incident learning. That accumulated knowledge is the real moat. Inference is a commodity you route work through, not an architecture you marry.
This isn’t a technology stack recommendation. It’s an operating model. The specific tools will change. The separation of concerns, the discipline around evaluation, and the principle that autonomy must be demonstrated before it’s granted shouldn’t.
The human stake#
AI transformation changes where work and cognitive burden land. But it doesn’t eliminate either one. It just moves the work around.
Preserve expertise formation. Junior engineers can’t learn to debug systems they’ve never built and never reviewed. Senior engineers can’t maintain calibration if they never encounter the surprises that automated gates filter out. Make it safe to surface uncertainty: the person who says “I don’t understand this generated change well enough to approve it” is doing the organization a service, not failing a metric.
Avoid surveillance metrics. Measuring individual prompt volume, lines generated, or acceptance rate creates incentives to generate code that passes review rather than code that’s correct. Don’t reduce review or operational staffing merely because generation became faster. The capacity freed by automation should be reinvested in specification quality, evaluation design, and outcome measurement, not cut as savings before the evidence is in.
The purpose is responsible capability expansion, not maximum automation. People are amplified unevenly by these tools. Strong judgment gets stronger. Weak judgment gets faster. The organization’s responsibility is to make the amplification visible, to strengthen judgment where it’s needed, and to ensure that no one is left cleaning up failures the dashboard was designed not to see.
What the assignment revealed#
As the working document behind that take-home took shape, I read my own point of view back to myself and recognized it. The articles, the frameworks, the open-source skills, the presentations: all of it had been converging on a coherent method while I was paying attention to something else. The assignment didn’t test whether I could invent a position. It surfaced one that was already there, developed in the open and tested against real systems.
What the method coheres around is earned autonomy. Each rung begins where the work stalls. The evidence decides whether the next boundary of authority opens.