Noelle Tolve has already written the epic when she asks AI to review it. She asks for criticism from an engineer’s or architect’s perspective, reads the suggestions, and uses the ones that make sense. Then she talks to her engineers.
“I let AI poke some holes in my epic,” she said.
She welcomes criticism from the engineers, too. They know the system they’re being asked to change. A suggestion from the model might give her something to ask them about, but it doesn’t settle the question.
I joined Noelle’s October 6 AgileRTP talk, “I Didn’t Become an AI Expert. I Became a Beginner Again.”, hosted by Catherine Louis and co-host Arjay Hinek. She described her move from testing into engineering and then product management, and how she uses AI in her work now.
An epic is a piece of work large enough that a team needs to break it down into smaller pieces. Before asking engineers to do that work, Noelle wants to find problems in what she’s proposing. She writes the epic herself. She was clear that she doesn’t want AI to take over her critical thinking.
Testing, engineering, then product#
Noelle’s deck, My Journey Across the SDLC, followed her through those jobs. In testing, she learned to look beyond the happy path, where the user does what you expect and the software behaves as intended. Engineering gave her a closer understanding of what happens inside the system. A shortcut that saves work now may leave more work for the next person who changes the code (the cost described by technical debt). She brings both kinds of experience to product management, where she decides which customer problem to solve and what work deserves priority.
Catherine asked about putting those perspectives together in a strategic meeting. Include a test engineer to represent risk, she said.
If a tester spots an edge case while you’re deciding what a feature should do, you can change the proposal before anyone builds it. Find the same problem after the code is written and you’re discussing rework. That’s a reason to invite the tester while there’s still a decision to make.
Noelle described making acceptance criteria clear and granular for test engineers. Those are the conditions the work must meet to be accepted. She leaves test design to the testers, who decide how to check the behavior and what else might go wrong.
The Agile Manifesto’s principles call for business people and developers to work together throughout a project. Catherine’s suggestion was to include someone who can assess testing risks in the strategic meeting.
What the model has access to#
Anthropic’s prompting guidance describes giving a model a role to focus its behavior and tone. Asking for an architect’s critique can direct the response, but your architect may know a constraint you haven’t put in the prompt.
Noelle offered a hypothetical Jira example. If an AI review suggested that an earlier defect might be relevant to the new work, the product manager could ask a tester about it.
For a tool to retrieve that defect, it needs access to the records. Google calls connecting model output to verifiable information sources grounding. The role prompt alone doesn’t give the model your Jira history. Check the issue before asking the team to act on it.
Noelle advocated discussing edge cases during discovery and refinement, before implementation. She also advocated test-driven development, where the engineer writes a test, writes code to make it pass, and refactors.
Her deck spelled out who remains responsible for the work: test engineering assesses risk, engineering owns architecture and tradeoffs, and product owns the problem and priority. AI suggestions still need someone to judge them.
There was a practical restriction, too. Noelle cautioned against putting detailed work information into personal AI tools. Use the tools your company permits and check what they return. The NIST Generative AI Profile has related guidance on acceptable use, approved providers, and verifying generated sources.
Still learning#
Catherine also asked about spec-driven development during the Q&A. A specification describes the behavior and constraints that guide implementation. GitHub’s Spec Kit, for example, organizes this into a specification, technical plan, and implementation tasks.
Noelle said she’d mainly encountered spec-driven development in engineering. She was still exploring its use in product management, including epics, features, and stories. She cautioned about prompts influencing the results and AI drifting from the intended work.
Her epic review is something you can try with work you’ve already written. Ask an approved tool to look for problems, read its suggestions, and take the relevant questions to your engineers. They may explain why an objection doesn’t apply, or find a problem the model missed. Either way, you can revise the proposal before they start building it.
You can find future events through AgileRTP on Meetup.
Based on Noelle Tolve’s October 6 AgileRTP talk, the audience discussion, and her accompanying presentation, forwarded by Catherine Louis.