Harnessing AI Agents P2: I stopped guessing what my AI agent should know
文章摘要
TL;DR: Your codebase already contains a feature list, scattered across routes, services, schema, tests, and issues. I'll show you how to extract those into a structured JSON inventory that anchors your AI agent and makes every task scoped and verifiable. All demonstrated on a real Cloudflare Workers project.
This post assumes you have an existing project and want to use AI coding agents (like Claude Code, Cursor, Copilot) more effectively. If you've read my previous post on building an AI agent workflow from scratch, this is the practical "how-to" for one of its core pieces.
The problem: why "just read my repo" doesn't work
I pointed Claude Code at mogpersonalweb (full-stack blog CMS, runs on Cloudflare Workers, uses Hono and Drizzle ORM and, uh, TanStack Start for the SSR bits) and asked it to add newsletter subscriptions.
It wandered. Spent tokens poking around the codebase like a new hire on day one with no onboarding doc, opening files at random, reading things that didn't matter, skipping things that did. Missed the patterns I'd spent weeks establishing. My Result<T, E> error type, the middleware chain, cache versioning. And built something that technically worked but felt like it belonged in a completely different project. The kind of output where you look at it and go, yeah, this is correct, but also, have you seen the rest of my codebase?
My first reaction was to blame context length. Or the model. But honestly the real problem was embarrassingly simple. I'd handed over code without ever telling it what any of that code was for.
And that's a distinction that took me way too long to see. Raw code tells an agent what exists. It says nothing about what actually matters. Or what you've already covered. And definitely nothing about when you're actually done with a feature. Without that structure, what you get is improvisation. And agent improvisation is just hallucination wearing a trench coat.
What I needed: some kind of anchor. A feature inventory that's structured (machine-readable, ideally JSON instead of just prose) something that actually pins down where the agent looks instead of letting it wander. Not documentation for humans to skim over coffee. More like a file where each entry says "here's a capability this app has, here's its current status, and here are the steps to prove it works." Intent, basically. The stuff that lives in your head but never makes it into the source files.
Once the agent has that, it stops guessing and starts working within boundaries. Still makes mistakes, yeah. But there's a world of difference between "guess and hope" and "work within constraints."
Great. So I sat down to write this inventory from scratch. Opened src/. Stared at it.
...where do you even start?
From memory, apparently. I tried. Got to about 8 items before everything started blurring together, descriptions getting vaguer with each one. By item 6 I was writing stuff like "comment system" with zero specifics, which, as we'll see later, is basically useless.
So I tried what felt like the obvious shortcut. (Obvious to me, anyway. Maybe nobody else would've done this.) I asked the agent itself to generate the feature list from the codebase. It came back with 24 items in clean JSON. Looked great, looked professional, looked like exactly the kind of thing you'd put in a README to impress someone. Then I checked them against the actual code:
This feature didn't exist. Neither did "dark mode toggle" or "post bookmarking." Three hallucinated features, conjured from whatever statistical soup of blog CMS patterns the model had ingested during training. And on the flip side, it had listed internal plumbing like "database migration system" and "middleware chain composition" as if they were product features a user would care about. Those things matter, but not as user-observable capabilities. (They do belong in a feature list, actually. Just framed differently. More on that in Rule 1.)
That's when it clicked, and I felt a little stupid for not seeing it earlier. The agent that builds features can't also be the one deciding what "done" means. That's writing your own exam and grading it too. What I needed was systematic extraction, starting from what the code actually does, filtered through a human who knows what matters versus what's noise.
The mental model: your code already knows what it does
Once I stopped trying to conjure the list from thin air and just, you know, looked at my project, things got easier fast. I opened the folder structure. Hono puts routes here. Drizzle puts schema there. Services live under each feature module. The shape of the inventory was already right there in front of me, I'd just been trying to write it from the wrong direction.
Every codebase, if you squint at it right, has five layers where features leave traces:
Source | What it reveals | Example |
|---|---|---|
Routes | What users can reach | /search, /admin/posts/edit.$id |
Service methods | What the app can do | moderateComment(), submitFriendLink() |
Schema / Models | What data the app manages | posts, tags, friend_links tables |
Tests | What behavior has been verified | "should return tags sorted by name" |
Issues / TODOs | What's broken or missing | "Search results have no highlighting" |
What about UI components? Fair question. Complex client-side interactions (drag-and-drop, live validation, modals) live in src/features/*/components/. I fold these into service or route items when I find them, since they almost always map to an existing capability. But if your app is heavily client-side (SPA, canvas-based, rich editor), components probably deserve their own pass.
This isn't a mechanical process where you run a script and get a feature list out the other end, though. I want to be clear about that because the table above makes it look cleaner than it is. Each source gives you raw material, and you still have to curate. Toss the internal helpers nobody cares about, merge items that describe the same thing from different angles, rewrite implementation details as capabilities someone could actually observe. Extraction plus editorial judgment. Not copy-paste.
The order that felt most natural to me: Routes first (broadest view), then Schema (fills structural gaps), Services (adds depth), Tests (validates what's been verified), Issues (captures what's missing). Wide to narrow, each pass filling holes the previous one left. But I could see someone starting with schema if that's how they think about their app. There's no magic to the ordering.
The 5 sources: how I extracted ~22 feature items from mogpersonalweb
For each source I'll walk through what I looked at, what I pulled out, and what caught me off guard. Every extracted item uses the same JSON schema (this is the actual format of the feature list file):
Three examples, three categories. The categories will make more sense as we go.
Status values: each item has one of four statuses:
Status | Meaning |
|---|---|
pending | Planned but no code exists yet. A slot in the backlog. |
failing | Code exists but the feature is broken, incomplete, or doesn't pass its verification steps. |
passing | Works correctly. Verification steps succeed. |
deprecated | Was passing, now intentionally removed or superseded. Kept in the list so the agent doesn't accidentally rebuild it. |
And the transitions between them:
Most items start as failing. You extracted them from existing code that works, so they go straight to passing, or you noted something broken, so it's failing. pending is for features you want but haven't started. deprecated is rare but prevents a specific failure mode: without it, the agent sees a capability that used to exist, finds no code for it, and helpfully rebuilds the thing you just spent an afternoon removing.
Source 1: Routes (user-reachable functionality)
If a user can navigate to it, it's a feature. Hard to argue with that, so I started here.
The layout groups basically handed me a role taxonomy for free: public/ is anyone, auth/ is authentication flows, _user/ is logged in, admin/ is admin only. That mapping went straight into each item's category field without me having to think about it, which was nice, because the next part required a lot more thinking.
For features that cross roles or describe system-level behavior (AI moderation, post tagging, search highlighting), I used "functional". And later I added "technical" for infrastructure items, but that whole distinction really belongs in the Rule 1 discussion, so I'll come back to it.
I poked through page components too, looking for interactions the route path alone doesn't reveal. Inline editing, keyboard shortcuts, that sort of thing. Short list, as it turned out. Folded them into existing service items and moved on.
I nearly skipped the SEO routes. rss.xml, sitemap.xml, robots.txt. No UI, no React component, not even something you'd demo to anyone. But they're endpoints a user (or a bot, or a feed reader) can hit, with behavior you can verify, and that makes them features whether they render pixels or not. Funny how the least glamorous routes were the ones I almost left out.
Routes produced 7 candidates. After de-duplication with services (coming up next), 5 survived as route-only items.
Source 2: Service methods (business capabilities)
Routes gave me the surface. Services were where I found the depth, and also where the de-duplication headaches started.
A single route like /post/$slug has a whole cascade happening behind it. Fetching the post, checking permissions, loading comments, incrementing view counts, maybe triggering a cache warm. Each public method on a *.service.ts file is one business capability, and I had 12 service files, north of 80 public methods total. posts.service.ts, comments.service.ts, tags.service.ts, friend-links.service.ts, search.service.ts, media.service.ts, cache.service.ts, ai.service.ts, and a handful more I'm probably forgetting.
Now, the de-duplication problem. POST /api/comments and createComment() are the same capability described twice. Lots of that. My rule ended up being: if the route is just an HTTP wrapper around a service call, kill the route item, keep the service one. Service names describe the business action (moderateComment) rather than the HTTP plumbing (POST /api/comments/moderate), and that's what you want in a feature description. I only kept route-only items for stuff that's purely routing. Redirects, permission gates, SEO endpoints with no service behind them.
This killed about a third of my initial list, which honestly felt great. Less noise in the JSON, fewer tokens wasted when the agent loads it. Services produced 10 items; combined with routes after the cull, 12 total.
Source 3: Schema / Models (data the app manages)
Almost skipped this one. Would've been a massive blind spot, in retrospect, because schema files contain information about your app that literally nothing else does.
Drizzle schema is basically a map of what the app cares enough about to persist. And what I found when I went through the tables was that some of them implied features I'd completely forgotten I'd built.
The friend_links table was the big one. It exposed an entire approval workflow (admin approve/reject, notification emails on status change) that had been invisible during route scanning because it was tucked behind a generic admin layout. I'd looked at admin/friend-links/index.tsx and thought "yeah, friend link management, already got that." But the approval flow was a separate feature with separate behavior, and I'd missed it.
And then there was email_unsubscriptions. I had to go re-read my own migration file to remember what that table was even for. (It handles opt-outs from comment notification emails. Past me apparently built that at some point.) Schema filled in 3 items that neither routes nor services had turned up. Not bad for a source I almost didn't check.
Source 4: Tests (verified behavior)
Test descriptions are feature descriptions in their pajamas. Rough around the edges, written for a different audience, but they contain the actual behavioral intent in a way that service signatures and route paths don't.
But here's the thing: what the tests didn't cover ended up being more useful than what they did. I had 50+ descriptions across tags, friend links, media, rate limiting, search. Nothing for comment creation. Nothing for post publishing. Nothing for the entire auth flow. Those gaps were a signal. They told me which features to give a mental asterisk, which areas to point the agent at first when I wanted better coverage. Tests confirmed 2 existing items and turned up 2 behavioral items I'd missed, but the real value was the negative space.
Source 5: Issues / TODOs (broken or missing features)
The first four sources extract what exists. This one gets at the stuff that isn't there yet, or that's there and broken.
I grep'd for TODO and FIXME markers, checked GitHub Issues. Codebase was clean. No outstanding TODOs in source files. I'm honestly not sure if that reflects discipline or if I just never write them. But I knew from actually using the app that certain things were busted or half-done. The kind of stuff you notice every time, wince at, and then go do something else because fixing it isn't urgent enough to bump whatever you're actually working on:
By far the easiest items to write. Past me had already done the hard work (noticing the problem, getting annoyed enough about it to remember). The agent could pick these up with zero ambiguity because the annoyance had already crystallized into specifics. 3 items, all "status": "failing".
After all five passes: 22 items. I felt good about the coverage, felt like I'd been thorough, felt like I could hand this to the agent and things would go well.
They did not go well. Half the items produced garbage. The features I'd extracted were real, the categories made sense, the coverage was solid. But the descriptions (the way I'd worded them) were leading the agent astray in ways I hadn't anticipated at all.
The 3 description rules: making extractions agent-ready
Extraction gives you the raw material. These rules are what make it actually usable, and I arrived at them the hard way: by writing bad descriptions, watching the agent misinterpret them, getting frustrated, and iterating until the failure modes stopped recurring. I'll show the failures alongside the fixes because the failures are where the learning is.
After my first pass I had 30+ items, and I'll be honest, most of them were junk. Too vague, too focused on implementation guts, or worded in a way that made verification impossible. I threw out a third entirely. Rewrote the rest.
Rule 1: Write what the user does, not what the code does
My gut was to write items that read like Jira tickets:
The agent built the endpoint. Just the endpoint. No form. No validation. No success message. No error states. It did exactly what I asked for, which (and I cannot stress this enough) was precisely the problem.
What happened concretely: it created a Hono route handler, wrote the database insert, returned a 200 JSON response, and called it done. No React component. No Zod schema on the input. No success or error UI. It had built the back half of a feature because the description only described the back half. I stared at the diff for a while before I realized the bug was in my spec, not its implementation.
Here's the rewrite, framed around what a user actually experiences:
Night and day. Implementation-focused descriptions let the agent stop at the code boundary and feel satisfied. User-intent descriptions drag it through the whole picture: form, validation, feedback, error states, the full loop.
Note: The newsletter feature doesn't exist yet in mogpersonalweb. This is an example of a "failing" item I'd add for future work.
When Rule 1 doesn't apply: technical features
So Rule 1 is great when a human is the actor. But not everything agents touch is user-facing, and this is where I got bitten in a completely different way.
I asked the agent to "refactor the cache invalidation layer to use version-based keys." My feature list was all user-observable stuff ("user can search posts," "admin can upload images") and it had nothing to say about how caching was supposed to work under the hood.
The agent went ahead and refactored caching. It "worked." Requests returned data. But it had silently dropped my waitUntil() pattern for background invalidation and swapped out versioned keys for timestamp-based expiry. A totally different architecture, one that would quietly break under load when stale cache entries piled up. The feature list had told the agent nothing about the intended design, so it designed its own, and the fact that it happened to return correct responses during testing was just luck.
For infrastructure (caching strategies, middleware chains, rate limiting, migration patterns), flip Rule 1. Write what the system does, not what the user sees. These get category: "technical":
Verification steps are still concrete, still things you could automate. They just test system behavior instead of UI flows. And the Rule 3 heuristic still holds: if you can't turn the steps into an automated test, you haven't finished writing them.
Quick gut check I use for categorization: would I explain this feature to a product manager? Then it's "functional". Would I only bring it up in a sprint retro or a pull request description? Probably "technical".
Rule 2: One observable behavior per item
Too broad:
When does "complete" happen? When can the agent mark this "passing"? It can't. Nobody can. The word "complete" is doing no work in that sentence.
I watched what the agent did with this one and it was kind of painful. Tried to build everything at once (creation, threading, moderation, notifications, deletion) in a single session. Around the third sub-feature it started losing the thread, referencing variables from thousands of tokens ago, duplicating logic it'd already written. End result: creation worked, moderation was half-broken, notifications never materialized. Context-window overreach, which, if you've spent any time with these agents, you know is one of the most common failure modes. The description invited it.
Too narrow:
That's not a feature. That's a line of code cosplaying as a feature. You're micromanaging, doing the agent's job for it, and in the process stripping it of the context it would need to make good decisions about how to implement that column.
Right-sized:
One behavior. Observable outcome. The definition of "done" fits in your head without squinting. The agent knows when to stop, which, I've learned, is half the battle.
Rule 3: Verification steps must walk the real UI
My early attempts:
Useless. "Tests pass" is a side effect, not proof that the feature works. It's like verifying a building is structurally sound by checking that the lights turn on.
What actually happened when I gave the agent this: it built search, wrote one unit test for the happy path, watched it go green, and moved on. The search page did render results. It just had no empty state when nothing matched, no partial-match handling, no edge cases at all. The agent had satisfied "tests pass" without ever getting close to "search works for a real person typing real queries."
Here's what useful verification looks like:
Do a thing, expect a thing. Do another thing, expect another thing. An agent can walk through it. A human can walk through it. A Playwright script can walk through it. That's the bar: if you can't turn the steps into a Playwright test, you aren't done writing them.
Trade-offs and common mistakes
This system isn't perfect. I've messed it up in every way you'd expect and a few you probably wouldn't. Here's the damage, in roughly the order I inflicted it on myself.
I went too big too fast. 35 items on day one because I was being "thorough." Also completely overwhelming, for me and for the agent. 35 feature items as context is a lot of tokens and a lot of diluted focus. The agent kept referencing the wrong items, confusing similar-sounding features, generally thrashing. Scaled back to 10-15 and things smoothed out immediately. You can always add more once the first batch is stable and you've calibrated on what the right granularity feels like for your project.
Then I mixed sizes wildly. Some items were 10-minute tasks. Others would've taken a week. When "add alt text to images" sits next to "implement full-text search with relevance ranking" in the same list, the agent can't gauge what kind of effort it's signing up for. It treats them the same, which means it either over-engineers the small one or under-engineers the big one. If something feels 5x bigger than its neighbors, that's a strong signal it should be broken down.
Worse, I let the agent write its own descriptions. God, this was tempting. "Hey, you already know the codebase, just generate the feature list." And it did! And the list was wrong in ways that didn't become obvious until I'd already started using it. The agent that builds features should not define what "done" means. Writing your own exam and grading it. Descriptions come from you, or from a separate planning session with a different hat on. The coding agent's only job is flipping "status": "failing" to "status": "passing".
The one that actually hurt, though, was not maintaining the list. I refactored three modules, renamed files, changed function signatures. Forgot to update the feature list. Next session, the agent spent 20 minutes (real minutes, real tokens, real context window) trying to import from files I'd deleted, calling functions with signatures I'd changed. It was faithfully following instructions that were now lies. All because I couldn't spend two minutes updating a JSON file. A stale feature list is worse than no feature list. At least without one the agent starts fresh instead of building on top of bad information.
When to skip all of this: not every project needs a feature list. Prototyping something, hacking on a weekend project, poking at an unfamiliar API? Just talk to the agent. This system earns its keep in a pretty specific set of circumstances:
Your codebase has patterns the agent should respect, not reinvent from scratch
You're tracking features across multiple sessions, conversations that don't share context
You care about actually verifying that work is correct, not just that it compiles and doesn't throw
If none of that rings true, don't bother. You can always bolt on a feature list later when the project outgrows the ad-hoc approach.
One more scenario worth mentioning: greenfield projects. No existing code, no extraction to do. The feature list is the spec. Write it first, build from it. Simpler in some ways, because you're defining intent from scratch rather than reconstructing it from code that already exists. Everything in this post is specifically for the messier case where you've got a codebase and need to build the inventory retroactively.
The list is the contract
The Agent runtime environment (the "harness") might contain auto-startup scripts and progress tracking and whatever else. But the feature list is what actually steers the agent. Not documentation. A control mechanism. It decides where the agent looks and what it considers done.
Here's what actually changed once I started doing this: I stopped re-explaining my codebase at the top of every session. Stopped reviewing diffs where the agent had solved some adjacent problem I never asked about. Stopped squinting at output trying to figure out if the agent actually finished or just got 80% there and moved on, because now there were verification steps, and either they passed or they didn't.
5 sources. 3 rules. ~22 items. One JSON file.
Nobody's reading this JSON file for fun. It's a contract between you and your agent. Each item says: this is what the app does, this is how we prove it, and this is where it stands right now. Product features anchor the user-facing work. Technical features anchor refactoring, migrations, and performance. Between them they cover basically everything an agent might touch. I load the file as context when I start a session, and the agent works within those boundaries instead of improvising its own.
Quick reference if you're building your own:
Status values: pending (planned, no code yet), failing (exists but broken or incomplete), passing (works, verification steps succeed), deprecated (intentionally removed, kept so the agent doesn't rebuild it).
Categories: public, user, admin (role-based, who can access it), functional (crosses roles or doesn't belong to a single actor), technical (infrastructure, not directly user-observable).
The follow-up post will cover how the feature list drives multi-session workflows. Assigning items across sessions, tracking progress between them, wiring verification steps into E2E tests that actually run. The list stops being a one-time exercise and becomes the spine of an ongoing development loop. Once you have this extracted map, the question becomes how to make it move, and that means building an automated harness that lets the agent auto-load the feature list, run verification steps, and update status as it goes.
But honestly, if you want to try this, just go open your project's route directory. Pick 5 routes. Write 5 feature items in the JSON format above. 15 minutes, maybe. You'll know pretty quickly whether this approach makes sense for your project.
I'm still figuring this out as I go. If you've tried something like this (or tried it and bailed) I'd genuinely like to hear what happened.
Further reading
You are reading the second part of the 5-part series on Harnessing AI Agents. Here are the other parts and relevant resources:
My AI agent kept "finishing" features that didn't work - The overview of the series covering the 4-pattern framework (feature list, progress tracking, environment init, verification loop).
My AI agent kept starting over from scratch - Covering the session protocol, git commits as backup memory, and why markdown beats plain text for agent consumption.
My AI agent coded for hours in a broken setup - Covering script design principles, the token-budget argument for minimal output, and the separation between init-agent and coding-agent roles.
I stopped letting my AI agent grade its own work - Covering the three verification layers, why agents skip verification, and a real before/after walkthrough.
Effective Harnesses for Long-Running Agents - The original inspiration by Barry Zhang on the Anthropic engineering blog.
The mogpersonalweb repository - The full-stack blog CMS codebase where this framework was built and tested.