Harnessing AI Agents P1: My AI agent kept "finishing" features that didn't work

This post assumes you're already using AI coding agents (Claude Code, Cursor, Copilot) and have hit the wall where they stop being reliably useful. If agents work fine for you, you probably don't need this.

The moment I stopped trusting "done"

I asked Claude Code to add email notifications for new comments on my blog. Straightforward feature. Someone leaves a comment, admin gets an email via Resend.

The agent wrote EmailService.sendCommentNotification(). Wired it into createComment(). Wrote unit tests. Tests passed. "Comment email notification feature is complete," it reported.

I submitted a comment. No email.

The Resend API key wasn't configured in the Workers environment. The unit tests had passed because they mocked the SDK. The agent had built a feature that worked in test-land and silently failed in reality. It wasn't lying to me. It just never checked.

That was annoying. What happened next was worse.

New session. I asked it to pick up where it left off. It couldn't. Fresh context window, no memory of what it had done before, no idea the notification code was half-broken. It read the codebase, saw the existing code, and started "improving" it. Created a second notification service alongside the first one. Duplicated the database query. Left the original broken code sitting right there.

Third session. Same story. By now I had three overlapping implementations of the same feature, none of which actually sent emails, and a git history that read like a crime scene.

The individual sessions weren't the problem. Each one, taken on its own, produced reasonable-looking code. The problem was between sessions. No memory. No shared definition of "done." No way to verify anything beyond "does it compile."

The framework that fixed it

In November 2025, Anthropic engineer Barry Zhang published Effective Harnesses for Long-Running Agents. The core argument: if you want an AI agent to reliably complete complex, multi-session tasks, you need to build a structured harness around it. Not smarter prompts. Not a better model. Structure.

I read it, nodded along, and then spent a few weeks actually trying to apply it to mogpersonalweb, a full-stack blog CMS I run on Cloudflare Workers (TanStack Start, Hono, Drizzle ORM, the whole stack). 12 feature modules, dual API layers, multi-tier caching. Complex enough that "one session, one feature" just isn't realistic. Simple enough that the framework doesn't need a PhD to maintain.

What came out of that was four patterns. None of them are new ideas. They're basically software engineering discipline, clear specs, incremental commits, reproducible environments, end-to-end tests, repackaged for an audience that has no long-term memory and an irresistible urge to declare victory early.

The 4 patterns

1. Feature list: what the agent is allowed to build

A JSON file where each entry describes one user-observable capability, its current status (passing or failing), and concrete verification steps. The agent can flip status. It cannot touch descriptions or verification steps.

This is probably the most important piece. It solves what I think of as the "writing your own exam" problem. Without it, the agent decides what "done" means. And its definition is always generous.

{
  "id": "comment-notification",
  "category": "functional",
  "description": "When a new comment is submitted, admin receives an email notification via Resend",
  "status": "failing",
  "verification_steps": [
    "Submit a comment on any post",
    "Check admin email inbox",
    "Confirm notification email arrived with comment content and post link"
  ]
}

The verification steps are the teeth. Not "run tests." Not "check it works." Specific actions with expected outcomes that the agent has to walk through before it's allowed to mark anything passing.

Deep dive: I stopped guessing what my AI agent should know covers how to extract a feature list from an existing codebase, the 3 rules for writing descriptions that actually work, and the mistakes I made along the way.

2. Progress tracking: memory that survives between sessions

A markdown file (claude-progress.md) that the agent reads at session start and updates at session end. What's done, what's in progress, what's broken, what comes next.

## Current state
- Branch: feature/comment-notifications
- Current task: comment-notification

## Completed
- [x] EmailService.sendCommentNotification method
- [x] Wired into createComment in comments.service.ts

## In progress
- [ ] Integration test: mock Resend API, verify email content

## Known issues
- Resend SDK needs fetch polyfill in Workers environment

Combined with git log, this gives the agent enough context to resume work without re-reading the entire codebase. It's not fancy. A checklist in a text file. But it's the difference between "pick up where we left off" and "start over from scratch and hope for the best."

Deep dive: My AI agent kept starting over from scratch Coming soon, covering the session protocol (what to read and update, and when), git commits as backup memory, and why markdown beats plain text for agent consumption.

3. Environment init: start every session from a known-good state

A shell script that verifies the development environment before the agent writes a single line of code. Dependencies installed? Type checking passes? Tests green? Dev server starts?

#!/bin/bash
set -e
bun install --silent
bun run tsc --noEmit    # type check
bun run test            # existing tests pass

Sounds obvious. But I once watched an agent code for two hours in an environment with a broken node_modules. Every import failed. It kept trying to "fix" the imports instead of fixing the environment. Two hours of context window, gone. I still think about that sometimes.

The script uses set -e to fail fast. Outputs only PASS or FAIL per step. Minimal output is deliberate: verbose logging eats tokens, and the agent really only needs two signals. Either "everything's fine, start working" or "this specific thing is broken, fix it first."

Deep dive: My AI agent coded for hours in a broken setup Coming soon, covering script design principles, the token-budget argument for minimal output, how to wire it into CLAUDE.md, and the separation between init-agent and coding-agent roles.

4. Verification loop: "done" means verified, not vibes

The agent's optimization target is "complete the task." The shortest path to "complete" skips verification. Code compiles, tests pass, looks right. Close enough. Except it's not, and you find out three sessions later when everything's tangled together.

Here's the fix: verification is not a nice-to-have final step. It's part of the definition of done. The feature list's verification_steps field is a contract. Every step must be executed. If any step fails, the feature stays failing. No exceptions, no "mostly works."

This is where the email notification story would have ended differently. The verification step says "confirm notification email arrived." So the agent submits a comment, checks the inbox, finds nothing, and keeps working on the problem instead of declaring victory.

Deep dive: I stopped letting my AI agent grade its own work Coming soon, covering the three verification layers (unit tests, type checking, end-to-end), why agents skip verification (it's not laziness, it's optimization), and a real before/after walkthrough.

Before and after

Without the framework, adding a newsletter subscription feature across three sessions produced: a broken database migration, an unauthenticated API endpoint, a deleted test file, and a form that didn't work. The agent declared success.

With the framework, same feature, same three sessions. Clean incremental commits. A progress file that guided each session. And a moment in session three where verification caught a missing notification trigger in the publish workflow. The agent found the bug because the verification steps told it to check. Fixed it. Marked it passing.

Three sessions either way. Same agent, same codebase, same feature. The difference was structure.

Further reading

You are reading the overview of the 5-part series on Harnessing AI Agents. Here are the other parts and relevant resources:

• I stopped guessing what my AI agent should know - How to extract a feature list from an existing codebase, the 3 rules for writing descriptions that actually work, and the mistakes I made along the way.

• My AI agent kept starting over from scratch - Covering the session protocol, git commits as backup memory, and why markdown beats plain text for agent consumption.

• My AI agent coded for hours in a broken setup - Covering script design principles, the token-budget argument for minimal output, and the separation between init-agent and coding-agent roles.

• I stopped letting my AI agent grade its own work - Covering the three verification layers, why agents skip verification, and a real before/after walkthrough.

• Effective Harnesses for Long-Running Agents - The original inspiration by Barry Zhang on the Anthropic engineering blog.

• The mogpersonalweb repository - The full-stack blog CMS codebase where this framework was built and tested.