Building an agent harness: What my weekend hackathon project taught me about wiring agents into real products
How I used Hermes Agent as a harness for a production web app and why treating an agent as an orchestration layer changed the way I made architectural decisions.
Last weekend I participated in the GrowthX x Hermes Buildathon and built an AI slop tweet detector app.
Though I didn’t win it, I learnt a bit or two about building agent harness and that’s what I will be sharing in this issue. (Please do not ask for the app link, I have turned off tunnelling and also out of OpenAI credits 😭)
Also, if you missed my writings all this while, I am sorry for being MIA. I had a BAD writer’s block. That’s a story for some other day. Let’s talk about some serious tech stuff. 😈
On April 25, 2026, a Cursor coding agent running Claude Opus 4.6 deleted PocketOS’ entire production database, and every backup, in nine seconds. It took one POST to Railway’s API and one volume-deletion call. There was no confirmation prompt. The founder ended up rebuilding customer reservations by hand, matching Stripe charges against calendar invites.
If you are building software, especially with the help of AI, you should read this till the end. PocketOS had every safeguard turned on: destructive guardrails, plan mode, the model’s own tool-use safety, and the founder’s explicit project rules. None of them worked. The agent simply had a credential broad enough to destroy the volume, and nothing in the system stopped it.
There’s another example of how the team at The New Stack poisoned their own vector store when an ingestion pipeline because of a badly scanned PDF. A normal parser would have thrown an error. The model did what models do when they are unsure: IT GUESSED. It wrote down a fiscal year of “2024” as if it were fact, and every query after that inherited the lie. The fix was not a smarter model, they pulled out the “validator agent” and replaced it with a plain schema check plus a rule that the value had to actually appear in the source text before anything got saved.
There is one pattern underneath the stories, and if you have built distributed systems you already know it: the split between a control plane and a data plane. The control plane decides what runs. It is fast, predictable, and boring on purpose, because it is the one place where a wrong decision spreads to everything downstream. The data plane does the actual work, where judgment and complexity live. Kubernetes is an example of this. A lot of agent products are repeating the mistake that infrastructure teams made twenty years ago, they put the judgment in the coordination layer.
A language model gives you judgment, creativity, and language understanding. Those are data plane strengths. It does not give you correctness, speed, or predictability, which is exactly what a control plane is for. Wire the model in as your control plane and you get a system that fails confidently, and in the exact spots you never tested.
So the interesting question was never “which model?” It was what part of this problem actually wants an agent, and what part wants plain code that fails before a damage is done? Get that boundary right and the model becomes a controller you can test and swap out.
How I used this in my weekend project “SlopScore”
SlopScore is a small web app where you type in your/anybody’s X handle, it scrapes your tweets and tells you how AI-generated your writing reads. It is a fun project with the same architecture we discussed above.
Here’s some of the major processes happening in the background:
1. Retrieval - Go get a handle’s recent posts using Linkup.
2. Judgment - Decide how AI-sounding the writing is.
3. Artifacts - Render a shareable card and write to a leaderboard.
4. Delivery - Send all of that back to the browser and make it feel instant.
The lazy instinct is to hand all four steps to one model: “here are some tweets, score them, explain why, and go update the board.” That is the fat orchestrator, the model as control plane. It gives you a product that is:
Unstable: The same handle scores 61 today and 68 tomorrow. The leaderboard turns into noise.
Impossible to defend: When someone disputes their score, you have no receipts, just the model’s opinion about the model’s opinion. This is the vector-store bug in miniature: a guess written down as fact.
Impossible to test: There is no unit test for “the model felt this tweet was slop.” You are back to “status: success” and hope.
Slow and expensive: Every score becomes a full generation, even the 90% that is plain string matching.
What “harness” means here
If you are active in the tech space, harness is not a new word to you. Let’s see how we are actually using it. In this project, Hermes Agent is a separate command-line agent. You install it, register your tools with it, and give it tasks in plain language. The app is wired as a Hermes harness. It hands the orchestration of a task to the agent, it exposes the things the agent is allowed to do as a small set of tools (using MCP), and it keeps everything else, the predictable and auditable parts, on the server.
When a browser asks to score a handle, the web server does not score it in-process. It runs the agent as a command:
hermes -z “call the score_slop tool for @handle, return the JSON”
Hermes connects over MCP to the app’s own tool server, sees the tools available, decides to call `score_slop`, and returns the result. The server then parses that result and takes over the rest.
The flow for a single score request is:
1. The browser sends the handle to the server.
2. The server runs Hermes. Hermes calls the `score_slop` tool.
3. That tool fetches the tweets and runs the scoring engine, then returns a plain report.
4. The server takes that report and does the predictable work: save it to the leaderboard, render the card image, and make the voice clip.
5. The server sends the report, the card, and the clip back to the browser.
The harness is the line between “the agent decides and orchestrates” and “the server executes and renders.”
The main decision: agent for thinking, server for artifacts
The most important choice in the codebase is that the pipeline is split into two phases.
Phase one is retrieval plus scoring. It fetches the posts and runs the engine, and returns a report. This is the part the agent drives through its tools. This is the “agent does the work” half.
Phase two is everything predictable: save the report and render the card. The server always runs this itself. The agent never touches the leaderboard and never renders an image. It orchestrates but does not create state.
The most important detail: the score itself is not the model’s opinion. The scoring engine is deliberately separate to prevent false negatives. That is what makes the roast defensible. “You used ‘delve’ four times, here are the tweets” is a fact you can point at, not a vibe. It is also what makes the whole thing testable.
This is the reframe worth keeping. The “AI” in this product is Hermes doing orchestration: deciding which tool to call, with what input, and shaping the interaction. The score is arithmetic over a set of rules with fixed weights. That split is why the leaderboard is stable and why every score comes with quotes. If you take one lesson from the harness, take this: an agent is a controller and not a calculator. Let it decide and route. Do not let it be the source of truth for anything you need to be stable, auditable, or testable.
MCP is the OG
The agent’s entire reach into the system is three tools:
`fetch_tweets`, which gets recent posts.
`score_slop`, which returns the full report.
`get_leaderboard`, which reads the shared board.
That is the whole contract. Everything the agent is allowed to do is listed, typed, and described in plain English so the agent knows when to use each one. Choosing this as the boundary pays off in a few ways:
First, the reach is explicit and minimal. The agent cannot do anything the tools do not expose. There is no “and also it can write to the database”, because the write path is simply not a tool.
Second, the same tools serve more than one caller. The tool server is reused by anything that speaks the protocol. The web app is just one client.
Third, the boundary is testable on its own. There is an integration test that starts the real tool server the same way Hermes connects to it, and checks that the tools are actually there and callable. That matters more than it sounds. It is very easy to claim your product “runs through an agent” while quietly calling a function in-process. The test keeps you honest.
The lesson for your own systems: treat the agent’s permissions like a public API you would be comfortable shipping. MCP nudges you toward that, because you have to name and type every capability before the agent can touch it.
Design for the agent’s bad days (being a pessimist is good at times)
Real agent integrations live or die on the moments the demo skips. Two choices here stand out.
First, constrain the output, then parse it defensively anyway. The instruction handed to Hermes is narrow: call one tool and return only the raw JSON. But the code assumes the agent will not fully comply. It scans the reply for the first complete JSON object that actually contains a score, so a stray sentence or a code fence around the answer does not break anything. Build the parser as if the agent will wrap its answer in an apology, because sometimes it will.
Second, make the boundary a config choice, not a code change. The harness has three modes, controlled by one setting:
- `cli` always uses the real agent, and fails loudly if it is not there.
- `direct` skips the agent and runs the same scoring in-process, which is handy for local development.
- `auto` uses the agent if it is installed and falls back to direct if it is not.
Notice that `cli` is strict. If the agent is required and missing, the request fails clearly instead of pretending. This is only possible because of the two-phase split. The in-process path runs the exact same code the tool calls, so “through the agent” and “in-process” are genuinely the same computation with a different driver. The agent is a swappable frontend to a predictable core.
Graceful fallback as a rule
The harness idea extends past the agent. Every external dependency has a fallback, and the whole app runs end to end with zero API keys. No tweet-search key means it uses a fixed mock timeline. No database means it writes to a local JSON file. No agent means it scores in-process.
This is not just a nice onboarding story (though it is one): clone it, run it, and it works. To make everything fall back cleanly, you are forced to define a clean interface at each boundary. The store is one interface with two backends, a hosted database and a local file. Retrieval is one function with a live source and a mock. Orchestration is one runner with an agent mode and a direct mode. Designing for the “no keys” case is what produces the seams that make the “all keys” case clean.
How this shapes architectural decisions
Wiring an agent in as a harness, instead of adding model calls through the code, changes the questions you ask when you design a feature.
What needs judgment, and what is just deterministic? Push everything you can into plain, testable code. Save the agent for orchestration and genuinely fuzzy decisions. In SlopScore that line sits right between deciding to score (agent) and the score itself (code). Let the model read and decide, but the moment something becomes ground truth, run it through a plain check.
What is the agent allowed to touch? Model its abilities as a small, typed set of tools. If something is not a tool, the agent cannot reach it. Anything that has to stay trustworthy, like saving data, stays outside that set.
What happens when the agent misbehaves or is missing? Design the fallback first. If your system cannot run without the agent, the agent is not a component. It is a single point of failure.
Can you prove the agent path is real? Test the boundary itself, the way the agent actually connects.
Is the output stable enough to build on? Anything downstream that has to be predictable, like leaderboards, disputed screenshots, or cached results, should be computed by code, with the agent orchestrating around it.
Answer these, and the agent becomes what it should be: a well-bounded controller behind a contract you can reason about, test, and swap. The industry has started calling this “harness engineering,” there’s a realization that the leverage in agent systems is in the plumbing around the model, not the next model release.
Closing
The word “harness” is deep. A harness does not replace the horse and it does not replace the cart. It is the set of connections that lets one pull the other in a controlled direction. It lets an agent orchestrate a real task through a small, explicit set of tools, while the predictable core it drives stays auditable, testable, and boring in all the ways production systems should be boring.
The trap with agents is treating them as a place to dump thinking you have not done yet. The harness pattern is the opposite move, do the thinking, draw the boundaries, decide what has to stay predictable, and then let the agent orchestrate inside those lines.
That’s it for this one, I will keep writing more such technical blogs, if you have any thoughts, feel free to comment below or DM me on X: @HaimantikaM