How I chose my agent system in one day

The question.

I run about a dozen long-lived agents on one Mac. Claude Code and Codex do most of the work, Hermes sits at the front desk, and Herdr keeps all of them in panes I can reach from my phone. Each agent has a role, and each role gets restarted often.

On 2026-10-11 I had a plan on hold to give those agents profiles, memory and persistence. I paused it and asked a different question. Should I adopt Pi Durable, a new project built for long-running agents that survive crashes? And behind that one sat three more. Does Pi work inside Herdr, can a role’s rules be injected into the system prompt on every start, and do my Claude and Codex subscriptions keep working if I change the harness?

I wrote the day down as a research, decide and codify flow, the same one Commune uses for any decision. Research first, with every claim labelled by how it was learned. Then a decision that names the options and what each costs. Then, only after I say yes, the files that change. This post follows that order.

Round one.

The first round was plain reading of primary sources. Two findings changed the picture.

Pi Durable is a library, not a runtime. It is an experimental TypeScript package for building your own agent host, and the project’s own README says the API changes without notice. The Pi coding agent does not run on it. So adopting it would mean rebuilding my roles, not swapping a tool.

Claude through Pi is billed as extra usage. Pi’s own documentation says third-party harness use draws from extra usage and is billed per token. Hermes says the same about its sign-in route. My rule is subscriptions only, so that route is out for Claude.

Round two and the Grok lesson.

I pasted in a note that said the ChatGPT side had changed, and that signing in with a ChatGPT plan now worked with Pi. It read well and it was specific. It also came from Grok.

I pointed out the problem in a line I added to the research folder. “The note above was written by Grok. Treat every line as an unverified model claim.”

A lane then took the note line by line against official pages fetched that day. There were 22 claims. Nine were verified with a quote from a primary source. Two were split, where the narrow claim held and the wider one could not be found. Eleven were not found, and none were contradicted. Some of the OpenAI pages returned an error to the fetcher, so a search summary agreed with a few rows, and the lane did not count a summary as evidence.

Zero contradicted sounds like a good score. I read it differently. Eleven of 22 claims could not be traced to any page, and the note gave no signal about which eleven. The lesson I took is a procedure. Anything a model hands me goes into the research folder marked as model output, and it moves a decision only after a row with a source and a quote says so.

I then ran a research prompt in Grok to sweep X for what people were saying about Pi, Hermes and the durable-agent idea. Its answer was a table of 14 claims, each with a link and its own verdict. I stored it the same way. The links and quotes are unverified until someone fetches the page, and the table is kept apart from the primary sources.

The manual and the source.

Then I shared the 190-page Pi Durable Technical Manual. It is a community document and not published by the Pi team, so we read it against the package source and the project’s announcement. It describes an agent harness in which every model call, tool call and compaction is checkpointed. Timers survive restarts. The system prompt is rebuilt from the project’s extensions before every request.

That last point mattered, because injecting a role’s rules on every start was one of my questions. Pi Durable does it natively. But my roles run in Claude Code and Codex, and Pi Durable cannot make those sessions durable. To use it I would have to rebuild each role on it and give up Claude Code and its Max subscription. It also has no recurring scheduler, only one-off timers, and its changelog already queued six breaking changes after the version the manual covers.

So the answer to my first question was “not now”, with a trigger to revisit it. If it leaves “Experimental” and the amount of API credit included in Max turns out to be substantial, I will look again.

The restart that forgot work.

Then I told the agents what had just happened. “Cake restarted and it forgot a whole bunch of ongoing work.”

That sentence changed the question. I had been asking which runtime would hold my agents’ memory. The thing that actually hurt was an orchestrator that came back from a restart without the open items it had been carrying. That is a different problem, and a new runtime might not touch it.

A choice that affects all my work deserves more than one agent’s opinion, so I ran it as a bigger study.

The UltraCode session.

I used an UltraCode session. It ran 30 read-only agents in about 45 minutes and used 4.8M subagent tokens. Nothing in it could write to my systems. It was built to do four things.

First, measure the forgetting. Second, research eight candidate systems, each with a second agent whose only job was to argue against the first. Third, design four answers from four different starting stances. Fourth, have three judges score the designs, then run a red team and a completeness critic over the result before a final draft.

Two separate analyses looked at frozen checkpoints and transcripts. They count differently.

• The first looked at one business orchestrator over 7 restarts. It found about 23 items it owned that were dropped or untouched. Thirteen fell out of a later checkpoint, one never appeared in any checkpoint, and nine were listed but never touched in the next session.
• The second looked at all orchestrators, and at 13 restarts for that same one. It counted about 650 handoff lines, about 140 untouched and 15 genuinely forgotten. It scored a short restart as too short to judge, so its number is lower.

Neither settles my own estimate that about half was getting lost. It depends on the denominator. Per handoff line the rate is low. Per open piece of work it is much higher. In that second study, 4 of 17 open pull requests in my lane were in none of the recent handoffs, plus one in another repo.

The worst case was a handoff that listed three pull requests as ready to enqueue. The next session never rechecked them for three hours and fifty minutes. When it finally looked, one was enqueued within a minute.

The analysis found three causes. The handoff is rewritten by hand at each restart, and items fall out. Items that are waiting have no alarm and no owner. And in one case a full list did not make the agent act, because it listed 60 open pull requests and acted on none of the dropped ones. Cron jobs being re-armed was not a cause. They were re-armed in 7 of 7 and 13 of 13 restarts.

The four designs ended in a tight pack.

I do not read much into the order. Nine points across three judges is probably noise, the judge passes were not saved to the repo, and all four designs worked from the same dossier. What I do read into it is that all four rejected a new runtime and arrived at the same core. That agreement is weak evidence, since they shared their inputs, and no judge found a fatal flaw in any of them.

The result.

Accepted on 2026-10-11. I said yes to all four lines.

The decision is no new runtime. Opus 5.5 stays in the unmodified Claude Code CLI inside Herdr. Codex lanes stay as they are, and Hermes stays as the front desk and scheduler.

The one new piece is a small command called open-work. It rebuilds each orchestrator’s list of open items from GitHub, Linear and my open asks. Every item must carry one of four dispositions. Own it with a next action. Park it with a wake time or a wake event. Close it with a reason. Or mark it as not mine, with a reason. A parked item cannot be written without a wake, and a wake in the past is refused, which removes the bad one-shot timers from the measured causes.

The list is injected when a session starts and when it compacts. It is re-checked on a cron that already existed. A limited stop hook blocks the end of a turn when an overdue item has sat untouched across two idle checks. The limit is that it blocks once per item per hour, so it cannot loop. That hook is aimed at the case where the agent saw the item and did nothing.

There is an honest limit. The model still writes the dispositions. The lists are derived from records, and the stop hook makes an item hard to lose silently, but a wrong “close” can still hide one. The pilot is meant to measure that.

The pilot is 7 days on one orchestrator, with numbers to hit. No live owned item without a disposition at restart. At most one item untouched for a full session without a dated wake. At most one real miss on the second study’s method, against 6 in 13 before. Stop rules end it early if the hooks block a start, if I call it noise twice, or if it needs more than three fix rounds in two weeks, which would mean it is turning into the machinery it replaces. The review date is 2026-10-24.

Profiles, memory and the skill keeper stay paused. They come later as reviewed changes. The measure I will judge them by is how often I repeat myself, which is currently about 20 to 24 times per 100 messages to one business orchestrator.

The rest of the options were not adopted now. OpenClaw, DeepSeek Harness and Goose are out for different reasons, each written in the decision. Letta Code has the best learning loop and I am watching it. I also asked about something I called “Grokbot” in a dictated message. My answer was that it is “a gated closed system with patterns to learn from”. We take its patterns into the open-work and learning phases and do not adopt the system itself.

Whether this is right will show in the pilot. If it fails its stop rules, this post gets revised.

How I run it from my phone.

The last piece is how this gets done. I added this to the brief after the decision was drafted, in these words. “Also forgot to mention, texting and iMessage has been very awesome.”

The agents do the steps and ask me only for credentials and approvals. When one needs a decision, it texts me a question with a short code at the start and the end, and I answer yes or no. The reply routes back to the agent that asked, with nothing for me to relay. Longer material goes in a write-up I can open on the phone. Herdr’s iPhone view shows the panes, so I can look at an agent without opening a laptop.

The whole decision in this post reached me that way. The decision file opens with a section headed “For your phone”, ten numbered lines and then four yes or no questions in a single text. The day began with a question about one library and ended with a decision I could answer from my phone.

What I would copy.

Four habits are worth taking from this, whatever you run.

1. Label every claim by how you know it. A source read, a number measured, a model’s output, an inference. The decision file marks each one.
2. Treat a model’s note as a lead. Half of mine could not be traced to any page, and it read just as confidently as the half that could.
3. Measure the problem before you pick the tool. The forgetting was real and had causes, and none of them were the thing I first asked about.
4. Say what the data cannot show. This one has no baseline for sessions that never restart, so causation is unproven.

I will update this post after the review on 2026-10-24.