The Harness Is the Thing

Reading this really makes me wish that I had a slightly better workflow. I'm really soley dependent of Fable to the point that I don't use other models, and I've already sort of hit a point where I'm running into usage limits every week. I am really living on borrowed time — when Anthropic finally collapses their 50% usage increase at the end of August, I'll definitely be forced to switch my workflow. When that happens, I have a hard time imagining that I'll be sticking with a single model on a single provider.

Author states, “Single developer projects can build to the caliber and consistency of large development teams.”

When I, as a single person, can produce a project in one month that would have taken a team of four people three months to produce, why would I care about token cost? I’m now spending $500/month instead of $40,000 month to get the same thing 3x faster. $500 for a project instead of $120,000. (Assumes my cost, $40k is the other three people)

It’s a no-brainer —- use frontier all the time.

What I've learned in last week is that a harness is basically a while loop.

In each iteration you make an LLM call, perform some work (e.g. tool call), augment the prompt (append or compact etc.)- not necessarily in that other- and continue.

Until an end condition is satisfied. Then you break out.

I've been very happy with Luna but my approach is "many bite sized edits" for which models basically hit saturation a year ago.

(I also tried the "let a massive model make massive changes" approach and am still psychologically recovering from the experience. The codebase may never recover!)

Also, Luna and DSV4 Flash seem to be on par now except Luna is faster and cheaper?

LoRA adaptive learning using open-weight models and your own reasoning traces is the thing. The big labs have a mammoth job ahead of them if they want to compete with running your own model - they will basically have to give every single user their own persistent virtual machine. When it's all said and done, I think their only really moat will be as inference/hardware providers. Stripe buying OpenRouter was a very smart bet.

We are talking about the process but not the result. I think the results will speak for themselves.

This guy knows harnesses!! I'm curious to know what your thoughts are on dsh(deepseek harness), where everything is a plugin. I read an article where the author described it as a motherboar in the Mobo combo purchases we use to do to save money, with the processor(deepseek), being replaceable by other llms but shipped with it by default. Did you evaluate it as a possible replacement? If not, why not?? In my nondev brain, it looks revolutionary, the equivalent of the x86, to the SoTAs attempt at keeping us in a walled garden similar to apple consumers.

I use Claude Code, Codex and opencode pretty much interchangeably. I am currently using Claude more this month because (stupidly) I paid for Max ($100) since I have a large client project.

I generally use larger models to plan. All my generated Epics have similar structure. All my repos have similar structure (https://github.com/brainless/akar and https://github.com/brainless/daftprompt are recent examples).

I barely spend time or thought in making prompts. I have a simple text file with a few combinations. They refer all the common files (README, AGENTS, DEVELOP, etc.)

All reference software is cloned locally and the docs mention that. The prompt templates then boil down to research mode (write Epic) or worker mode (write software) or review mode (leave review notes in Epic). That's it.

Many of my harness experiments are about text manipulation, text search, graph on text. Because that is what LLMs are - text processing systems. Cut parts of prompts, cut parts of response, cut parts of user's intent. Join, break into epics/tasks, run with LLMs, repeat.

i doubt it. someday harnesses get complex enough that you start asking models to interpret and modify them.

the pattern of layering of deterministic, probabilistic, deterministic, probabilistic. it's a strange pattern but it seems somehow natural.

More agentic than this. The harness is the thing / Wherein I'll grant Claude access to Bing.

Mine is more granular. I start with a scoping brief before planning. But yeah, I'm all-in in this same thesis.

am I the only one here who is perfectly fine with deepseek and qwen? generally I do all the planning in my head and a few sentences describing what I want is enough for deepseek to save me hours of work. Hell I get away with flash for most of it and use pro for some reviews.

People are doing too much.

Everyone now, more or less, has a harness, and the beauty is that you can afford to have your own version.

Heh. Interesting article.

> Single developer projects can build to the caliber and consistency of large development teams.

This has always been true. Good developers, like truly good devs, could run rings around a team of mediocre devs. It’s a multiplier, a team of 10 1x devs will get dominated by a single 10x dev no matter how much AI they use. Nothing has changed here, if anything it benefits the good developers.

> At the moment my rig is supported by two subscriptions (Cursor, Claude) that I can augment with Pi as needed.

This conflicts subscriptions with an actual harness, doesn’t bode well for the rest of the article…

> Recently I learned about prewalk, Can Bölük's technique that uses frontier for the planning phase and first task, then hands off once the pattern is set.

This pattern has been known for years and is not attributable to a single person.

> Exploration leads to a plan formalized into an explicit DAG (directed acyclic graph) task list. Then a worker takes over, focusing on implementing the DAG one node at a time. Once complete, I bring in the critic to simplify and question what was implemented. Often this phase will push back enough that the worker phase is revisited. But once satisfied, the critic gives way to a promoter, which is my reminder that a job is not complete until you've properly communicated it to others.

I’ve tried all these complicated workflows. In the end the best way to use LLMs is to give it some instructions, take a look at the code, and then ask it for changes. At the end, ask it (in a fresh session) to review the changes for bugs or incorrect assumptions and architecture. Rinse and repeat. Anything more complex is over engineering.

None of the rest of the article seems particularly interesting. Just more busywork.

> HUMANIST SOFTWARE DEVELOPER

Uh huh?