The Return on Investment of Harness Engineering: How Investing in Configuration Files Reduced Workload
About This Article
To be honest, the time I spent building CLAUDE.md and rule files felt like wasted effort at first.
Creating files with no direct connection to the site’s code or articles seemed less productive than writing actual content. Over three months, that feeling changed. This article describes concretely what was different before and after building the harness.
What Was Happening Before the Harness
AI Made the Same Mistakes Repeatedly
With each similar task I assigned, the AI would make the same kind of error. For example, when creating articles in Japanese and English simultaneously, I had to include the same explanation in every instruction to enforce the rule that “the English first person is I only — do not use we.”
Even after explaining it once, the same mistake would appear again the next time I requested an article.
I Repeated the Same Explanations
Every time I asked the AI to make code changes, I had to re-explain the same context: “This site is built with Astro + Starlight,” “Japanese is the base language and URL paths follow this format,” “Do not change the existing navigation.”
Sometimes the prerequisite explanation became as long as the actual instruction itself.
Reviewing AI Changes Took a Long Time
Because I was not always conveying the full context before requesting a change, the AI would sometimes make unintended modifications. For example, an instruction to change one file would result in a related file also being modified. Reviewing these changes sometimes took longer than making the change itself.
What Changed After Building the Harness
The Same Mistakes Decreased
Once I wrote rules like “English first person is I only,” “do not modify existing UI,” and “slug format is blog/name” into CLAUDE.md and the rule files, I no longer needed to repeat these explanations. The AI began referencing these files at the start of each session and acting according to the rules.
More Tasks Could Proceed Without Explanation
With the prerequisites written in the harness, new article requests or change requests no longer needed to include background explanations for the AI to respond correctly. Instructions became shorter, and intent was easier to convey.
Review Effort Decreased
After I wrote rules into the harness limiting the scope of AI changes to “this file only” or “this component only,” unintended modifications decreased. The number of places to check after a change narrowed, and time spent on review shortened.
A Sense of the Investment and Return
The time I spent building the harness was concentrated in the first one to two weeks.
Creating an initial version of CLAUDE.md took a few hours. Organizing the individual rule files took roughly a day in total. After that, time continues to be spent adding and revising rules as new issues surface.
On the other hand, the time saved by no longer needing to re-explain context with each request was enough to recover the setup cost within one to two weeks. I did not measure exact times, so this is a description of the trend rather than a precise figure, but the return on the investment was real.
Which Rules Had the Greatest Effect
Not all rules produced the same effect. The rules that made the greatest difference shared these characteristics.
- They addressed a problem that had been repeating
- The rule content was specific enough for the AI to apply without ambiguity
- Whether the rule was being followed was easy to check
Conversely, rules that stated abstract principles, or rules with vague conditions, had limited effect. When writing a rule, asking “what does a concrete example of this rule being broken look like?” helps produce rules that actually work.
Summary
Investment in the harness is work whose direct results are hard to see at first. But when the same types of tasks repeat, the setup cost can be recovered relatively quickly. A useful criterion for deciding whether to write a rule is: “Will this problem come up again?” If a problem is a one-time occurrence, handling it in the moment is more efficient. If it will repeat, writing it as a rule is worth the effort.