What Is Harness Drift: Detecting and Preventing Gaps Between AI Configuration and Code
When configuration and code stop matching
As I built this site, a situation kept recurring: “what is written in the configuration files no longer matches what the code actually does.”
I call this “harness drift.” Drift refers to the state where configuration files and code gradually diverge from each other over time.
What drift looks like in practice
Here are examples of drift that actually occurred on this site.
A rule that no longer matched the code
CLAUDE.md stated “do not modify the src/components/ directory directly.” During a refactoring of one component, I forgot to update that rule, and for a period of time the rule’s text and the actual code were out of alignment.
A skill file that was behind the implementation
The blog writing skill file specified “add field XXX to the frontmatter.” When the site’s code changed and that field became unnecessary, the skill file retained the outdated instruction, so AI continued adding the field.
A path reference left behind after a move
When a file was moved to a different directory, I forgot to update the reference path in CLAUDE.md, and AI continued using the old path.
Why drift happens
The main reason drift occurs is that when code changes, updating the configuration files gets forgotten.
Code changes happen because they have a clear purpose. Configuration file updates are secondary work, so they tend to be deferred or overlooked.
AI-assisted development introduces an additional factor: AI sometimes modifies code on its own. When AI makes a code change, it does not always update the related configuration files at the same time. This is a contributing cause of drift.
Prevention: making updates and changes a pair
The first prevention measure I introduced was adding an explicit rule to CLAUDE.md: “when changing file structure or directories, update the relevant sections of CLAUDE.md at the same time.”
With this instruction present, AI takes configuration file updates into account when performing changes. That said, this rule alone is not a complete solution. Rules being in place does not guarantee they are always followed — that is a realistic constraint when working with AI.
Prevention: automated detection (harness:check)
Alongside rule-based prevention, I use npm run harness:check to detect drift automatically. I cover the check design and implementation process in Building a Validation Script with AI; this article stays focused on why drift occurs and why early detection matters.
Complete prevention is not realistic
At this point, I do not believe drift can be completely prevented. As long as code continues to change, some risk of divergence from configuration files remains.
The realistic approach is to shorten the time between when drift occurs and when it is detected. Building a habit of running the check script after each change makes it possible to catch drift before it compounds.
Summary
Harness drift is the phenomenon where configuration files and actual code diverge over time. The primary cause is forgetting to update configuration files when code changes. Combining a rule that pairs code changes with configuration updates and an automated detection script reduces the chance that drift accumulates unnoticed over a long period.