Balancing Token Savings and Code Quality: A Real Example of Quality Declining from Over-Trimming
What This Article Covers
A token is the unit by which AI measures the volume of information it processes. Both input — instructions and configuration — and output — generated code or text — are counted as tokens. Because the number of tokens used affects cost, trimming input context to reduce volume can be a practical cost management measure.
This article records a case where removing too much input context caused output quality to decline, and the criteria I use now to decide what to keep and what to remove.
What Caused Problems When Removed
Part of the Rule Definitions
While organizing the content of the project configuration file (CLAUDE.md), I removed some rules I judged as infrequently applied.
Some time later, problems related to those removed rules reappeared. Specifically, expressions that the rules had defined as off-limits started appearing again in AI output.
The right criterion for keeping a rule is not how often it is applied, but how significant the problem is when it is absent. A rule that was put in place because a problem occurred once is doing its job as a preventive measure. Removing it removes that prevention.
References to Past Lessons Logs
I had a log file summarizing past problems and their resolutions. I excluded it from AI’s reference context on the grounds that it contained old information.
Afterward, a similar problem occurred again. Without access to the lessons log, AI had no basis for the reasoning behind past decisions and repeated the same pattern.
A lessons log carries its meaning as a pairing of context — why a problem happened — and resolution. Removing it based solely on the age of the information, without checking whether the judgment it records is still valid, meant losing context that was still relevant.
What Could Be Removed Without Issue
For comparison, here is what I found could be removed without affecting output quality:
- Abstract style descriptions where concrete examples were still present
- Duplicate passages that restated the same content in different words
- Descriptions of past project structure that no longer matched the current configuration
Removing these produced no change in AI output and reduced token count.
The Criteria I Use Now
When deciding what to keep and what to remove, I apply the following framework:
Keep:
- Anything that addresses a problem that occurred before, where the rule prevents recurrence
- Anything where AI would likely make an incorrect decision without the project-specific context
- Lessons logs where a past problem and its resolution are recorded together
Candidates for removal:
- Duplicate content where the same meaning appears in multiple places
- Descriptions of configuration or specifications that no longer match the current state
- Explanatory text confirmed to have no effect on AI output
Summary
Reducing tokens is a valid cost management approach, but removing the wrong content affects output quality. Removing rules based on low frequency or removing logs based on age can eliminate the context that was preventing a problem from recurring. Before removing anything, asking “how would AI behave without this?” is a practical check for maintaining quality while still reducing token use.