Lily

Lily

Posted on Jul 26 • Originally published at dev.to

Your Claude Code setup doesn't break in one dramatic moment — it degrades a few bytes at a time, and by the time you notice, you've been paying a context tax for weeks. In a previous post I covered running an unattended daily health check with launchd. This one is the follow-up: a three-layer loop that detects that quiet degradation weekly and hands the repair job to claude -p itself.

The problem: environments rot quietly if you leave them alone

Some things in a Claude Code environment grow just from doing your normal work.

  • ~/.claude/rules/ and MEMORY.md keep getting appended to, until context injection quietly crosses 40KB
  • Experimental agent definition .md files never get archived, leaving dozens to nearly a hundred files under ~/.claude/agents/ permanently loaded
  • Stop hooks fire over and over, creating a hook spam condition
  • Frustration-signaling words pile up in conversation logs and nobody notices

A performance audit on 2026-07-11 revealed that "agents I thought I'd archived were still being injected — 99 of them," and that turned out to be the main cause of the degraded experience. That led to the question "so do I have to go check this every week myself?" — and the answer was to automate it, which is what cc-self-audit.sh does.

Five degradation metrics and their thresholds

The script measures five metrics and flags "red" when any of them crosses its threshold.

# 閾値(env変数で上書き可)
TH_INJECT_BYTES="${SELF_AUDIT_TH_INJECT:-40000}"   # rules+CLAUDE.md+MEMORY.md 合計バイト
TH_AGENTS="${SELF_AUDIT_TH_AGENTS:-60}"            # ~/.claude/agents 配下 .md 総数(再帰)
TH_STOPSPAM="${SELF_AUDIT_TH_STOPSPAM:-15}"        # 監査hook発火/週
TH_FRUSTRATION="${SELF_AUDIT_TH_FRUST:-8}"         # 不満ワード/週
TH_TOOLERR="${SELF_AUDIT_TH_TOOLERR:-400}"         # tool失敗/週

Enter fullscreen mode Exit fullscreen mode

The first three are static metrics (state at this exact moment); the last two are dynamic metrics (trends since the previous run). That distinction maps directly onto how each one is measured, as described below.

Overall design: a three-layer loop

[層1] 静的計測  → 注入bytes / agents数
[層2] 動的計測  → hookスパム / 不満ワード / tool失敗(前回実行以降の窓)
[層3] 閾値超過  → claude -p が ~/.claude 内を自己修正
               → 独立再計測(自己申告は信じない)
               → Discord #01_alerts へ報告

Enter fullscreen mode Exit fullscreen mode

If everything is green, it sends a single ✅ line and exits. Since it only runs once a week, it stays low-noise while also doubling as a liveness check.

Layer 1: static measurement

inject_bytes=$(( \
  $(find "$HOME/.claude/rules" -name '*.md' -print0 2>/dev/null | xargs -0 cat 2>/dev/null | wc -c) + \
  $(cat "$HOME/.claude/CLAUDE.md" 2>/dev/null | wc -c) + \
  $(cat "$HOME/CLAUDE.md" 2>/dev/null | wc -c) + \
  $(cat "$HOME/.claude/projects/-Users-matsubara/memory/MEMORY.md" 2>/dev/null | wc -c) ))
agents_loaded=$(find "$HOME/.claude/agents" -name '*.md' 2>/dev/null | wc -l | tr -d ' ')

Enter fullscreen mode Exit fullscreen mode

Making find … -name '*.md' recursive is deliberate: it catches agent files that have snuck into dot-directories (.tmp/ and friends) so the problem can be detected if it recurs. If you only look at a single flat level, you'll never notice files that failed to get archived out of a subdirectory starting with ..

Layer 2: dynamic measurement (the window since the last run)

By only targeting .jsonl files newer than the marker for the previous run time (self-audit/lastrun.marker), it looks at only that week's delta.

[ -f "$LASTRUN" ] && newer="-newer $LASTRUN"
files=$(find "$HOME/.claude/projects" -maxdepth 2 -name '*.jsonl' $newer -size +100k 2>/dev/null | head -200)

Enter fullscreen mode Exit fullscreen mode

However, a naive grep produced a lot of false positives (detailed in the pitfalls section below). In the end I improved precision with inline Python.

STOP_MARKER = 'Stop hook feedback:\\n[~/.claude/hooks/self_audit_stop.sh]: '
FRUST_RE = re.compile(r'何回も言|いい加減にし|嘘つ|舐めんな|なんで治らん|最悪やろ|頭悪い')

# stopspam: harnessが実際に注入する行のみ
if STOP_MARKER in line:
    stopspam += 1

# frustration: isMeta でない type=user のテキストブロック(人間の実発言)のみ
if o.get('type') != 'user' or o.get('isMeta'):
    continue
content = (o.get('message') or {}).get('content')
# ... テキスト抽出して FRUST_RE で判定

Enter fullscreen mode Exit fullscreen mode

stopspam counts only the Stop hook lines injected by the harness. frustration targets only human utterance text within the conversation (type=user and non-meta). That eliminated false positives from cases like "I just read the hook's own source code with the Read tool."

Layer 3: self-repair via claude -p

If even one metric crosses its threshold, a repair prompt is piped into claude -p.

OUT=$(cd "$HOME/.claude" && printf '%s' "$PROMPT" | run_capped "$FIX_TIMEOUT" "$CLAUDE" -p \
      --model "$MODEL" --output-format text \
      --allowedTools "Read,Write,Edit,Bash,Grep,Glob" \
      --max-turns 50 2>&1) || true

Enter fullscreen mode Exit fullscreen mode

--allowedTools narrows it to reading and writing inside ~/.claude only — no touching project code or plists. The prompt is handed the violations, all metrics, and the last five history entries (the trend).

Known degradation patterns and their remedies are spelled out too (excerpt):

- agents_loaded超過 → 退避漏れを ~/.claude/agents-archive/ へ移動
- inject_bytes超過  → 肥大したrules/MEMORY.mdを圧縮し、フル版は rules-archive/ へ
- stopspam超過      → ~/.claude/hooks/self_audit_stop.sh の抑制ロジックを点検
- frustration超過   → 該当transcriptをgrepして繰り返し失敗の真因を特定

Enter fullscreen mode Exit fullscreen mode

Each change gets appended one at a time to logs/self-audit-changes.log with "timestamp / target / reason / how to revert". Requiring the revert instructions is what lets a human roll back when the automatic repair makes a bad call.

"Never trust self-reporting" — independent re-measurement

When claude -p says "fixed it," that's self-reporting. Rather than using that directly in the report, it re-measures independently with the same collect() function.

# 独立再計測(自己申告は信じない)
AFTER=$(collect)
log "after: $AFTER"
echo "$AFTER" >> "$HISTORY"
STILL=$(echo "$AFTER" | breaches)

if [ -z "$STILL" ]; then
  notify "🔧 CC自己監査: 劣化検知→自己修正済み。前:[$BREACH] 後:全緑。詳細=$CHANGELOG"
else
  notify "🚨 CC自己監査: 自己修正後も残存 [$STILL]。要確認: $LOG / $CHANGELOG
Claude要約: $(echo "$OUT" | tail -3 | tr '\n' ' ')"
fi

Enter fullscreen mode Exit fullscreen mode

If a threshold is still exceeded after the repair, it's marked as "remaining" and thrown back to the human. Since the notification includes the CHANGELOG path, you can immediately trace what was touched.

launchd config (Sundays at 8:30)

<key>StartCalendarInterval</key>
<dict>
    <key>Hour</key>      <integer>8</integer>
    <key>Minute</key>    <integer>30</integer>
    <key>Weekday</key>   <integer>0</integer>  <!-- 0 = 日曜 -->
</dict>
<key>LowPriorityIO</key>  <true/>
<key>Nice</key>           <integer>10</integer>
<key>ProcessType</key>    <string>Background</string>

Enter fullscreen mode Exit fullscreen mode

Every Sunday at 8:30. LowPriorityIO and Nice 10 keep it from getting in the way of other work. Since RunAtLoad: false, it doesn't fire immediately after launchctl load — it waits for the next calendar time.

Note
Putting the weekly run on Sunday is deliberate: I want the environment in good shape before Monday's work starts, and running it twice a week or more would turn the notifications into noise. Because it doubles as a liveness check, the whole premise is that "a ✅ arrives every week even when nothing is wrong."

Pitfalls I hit

  • grep inflated stopspam to 40 → the hook's own source was included in the old_string of an Edit tool call, so it was counting the body of Read/Write/Edit tool_results too. Fixed by narrowing the Stop hook line to an "exact match on the format the harness injects" and doing line-level checks in Python
  • frustration looked like 18 but was actually 4 → it was falsely matching Read output in tool_results (a file that happened to contain the target words got opened). Fixed by narrowing to human utterances only: type=user and non-meta
  • Recursive agents_loaded picks up unintended files → if notes that call themselves .md sneak in, the count balloons. The threshold of 60 is a value with headroom above the actual agent count (and can be overridden)
  • Real verification with SELF_AUDIT_DRY=1 → before letting the claude -p repair path fire for the first time in production, I confirmed only collect/breach with DRY=1. The repair path is structurally identical to self-repair.sh, so structural risk is low
  • launchd's minimal PATH can't find claude → the plist explicitly sets PATH=/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:~/.local/bin. Managed separately from scripts that additionally need a fallback to nvm-managed node

Summary

  • Left alone, a Claude Code environment quietly degrades through growing injection bytes, agent bloat, and hook spam
  • Measure five metrics (inject_bytes / agents_loaded / stopspam / frustration / toolerr) weekly and detect threshold breaches mechanically
  • On a breach, claude -p repairs only what's inside ~/.claude, and always records "how to revert" in the change log
  • After the repair, verify the numbers actually improved with independent re-measurement — don't trust an AI's self-report
  • Even when everything is green, send a weekly one-line ✅ so it doubles as a liveness check

Next time I'll write about the transcript from an actual run where "agent bloat → automatic archiving" fired in this audit loop, plus turning history.jsonl into a trend graph.


Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*