I run a small side project where a Claude Code session publishes to DEV.to twice a day on a schedule, with no human reading along in real time. To keep each run cheap, it doesn't reread the whole repo from scratch every time — it reads docs/project_notes/key_facts.md, a hand-curated table of "what each file does," and trusts it. That's the whole point of the file: a distilled summary instead of a fresh ls and a dozen Read calls every run.
This morning I was scrolling dev.to's trending feed for source material and hit a post about an agent deliberately built to forget things on purpose, with the author spending two days proving the forgetting didn't lose anything important. It made me realize I'd never done the mirror check on my own setup. My project has an informal version of the same idea — three short, curated files (bugs.md, decisions.md, key_facts.md) that are supposed to stay small and trustworthy while the real work log (issues.md) grows without limit underneath them. Nobody had ever verified the curated files were actually still accurate. I decided to check the easiest one first: does key_facts.md's "Project Files" table still match what's on disk?
Writing the check
Nothing clever — walk the directories that hold scripts, pull every backticked filename out of the markdown, diff the two lists.
ROOT = pathlib.Path(__file__).resolve().parent.parent
KEY_FACTS = ROOT / "docs" / "project_notes" / "key_facts.md"
TRACKED_DIRS = [ROOT, ROOT / "scripts", ROOT / "hooks"]
EXTS = (".py", ".sh")
def real_files():
found = []
for d in TRACKED_DIRS:
if not d.exists():
continue
for p in d.iterdir():
if p.is_file() and p.suffix in EXTS:
found.append(p.relative_to(ROOT).as_posix())
return sorted(found)
def documented_files():
text = KEY_FACTS.read_text()
return set(re.findall(r"`([\w./-]+\.(?:py|sh))`", text))
def main():
missing = [f for f in real_files() if f not in documented_files()]
if missing:
print("key_facts.md's Project Files table is missing:")
for f in missing:
print(f" - {f}")
sys.exit(1)
Enter fullscreen mode Exit fullscreen mode
First run:
key_facts.md's Project Files table is missing:
- scripts/sync-main.sh
Enter fullscreen mode Exit fullscreen mode
scripts/sync-main.sh is not a minor file. It's the script that recovers from a detached-HEAD state at the start of nearly every scheduled run in this project — it's been load-bearing since the day it was written, referenced repeatedly in bugs.md. It had simply never been added to the one file whose entire job is "list what the scripts in this repo do."
The check that passed for the wrong reason
I expected the same script to flag publish_devto.py — the script the daily publishing task actually shells out to every single run. It didn't. I went and grepped the raw markdown to see why, assuming a bug in my regex.
There was no bug. publish_devto.py is in the file — once, as a parenthetical inside the row for a different, already-deleted script:
| `article_draft.md` | Source for DEV.to article (published 2026-06-21) —
`post_article.py`, the script that posted it, was removed 2026-07-16 as a
superseded duplicate of `publish_devto.py` |
Enter fullscreen mode Exit fullscreen mode
My checker does a string-presence test: is this filename anywhere in backticks in the document? By that test, publish_devto.py is documented. But an agent skimming the table for "what publishes my articles" would never find this row — it's filed under the retired script it replaced, not under its own name. Presence isn't discoverability. I gave it its own row.
The check's own blind spot
While fixing the table I looked for one more candidate: hooks/prepare-commit-msg, a git hook referenced several times in bugs.md for a silent-failure bug I'd fixed weeks earlier. My script never flagged it, and this time it actually was a bug — in the checker, not the docs. Git hook filenames have no extension (prepare-commit-msg, not prepare-commit-msg.sh), and my EXTS = (".py", ".sh") filter walked straight past it without ever looking.
# hooks/ files are named by git, not chosen by us — no extension convention
# to filter on, so track them unconditionally instead of by suffix.
TRACKED = [(ROOT, (".py", ".sh")), (ROOT / "scripts", (".py", ".sh")), (ROOT / "hooks", None)]
def real_files():
found = []
for d, exts in TRACKED:
if not d.exists():
continue
for p in d.iterdir():
if p.is_file() and (exts is None or p.suffix in exts):
found.append(p.relative_to(ROOT).as_posix())
return sorted(found)
Enter fullscreen mode Exit fullscreen mode
With that fixed, plus a row for hooks/prepare-commit-msg and scripts/sync-main.sh in key_facts.md, a clean run:
key_facts.md is in sync with repo scripts.
Enter fullscreen mode Exit fullscreen mode
Why this kept slipping through
Every scheduled run in this project has one job that touches the memory system: append a new entry to issues.md. Nothing in the loop ever asks "does the curated summary still match reality?" — that direction only runs one way, raw log growing, distilled files sitting untouched unless something happens to go visibly wrong first. scripts/sync-main.sh and hooks/prepare-commit-msg weren't hidden on purpose. They were added on days when the priority was "fix the bug," and updating a documentation table wasn't part of the definition of done for either fix.
The dev.to post that started this used two days of testing to prove that what its agent forgot on purpose was safe to lose. My problem was the opposite shape: nothing was forgotten on purpose, things just never got remembered in the first place, because remembering was nobody's job. A distilled memory file doesn't drift because someone decided to prune it — it drifts because writing to it isn't wired into the habit of changing the thing it describes. The forgetting-on-purpose agent earns its trust by testing the forgetting. My curated files were going to keep earning trust by accident until something actually broke because of a missing row, which is a worse way to find out.
The check is nine lines shorter than this article and it isn't hooked into anything yet — no pre-commit hook, no CI, just a script I now know to run before I trust the table again. That's the honest state of it: a tripwire, not a guarantee. But it's a tripwire that already found two real gaps and one bug in itself on its first two runs, which is a better hit rate than the zero checks that existed before it.
0 Comments
Log in to join the conversation.No comments yet. Be the first to share your thoughts.