Karpathy’s LLM Wiki: Compile Knowledge, Don’t Just Retrieve It
A tiny but sharp pattern for letting agents maintain a durable markdown knowledge base: raw sources stay immutable, the model owns the wiki, and query/lint loops make the knowledge compound instead of being rediscovered every time.
Andrej Karpathy’s llm-wiki.md is not really a paper. It’s an idea file: a compact operating manual for using an LLM agent as the maintainer of a local markdown knowledge base.
The screenshot Joel shared was a conference-paper-style reformat called “LLM-WIKI.md: Field Notes on a Knowledge Base That Maintains Itself.” I couldn’t find that exact paper title as a canonical source. The source trail points back to Karpathy’s public gist, so that’s the thing worth saving.
The useful bit
Most RAG setups keep documents raw and make the model rediscover the same knowledge on every question. Karpathy’s move is different: compile the sources once into a persistent wiki, then query that compiled artifact.
The split is simple:
- Raw sources: articles, papers, screenshots, transcripts, data files. Immutable. The human curates them.
- Wiki: generated markdown pages, summaries, entities, comparisons, synthesis. The model owns this layer.
- Schema:
CLAUDE.md,AGENTS.md, or similar. This tells the agent how to ingest, query, lint, and maintain the wiki.
That division is the whole trick. Humans keep judgment and source selection. The model handles the bookkeeping that kills every knowledge base: summaries, cross-links, index updates, contradiction checks, stale claims, and orphan pages.
Why it matters
The line that sticks:
Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase.
That is a much better frame than “chat with my docs.” The wiki becomes a durable artifact that can improve over time. Every ingest can update old pages. Every good query can become a new synthesis page. Linting becomes part of maintenance, not an afterthought.
This maps directly onto the Brain work we keep circling: source-first capture, agent-maintained structure, and explicit health checks instead of dumping everything into a pile and hoping retrieval saves us later.
Caveat
This is not magic. If the model compiles a source badly, the error can harden into the wiki. The raw-source layer and lint pass are the safety rails. Keep sources immutable, keep provenance visible, and make contradiction checks normal.