Close the Loop — Reflect Brain, Failed Targets, and Mise Brief

2026-03-03 Reality Check and Extension

This ADR remains accepted for retrieval/indexing direction, but the write-side coverage is only partially implemented as of March 3, 2026:

• joelclaw knowledge sync currently indexes ADR + skill documents only.
• Live audits still show sparse/empty lesson, pattern, and failed_target query results in system_knowledge.
• Turn-level knowledge writes are not yet universally enforced across gateway + system-agent turns.

ADR-0202 extends ADR-0199 with non-optional turn-level write enforcement (knowledge note or explicit knowledge.skip reason) plus compliance verification.

2026-03-04 Observation: lesson/pattern docs are write-on-loop-completion only

All checklist items in the 2026-03-03 section are accurate for the implementation. However:

• lesson, pattern, failed_target document counts in system_knowledge are 0 because no loops have completed recently — these documents are produced by agent-loop-retro and will accumulate as loops run
• The infrastructure is correct; the collection will populate naturally
• recall budget profiles (lean/balanced/deep) are implemented in the CLI flag handling (joelclaw recall --budget) but the typesense-recall.ts adapter does not yet vary query depth by budget profile — all profiles use the same Typesense query parameters. See ADR-0096 for the full budget spec; that gap remains open.

Context and Problem Statement

joelclaw has extensive infrastructure for agent loops (Inngest, Redis, OTEL, Typesense) and well-specified designs for post-loop learning (ADR-0008), memory (ADR-0021), and friction auto-fix (ADR-0065). But the feedback cycle is broken at three specific points:

1. Retro writes to Redis and Vault, but nothing reads it back at the right moment. The agent-loop-retro function (400 lines, shipped) writes a retrospective to ~/Vault/system/retrospectives/ and stores codebase patterns in Redis via writePatterns(). The implementor reads patterns via readPatterns(). But: patterns are per-project Redis keys that get overwritten each loop. There’s no durable, browsable knowledge base that agents consult before starting new work. Retrospective notes in Vault are write-only — nothing reads them.

2. No failed target tracking across loops. Within a single loop, idempotency guards (ADR-0016) prevent duplicate story dispatch. But across loops, nothing prevents the pitch system or planner from proposing the same broken work repeatedly. Noodle tracks failedTargets persistently and feeds them to the scheduler as “don’t retry these” context.

3. Pitch decisions lack situational awareness. The adr-work-pitch function reads ADR files from the Vault filesystem and applies the ADR-0183 rubric. It does not consider: recent loop outcomes, active loops, failed targets, system health, recent retro findings, or capacity. Noodle’s “mise en place” pattern gathers ALL of this into one brief before the scheduler makes any decision.

What exists today (audit)

The noodle insight

Noodle (github.com/poteto/noodle) by Lauren Tan solves this with three structural patterns:

• brain/codebase/ — A directory of markdown notes written by reflect agents after sessions. Each note captures a specific operational lesson (e.g., “schedule bootstrap causes hot loop because fsnotify triggers on every orders.json write”). Agents read relevant brain notes before starting work. Knowledge accumulates; it doesn’t get overwritten.
• mise en place — Before any scheduling decision, gather the full picture: backlog, active agents, recent failures, capacity, history, failed targets. One snapshot brief that the scheduler reads.
• failed targets — Permanently failed work items tracked in a set. The scheduler sees them as context and won’t recreate the same broken orders.

The key principle: the feedback loop must be structural, not aspirational. Make it involuntary. If reflect doesn’t complete, the next work item doesn’t dispatch. If the mise brief isn’t gathered, the scheduler doesn’t run.

Decision

Wire up the existing pieces and add four missing components. The core insight: we have 10 Typesense collections, a working joelclaw recall command, memory observations, vault notes — but system knowledge from loops (retros, lessons, patterns, failed targets) never enters the search layer. And retrieval is opt-in when it should be involuntary.

This is a wiring ADR, not a design ADR. Two principles from noodle apply:

1. Knowledge must accumulate, not overwrite. Current writePatterns() clobbers the previous loop’s patterns. Each learning is a document that persists.
2. Retrieval must be involuntary. Not “agents should read brain notes” but “the harness injects relevant context before dispatch.” Same enforcement pattern as ADR-0195 (mandatory memory participation).

0. System knowledge in Typesense — the brain IS the index

Create a system_knowledge Typesense collection as the unified brain. Single collection, type-tagged, vector-embedded for semantic search.

Schema:

{
  "name": "system_knowledge",
  "fields": [
    { "name": "id", "type": "string" },
    { "name": "type", "type": "string", "facet": true },
    { "name": "title", "type": "string" },
    { "name": "content", "type": "string" },
    { "name": "source", "type": "string", "optional": true },
    { "name": "project", "type": "string", "optional": true, "facet": true },
    { "name": "loop_id", "type": "string", "optional": true },
    { "name": "status", "type": "string", "optional": true, "facet": true },
    { "name": "score", "type": "int32", "optional": true },
    { "name": "tags", "type": "string[]", "optional": true, "facet": true },
    { "name": "created_at", "type": "int64" },
    { "name": "embedding", "type": "float[]", "num_dim": 768, "optional": true }
  ],
  "default_sorting_field": "created_at"
}

Document types:

Vault files remain source of truth. Typesense is the search/retrieval layer. Retros write to Vault AND index to Typesense. ADRs sync from Vault files. Skills sync from skills/ directory.

Mandatory retrieval via extension hooks:

The memory-enforcer extension (ADR-0195) pattern extends to system knowledge:

• Before codex dispatch: Query system_knowledge with story title + domain keywords. Inject top 5 results as ## System Context section in the prompt. Non-optional.
• Before pitch evaluation: Query for retros + failed targets related to candidate ADRs.
• Before loop planning: Query for patterns + lessons for the project.
• OTEL enforcement: Every dispatch must have a system_knowledge.retrieval OTEL event. Zero-retrieval is an alert.

Add system_knowledge to the COLLECTIONS array in packages/cli/src/commands/search.ts so joelclaw recall can query it alongside existing collections.

1. Brain notes — accumulated codebase knowledge (indexed to Typesense)

Each retro produces 0-N knowledge documents indexed to system_knowledge with type: "lesson" or type: "pattern". Old documents persist. No overwrite.

Source: The retro function already extracts codebasePatterns from progress.txt and runs LLM reflection. Currently stored as a single overwritten Redis key. Change: index individual findings as Typesense documents.

Also write to Vault at ~/Vault/system/brain/codebase/ as readable markdown backup. Typesense is the query layer; Vault is the durable source.

Read path: Extension hook queries Typesense before dispatch. No manual file reading required.

2. Failed target tracking

Add a Redis set loop:failed:targets:{project} that tracks story identifiers (or ADR numbers, for pitch) that have permanently failed.

Write path: When the retro function detects a story that failed after max attempts and was skipped, add its identifier to the set.

Read path (loops): The plan function checks SISMEMBER before including a story in the next loop’s PRD.

Read path (pitch): The pitch function checks before proposing an ADR for work.

Expiry: Failed targets expire after 7 days (Redis TTL on each member via sorted set with timestamp scores). This prevents permanent blacklisting while stopping immediate re-attempts.

Override: joelclaw loop clear-failed <target> CLI command to manually remove a failed target.

3. Mise brief for pitch

Before the pitch function evaluates ADRs, gather situational context into a brief object:

type MiseBrief = {
  recentRetros: Array<{ loopId: string; summary: string; storiesCompleted: number; storiesFailed: number; date: string }>;
  activeLoops: Array<{ loopId: string; project: string; status: string }>;
  failedTargets: string[];
  recentPitchHistory: PitchRecord[];
  systemHealth: { workerStatus: string; inngestReachable: boolean; redisReachable: boolean };
  capacity: { activeLoops: number; maxConcurrentLoops: number };
};

Sources: Redis (loop state, pitch history, failed targets), Inngest API (recent runs), system health checks (existing in joelclaw status).

Usage: The pitch function passes the mise brief to the ADR evaluation step. ADRs related to recently-failed work are deprioritized. ADRs related to successful patterns are boosted. If capacity is zero (active loop running), don’t pitch.

4. Wire retro → appendLessons

The appendLessons() function exists but is never called by the retro function. Add a step to retro that extracts key lessons from the LLM reflection and appends them via appendLessons(). The implementor already reads lessons via readLessons() — this wire just needs connecting.

5. Wire retro → skill update proposals

ADR-0008 §4 specifies a safe skill update policy (additive-only auto-apply, structural requires review). The retro function doesn’t implement it. Add a step that:

• Identifies candidate skills from files touched and tools used (signals already available in retro)
• Compares retro findings against current skill bodies
• For additive_drift: emits skill/update.proposed event with the proposed addition
• For structural_drift: writes a proposal note to ~/Vault/system/brain/skill-proposals/

Gate: Skill mutations require human approval. The proposed event triggers a pitch-style Telegram notification with 👍/👎.

6. Mandatory retrieval enforcement

Extend the memory-enforcer extension pattern (ADR-0195) to system knowledge:

• Extension hook (before_agent_start): Query system_knowledge collection with task context. Inject top results as system message.
• Codex delegation: buildPrompt() in implement.ts queries Typesense before constructing the prompt. Results go in a ## System Context section that cannot be removed.
• Pitch function: buildMiseBrief() queries for related retros, failed targets, and lessons.
• OTEL gate: Every dispatch emits system_knowledge.retrieval event with query, result count, latency. A scheduled function alerts on zero-retrieval dispatches.

This makes system knowledge retrieval involuntary — the same enforcement philosophy as ADR-0195 for memory.

Implementation Plan

Phase 1: Typesense Collection + Index Sync (foundation)

1. Create system_knowledge Typesense collection with schema above
2. Write sync function: scan Vault ADRs → index as type: "adr" documents with status, scores, tags
3. Write sync function: scan skills/*/SKILL.md → index as type: "skill" documents
4. Register system_knowledge in CLI search collections
5. Verify: joelclaw recall "failed targets" returns results from new collection

Phase 2: Retro → Index + Failed Targets (close the write side)

6. Modify agent-loop-retro → index-findings step: extract lessons and patterns, index to system_knowledge as individual documents
7. Also write brain notes to ~/Vault/system/brain/codebase/ as durable markdown backup
8. Modify agent-loop-retro → track-failed-targets step: index permanently failed stories as type: "failed_target" documents (with TTL tag for 7-day expiry)
9. Wire retro → appendLessons() call (one-line fix — the function exists, never invoked)
10. Add OTEL events for all index writes

Phase 3: Mandatory Retrieval + Mise Brief (close the read side)

11. Modify buildPrompt() in implement.ts: query system_knowledge for relevant context, inject as ## System Context
12. Modify plan.ts: query system_knowledge for type: "failed_target" before including stories
13. Create buildMiseBrief(): query system_knowledge for retros + failed targets + lessons, plus Redis for active loops + pitch history
14. Modify adr-work-pitch → gather-mise step before ADR evaluation
15. Add capacity check to pitch (don’t pitch if loop is actively running)
16. Extend memory-enforcer extension: query system_knowledge on before_agent_start, inject results
17. Add OTEL gate: every dispatch must have system_knowledge.retrieval event. Alert on zero-retrieval.

Phase 4: Skill Proposals + Verification (prove it works)

18. Wire retro → skill update proposals (additive drift detection)
19. Add skill/update.proposed event + Telegram notification
20. Add joelclaw loop clear-failed <target> CLI command
21. Run a loop end-to-end, verify:
    • Findings indexed to Typesense
    • Failed targets tracked and skipped
    • Implementor prompt includes system context
    • Mise brief includes retro findings
    • OTEL shows retrieval events on every dispatch

Affected Paths

• packages/system-bus/src/inngest/functions/agent-loop/retro.ts — add Typesense indexing + failed target + lessons steps
• packages/system-bus/src/inngest/functions/agent-loop/implement.ts — query system_knowledge in buildPrompt()
• packages/system-bus/src/inngest/functions/agent-loop/plan.ts — query failed targets from Typesense
• packages/system-bus/src/inngest/functions/adr-daily-pitch.ts — add mise brief gathering
• packages/system-bus/src/inngest/functions/agent-loop/utils.ts — add Typesense indexing helpers
• packages/system-bus/src/inngest/functions/system-knowledge-sync.ts — new: periodic ADR + skill sync to Typesense
• packages/cli/src/commands/search.ts — add system_knowledge to COLLECTIONS
• packages/cli/src/cli.ts — add loop clear-failed command
• ~/.pi/agent/extensions/memory-enforcer/index.ts — extend with system knowledge retrieval
• ~/Vault/system/brain/codebase/ — new directory (durable markdown backup)
• ~/Vault/system/brain/skill-proposals/ — new directory (Phase 4)

Verification

Typesense Collection

• system_knowledge collection created with schema (commit fa0bdc0)
• ADRs synced (195 documents, type: “adr”) via joelclaw knowledge sync
• Skills synced (64 documents, type: “skill”) via joelclaw knowledge sync
• joelclaw recall queries system_knowledge alongside existing collections (commit fa0bdc0)

Brain Notes + Index

• Retro indexes findings as individual type: "lesson" and type: "pattern" documents (commit 8bce83a)
• Documents accumulate (no overwrite — each lesson gets unique lesson:{loopId}:{i} ID)
• Vault backup written to ~/Vault/system/brain/codebase/ — retro write-brain-notes step
• Documents have embeddings for semantic search (ts/all-MiniLM-L12-v2 auto-embed)

Failed Targets

• Permanently failed story indexed as type: "failed_target" (commit 8bce83a)
• Plan function queries Typesense for failed targets before including stories (commit 2ce49bc)
• Failed targets effectively expire after 7 days — knowledge-watchdog prunes expired
• joelclaw knowledge clear-failed removes a target
• Pitch function checks failed targets before proposing ADRs

Mandatory Retrieval

• buildPrompt() queries system_knowledge and injects ## System Context (commit fa0bdc0)
• Memory-enforcer extension queries system_knowledge on session_start (commit fa0bdc0)
• Pitch function gathers mise brief (retros + failed targets + lessons + capacity)
• Pitch does not fire when a loop is actively running (capacity check in 12d71eb)
• EVERY dispatch has a system_knowledge.retrieval OTEL event (commit 2ce49bc)
• Zero-retrieval dispatches trigger alert (knowledge-watchdog, commit e535002)

Lessons Wire

• Retro calls appendLessons() with key findings (commit 8bce83a)
• Implementor reads lessons and includes in prompt (pre-existing)

ensureKnowledge Invariant (added during implementation)

• ensureKnowledge() utility: check-or-upsert single doc (commit 631481b)
• ensureKnowledgeBatch(): bulk version for >20 docs
• Discovery capture → system_knowledge on capture
• Memory promote → system_knowledge on promotion
• Meeting decisions → system_knowledge on analysis
• Pitch approval/rejection → updates ADR status in system_knowledge
• All enforce graceful degradation if Typesense unavailable

Multi-stage Retrieval

• Implementor queries system_knowledge (commit fa0bdc0)
• Reviewer queries system_knowledge for evaluation context (commit 2ce49bc)
• Judge queries system_knowledge for verdict context (commit 2ce49bc)
• Test writer queries system_knowledge for test patterns (commit 2ce49bc)
• Fan-out multi_search across 4 collections in one call (commit 2ce49bc)

Sync Triggers

• Daily 3am cron syncs ADRs + skills to system_knowledge (commit d0a9e23)
• system/adr.sync.requested triggers system_knowledge sync (commit d0a9e23)
• Pitch function syncs scored ADRs with rubric data (commit d0a9e23)
• ADR skill mandates sync after every lifecycle change (commit d0a9e23)

Observability

• OTEL events for: retrieval executed (implement.ts), watchdog check
• Knowledge watchdog every 4h: collection health, zero-retrieval detection, staleness (commit e535002)
• joelclaw otel stats shows system_knowledge retrieval + watchdog counts

Consequences

• Good, because existing infrastructure (Redis, Vault, Inngest, retro function) is reused — minimal new code
• Good, because the feedback loop becomes structural: retro writes brain notes → implementor reads them → next loop is smarter
• Good, because failed target tracking prevents the system from banging its head against the same wall
• Good, because mise brief gives pitch decisions situational awareness beyond static ADR scores
• Bad, because brain note accumulation requires eventual pruning/meditate (deferred)
• Bad, because failed target TTL (7 days) is a guess — may need tuning
• Neutral, because skill update proposals (Phase 3) still require human approval — no autonomous skill mutation yet

Revisit Triggers

• Brain notes exceed 100 files without pruning → need meditate/consolidation function
• Failed target 7-day TTL is too short (things get re-proposed too fast) or too long (legitimate retries blocked)
• Mise brief adds >2s latency to pitch function → optimize or cache
• Skill update proposals create review fatigue → adjust threshold or batch