ADR-0225proposed

Conversation Annotation Pipeline

Status: Accepted Date: 2026-03-11 Deciders: Joel Hooks, Panda

Context

When Front conversation IDs (e.g. cnv_xxx) appear in gateway messages, or when VIP emails arrive via webhooks, the operator has no automatic context — they must manually ask the agent to pull the conversation. This breaks flow and wastes time.

The system already receives Front webhooks and has a VIP email pipeline. The gap is automated summarization and annotation when conversations are referenced or arrive.

Decision

Build a conversation annotation pipeline as an Inngest durable function that:

Triggers

  1. Front webhook — VIP email or any inbound conversation event (vip.email.received, email/received)
  2. Pattern detection — gateway detects cnv_*, msg_* patterns in any inbound message and emits conversation/annotate.requested
  3. Explicit — operator sends conversation ID directly

Pipeline Steps

  1. Fetch — joelclaw email read --id <cnv_id> to pull full conversation thread
  2. Dedup check — store last-seen message count per conversation in Redis. Skip if no new messages since last annotation.
  3. Summarize — LLM inference to extract:
    • Participants (cross-reference with Vault contacts)
    • Subject/topic
    • Key decisions and action items
    • Extracted links (Google Docs, GitHub, etc.)
    • Urgency/priority assessment
    • What (if anything) needs Joel’s input
  4. Notify gateway — push summary card to Telegram in the relevant thread
  5. Persist — create/update Vault note at ~/Vault/Resources/conversations/{subject-slug}.md
  6. Extract actions — if actionable items found, surface them separately

Observability (heavy logging)

Every step emits structured OTEL events via emitOtelEvent():

  • conversation.annotate.triggered — source (webhook/pattern/explicit), conversation ID
  • conversation.annotate.fetched — message count, participants, has_new_messages
  • conversation.annotate.dedup_skip — when no new messages
  • conversation.annotate.summarized — token usage, model, summary length
  • conversation.annotate.gateway_notified — telegram delivery status
  • conversation.annotate.vault_persisted — vault note path
  • conversation.annotate.actions_extracted — count and types of action items
  • conversation.annotate.failed — any step failure with full context

Gateway Integration

The function pushes structured messages to the gateway via pushGatewayEvent() at each significant step:

  • Annotation start (so the operator knows it’s working)
  • Summary card (the main output)
  • Action items (if any)
  • Errors (if fetch or summarize fails)

Pattern Detection (Gateway Side)

The gateway message handler gains a pattern detector that recognizes:

  • cnv_[a-z0-9]+ — Front conversation IDs
  • msg_[a-z0-9]+ — Front message IDs
  • Future: GitHub issue URLs, PR URLs, other ID-addressable resources

When detected, emits conversation/annotate.requested with the extracted ID and source context.

Idempotency

  • Dedup by conversation ID + message count hash in Redis
  • TTL: 4 hours (re-annotate if referenced again after cooldown, in case thread grew)
  • Inngest idempotency key: annotate-{cnv_id}-{message_count}

Technical Details

  • Event: conversation/annotate.requested
  • Function ID: conversation/annotate
  • Concurrency: limit 2 (don’t overwhelm Front API)
  • Retries: 2
  • Redis keys: conversation:annotate:{cnv_id}:last_count, conversation:annotate:{cnv_id}:last_hash
  • Vault path: ~/Vault/Resources/conversations/{subject-slug}.md
  • Inference: Use infer() with agent profile, not direct API calls

Consequences

  • VIP emails and referenced conversations get automatic context — no manual asking
  • Every annotation is observable via OTEL
  • Gateway stays informed of pipeline progress
  • Vault accumulates a durable record of important conversations
  • Pattern detection in gateway adds a small amount of processing per message (regex, negligible)

Files to Create/Modify

  • packages/system-bus/src/inngest/functions/conversation-annotate.ts — main function
  • packages/system-bus/src/inngest/functions/index.ts — register function
  • packages/system-bus/src/inngest/client.ts — add event type
  • packages/gateway/src/pattern-detector.ts — ID pattern detection (or add to existing message handler)
  • ~/Vault/Resources/conversations/ — output directory

Related

  • ADR-0045: TaskPort hexagonal interface (pattern for port-based access)
  • ADR-0144: Gateway hexagonal architecture
  • VIP email pipeline (existing, this extends it)