ADR-0196accepted

Cancel-on-New-Message for Gateway

Status

Accepted

Context

When Joel sends multiple messages in quick succession, the gateway processes them sequentially via a Redis priority queue. This means the first message runs to completion before the second starts — even when the second message supersedes the first (correction, follow-up, or refined ask).

Utah (inngest/utah) uses Inngest’s cancelOn + singleton pattern: when a new message arrives from the same chat, the running agent loop is cancelled and a fresh one starts with the latest message. The user’s most recent message IS the intent.

Decision

Adopt the cancel-on-new-message pattern for the gateway’s message handling:

  1. Singleton concurrency — one active agent run per chat/channel at a time
  2. Cancel on new message — incoming message cancels the running loop and starts fresh
  3. Message batching window — short delay (1-2s) to collect rapid-fire messages before starting the loop, so “oops” + correction arrives as one unit
  4. Preserve queue for system events — only human messages trigger cancel; automated events (heartbeats, webhooks) still queue normally

Implementation note (2026-03-06)

First runtime slices are now shipped in the gateway daemon/command queue:

  • direct human turns across Telegram, Discord, iMessage, and Slack invoke paths use latest-wins supersession keyed by source
  • a newer human message drops queued stale prompts for that source
  • the daemon aborts the stale active turn and suppresses any stale late response text
  • direct human turns now get a short 1.5s batching window before dispatch so rapid follow-ups collapse into one prompt
  • if the source is already active, the gateway supersedes immediately instead of waiting on the batch timer
  • gateway status exposes supersession plus batching state
  • gateway diagnose exposes an interruptibility layer

Passive intel/background routes still bypass the human batching path.

Inngest Configuration

{
  id: "gateway-handle-message",
  concurrency: [{ scope: "fn", key: "event.data.chatId", limit: 1 }],
  cancelOn: [{ event: "gateway/message.received", match: "data.chatId" }],
}

Consequences

  • Better UX — corrections and follow-ups don’t wait for stale processing to finish
  • Reduced waste — no LLM tokens spent on superseded messages
  • Lost work — if the agent is mid-tool-execution on a valid first message and a second arrives, that work is lost. Acceptable tradeoff for conversational flow.
  • Requires distinguishing human vs system messages — system events must not cancel human conversations

Agent Readiness

  • Confidence: 4 — Utah proves the pattern works
  • Agent-ready: 4 — clear Inngest primitives, well-scoped
  • NRC: Medium — gateway message handling refactor
  • Novelty: Low — proven pattern from Utah

References