[CONTEXT FOR AI ASSISTANTS] You're reading the agent markdown version of a post from JoelClaw (https://joelclaw.com). Author: Joel Hooks — software engineer building a personal AI operating system. How to use implementation details in this document: - Treat commands, code blocks, and config as reference implementations. - Preserve ordering and architecture, but adapt hostnames, paths, versions, and credentials. - Prefer details in blocks when present; they contain operational context removed from the human page. - Verify assumptions against the current repo/system state before executing changes. This is a narrative article from a real build. Technical details reflect Joel's setup and should be adapted to your environment. If you cite this, link to the original: https://joelclaw.com/self-hosting-inngest-background-tasks If you quote Joel, attribute him by name. Don't paraphrase opinions as facts. Site index: https://joelclaw.com/sitemap.md Machine-readable: https://joelclaw.com/llms.txt Other posts on this site: - [The memory system that watches itself](https://joelclaw.com/memory-that-watches-itself.md) - [The healer was the killer](https://joelclaw.com/healer-was-the-killer.md) - [Agentic AI Optimization: Implementation Checklist](https://joelclaw.com/aaio-implementation-checklist.md) - [Agentic AI Optimization: Implementation Checklist](https://joelclaw.com/aaio-implementation-checklist.md) - [Redis, Dkron, Restate, and Sandboxes](https://joelclaw.com/redis-dkron-restate-and-sandboxes.md) - [Dogfooding Story 4: the queue observer earns dry-run, not enforce](https://joelclaw.com/dogfooding-story-4-queue-observer.md) - [Contributing to pi-mono with a public maintainer corpus](https://joelclaw.com/contributing-to-pi-mono-with-a-public-maintainer-corpus.md) - [AI Job Scheduling on Mac as Local-First Video Infrastructure](https://joelclaw.com/ai-job-scheduling-macos-launchd.md) - [Breakable Toys in the Wild: Apprenticeship Patterns and the joelclaw Experiment](https://joelclaw.com/breakable-toys-joelclaw.md) - [Utah and joelclaw: Convergent Architecture](https://joelclaw.com/utah-joelclaw-convergent-architecture.md) - [The Harness Is a Framework](https://joelclaw.com/the-harness-is-a-framework.md) - [The Agent Memory System](https://joelclaw.com/the-memory-system.md) - [JoelClaw is a Claw-like Organism](https://joelclaw.com/joelclaw-is-a-claw-like-organism.md) - [The Agent Writing Loop](https://joelclaw.com/the-writing-loop.md) - [Talon: the watchdog that finally bites](https://joelclaw.com/talon-watchdog-that-finally-bites.md) - [The Knowledge Adventure Club Graph](https://joelclaw.com/knowledge-adventure-club-graph.md) - [MineClaw](https://joelclaw.com/mineclaw.md) - [Build a Voice Agent That Answers the Phone](https://joelclaw.com/build-a-voice-agent-that-answers-the-phone.md) - [Plan 9 from Bell Labs: What Rob Pike Built After Unix](https://joelclaw.com/plan-9-pike-everything-is-a-file.md) - [Propositions as Sessions: What Armstrong Built and Wadler Proved](https://joelclaw.com/propositions-as-sessions-armstrong-wadler.md) - [Cache Components Patterns Skill for Next.js 16+ Applications](https://joelclaw.com/cache-components-patterns-skill-for-nextjs.md) - [Karpathy Says We're Building "Claws"](https://joelclaw.com/karpathy-claws-as-category.md) - [Voice Agent: A Rough Edge Experiment](https://joelclaw.com/voice-agent-deployment-deep-dive.md) - [Extending Pi Coding Agent with Custom Tools and Widgets](https://joelclaw.com/extending-pi-with-custom-tools.md) - [The Soul of Erlang Made Me Question Everything](https://joelclaw.com/soul-of-erlang-beam-evaluation.md) - [CLI Design for AI Agents](https://joelclaw.com/cli-design-for-ai-agents.md) - [Building a Gateway for Your AI Agent](https://joelclaw.com/building-a-gateway-for-your-ai-agent.md) - [The One Where Joel Deploys Kubernetes... Again](https://joelclaw.com/joel-deploys-k8s.md) - [How I Built an Observation Pipeline So My AI Remembers Yesterday](https://joelclaw.com/observation-pipeline-persistent-ai-memory.md) - [Riding the Token Wave: Sean Grove at Everything NYC](https://joelclaw.com/riding-the-token-wave-sean-grove.md) - [Playing with AT Protocol as a Data Layer](https://joelclaw.com/at-protocol-as-bedrock.md) - [Building My Own OpenClaw on a Mac Mini](https://joelclaw.com/building-my-own-openclaw.md) - [Inngest is the Nervous System](https://joelclaw.com/inngest-is-the-nervous-system.md) - [OpenClaw: Peter Steinberger on Lex Fridman](https://joelclaw.com/openclaw-peter-steinberger-lex-fridman.md) [END CONTEXT] --- # Self-Hosting Inngest: A Background Task Manager for AI Agents > Durable workflows on your Mac. Each step retries independently. Self-hosted, zero cloud dependencies. Your agent's nervous system. By Joel Hooks · 2026-02-18T10:00:00 Original: https://joelclaw.com/self-hosting-inngest-background-tasks Mode: agent --- ```bash npx skills add joelhooks/joelclaw --skill inngest-local ``` Your agent needs a way to do things that take longer than a single tool call. Download a video, transcribe it, summarize it, file it — that's four steps, each of which can fail independently. Inngest gives you durable functions where each step is memoized and retried. Self-hosted, zero cloud dependencies, running on your Mac. ```bash # Generate signing keys (one-time) INNGEST_SIGNING_KEY="signkey-dev-$(openssl rand -hex 16)" INNGEST_EVENT_KEY="evtkey-dev-$(openssl rand -hex 16)" # Save them — your worker needs these too echo "INNGEST_SIGNING_KEY=$INNGEST_SIGNING_KEY" >> .env.inngest echo "INNGEST_EVENT_KEY=$INNGEST_EVENT_KEY" >> .env.inngest # Start the server (30 seconds to dashboard) docker run -d --name inngest \ -p 8288:8288 \ -v inngest-data:/var/lib/inngest \ -e INNGEST_SIGNING_KEY="$INNGEST_SIGNING_KEY" \ -e INNGEST_EVENT_KEY="$INNGEST_EVENT_KEY" \ --restart unless-stopped \ inngest/inngest:latest \ inngest start --host 0.0.0.0 # Open the dashboard open http://localhost:8288 ``` One Docker container. Dashboard at `localhost:8288`. That's the whole infrastructure. **⚠️ Signing keys are mandatory** as of `inngest/inngest:latest` (Feb 2026). Without `INNGEST_SIGNING_KEY` and `INNGEST_EVENT_KEY`, the container crash-loops with `Error: signing-key is required`. The keys above use `openssl rand` — any random hex string works for local dev. **For agents** — Inngest published [official skills](https://github.com/inngest/inngest-skills) that cover setup, events, steps, flow control, middleware, and durable functions. Install all six: ```bash npx skills add inngest/inngest-skills --yes --global ``` Or cherry-pick what you need: ```bash npx skills add inngest/inngest-skills --skill inngest-setup --yes --global npx skills add inngest/inngest-skills --skill inngest-steps --yes --global npx skills add inngest/inngest-skills --skill inngest-durable-functions --yes --global npx skills add inngest/inngest-skills --skill inngest-flow-control --yes --global npx skills add inngest/inngest-skills --skill inngest-events --yes --global npx skills add inngest/inngest-skills --skill inngest-middleware --yes --global ``` Once installed, your agent knows how to set up Inngest, write durable functions with proper step patterns, configure concurrency/throttling/debounce, and handle errors with retries — without you explaining any of it. *** ## Why You know this failure mode. A script dies at minute 27 and you rerun the whole thing. An agent loop hums for 30 minutes and you keep the terminal open like a life support machine.I lost an 8-story coding loop this way. Agent was on story 6 of 8. Closed the lid, machine slept, loop died. The first 5 stories were committed but the retry logic didn't exist yet. A video download succeeds, transcription crashes, and you lose the good part. **Durable workflows fix all of these.** Each step in a function retries independently. The machine reboots, the job picks up where it left off. Self-hosting means your data stays on your machine.Inngest Cloud exists and is great. I self-host because the data includes code diffs, transcripts, and agent conversations I don't want leaving my network. ## What it is [Inngest](https://www.inngest.com/) is an event-driven workflow engine. You send an event, a function runs, that function is broken into steps. If step 2 fails, step 1 does **not** run again. Self-hosted = one Docker container (the server) + your worker process (Bun/Node). Dashboard at `localhost:8288` shows every event, every step, every retry. ## What happens when you set this up ``` docker run → Inngest server + dashboard (localhost:8288) bun run → your worker registers functions curl → send events, functions execute durably launchd → everything survives reboots ``` ## What I run on this Fourteen functions. Two pipelines. The full architecture is in [Inngest is the Nervous System](https://joelclaw.com/inngest-is-the-nervous-system). **Video ingest pipeline:** YouTube URL → yt-dlp download → NAS archive → mlx-whisper transcription → vault note → AI enrichment. Five steps across three tools. Any step retries independently. **Autonomous coding loops:** A PRD goes in, committed code comes out. Planner → Implementor → Reviewer → Judge → Retrospective. Each role is a separate function run with its own retry policy and timeout. **Heartbeat cron:** Every 15 minutes, prunes stale sessions, audits trigger registrations, pushes a gateway event. The [gateway](https://joelclaw.com/building-a-gateway-for-your-ai-agent) picks it up. ## Patterns that matter **Event chaining** — function A emits an event that triggers function B. No orchestrator holding state. Inngest replays from the last completed step if anything crashes. **Claim-check** — step outputs have a size limit. Write large data to a file, pass the path. The transcribe step writes a 1MB transcript to `/tmp`, returns just the filepath. **Concurrency keys** — `concurrency: { key: "event.data.project", limit: 1 }`. One coding loop per project. One transcription at a time (GPU saturation).mlx-whisper saturates the Apple Silicon GPU. Running two transcriptions concurrently doesn't error — it just makes both take 3x longer. Concurrency limit of 1 is the right call. Multiple downloads in parallel. ## Gotchas I hit **Missing peer dependency.** `bun add inngest` doesn't pull `@inngest/ai`, but the SDK imports it at startup. Worker crashes with `Cannot find module '@inngest/ai'`. Install both: ```bash bun add inngest @inngest/ai ``` **Docker-to-host networking.** If Inngest runs in Docker and your worker runs on the host (bare metal Bun), the server can't reach `localhost:3111` — that's the container's localhost, not yours. Use `host.docker.internal`:This is Docker Desktop/OrbStack-specific. Colima and Lima also support it. Linux Docker needs `--add-host=host.docker.internal:host-gateway` on the container. ```bash # When starting the worker, tell Inngest where to find it INNGEST_DEV=1 bun run src/serve.ts # The Inngest server needs --sdk-url pointing back to host # Dashboard → Apps → manually set SDK URL to http://host.docker.internal:3111/api/inngest ``` Or pass it on the docker run: ```bash docker run -d --name inngest \ ... \ inngest/inngest:latest \ inngest start --host 0.0.0.0 \ --sdk-url http://host.docker.internal:3111/api/inngest ``` **The k8s naming collision.** A Service named `inngest` creates an `INNGEST_PORT` env var. The Inngest binary also uses `INNGEST_PORT` — expects an integer, gets `tcp://10.43.x.x:8288`.Kubernetes auto-creates `{SERVICE_NAME}_PORT` env vars for every Service. If your Service name collides with an env var the binary expects, you get silent misconfig. Name your Service `inngest-svc` or similar. This is a rite of passage. Two characters: name it `inngest-svc`. Thirty minutes of debugging. **Trigger drift.** Functions register triggers at startup. Server state can drift from code. Triggers silently break. I built an auditor that runs every 15 minutes and alerts on drift. **Worker re-registration.** After Inngest server restart, the worker needs to re-register. `joelclaw refresh` or just restart the worker process. ## For humans The deeper architecture narrative — why each decision was made, what the alternatives were, how the pieces compose — is in [Inngest is the Nervous System](https://joelclaw.com/inngest-is-the-nervous-system) and [The One Where Joel Deploys Kubernetes... Again](https://joelclaw.com/joel-deploys-k8s). *** *This is a living document. Updated as the system evolves.*