Article Content Migration — MDX to Convex ContentResource
Superseded by 0168-convex-canonical-content-lifecycle
Status
superseded by ADR-0168
Context
joelclaw.com articles live as .mdx files in apps/web/content/. The content review pipeline (ADR-0106) needs articles in Convex so agents can read, edit, and write revisions. The feedback form on joelclaw.com currently triggers no work — this migration enables closing that loop.
Auth is shipped. Convex is deployed in k8s (ADR-0039). ContentResource schema exists (ADR-0084). The missing piece is getting article content into Convex and wiring the rendering pipeline to read from it.
Decisions Made
- Convex is source of truth — filesystem MDX becomes seed input only, not read at runtime
- Feedback is Joel-only — auth-gated, single user
- Agent edits auto-publish — no approval step, revision history provides safety net
- Raw MDX stored in Convex — preserves component imports and JSX
Decision
Phase 1: Seed Script
A one-shot script reads all .mdx files from apps/web/content/, parses frontmatter, and writes each as a ContentResource:
// scripts/seed-articles.ts
import { ConvexHttpClient } from "convex/browser"
import { api } from "../convex/_generated/api"
import fs from "fs"
import path from "path"
import matter from "gray-matter"
const client = new ConvexHttpClient(process.env.CONVEX_URL!)
const contentDir = path.join(process.cwd(), "apps/web/content")
const files = fs.readdirSync(contentDir).filter(f => f.endsWith(".mdx"))
for (const file of files) {
const raw = fs.readFileSync(path.join(contentDir, file), "utf-8")
const { data: meta, content } = matter(raw)
const slug = file.replace(/\.mdx$/, "")
await client.mutation(api.content.upsert, {
resourceId: `article:${slug}`,
type: "article",
fields: {
slug,
title: meta.title,
description: meta.description,
content, // raw MDX body (no frontmatter)
image: meta.image || null,
tags: meta.tags || [],
type: meta.type || "post",
date: meta.date,
updated: meta.updated || null,
draft: meta.draft || false,
},
})
console.log(`Seeded: ${slug}`)
}Phase 2: Read Path — getPost() from Convex
Replace filesystem reads with Convex queries:
// apps/web/lib/posts.ts (new)
import { fetchQuery } from "convex/nextjs"
import { api } from "../../convex/_generated/api"
export async function getPost(slug: string): Promise<Post | null> {
const resource = await fetchQuery(api.content.getByResourceId, {
resourceId: `article:${slug}`,
})
if (!resource || resource.type !== "article") return null
return {
meta: {
slug: resource.fields.slug,
title: resource.fields.title,
description: resource.fields.description,
date: resource.fields.date,
updated: resource.fields.updated,
type: resource.fields.type,
tags: resource.fields.tags,
image: resource.fields.image,
draft: resource.fields.draft,
},
content: resource.fields.content,
}
}
export async function getAllPosts(): Promise<PostMeta[]> {
const resources = await fetchQuery(api.content.listByType, {
type: "article",
})
return resources
.filter(r => !r.fields.draft)
.sort((a, b) => new Date(b.fields.date).getTime() - new Date(a.fields.date).getTime())
.map(r => r.fields as PostMeta)
}Phase 3: Rendering — Static Shell + Dynamic Slots
// app/[slug]/page.tsx
import { Suspense } from "react"
export default async function ArticlePage({ params }) {
const { slug } = await params
return (
<article>
{/* CACHED: article content */}
<Suspense fallback={<ArticleSkeleton />}>
<CachedArticle slug={slug} />
</Suspense>
{/* DYNAMIC: feedback status (Convex subscription, client island) */}
<Suspense fallback={null}>
<FeedbackStatusIsland slug={slug} />
</Suspense>
{/* STATIC: feedback form */}
<FeedbackForm slug={slug} />
</article>
)
}
async function CachedArticle({ slug }: { slug: string }) {
'use cache'
cacheLife('max')
cacheTag(`article:${slug}`)
const post = await getPost(slug)
if (!post) notFound()
return <MDXRenderer source={post.content} />
}Phase 4: MDX Rendering from String
Currently MDX is compiled at build time via next-mdx-remote or similar. With content from Convex, we need runtime MDX compilation:
// components/mdx-renderer.tsx
import { compileMDX } from 'next-mdx-remote/rsc'
import { components } from './mdx-components'
export async function MDXRenderer({ source }: { source: string }) {
const { content } = await compileMDX({
source,
components,
options: {
parseFrontmatter: false, // already parsed
},
})
return content
}This runs server-side inside the 'use cache' boundary — compiled once, cached until tag invalidation.
Phase 5: Cache Invalidation Endpoint
// app/api/revalidate/route.ts
import { revalidateTag } from 'next/cache'
import { headers } from 'next/headers'
export async function POST(req: Request) {
const h = await headers()
const secret = h.get('x-revalidation-secret')
if (secret !== process.env.REVALIDATION_SECRET) {
return Response.json({ error: 'unauthorized' }, { status: 401 })
}
const { tag } = await req.json()
revalidateTag(tag)
return Response.json({ revalidated: true, tag })
}Phase 6: Wire Feedback Pipeline (ADR-0106)
With content in Convex and cache invalidation working, the feedback form → Inngest → agent → revision → revalidate loop is complete.
generateStaticParams
export async function generateStaticParams() {
// At build time, Convex HTTP client fetches all article slugs
const client = new ConvexHttpClient(process.env.CONVEX_URL!)
const articles = await client.query(api.content.listSlugs, { type: "article" })
return articles.map(slug => ({ slug }))
}This works because ConvexHttpClient is a plain HTTP client — no WebSocket, no provider needed.
Filesystem MDX After Migration
The .mdx files in apps/web/content/ become:
- Seed source for initial migration
- Git history for article provenance
- Not read at runtime — Convex is the live source
New articles are created in Convex directly (via dashboard, API, or agent). The MDX files can stay in the repo as archival artifacts but are not part of the build.
Consequences
Good:
- Articles are writable by agents — enables the entire feedback pipeline
- Static shell with cached content — same performance as current filesystem reads
cacheTaginvalidation is surgical — only the edited article re-rendersConvexHttpClientfor build-time queries means no provider complexity at the page level- Revision history in Convex provides full audit trail
Bad:
- Runtime MDX compilation adds ~50-100ms per cache miss (amortized by
cacheLife('max')) - Convex becomes a hard dependency for the site (currently zero runtime deps)
- MDX component imports (
import X from './component') won’t work from Convex strings — only pre-registered components via the components map
Mitigations:
- Cache miss cost is acceptable — happens once per deploy or revalidation
- Convex in k8s is on the same machine — sub-ms latency
- All current MDX components are already in a shared components map — no dynamic imports in article content
Resolved Questions
- Discoveries: Yes, but in a later phase. Articles first to prove the pipeline.
- Seed script: Idempotent upsert — re-runnable during development and if MDX files update before cutover.
- Feedback scope: Joel-only (auth-gated, single user).
- Agent edits: Auto-publish, no approval step. Revision history is the safety net.
- Content format: Raw MDX in Convex — preserves component imports and JSX.
Technical Notes
fetchQueryfromconvex/nextjssetscache: "no-store"internally. This is fine inside a'use cache'boundary — the outer cache directive handles caching. ButfetchQueryoutside a cache boundary hits Convex on every request.compileMDXfromnext-mdx-remote/rscusesFunction()constructor to eval compiled MDX. Inside'use cache'this works because the output is a serializable React tree. The components map must be passed at compile time, not closed over.ConvexHttpClientfromconvex/browseris correct forgenerateStaticParamsand seed scripts — plain HTTP, no WebSocket, no provider. Already used inapps/web/lib/convex-content.tsandpackages/system-bus/src/lib/convex.ts.
Related
- ADR-0039: Self-host Convex (shipped — infrastructure)
- ADR-0084: ContentResource schema (shipped —
articletype added) - ADR-0106: Content Review Pipeline (accepted — the consumer of this migration)
- ADR-0075: Better Auth (shipped — auth for feedback)
- ADR-0112: Unified caching layer