Build Log

A Token Rotation That Found Two Silent Holes

September 27, 2026

Rotating an expiring PAT across 6 apps on 2 platforms surfaced a missed consumer, a 24-day-old deploy check that passed on 404, and produced the mechanism that makes the next rotation enumerable.

OSSecurityOwner LoopDeploy

consumers rotated (2 platforms)

days canvas-1's deploy check passed on a 404

consumer missed on the first pass

deploy targets re-verified, 0 regressions

Timeline

Start

Ignite v2 blocked on a stale ownership claim

9h53m-old claim from the 2026-09-26 retro session; force-released only after Zeb confirmed that session was closed

Charge

GitHub PAT expires in ~16 hours

medinachf-owner-loop-dispatch — regenerate and update every consumer

Mid

Rotation reported complete — and it wasn't

A backgrounded grep finished late and revealed Alloy on Railway still held the old token

Then

canvas-1's health URL had been 404ing since 2026-09-03

verify-deploy.mjs only failed on 5xx, so it passed with a checkmark every time

Two mechanisms built, both tested against real failure

4xx now fails deploys; credential consumers are enumerable from source

What shipped

Rotated GITHUB_OWNER_LOOP_DISPATCH_TOKEN across all 6 real consumers: products/medinaCHF, experiments/canvas-1, experiments/canvas-2, experiments/canvas-3, experiments/canvas-4 (Vercel Production) and products/alloy (Railway production)

Redeployed all 6 so the running deployments actually hold the new value, then verified the credential end-to-end by POSTing a real workflow_dispatch and confirming run 36314731679 appeared

Fixed experiments/canvas-1/proxy.ts — added /api/health to isPublicRoute, the exemption canvas-2/3/4 received on 2026-09-03 but canvas-1 never did

Tightened forgekit-os/scripts/verify-deploy.mjs to fail on 4xx (not only 5xx), with a per-target expectStatus escape hatch for targets that legitimately answer non-2xx

Recorded confirmed live statuses and the incident in forgekit-os/scripts/deploy-targets.config.mjs

Added forgekit-os/scripts/check-shared-credential-consumers.mjs + npm run check:credential-consumers — derives a shared credential's consumer set and deploy platforms from real process.env reads

Documented the rotation-enumeration procedure and the incident in .claude/rules/security.md