Unhandled rejection in status fanout timer can crash the controller #66

Closed
opened 2026-10-08 16:33:29 -04:00 by goddard · 1 comment
Collaborator

setInterval(fanoutStatus, …) and void fanoutStatus() (worker.ts:278,280) drop a promise that rejects when the snapshot query throws. Nothing handles unhandledRejection, so Node 22 exits, and a full disk becomes a crash loop.

Fix

  • Add .catch at both call sites
  • Add a process-level unhandledRejection handler that logs
`setInterval(fanoutStatus, …)` and `void fanoutStatus()` (`worker.ts:278,280`) drop a promise that rejects when the snapshot query throws. Nothing handles `unhandledRejection`, so Node 22 exits, and a full disk becomes a crash loop. **Fix** - [x] Add `.catch` at both call sites - [x] Add a process-level `unhandledRejection` handler that logs
goddard added the
priority
medium
type/bug
labels 2026-10-08 16:39:11 -04:00
josh was assigned by goddard 2026-10-08 17:31:46 -04:00
Author
Collaborator

Reviewed 96d21d0514. Both fix items are done, so I'm closing this.

Checklist

  • .catch at both call sites. The startup call and the interval callback now catch and log a failed status fanout (worker.ts:279,281-282). fanoutStatus is async, so a query that throws synchronously inside it still becomes a caught rejection. The new worker-scheduling test proves this for both the startup and the scheduled call, and that the next interval runs again.
  • Process-level logger. hooks.server.ts:16 registers an unhandledRejection handler that logs. After #65's change it's guarded so it registers only once, even when boot is retried (hooks.server.ts:19). The new hooks-server.test.ts checks it's registered once, before the workers start.

Review notes

  • The 5 affected test files pass locally (105 tests), the full suite passes (208 files, 2,625 tests), and npm run check reports 0 errors.
Reviewed 96d21d0514ccddccacbbf8619df01fb050f6e472. Both fix items are done, so I'm closing this. **Checklist** - [x] **`.catch` at both call sites.** The startup call and the interval callback now catch and log a failed status fanout (`worker.ts:279,281-282`). `fanoutStatus` is async, so a query that throws synchronously inside it still becomes a caught rejection. The new worker-scheduling test proves this for both the startup and the scheduled call, and that the next interval runs again. - [x] **Process-level logger.** `hooks.server.ts:16` registers an `unhandledRejection` handler that logs. After #65's change it's guarded so it registers only once, even when boot is retried (`hooks.server.ts:19`). The new `hooks-server.test.ts` checks it's registered once, before the workers start. **Review notes** - The 5 affected test files pass locally (105 tests), the full suite passes (208 files, 2,625 tests), and `npm run check` reports 0 errors.
Sign in to join this conversation.