Rust Task Runner

My deep-dive into Rust: a CLI worker that claims jobs from a PostgreSQL-backed queue using SKIP LOCKED and executes them across a configurable rayon thread pool. Roughly twenty times faster than the PHP worker it was built to replace.

Highlights

20× throughput vs the PHP worker it replaces
SKIP LOCKED-based job claiming — no lock contention
Graceful shutdown drains in-flight jobs before exit
Structured tracing output with per-job spans

Why Rust

PHP is superb for web request cycles, but long-running CPU-bound batch work is exactly where its shared-nothing model starts to cost money. I wanted first-hand experience with ownership, async runtimes and fearless concurrency — and a real tool at the end of it.

Design notes

Jobs are claimed in batches with FOR UPDATE SKIP LOCKED, then fanned out to a rayon pool. Failures increment an attempts column with exponential backoff computed in SQL, so retry logic survives crashes without external state.

Single static binary — deploy is scp and done
Connection pooling via deadpool-postgres
Integration-tested against ephemeral Postgres containers