Skip to content

We replaced a cloud queue with Redis, and mostly gained

Distributed systems · Queues · Architecture

We had two services that talked to each other through a managed cloud message queue. It worked. But the two services live in the same cluster, a few milliseconds apart on the network — and we were round-tripping their messages out to a cloud region and back, paying for the privilege, and making every local dev setup depend on cloud credentials. So we moved the queue in-house, onto Redis.

What we gained

The obvious wins landed immediately. Latency dropped — in-cluster messaging instead of a cloud round trip. Cost went to roughly zero, because it’s just Redis we already run. And local development got simpler: spin up Redis and the whole pipeline works on a laptop, no cloud account required, no surprise bill from an integration test that left a queue draining over the weekend.

What we had to pay back

Here’s the part worth writing down, because it’s the part you don’t see in the “just use Redis” blog posts. A managed cloud queue is doing a lot of quiet work on your behalf, and when you self-host you inherit all of it:

  • Retries and visibility timeouts. The cloud queue re-delivers a message if you don’t acknowledge it in time. You have to rebuild that — a job that’s picked up but not completed has to become visible again, without being lost or double-run.
  • A dead-letter path. Messages that fail repeatedly need somewhere to go besides “infinite retry loop.” The cloud gave us a dead-letter queue for free; now it’s our responsibility.
  • Backpressure and ordering. What happens when the consumer falls behind? What ordering guarantees do you actually need — and which ones were you silently relying on without realising?

We used a mature Redis-backed job-queue library rather than hand-rolling this, which covers most of it. But the thinking still has to happen: you have to decide, explicitly, what delivery and ordering semantics you need, because nobody is providing them by default anymore.

The takeaway

Self-hosting a queue is a good trade when the producer and consumer are close, the volume is yours to handle, and you value simplicity and cost over managed guarantees. Just go in clear-eyed that “the queue” is not one feature — it’s a bundle of retry, dead-lettering, visibility, and ordering semantics, and moving it in-house means owning every item in that bundle on purpose.