Redis Consumer Groups and a Real-Time Notification Feed

A Mailroom That Never Drops a Letter

The Deliver-and-Acknowledge Cycle

A consumer group tracks a Pending Entries List (the PEL). When a worker reads a new message with >, Redis marks it delivered and puts it in the PEL. The message stays there until the worker sends XACK. Only then is it considered done.

A message is delivered to one worker and held in the pending list. It is removed only after the worker acknowledges it, so a crash never loses work.

# create the group, starting from new messages ($). MKSTREAM makes the stream if needed
XGROUP CREATE notifications g1 $ MKSTREAM

# a worker reads NEW messages ( > ), up to 10, waiting up to 5s
XREADGROUP GROUP g1 worker-1 COUNT 10 BLOCK 5000 STREAMS notifications >

# after the work is done, acknowledge so it leaves the pending list
XACK notifications g1 1762-0

# see what is still pending for the group
XPENDING notifications g1

When a Worker Crashes — Claim and Recover

A worker can die mid-message. Its message is still in the pending list, owned by the dead worker, its idle time growing. Another worker notices and takes it over with XAUTOCLAIM (or XCLAIM). Nothing is lost.

worker-1 crashed holding 1762-0. It has been idle too long, so worker-2 claims it from the pending list and finishes the job.

# find messages idle longer than 60s and hand them to worker-2
XAUTOCLAIM notifications g1 worker-2 60000 0

# or claim one specific message by ID
XCLAIM notifications g1 worker-2 60000 1762-0

# inspect pending details: how many times each was delivered
XPENDING notifications g1 - + 10

Building a Real-Time Notification Feed

Now let us use these pieces. A notification feed must do two things at once: store every notification so a user can see it later, and push it live to a user who is online right now. We use a stream for storage and Pub/Sub for the live ping.

One event writes to the user's stream (durable) and fires a live ping. Online users get it instantly; offline users read the stored feed when they return.

The Feed Data Model and Commands

Keep one stream per user for their notifications, a counter for unread, and a Pub/Sub channel for the live ping. Reading the feed is a reverse range; marking it read resets the counter.

# 1) a new notification for user 88 (store it, durably)
XADD notif:user:88 MAXLEN ~ 200 * type "like" actor 57 post 9001

# 2) bump the unread badge
INCR unread:user:88

# 3) ping any live client for this user
PUBLISH user:88 "new"

# --- when the user opens their notifications ---

# newest 20 notifications, newest first
XREVRANGE notif:user:88 + - COUNT 20

# mark all as read: reset the unread badge to 0
SET unread:user:88 0

# show the badge number any time
GET unread:user:88

Putting It Together

Here is the whole flow when Asha comments on Ben's post. The comment is saved in your database as usual; the notification uses Redis for speed and live delivery.

# after saving the comment in your main database ...

# build Ben's notification (user 88), keep his feed capped
XADD notif:user:88 MAXLEN ~ 200 * type "comment" actor 57 post 9001 text "Nice shot!"
INCR unread:user:88
PUBLISH user:88 "new"

# Ben's phone is online: its WebSocket server is subscribed and pushes at once
# Ben's laptop is offline: next time it opens, it calls XREVRANGE and sees the comment

Golden Rules