Skip to content

Redis consumer does not recover unacknowledged events after worker failure #351

Open

Description

@fkautz

Problem

The Redis subscription reads only new events using XREADGROUP with >. There is no pending-entry read or claim path. Events delivered to a worker before it exits without acknowledging them remain assigned to that worker.

Reproduction

Create a consumer group, save a task, and read its event as dead-worker without acknowledging it. Close that subscription, start replacement in the same group, and publish another task.

Observed behavior

The replacement processes the fresh event but does not recover the stranded event. Redis XPENDING still shows the original event owned by dead-worker with RetryCount=1.

Expected behavior

Replacement workers should recover eligible unacknowledged events with a bounded recovery policy.

Executable regression test

TestReviewSubscriptionRecoversUnacknowledgedEvents (includes fixture in the same file).

git clone --branch test/review-regressions https://github.com/fkautz/ax.git ax-review
cd ax-review
git checkout 1dc2613d29e12f306154a7e1183c1825a55d24ed
go test ./internal/store/redis -run '^TestReviewSubscriptionRecoversUnacknowledgedEvents$' -count=1 -v

Requires the Go toolchain specified by go.mod and redis-server on PATH. The test asserts the desired behavior and fails on the published revision. Production code is unchanged.

The regression launches an isolated real redis-server over a private Unix socket. Closing the subscription without Ack models worker loss after delivery. The test allows a 300 ms recovery observation window; this alone is not proof of infinite delay. The production read path has no recovery operation, which explains the stranded pending entry. A future intentional claim delay should be reflected in the test.

Activity

  1. arbazkhan971 commented on Sep 22, 2026

    @arbazkhan971

    Fix ready: implemented on my fork (branch fix/351-recover-unacknowledged-events, regression test included, make test green) and ready to PR — but this repo's collaborators_only PR creation policy rejects PRs from non-collaborators (see #358), so I cannot open it. If PR creation opens for external contributors, I will open the PR immediately. Happy to adjust scope per maintainer feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions