Skip to content

Apply each sync Change in one task at a time per channel and user - #6424

Open
rtibblesbot wants to merge 3 commits into
learningequality:hotfixesfrom
rtibblesbot:issue-6423-e948e7
Open

rtibblesbot wants to merge 3 commits into
learningequality:hotfixesfrom
rtibblesbot:issue-6423-e948e7

Conversation

@rtibblesbot

@rtibblesbot rtibblesbot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Requeued tasks are signed with their full kwargs, so sync finds them instead of enqueuing a duplicate.
  • Channel and user change tasks hold a per-channel or per-user session advisory lock.

References

Fixes #6423. Fixes STUDIO-J4J.

Reviewer guidance

Suggested review: Full code review.

Engineer review

  • The issue guessed a frontend double-send to dedupe on session + client_rev; the branch makes change tasks exclusive per channel/user instead, since Sentry shows concurrent tasks.
  • Session advisory locks need workers to reach production Postgres without a transaction-mode pooler (pgbouncer) — contentcuration/contentcuration/db/advisory_lock.py:109.
  • A change task that finds its lock busy exits; only the holder's post-unlock exists() requeues that work — contentcuration/contentcuration/tasks.py:51.
  • apply_user_changes takes a per-user lock too, though Sentry shows no events for it — contentcuration/contentcuration/tasks.py:64.
Engineer evidence
  • Session advisory locks leak across clients behind a transaction-mode pooler such as pgbouncer; does production Postgres sit behind one?
  • A task finding the lock busy exits without touching Changes; only the holder's post-unlock exists() check requeues them. Is exiting rather than waiting wanted?
  • apply_user_changes takes a per-user lock too, though Sentry shows no failures for it; keep it?

AI usage

Used Claude Code to trace the duplicate tasks from Sentry events and write the fix and tests. Verified with the full Python test suite and a real-Postgres lock contention test.


Written by rtibblesbot, an LLM-based coding agent.

@rtibblesbot

🟡 Waiting for feedback

WAITING · for review
Last updated: 2026-11-18 03:14 UTC

@rtibblesbot
rtibblesbot marked this pull request as ready for review November 16, 2026 18:32
@rtibblesbot
rtibblesbot requested review from a team and leq-bot November 16, 2026 18:32
rtibblesbot and others added 3 commits November 17, 2026 19:13
Requeue built the signature from override kwargs only, so the requeued
task's signature didn't match its kwargs and deduplication missed it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Holds a non-blocking session-level lock across the transactions
committed inside it, releasing it on exit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Concurrent apply_*_changes tasks for the same channel or user applied
the same changes twice, raising IntegrityError on duplicate creates.
A task that can't take the lock exits; the holder requeues for changes
that arrive before it releases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant