You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 858303f
Browse filesBrowse the repository at this point in the historyBrowse files
perf: cheaper per-message consume and publish paths (#7)
**Stack** (merge in order; each PR is based on the previous one):
1. #2 fix(consumer): wake the dispatcher and the event pump immediately
when stopping
2. #3 perf(consumer): free handler slots when settlement starts and
bound the settling backlog
3. #4 perf(consumer): batch acks and retries (full batches at once, 5 ms
linger)
4. #5 feat(consumer): full receive batches (min_receive_batch,
receive_batch_wait, default prefetch)
5. #6 feat!: lean defaults - raw envelope, opt-in CloudEvents and MD5
checks, immediate stop
6. #7 perf: cheaper per-message consume and publish paths 👈 this PR
---
Part 6 of 6 of the **fast defaults** work for 0.2.0 (overview in #2),
driven by profiles of the
consume and publish paths (single CPU, py-spy and cProfile).
- Settlement awaitables are resolved in batches (`runtime::Completions`:
one interpreter attach and
one `call_soon_threadsafe` for everything that completed meanwhile) and
run as tracked engine
tasks (`close()` waits for them; cancelling the awaitable no longer
stops a started settlement).
- Automatic acks are not awaited per message:
`Delivery.ack_in_background()` queues the delete and
the settle loop finishes it (failures: `message.ack_failed` events and
metrics).
- Acks are queued synchronously when a settling slot is free, so they
reach the settle loop in
request order; the send-at-once rule now counts the awaited settlements
in a batch
(`mark_awaited`), which keeps explicit-ack batches full.
- Python: cached event enablement and log targets, cached type names and
encoders (built-in codecs
only), no route-list copies, guarded event emission, telemetry fast
paths without spans.
- Rust: decoding no longer copies messages; interned dict keys.
**Tests:** `a_batch_every_handler_waits_for_is_sent_at_once`,
`background_acks_are_tracked_until_close`,
`background_acks_wait_for_a_settling_slot` (Rust);
`test_stop_completes_automatic_acks`, handlers do
not wait for automatic acks, encoder cache, event enablement, routing
caches.
**Measured** (fantest, all scenarios, the whole stack against 0.1.0; all
360 trials independently
verified: every message delivered exactly once with an identical body,
every consumed message
deleted):
| | 0.1.0 defaults | 0.1.0 tuned | defaults | auto-ack | tuned config |
|---|---:|---:|---:|---:|---:|
| consume, 1 poller: client CPU µs/msg | 171.5 | 85.7 | 64.2 | 44.8 |
50.4 |
| consume, 8 pollers: client CPU µs/msg | 126.4 | 93.9 | 60.4 | 52.6 |
54.8 |
| consume, 1 poller: MiniStack CPU µs/msg | 335 | 130 | 135 | 125 | 135
|
| messages per receive (1 / 8 pollers) | 2.9 / 3.8 | 10.0 / 9.6 | 10.0 /
9.6 | 10.0 / 9.6 | 10.0 / 9.6 |
| publish, 1 caller: client CPU µs/msg | 221.8 | 217.6 | 186.9 | 196.2 |
176.7 |
| publish_batch ×10, 1 caller: client CPU µs/msg | 45.5 | 35.0 | 37.0 |
38.3 | 30.5 |
| publish_batch ×10, 16 callers: client CPU µs/msg | 45.8 | 32.1 | 36.0
| 35.9 | 29.3 |
| teardown ms | 528 | 534 | 10 | 11 | 10 |
| threads (max) | 49 | 93 | 15 | 14 | 17 |
Compared with #6: consuming costs 119.7 → 64.2 µs/msg with `await
message.ack()` and 102.7 → 44.8
µs/msg with returning handlers; publishing 220.9 → 186.9 (single) and
41.6 → 37.0 µs/msg (batch). A
probe with this build (2,000 messages, 1 poller, MiniStack) sends 200
`DeleteMessageBatch` calls with
either handler style, i.e. 100 per 1,000 messages.
Against the plan's acceptance checks (targets measured on a slower
machine):
| check | target | result |
|---|---|---|
| messages per receive, 1 and 8 pollers | ≥ 9.5 | 10.0 / 9.6 |
| client CPU per consumed message, 1 poller | ≤ 117 µs, goal ≤ 80 µs |
64 µs (`await message.ack()`), 45 µs (auto-ack) |
| MiniStack CPU per consumed message, 1 poller | ≈ tuned | 135 µs (tuned
130, 0.1.0 defaults 335) |
| `DeleteMessageBatch` calls per 1,000 messages | ≤ 120 | 100 |
| publish CPU per message | within 10 % of tuned | single −14 %; batch
+6 % (16 callers +12 %: the defaults JSON-encode `dict`s, tuned sends
bytes) |
| `consumer.stop()` with no running handlers | < 50 ms | 10 ms process
teardown |
<sub>Benchmark: the fantest SNS/SQS client benchmark (not part of this
repo) against MiniStack 1.5.27, run without Docker on an OrbStack VM on
Apple Silicon (MiniStack pinned to 2 vCPUs, client to 4, CPython 3.13,
`--profile dist` builds). Every run measured 0.1.0 next to the build
under test; medians over 3 passes × 3 rounds of 2,000 consumed messages.
*defaults*: the build's defaults with handlers that `await
message.ack()` (the harness's drivers); *auto-ack*: the same with
returning handlers; *tuned*: the harness's hand-tuned configuration
(pre-encoded bytes, no attributes, 50 slots per poller). Across runs,
0.1.0 tuned stayed within 85.7-89.4 µs per consumed message, so the runs
compare.</sub>
**Checked locally on this commit:** cargo fmt, clippy `-D warnings`,
ruff, pyrefly,
`scripts/check_consistency.py`, 147 Rust tests plus the 4 Rust e2e tests
against MiniStack, 223 pytest
tests (memory broker and MiniStack) on Python 3.12, 3.13, 3.14 and 3.15.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
0 commit comments