Repository navigation
The index memory amplification asymptotes at 7.04x, and 4.0x needs 91.7 B/key removed - #2390
Merged
Merged
Conversation
….7 B/key
The published space-in-memory quadrant is read at one corpus, and one corpus
cannot say whether the figure is a property of the index or of the corpus.
Two corpora usually cannot either: the per-key cost is a step function of n,
so two arms in different phases differ by up to 2x with no constant term to
give it away.
So the two arms here are phase-matched by construction. 1,792 and 7,168 keys
are both exactly 7/8 of a power of two -- the most a 2,048- and an
8,192-bucket table hold -- so both string tables sit at exactly 100% usable
load, and they are exactly 4.0x apart. Both facts are asserted, so a change
to hashbrown's growth policy fails the arm instead of silently putting the
arms in different phases. Measured on the shipped 0..1023 range, after a full
dump round, with the shared Arc<str> key characters added back in.
B/key resident 309.4 236.7
amplification 9.980x 7.790x
marginal: 212.4 B/key resident over 30.2 B/key encoded
ASYMPTOTE: 7.04x
bucket_recency and the rest of CoreIndex contribute identical bytes at both
arms, confirming they are per-bucket and capped at 1,024 buckets. That
saturation is why the ratio falls between the arms, which the arm asserts: if
it rises, something per-key grew superlinearly and the asymptote is not a
bound.
At 30.2 encoded B/key a 4.0x target allows 120.7 resident B/key, so reaching
it needs 91.7 B/key removed, 43% of the per-key index. Four rows survive to
the steady state and sum to the marginal cost exactly:
object_block_lookup 78.4, bucket_map 55.2, strings 46.9, key characters 32.0.
The arm asserts the asymptote is still above 4.0x, so a change that reaches
the target fails it and says so rather than passing quietly.
Two levers, for the record. Folding the per-key tables together is priced and
LOSES: break-even is 78.0% key overlap against a measured workload overlap of
50.0%, so it increases resident footprint here. I expected that number to
have moved, because ObjectSlot went 32 B to 24 B and the prior derivation
named that width as what would invalidate it -- it does not, because the
union entry is built from ElementEntry, OwnedTimestampedSeries and a 40 B
selection and never touches ObjectSlot.
The only row big enough to matter alone is object_block_lookup, 85% of what
the target needs. It is skip_serializing and rebuilt from the bucket map, and
many consumers already carry an is_empty() fallback arm. But it is wired at
294 sites with eleven production insert callers and reads across six modules,
and it exists to avoid O(total pages) rebuild walks -- so it is a
memory-for-recomputation trade whose recompute half is not measured. This
establishes the resident half only; it does not make that call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
matrixarkai
force-pushed
the
index-memory-amplification-asymptote
branch
from
October 30, 2026 03:49
3f6e218 to
66ddf90
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
the_amplification_set_in_the_published_formatreports the index's space-in-memory quadrant at onecorpus. One corpus cannot say whether that figure is a property of the index or of the corpus — and
the_per_key_index_figure_is_a_sawtoothis why two corpora usually cannot either: a hashbrowntable's cost per key is a step function of n, so two arms in different phases differ by up to 2x
with no constant term to give it away.
This measures the quadrant at two arms exactly 4.0x apart, at a matched table phase, and
derives the figure a large corpus converges on.
The arms are phase-matched by construction
1,792 and 7,168 keys are both exactly 7/8 of a power of two — the most a 2,048- and an
8,192-bucket table hold — so both string tables sit at exactly 100% usable load, and they are
exactly 4.0x apart, the arm spacing this campaign uses. Both facts are asserted rather than arranged
and trusted: if hashbrown's growth policy moves, the capacity assertion fails and says the phase
match is gone.
It is also the steady state, not the fixture state: shipped
0..1023routing range rather thanload_shard's whole keyspace, after a full dump round, with the sharedArc<str>key charactersadded back in because
deep_heap_bytesis a clone andArc::clonebumps a refcount.What it measures
Two rows are confirmed per-bucket, not per-key:
bucket_recencyand the rest ofCoreIndexcontribute identical bytes at both arms, so they are capped at 1,024 buckets and stop contributing
as the corpus grows. That saturation is why the ratio falls between the arms, which the arm
asserts — if it ever rises, something per-key grew superlinearly and the asymptote is not a bound.
What a 4.0x target requires
Only four rows survive to the steady state, and they sum to the marginal cost exactly. The arm
asserts the asymptote is still above 4.0x — if a change ever brings it under, that assertion
fails and says the goal is reached, rather than passing quietly.
4.0x is not reachable by any lever that is currently priced
Folding the per-key tables together is priced and it loses.
what_consolidating_three_holders_would_cost, re-run on this tree: full-overlap saving 29.5 B/key,no-overlap cost 104.9 B/key, break-even at 78.0% key overlap — against a measured workload
overlap of 50.0%. Consolidation therefore increases resident footprint on this workload. I
expected the break-even to have moved, because
ObjectSlotwent 32 B → 24 B and the priorderivation named that width as the thing that would invalidate it; it does not, because this arm's
union entry is built from
ElementEntry(16 B),OwnedTimestampedSeries(8 B) and a 40 B selectionand never touches
ObjectSlot. The number is unchanged at 78.0%.And the one row large enough to matter on its own is not droppable.
object_block_lookupis78.4 B/key, 37% of the per-key index and 85% of what a 4.0x target needs. It is
skip_serializing,and several consumers branch on
is_empty(), which together make it look like a derived cachewhose absence is handled. Checked rather than assumed, it is neither:
rebuild_object_block_lookupdoes not exist. It was removed when the page entry stoppedrestating the key it is filed under, so there is no function that can reconstruct this map from
the bucket map. The keys reach it only as the stored bucket nodes decode.
is_empty()arms are a degraded mode, not a fallback. Inobject_manager_stats(
engine.rs:7665) the empty arm reports the object count and block-ref count as zero — itsown comment says so: "an empty lookup names no position at all, so no page has an identity".
Published metrics would read zero objects, not a recomputed figure.
target_buckets(engine.rs:6150)degrades from the object's own buckets to every bucket in the bucket map — up to 1,024 on the
shipped range, each doing a
retainover its block index.So this is not the memory-for-recomputation trade I described when I opened this PR. Since the entry
stopped carrying its key, this map is the sole resident holder of page→object identity, and the
78.4 B/key is not recoverable by removing it. It could only be made cheaper by a narrower
representation of that identity.
That closes the arithmetic. No remaining row is individually large enough (55.2, 46.9 and 32.0
all fall short of 91.7), and the largest is load-bearing. Reaching 4.0x requires changing how
page→object identity is represented — which is a specific, well-posed piece of work, and not one
this PR takes on. What this PR establishes is the target, the asymptote, and which rows the target
has to come out of.
Checks
Three arms green:
--lib,--all-targets,--all-targets --features alloc-probe. CI here does notrun the library suite, so the arms were run locally by name, and
index_bytes_per_keyran serially at12 passed; 0 failed.