Skip to content

The index memory amplification asymptotes at 7.04x, and 4.0x needs 91.7 B/key removed - #2390

Merged
matrixarkai merged 1 commit into
mainfrom
index-memory-amplification-asymptote
Oct 30, 2026
Merged

matrixarkai merged 1 commit into
mainfrom
index-memory-amplification-asymptote

Conversation

@matrixarkai

@matrixarkai matrixarkai commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

the_amplification_set_in_the_published_format reports the index's space-in-memory quadrant at one
corpus. One corpus cannot say whether that figure is a property of the index or of the corpus — and
the_per_key_index_figure_is_a_sawtooth is why two corpora usually cannot either: a hashbrown
table's cost per key is a step function of n, so two arms in different phases differ by up to 2x
with no constant term to give it away.

This measures the quadrant at two arms exactly 4.0x apart, at a matched table phase, and
derives the figure a large corpus converges on.

The arms are phase-matched by construction

1,792 and 7,168 keys are both exactly 7/8 of a power of two — the most a 2,048- and an
8,192-bucket table hold — so both string tables sit at exactly 100% usable load, and they are
exactly 4.0x apart, the arm spacing this campaign uses. Both facts are asserted rather than arranged
and trusted: if hashbrown's growth policy moves, the capacity assertion fails and says the phase
match is gone.

It is also the steady state, not the fixture state: shipped 0..1023 routing range rather than
load_shard's whole keyspace, after a full dump round, with the shared Arc<str> key characters
added back in because deep_heap_bytes is a clone and Arc::clone bumps a refcount.

What it measures

    row                                       1x (1,792)  4x (7,168)  slope B/k
    strings                                        83984      335888       46.9
    bucket_index.bucket_map                       236520      533128       55.2
    bucket_index.object_block_lookup              140376      561792       78.4
    bucket_index, the rest of CoreIndex              364         364        0.0
    dirty_objects                                      0           0        0.0
    bucket_recency                                 34832       34832        0.0
    bucket-index Arc<str> payloads                 57368      229400       32.0

    buckets                                          906        1024
    encoded image                                  55567      217787
    TRUE resident                                 554532     1696492

    B/key resident      309.4       236.7
    amplification       9.980x     7.790x

    marginal: 212.4 B/key resident over 30.2 B/key encoded
    ASYMPTOTE: a large corpus on this structure converges on 7.04x

Two rows are confirmed per-bucket, not per-key: bucket_recency and the rest of CoreIndex
contribute identical bytes at both arms, so they are capped at 1,024 buckets and stop contributing
as the corpus grows. That saturation is why the ratio falls between the arms, which the arm
asserts — if it ever rises, something per-key grew superlinearly and the asymptote is not a bound.

What a 4.0x target requires

    at 30.2 encoded B/key, 4.0x allows 120.7 resident B/key
    the marginal cost is 212.4, so reaching it needs 91.7 B/key removed (43% of the per-key index)
    the rows still per-key at the steady state, largest first:
      bucket_index.object_block_lookup                78.4 B/key
      bucket_index.bucket_map                         55.2 B/key
      strings                                         46.9 B/key
      bucket-index Arc<str> payloads                  32.0 B/key

Only four rows survive to the steady state, and they sum to the marginal cost exactly. The arm
asserts the asymptote is still above 4.0x — if a change ever brings it under, that assertion
fails and says the goal is reached, rather than passing quietly.

4.0x is not reachable by any lever that is currently priced

Folding the per-key tables together is priced and it loses.
what_consolidating_three_holders_would_cost, re-run on this tree: full-overlap saving 29.5 B/key,
no-overlap cost 104.9 B/key, break-even at 78.0% key overlap — against a measured workload
overlap of 50.0%. Consolidation therefore increases resident footprint on this workload. I
expected the break-even to have moved, because ObjectSlot went 32 B → 24 B and the prior
derivation named that width as the thing that would invalidate it; it does not, because this arm's
union entry is built from ElementEntry (16 B), OwnedTimestampedSeries (8 B) and a 40 B selection
and never touches ObjectSlot. The number is unchanged at 78.0%.

And the one row large enough to matter on its own is not droppable. object_block_lookup is
78.4 B/key, 37% of the per-key index and 85% of what a 4.0x target needs. It is skip_serializing,
and several consumers branch on is_empty(), which together make it look like a derived cache
whose absence is handled. Checked rather than assumed, it is neither:

  • rebuild_object_block_lookup does not exist. It was removed when the page entry stopped
    restating the key it is filed under, so there is no function that can reconstruct this map from
    the bucket map. The keys reach it only as the stored bucket nodes decode.
  • The is_empty() arms are a degraded mode, not a fallback. In object_manager_stats
    (engine.rs:7665) the empty arm reports the object count and block-ref count as zero — its
    own comment says so: "an empty lookup names no position at all, so no page has an identity".
    Published metrics would read zero objects, not a recomputed figure.
  • Where it does fall back, it falls back to a sweep. target_buckets (engine.rs:6150)
    degrades from the object's own buckets to every bucket in the bucket map — up to 1,024 on the
    shipped range, each doing a retain over its block index.

So this is not the memory-for-recomputation trade I described when I opened this PR. Since the entry
stopped carrying its key, this map is the sole resident holder of page→object identity, and the
78.4 B/key is not recoverable by removing it. It could only be made cheaper by a narrower
representation of that identity.

That closes the arithmetic. No remaining row is individually large enough (55.2, 46.9 and 32.0
all fall short of 91.7), and the largest is load-bearing. Reaching 4.0x requires changing how
page→object identity is represented — which is a specific, well-posed piece of work, and not one
this PR takes on. What this PR establishes is the target, the asymptote, and which rows the target
has to come out of.

Checks

Three arms green: --lib, --all-targets, --all-targets --features alloc-probe. CI here does not
run the library suite, so the arms were run locally by name, and
index_bytes_per_key ran serially at 12 passed; 0 failed.

….7 B/key

The published space-in-memory quadrant is read at one corpus, and one corpus
cannot say whether the figure is a property of the index or of the corpus.
Two corpora usually cannot either: the per-key cost is a step function of n,
so two arms in different phases differ by up to 2x with no constant term to
give it away.

So the two arms here are phase-matched by construction. 1,792 and 7,168 keys
are both exactly 7/8 of a power of two -- the most a 2,048- and an
8,192-bucket table hold -- so both string tables sit at exactly 100% usable
load, and they are exactly 4.0x apart. Both facts are asserted, so a change
to hashbrown's growth policy fails the arm instead of silently putting the
arms in different phases. Measured on the shipped 0..1023 range, after a full
dump round, with the shared Arc<str> key characters added back in.

    B/key resident      309.4       236.7
    amplification       9.980x     7.790x
    marginal: 212.4 B/key resident over 30.2 B/key encoded
    ASYMPTOTE: 7.04x

bucket_recency and the rest of CoreIndex contribute identical bytes at both
arms, confirming they are per-bucket and capped at 1,024 buckets. That
saturation is why the ratio falls between the arms, which the arm asserts: if
it rises, something per-key grew superlinearly and the asymptote is not a
bound.

At 30.2 encoded B/key a 4.0x target allows 120.7 resident B/key, so reaching
it needs 91.7 B/key removed, 43% of the per-key index. Four rows survive to
the steady state and sum to the marginal cost exactly:
object_block_lookup 78.4, bucket_map 55.2, strings 46.9, key characters 32.0.
The arm asserts the asymptote is still above 4.0x, so a change that reaches
the target fails it and says so rather than passing quietly.

Two levers, for the record. Folding the per-key tables together is priced and
LOSES: break-even is 78.0% key overlap against a measured workload overlap of
50.0%, so it increases resident footprint here. I expected that number to
have moved, because ObjectSlot went 32 B to 24 B and the prior derivation
named that width as what would invalidate it -- it does not, because the
union entry is built from ElementEntry, OwnedTimestampedSeries and a 40 B
selection and never touches ObjectSlot.

The only row big enough to matter alone is object_block_lookup, 85% of what
the target needs. It is skip_serializing and rebuilt from the bucket map, and
many consumers already carry an is_empty() fallback arm. But it is wired at
294 sites with eleven production insert callers and reads across six modules,
and it exists to avoid O(total pages) rebuild walks -- so it is a
memory-for-recomputation trade whose recompute half is not measured. This
establishes the resident half only; it does not make that call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@matrixarkai
matrixarkai force-pushed the index-memory-amplification-asymptote branch from 3f6e218 to 66ddf90 Compare October 30, 2026 03:49
@matrixarkai
matrixarkai merged commit fb1aba8 into main Oct 30, 2026
7 checks passed
@matrixarkai
matrixarkai deleted the index-memory-amplification-asymptote branch October 30, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants