Skip to content

A mirror pair cannot leave the published population unnoticed - #2392

Merged
matrixarkai merged 1 commit into
mainfrom
guard/a-mirror-pair-cannot-leave-the-published-population-unnoticed
Oct 30, 2026
Merged

matrixarkai merged 1 commit into
mainfrom
guard/a-mirror-pair-cannot-leave-the-published-population-unnoticed

Conversation

@matrixarkai

Copy link
Copy Markdown
Owner

page and block name one storage tier in this crate, block is the canonical
spelling, and the published page_* statistics are a mirror of the block_* ones.
every_page_named_shard_stat_agrees_with_its_block_twin is what holds that mirror
together. It could be switched off by shrinking the population it checks.

The defect

The test built its groups like this:

groups.retain(|_, members| members.len() >= 2);

and then asserted the group COUNT:

assert_eq!(groups.len(), 3, "expected three mirror pairs on the canonical shard stats");

Retire one spelling of a pair and the group falls below two members, so it is dropped
from the map entirely. The count then no longer matches, and the obvious repair is to
lower the number. The moment the number is lowered, nothing in the file compares the two
spellings of that pair any more, and every assertion is green. The count moved and the
coverage went with it — and no failing test anywhere says so.

It also could not tell the two halves of a retirement apart. Dropping block_reads and
keeping page_reads lowers the count by exactly as much as dropping page_reads does,
and leaves the tier published only under the word this crate does not use.

What replaces it

The filter is gone, so a one-member group is still a group, and the population is
DECLARED rather than measured:

  • TWINNED_ON_SHARD_STATS — the canonical names published under both spellings, with
    the exact pair each one must carry.
  • SINGLE_ON_SHARD_STATS — the canonical names that are down to one spelling, with the
    spelling that survived. Empty today, and every clause runs over it, so the first
    retirement adds one line and writes no new machinery.

Retiring a spelling means moving a name between those two lists. That move is the
reviewable artefact, and five clauses refuse every other way of shrinking the
population:

  1. the rosters are well formed — every listed spelling canonicalises to the name it is
    listed under, no name sits on both lists, and a retired name's survivor is the block
    spelling;
  2. the set of canonical names published under two or more spellings IS the twinned
    roster, in both directions, so a pair that leaves without a roster move fails, and a
    pair that arrives unnoticed fails too;
  3. each declared pair is present under exactly the two spellings declared;
  4. each retired name is present under exactly its surviving spelling;
  5. the spellings of each declared pair hold one value, and that value is not zero on
    every spelling.

The same defect sat on the published contract test, which asserted that two
page-spelled keys exist by counting them. The count becomes a named set,
PAGE_SPELLED_CONTRACT_KEYS, for the same reason: a 2 can be written as a 1 without
saying which key left.

Three facts that were prose are now scanned for

The module header used to rest on five type-level facts, and three of its citations named
things that had already stopped existing: PageIndexNamedBy and PageEntryAt had been
renamed to BlockIndexNamedBy and BlockEntryAt, and PAGE_SIZE and page_ordinal were
only ever named in order to be denied. Each of those four tokens occurred exactly ONCE in
this crate, and the occurrence was the header itself. #2386 removed that prose, which
leaves the facts unstated rather than untrue.

no_page_named_identity_type_field_or_size_constant_exists_in_this_crate scans the
crate's src for the three shapes instead: a type declared with a name beginning Page,
a page_id or page_ordinal FIELD, and a const/static PAGE_SIZE. Each matcher
requires the line to BE a declaration, so a doc line, a serde alias and a JSON key inside
a fixture string cannot match and no path has to be excluded by hand — including this
file, which names all three shapes in its own comments.

Five non-vacuity clauses, because a scan that finds nothing looks the same as a clean
tree: the walk reaches a floor of files and recurses, each matcher is exercised on a line
it must report and on a line it must not, and the walk is required to find the Block
spelling of two of the three shapes, so a clean page-named result is a fact about the
spelling rather than about the scan. It reports 461 files scanned, 60 Block-named type
declarations and 53 block_id fields.

StoragePageAddressSample and StoragePageIndexEntrySample do not begin with Page and
are deliberately untouched: they are report projections, not identities.

What does not move

No published key, no stored key, no format stamp and no value. The three mirror pairs on
ShardCanonicalStorageStats are exactly the three that were there before:
block_index_entries/page_index_entries at 20, block_reads/page_reads at 10 and
block_writes/page_writes at 10.

The plants

Six, each against one clause, run in release and reverted:

plant what it does what fails
retire page_reads with #[serde(skip)] the exact disarmament the old shape allowed clause 2, naming block_reads as the pair that left
drop block_writes from the twinned roster a published pair nothing declares clause 2, naming block_writes as the pair that arrived
record block_reads as retired while it is also twinned both rosters at once clause 1, "it is either twinned or down to one spelling"
record a survivor that is the page spelling retiring the wrong half clause 1, "which is not the block spelling"
record a retirement for a name that is not published a roster entry about nothing clause 4, "retiring a twin publishes one name, not none"
declare pub struct PageOrdinalIdentity { page_id: u64 } and const PAGE_SIZE in engine/lifecycle.rs all three structural shapes the scan, naming the file and line of each

And a positive arm: page_reads retired AND block_reads moved to SINGLE_ON_SHARD_STATS
PASSES, printing mirror block_reads: block_reads=10 — a one-member group still named and
still checked. So the designed repair is a repair, and it is the only one.

Validation

cargo check -p temporalstore-rust --all-targets
cargo test --release -p temporalstore-rust --lib data_node::tests::part3 -- --test-threads=1
cargo test --release -p temporalstore-rust --lib page_and_block_name_one_tier -- --test-threads=1
cargo test --release -p temporalstore-rust --lib block_entry_names -- --test-threads=1
cargo test --release -p temporalstore-rust --lib page_and_block_counter_ratio -- --test-threads=1

🤖 Generated with Claude Code

`every_page_named_shard_stat_agrees_with_its_block_twin` built its groups with
`groups.retain(|_, members| members.len() >= 2)` and then asserted the group COUNT.
Retiring one spelling of a pair drops the group below two members, the count stops
matching, and the obvious repair is to lower the number -- at which point nothing
compares that pair's two spellings any more and every assertion in the file is green.
The count moved and the coverage went with it.

It also could not tell the two halves of a retirement apart: dropping `block_reads` and
keeping `page_reads` lowers the count by exactly as much as dropping `page_reads` does.

The filter is gone, so a one-member group is still a group, and the population is
declared by TWINNED_ON_SHARD_STATS and SINGLE_ON_SHARD_STATS. Retiring a spelling means
moving a name between those two lists, and five clauses refuse every other way of
shrinking the population: the rosters are well formed and a retired name's survivor is
the block spelling; the set of names published under two or more spellings IS the twinned
roster, in both directions; each declared pair carries exactly its two spellings; each
retired name carries exactly its survivor; and the pair agrees and is non-zero.

The published contract test had the same defect -- it counted its two page-spelled keys --
so the count becomes the named set PAGE_SPELLED_CONTRACT_KEYS.

Three facts the module header used to carry as prose are now scanned for instead:
no type declared with a name beginning `Page`, no `page_id` or `page_ordinal` FIELD, and
no `const`/`static PAGE_SIZE`. Each matcher requires the line to BE a declaration, so a
doc line, a serde alias and a JSON key inside a fixture string cannot match and no path
needs excluding by hand. Five non-vacuity clauses: the walk reaches a file floor and
recurses, each matcher is exercised in both directions, and the walk is required to find
the Block spelling of two of the three shapes.

No published key, no stored key, no format stamp and no value moves.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@matrixarkai
matrixarkai merged commit 6e8380b into main Oct 30, 2026
7 checks passed
@matrixarkai
matrixarkai deleted the guard/a-mirror-pair-cannot-leave-the-published-population-unnoticed branch October 30, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants