Repository navigation
Conversation
…nerator Takes the values generate_params.py derives for 300 clients, 16 servers, 16 KiB messages, 2^-30 decryption and 2^-20 decoding failure: Gaussian width 6.70, leaf commitment over p = 57637 with beta = 39 and kappa = 6, Chipmunk tree over q = 557057 with base 769 (two digits), and an IBLT shape chosen per capacity from the paper's exact stopping-set bound instead of the fixed 4 x 3rho/4 layout. Chipmunk is pointed at a local checkout until its fork is published.
The paper needs t prime and above the capacity, so the peeling table and the Newton-Girard sketch now share t = 2^31 + 11 and chunk messages into 31-bit symbols (a bit-level chunker replaces the 4-byte packing in the channel and the slot codec). The peeling decoder re-hashes each pure cell's key before trusting it and stops at the table's capacity, as the paper's decode does, so a forged table fails instead of looping or yielding garbage. Decoded symbols are canonical in [0, t).
The paper needs q prime, and with sigma = 6.70 the ring-LWE hardness check fails at the old 2^48 composite modulus anyway. q is now the generator's 79718129102849 (2^46.2, 1 mod 2N), so the two-prime RNS representation goes: KAHE NTT polynomials hold one 64-bit coefficient per slot and run on the ring library's 64-bit NTT. The share commitment decomposes each coefficient into six base-769 digits directly, and the erasure-coding layer does its Lagrange arithmetic mod q with six-byte coefficients on the wire.
Each measured phase now also records process CPU time (user plus system over all threads, via getrusage) and its peak heap growth, counted by a global allocator the bench binaries install, so per-phase working sets do not vanish behind the allocator's page reuse. The recipient's verify call is measured as a whole, since its splits are timed inside the protocol. A resources table reports per-role CPU, CPU-to-wall ratio, peak heap, the derived rho times ciphertext a recipient holds, and the process peak RSS per cell; the CSV gains the matching columns and older sweeps read as zero.
noot
changed the title
Pins Chipmunk to the published fork carrying the HVC parameter change, drops the rns feature from the workflow (removed with the RNS switch, so CI on main had failed since), formats the tree, and clears the two is_multiple_of lints and the unequal hex grouping clippy rejects.
# Conflicts: # src/rs.rs
There was a problem hiding this comment.
🟡 Changes recommended
Unconditional POSIX resource APIs break Windows library compilation, while active defects also affect midpoint decoding, legacy CSV loading, and the public commitment setup default.
4 open issues
What changed in this PR
This PR aligns Panetière’s cryptographic parameters and multiset encodings with the paper’s parameter generator, and extends benchmark resource reporting.
Changes:
- Updates prime moduli and commitment parameters, replacing KAHE’s two-prime representation with a single 64-bit NTT.
- Adds capacity-based IBLT sizing, stricter peeling checks, and shared 31-bit message packing.
- Adds CPU-time and heap measurements, with corresponding benchmark tables and CSV fields.
| File | Description |
|---|---|
tests/rs_mode_e2e.rs |
Supplies explicit MSE capacity. |
tests/prony_e2e.rs |
Adjusts messages to the smaller plaintext field. |
tests/mse_e2e.rs |
Updates capacity arguments and channel-based sizing. |
tests/end_to_end.rs |
Adapts slot tests to polynomial byte capacity and padding. |
src/share_commitment.rs |
Uses six base-769 digits over the KAHE prime ring. |
src/scaling_bench.rs |
Adds resource measurements and updates parameter accounting. |
src/rs.rs |
Moves erasure coding to single-prime arithmetic and serialization. |
src/rings.rs |
Updates moduli and implements KAHE’s 64-bit NTT representation. |
src/protocol/verify.rs |
Removes obsolete two-channel wording from a comment. |
src/protocol/mod.rs |
Increases commitment randomness width and adjusts a budget test. |
src/prony.rs |
Uses the shared prime plaintext modulus. |
src/mse.rs |
Adds capacity limits, key checks, and canonical decoded symbols. |
src/lib.rs |
Registers the chunking and IBLT-shape modules. |
src/kahe.rs |
Updates Gaussian widths and the plaintext modulus. |
src/iblt_shape.rs |
Adds stopping-set-bound shape selection and reference tests. |
src/cs.rs |
Updates leaf commitment bounds and HVC documentation. |
src/codec.rs |
Introduces 31-bit packing and byte-granular slot allocation. |
src/chunking.rs |
Adds shared bit-stream packing helpers and tests. |
src/channel.rs |
Integrates generated IBLT shapes and bit-level message packing. |
src/bulletin.rs |
Derives share wire sizes from six-byte coefficients. |
src/bin/panetiere_scaling.rs |
Installs the counting allocator. |
README.md |
Documents the revised ring representation. |
Cargo.toml |
Pins updated Chipmunk parameters and adds libc. |
Cargo.lock |
Records dependency changes. |
benches/protocol_sweep.rs |
Installs the counting allocator for benchmark runs. |
Review effort: Balanced
| fn cpu_time_us() -> f64 { | ||
| let mut ru = std::mem::MaybeUninit::<libc::rusage>::uninit(); | ||
| // SAFETY: `getrusage` only writes the struct it is handed; no pointer | ||
| // outlives the call. | ||
| let rc = unsafe { libc::getrusage(libc::RUSAGE_SELF, ru.as_mut_ptr()) }; | ||
| if rc != 0 { | ||
| return 0.0; | ||
| } | ||
| // SAFETY: a zero return means the struct was fully written. | ||
| let ru = unsafe { ru.assume_init() }; | ||
| (ru.ru_utime.tv_sec + ru.ru_stime.tv_sec) as f64 * 1e6 | ||
| + (ru.ru_utime.tv_usec + ru.ru_stime.tv_usec) as f64 | ||
| } |
| // `poly_mod_t` has centered them, so the upper half of that range | ||
| // shows up in `[-t/2, 0)`. Anything else is genuine overflow. | ||
| let v = if c < 0 { c + t } else { c }; | ||
| if c < -(t / 2) || !(0..SYMBOL_MOD).contains(&v) { |
There was a problem hiding this comment.
Agree with copilot on this - t / 2 will be rounded down. The suggested change looks good. i tried applying it and tests pass.
|
|
||
| /// Fresh per-opening randomness radius. | ||
| pub const BETA_CS: u32 = 116; | ||
| pub const BETA_CS: u32 = 39; |
There was a problem hiding this comment.
| let stat = |name: &str| -> Result<Stat, String> { | ||
| Ok(Stat { | ||
| med: num(&format!("m_{name}_us_med"))?, | ||
| min: num(&format!("m_{name}_us_min"))?, | ||
| max: num(&format!("m_{name}_us_max"))?, | ||
| cpu: opt_num(&format!("m_{name}_cpu_us"))?, | ||
| mem_kib: opt_num(&format!("m_{name}_mem_kib"))?, | ||
| }) | ||
| }; |
|
I realize that's not the point of the comparison, but am I reading the numbers correctly that at 16KiB messages, the Newton-Girard encoding is actually faster to decode than the IBLT peeling? |
ameba23
left a comment
ameba23
left a comment
There was a problem hiding this comment.
🌟 great work!
Leaving this as comment rather than approve because i think the decoding failure target for prony encoding needs thinking about more (in the paper as well as this PR). And also because my understanding of this is not good enough.
| // `poly_mod_t` has centered them, so the upper half of that range | ||
| // shows up in `[-t/2, 0)`. Anything else is genuine overflow. | ||
| let v = if c < 0 { c + t } else { c }; | ||
| if c < -(t / 2) || !(0..SYMBOL_MOD).contains(&v) { |
There was a problem hiding this comment.
Agree with copilot on this - t / 2 will be rounded down. The suggested change looks good. i tried applying it and tests pass.
|
|
||
| /// Fresh per-opening randomness radius. | ||
| pub const BETA_CS: u32 = 116; | ||
| pub const BETA_CS: u32 = 39; |
There was a problem hiding this comment.
| /// 35-bit: a 36-bit `p` exceeds the KAHE budget at q = 2^47.995. | ||
| pub const PRONY_PRIME: u64 = (32_788u64 << 20) + 1; | ||
| /// The sketch's field: the shared prime plaintext modulus. | ||
| pub const PRONY_PRIME: u64 = T_MODULUS_DEFAULT; |
There was a problem hiding this comment.
This i think needs thinking about. This is making a much smaller key-space than before, increasing possible collisions.
It looks like its not yet totally decided what the decoding failure target is for prony. The PR description says 2^-20 for IBLT. The paper says (in red text on page 33):
Decision: εKAHE and εMSE . The script currently defaults to 2^−11 for both, which is not negligible as required by Theorem 17
The proposed change here would meet 2^-11 as described in the paper but not 2^-20 as imposed on IBLT encoding in the PR description. If choosing this, we should document that prony configuration has more likely decoding failure.
Later the paper says (section 7.1, page 33):
For the Newton–Girard encoding (Section 2.7), ω is the smallest extension degree with ρ2 t−ω ≤ 2−εMSE, and roots are found with the Cantor–Zassenhaus algorithm (see Section 2.7).
There was a problem hiding this comment.
This is exactly the reason why the paper uses an extension field, so the key can occupy more than one Z_t element and correctness is not limited by the size of t. But this is currently not implemented as I just noted in #27. For the Newton-Girard sketch having negligible correctness error is very cheap, so I think we should definitely do it. (But nobody has ever discussed which parameters to use with me.) Note that Section 7.1 of the draft does not currently reflect the actual implementation, but rather what I would have expected before looking at the implementation.
Aligns the crate's parameters with the paper's generator (
scripts/generate_params.pyin the paper repo), for flashbots/flashnet#9.What changes
iblt_shape.rs, a port of the paper's search, unit-tested against it) instead of the fixed 4 × ⌈3ρ/4⌉ layout. For 300 clients: 6 × 101 = 606 cells at 2^-20. The peeling decoder re-hashes each pure cell's key and stops at capacity, as the paper's decode does.Verification
q prime,t prime, the IBLT bound, leaf SIS and both HVC SIS checks.Benchmarks vs main
100 clients, 8 servers, 8 threads, medians of 5. Peeling IBLT encoding:
Newton-Girard sketch encoding:
The commitment grows because the HVC modulus went from 17 to 20 bits (the old one is SIS-broken). The 16 KiB peel is slower because the table has six rows instead of four (the old shape was undersized: 2^-15.8 against the 2^-20 target) and symbols are 31-bit. The sketch's throughput dip is an accounting change: main counted 35 bits per symbol while packing 32.
Before undrafting
noot/Chipmunk; it should move to a flashbots fork or upstream before merge.--features rnssteps are dropped: the feature went away with the RNS switch, so CI on main had been failing since.