[mitmul]

Summary

PLaMo 3 applies YaRN in its three full-attention layers and keeps the correction range fractional (truncate: false; dimension pairs 13.96 to 30.02). No llama.cpp release honors the GGUF's yarn_truncate = false yet. The upstream fixes, ggml-org/llama.cpp#30206 (ggml, CPU and CUDA only) and #30337 (PLaMo 3), are both open. As a result, llama.cpp's floor and ceil widen the range to 13 to 31 in b11960 and in Homebrew's 0.9.0 (b12110). The error grows with input length.

#35 called the effect negligible. That measurement covered only confident translation outputs of sequences up to 2,700 tokens. This PR corrects it and works around the rounding:

• The backend reads the GGUF's RoPE metadata. For YaRN GGUFs with yarn_truncate = false, it passes --yarn-beta-fast 31.738495 --yarn-beta-slow 1.0040927, which make llama.cpp's floor and ceil land on 14 and 30, the nearest whole dimensions. Other GGUFs, including PLaMo 2, get no options. The 0.8B draft has the same RoPE metadata.
• gguf_metadata() reads every scalar and string of a GGUF header, decoding strings leniently as llama.cpp does. gguf_architecture() still stops at the architecture.
• README, a correction in the PLaMo 3 llama.cpp summary, and a new YaRN range summary.

Measurements (M1 Max, llama.cpp b11960)

Source-text predictions in PFML3 translation prompts, FP32 MLX with each range. Values are KL divergence from the exact range / top-1 agreement over the last 256 prompt positions, median of three documents:

Prompt tokens	Rounding (13–31)	Options (14–30)	MLX 4bit, exact range
2,033–2,083	0.038 / 96.5%	3.0e-4 / 100%	0.41 / 69.5%
8,205–8,241	0.23 / 83.2%	1.6e-3 / 98.8%	0.53 / 70.3%
16,365–16,388	0.66 / 76.2%	5.2e-3 / 98.0%	0.43 / 72.3%

Inside llama.cpp (llama-perplexity against BF16 with the options, scored positions 8,192–16,383):

Setup	KL / top-1
BF16 with rounding	0.85 / 71.6%
Q4_K_M with the options	0.38 / 77.4%
Q4_K_M with rounding (previous default)	1.14 / 64.2%

Beyond about 8,000 tokens, the rounding moves predictions more than Q4_K_M quantization does.

Greedy English-to-Japanese translations, lines that differ from the MLX FP16 backend (exact range):

Source tokens	Lines	llama-server BF16, rounding	llama-server BF16, options	MLX, 14–30
2,064	75	9	1	1
4,115	208	18	9	14
8,186	315	66	2	24

With the rounding, the 8,186-token translation also turned "zero in-loop vector spills" into ループ内量子化がゼロ and dropped a parenthetical. Applying llama.cpp's rounding to the MLX model reproduced llama-server's output byte for byte, so the rounding causes it. Greedy runs diverge after the first changed token, so the counts vary between runs. With the same 14–30 range, MLX and llama-server split at character 7,724, giving 2 or 24 lines.

Validation

• With the options, the BF16 GGUF still translates both fixtures byte-identically to MLX FP16.

• Q4_K_M decodes the English fixture at 112 tok/s (111 before) with the same output.

• tests/test_llama_cpp_integration.py passes locally on the b11960 build.

• The values in float32, as ggml computes them, give corr dims 14.000999 and 29.999, so floor and ceil give 14 and 30.

• New unit tests cover:

  • non-default betas and rope.dimension_count;
  • yarn_truncate = true and missing keys;
  • PLaMo 2;
  • early stop at the architecture;
  • non-UTF-8 strings.

• pytest (unit and mock suites): 96 passed. ruff check src tests: clean.

• A multi-agent review (numerics against the llama.cpp source, claims against the raw measurement outputs, code and tests, each checked by an independent verifier) found:

  • wrong translation-diff descriptions;
  • an overstated headline;
  • a reversed ramp direction;
  • a strict UTF-8 decode that could block GGUFs llama-server accepts;
  • test gaps.

  All are fixed here.

Notes

• With #30206 and #30337, these options would still decide the range: llama.cpp would use 14.001–29.999 instead of the exact range. They must be removed once a release honors yarn_truncate; the docstring, README and summary say so.
• #30206 has no Metal kernel, so even after it merges, Apple Silicon would run that RoPE on the CPU (not measured).
• The approximation's error also grows with length (KL 5e-3 at 16k tokens), but stays about 100 times below 4bit quantization.