[faresobeid]

Problem

<|image_pad|> is an ordinary token id the policy can sample (about 1 in 300k tokens on Qwen3.5-4B VLM RL). Tool or user text containing the literal string also tokenizes to it. Several consumers identify images by that token id:

• MultimodalRenderer.mm_token_type_id_map, which verifiers uses to build mm_token_type_ids;
• the bridges' previous_ids.count(image_pad);
• vLLM's raw-media matcher, which binds images to the first N placeholder tokens it finds.

A stray placeholder token then becomes an extra "image".

Design

• Every renderer that places media records PlaceholderRange(offset, length) at emission, in every mode. With process_multimodal=False this is one logical token per image, for qwen3_vl, qwen3.5/3.6/3.8, kimi_k25, GLM-5.3 Flash and Muse Glimmer. Nemotron 3.5 records its <image> tokens: its template places images as <image> text, and HF treats a typed <image> the same way.
• GLM-5 and Muse record only tokens placed through emit_special or placeholder, never text.
• Bridges merge previous_multi_modal_data.mm_placeholders in both modes, and seed add_vision_id counts from them instead of counting token ids.
• generate(process_multimodal=False) attaches each raw media part's range to its content part ("placeholder": {"offset", "length"}). It raises if the counts differ. A prebuilt prompt with no multi-modal data sends no ranges, which keeps today's behavior.
• mm_token_type_id_map is removed from the protocol and from every renderer. This is a clean break, with no shim. is_multimodal = True is now an explicit class attribute, because the runtime protocol check needed a discriminating member. Org-wide code search found consumers only in verifiers (updated in the linked PR) and in PrimeIntellect-ai/internal-prime-rl-elie-test, a test fork, which is flagged.

Validation

• New tests/test_placeholders.py, Qwen3.5-0.8B and Qwen3-VL-4B:
  • a sampled <|image_pad|> in the previous completion and a literal one in the next user text produce exactly 2 placeholders, each right after <|vision_start|>;
  • bridge and full render agree.
• Updated tests: glm53_flash, muse, nemotron35, client.
• tests/test_client.py test_placeholders test_glm53_flash test_muse_glimmer test_nemotron35_vlm test_multimodal test_bridge test_qwen38 test_is_content test_incremental: 940 passed, 48 skipped. Main gives 938 passed on the same files minus the new one.
• ruff check / ruff format --check: clean.

Stack / merge order

1. this PR
2. verifiers: feat(v1): carry media placeholder positions on message nodes verifiers#3013
3. prime-rl: feat: train on explicit multimodal placeholder positions, not image token ids prime-rl#4343 (bumps deps/renderers to this branch)

The verifiers PR needs this renderers change. Bump the verifiers renderers floor after a release.