[rollout, perf] fix: cache the generation-prompt delta in incremental tokenization

[ruiling-smartbear]

What does this PR do?

Fixes #7617. tokenize_non_assistant_incremental_messages() rendered the full conversation twice after every appended tool/user/system group, only to obtain the tokens that add_generation_prompt=True adds. Over a long tool loop that is a quadratic amount of chat-template rendering; the issue measured it at ~34% of rollout wall time on a 200-turn Qwen3.5-9B agent loop.

Following the discussion on the issue (@gxlvera, @wuxibin89): the generation-prompt delta is now cached on the builder by default, keyed by the role of the final message and the tool schemas, and filled by the existing full-history render. Builders keep template-specific behaviour through an override hook.

Checklist Before Starting

• Search for similar PRs: https://github.com/verl-project/verl/pulls?q=is%3Apr+generation+prompt+delta+tokeniz
• PR title formatted as [{modules}] {type}: {description}

Test

• tests/utils/test_continuous_token_on_cpu.py: 128 passed (119 existing + 9 new: a Qwen tool loop no longer renders the full history every turn, the cached delta equals the rendered one turn by turn, a template whose generation prompt drifts is detected and falls back, drift after the warm-up is still caught, roles and tool schemas get separate entries, the default builder still renders the full history on first use, VL builders stay uncached, Gemma4's own last-message render is what the cache serves).
• Real tokenizers, this branch vs main, 100-turn tool loops with 800-char tool results through the public incremental API — token ids identical on every turn:

builder	model	main	this PR	last-turn cost
qwen3	Qwen3-8B	6.31s	0.19s	147 ms -> 1.5 ms
qwen35	Qwen3.5-9B	6.16s	0.25s	117 ms -> 1.9 ms
glm47	GLM-4.7	4.50s	0.30s	91 ms -> 1.4 ms
minimaxm2	MiniMax-M2	4.94s	0.31s	98 ms -> 1.4 ms
gptoss	gpt-oss-20b	5.55s	0.22s	111 ms -> 0.3 ms
deepseek	DeepSeek-V3	4.80s	0.22s	96 ms -> 0.7 ms
default	Qwen3-8B	6.14s	0.35s	112 ms -> 1.6 ms

Tokenization only, CPU. Rollout-level numbers are the issue's, not mine.

• The cache key has to include the final role: DeepSeek-V3.2-Exp emits <｜Assistant｜></think> after a user turn and nothing after a tool output. Keyed on (role, tools), every template measured is stable over 20 turns.

API and Usage Example

No API change.

Design & Code Changes

• ContinuousTokenBuilder._tokenize_generation_prompt_delta: cache keyed by (final message role, tools fingerprint). The first 4 uses of a key re-render and compare, then re-validation happens at powers of two (8, 16, 32, ...). One disagreement marks the key unstable and it renders in full for the rest of the rollout, with a warning. O(log n) full renders per rollout instead of O(n).
• _render_generation_prompt_delta: the uncached renderer, which is the override hook for template-specific behaviour. Base = full-history render; Gemma4ContinuousTokenBuilder keeps its existing last-message render there.
• cache_generation_prompt_delta class flag: VLContinuousTokenMixin sets it to False, so processor-backed builders keep today's behaviour until someone checks them.

Checklist Before Submitting

• Read the Contribute Guide.
• ruff check / ruff format pass on the touched files.
• Docs: no user-facing change (internal tokenizer utility).
• Unit tests added to tests/utils/test_continuous_token_on_cpu.py, covered by the CPU unit-test workflow.
• CI request in ci-request once ready.

[CLAassistant]

All committers have signed the CLA.

[wuxibin89]

Fixed in #7628 (comment)

[ruiling-smartbear]

Sounds good, #7628 does the job. I ran the merge commit against its parent on the same trajectories (token ids identical on every row) and left the notes plus a DeepSeek caveat over there. Thanks for taking a look.