[bighead-liat]

Motivation

Ling-3.0 (inclusionAI, BailingMoeV3 / model_type: bailing_hybrid) has no chat template and no draft recipe in SpecForge, and the existing ling-flash-2.0 entry does not carry over to it.

Ling-3.0 keeps Ling-2.0's <role> headers, but the Bailing V3 template always emits a think block on every assistant turn:

<role>ASSISTANT</role>\n<think></think>{answer}                    # no reasoning
<role>ASSISTANT</role>\n<think>{reasoning}</think>{answer}         # reasoning

The opening tag — and, with thinking off, the whole empty block — is part of the generation prompt, so it is never model output. Reusing ling-flash-2.0, whose assistant_header stops at </role>, therefore supervises template boilerplate instead of the answer. Measured with the real tokenizer:

template	assistant_header	supervised spans (non-thinking data)
ling-flash-2.0 (existing)	<role>ASSISTANT</role>	\n<think></think>It is 4.<|role_end|> ← boilerplate supervised
ling-3.0 (this PR)	<role>ASSISTANT</role>\n<think></think>	It is 4.<|role_end|>
ling-3.0-thinking (this PR)	<role>ASSISTANT</role>\n<think>	{reasoning}</think>It is 4.<|role_end|>

The two Ling-3.0 templates have to stay separate: applying the non-thinking one to data that carries reasoning matches nothing and yields zero supervised tokens, silently. tests/test_data/test_ling3_template.py pins that.

Modifications

• specforge/data/template.py: register ling-3.0 and ling-3.0-thinking.
• specforge/data/parse.py: add LingParser, mirroring the existing GLMParser, so the Bailing V3 thinking option is passed explicitly. Without it the template defaults to detailed thinking on in the system line while the data carries empty think blocks, and serving with thinking off renders detailed thinking off — the draft would train on a prefix it never sees. specforge/data/preprocessing.py dispatches parser_type: "ling".
• configs/ling-3.0-tiny-dspark.json and examples/configs/offline/colocated/ling-3.0-tiny-dspark-offline.yaml: a DSpark draft config and offline recipe for Ling-3.0-tiny (24 target layers, hidden 1536, vocab 157184, block size 8, target_layer_ids [1, 6, 13, 17, 20]). num_anchors is 128 rather than 512: with a 157k vocabulary the anchor loss is the peak allocation of the step, and 128 keeps it inside a 24 GB card.
• tests/test_runtime/test_package_architecture.py: the repo-wide DSpark guard asserted attention_mode == "gqa" for every checked-in DSpark config. Ling-3.0-tiny is a 12/12-head draft, which the library itself requires to be mha: specforge/modeling/draft/dspark.py rejects gqa when num_key_value_heads >= num_attention_heads and says to set dflash_config.attention_mode='mha', and validate_dflash_attention_config in dflash.py then requires kv == q for that mode. The per-config assertions now branch on attention_mode — gqa keeps kv < q, mha requires equality, both keep model_type == "qwen3" and head divisibility, mla is still rejected — and the test is renamed test_dspark_configs_are_qwen3_family.
• Tests: tests/test_data/test_ling3_template.py (registration contract + loss-mask spans + the zero-mask regression guard), plus test_ling_3_0 / test_ling_3_0_thinking in tests/test_data/test_parsers.py with reference snapshots generated under the pinned transformers==5.12.1.

Related Issues

• offline capture: compat shims for sglang-main API drift (sglang 0.5.18 path unchanged) #876 — the sglang-main capture shims. Not a merge blocker here: training in this PR runs on the pinned sglang==0.5.18, but the offline capture step needs SGLang main, because 0.5.18 has no bailing_moe_v3.
• [Bug] There is a gap between the acceptance rates of training and inference #533 — the gap between training and inference acceptance. The thinking-flag mismatch LingParser prevents is one concrete instance of it.

Accuracy Test

Loss masks were measured against the real inclusionAI/Ling-3.0-tiny tokenizer (the table above is that output, not a design sketch); the committed reference snapshots are the token-level form of it.

Trainer side, on one RTX 4090: specforge train with the recipe in this PR runs 5 optimizer steps, initialises DSparkDraftModel from the new config (block size 8 gives the expected 8 supervised positions ce_position_0..7, confidence and Markov heads active), loads model.word_embeddings.weight and lm_head.weight from the Ling-3.0-tiny checkpoint, and fits in 24 GB at num_anchors: 128. Those steps ran on synthetic offline features (random tensors shaped to the DSpark contract), so they validate the config/recipe wiring only — no convergence or acceptance claim comes from them.

Since then the recipe has been trained through SpecForge end to end on features captured by it, on one RTX 4090 at num_anchors: 128, for 10 epochs rather than the 20 the recipe ships (reproduce with --override training.num_epochs=10). Greedy acceptance length, 60 prompts per set, against NGRAM on the same target:

eval set (n = 60 prompts)	NGRAM	2k convs	5k convs
MT-Bench	1.42	1.95	2.28
HumanEval	1.68	2.11	2.57
GSM8K	1.64	2.61	3.16

2.5x the data buys about 20% on every set, so data volume is still the binding constraint at this scale. The export serves in SGLang after scripts/gates/normalize_dflash_export.py --block-size 8 with no code change, and the numbers above are from that served path.

For scale, a DSpark draft I trained for Ling-3.0-tiny outside SpecForge (11.4k conversations, 20 epochs, same shape and same target layers) on the same three sets. This is one serving run reported two ways, because the prompt set matters for the comparison:

reference draft	MT-Bench	HumanEval	GSM8K
on the same 60 prompts as the 5k column	3.30	4.27	4.68
on the full benchmarks (n = 80 / 100 / 100)	3.03	4.24	4.75

The first row is the like-for-like number; the second is that draft given every prompt.

Note: offline capture needs an SGLang newer than the pinned one

sglang==0.5.18, the pinned dependency, has no bailing_moe_v3 at all, so Ling-3.0 can neither be served nor captured with it. Ling-3.0 support — including set_dspark_layers_to_capture, which is what specforge/offline_capture/sglang_backend/capture.py needs for this strategy — is on SGLang main.

Running scripts/prepare_hidden_states.py against main needed four adaptations, which I can send as a separate PR if SpecForge wants to track main:

1. ModelRunner now asserts a published runtime context: publish(server_args, role="scheduler") (sglang.srt.runtime_context) must be called before it is constructed.
2. ServerArgs.device is left None when ServerArgs is built directly, and initialize_dp_attention does torch.device(server_args.device).
3. With page_size=1 a hybrid target lands on an allocator without alloc_extend (NotImplementedError: alloc_extend is only for paged allocator); a paged size works.
4. sglang.srt.utils.require_mlp_sync() no longer takes server_args.

Past those, capture runs end to end. The one local trap was the toolchain rather than SpecForge: SGLang main's JIT needs a CUDA 13 toolkit, so CUDA_HOME has to point at one instead of at a system CUDA 11 nvcc. The acceptance numbers above come from that path.

Benchmark & Profiling

Not applicable — this PR adds data-side templates, a draft config and a recipe; it changes no kernels or model code.

Checklist

• Format your code according to the Code Formatting with Pre-Commit (pre-commit run --all-files clean).
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed — acceptance numbers are in the Accuracy Test section; no throughput or latency benchmark, since this PR changes no kernels or model code.

[Copilot]

🟢 Approval recommended

The changes are well-scoped with strong regression coverage and only minor maintainability nits identified.

Pull request overview

Adds first-class SpecForge support for Ling-3.0 (Bailing V3) draft training by introducing dedicated chat templates (thinking + non-thinking), a parser path to explicitly control the thinking option, and a ready-to-run DSpark offline recipe with regression tests and tokenizer-level reference snapshots.

Changes:

• Register new ling-3.0 and ling-3.0-thinking templates so loss masks exclude template boilerplate and correctly supervise answers/reasoning.
• Add LingParser and preprocessing dispatch to explicitly pass enable_thinking=False for Bailing V3’s default-on behavior.
• Add DSpark draft config + offline recipe for inclusionAI/Ling-3.0-tiny, plus new tests and reference snapshots.

File summaries

File	Description
specforge/data/template.py	Registers the two Ling-3.0 templates with headers that account for the mandatory <think> block.
specforge/data/parse.py	Adds LingParser to force enable_thinking=False when rendering Ling-3.0 non-thinking prompts.
specforge/data/preprocessing.py	Routes parser_type: "ling" templates through the new LingParser.
configs/ling-3.0-tiny-dspark.json	Introduces a DSpark draft-model config for Ling-3.0-tiny (block size 8, target layers, vocab, etc.).
examples/configs/offline/colocated/ling-3.0-tiny-dspark-offline.yaml	Adds an offline colocated training recipe wired to the new config/template.
tests/test_data/test_parsers.py	Adds regression tests that compare tokenizer output against reference snapshots for both templates.
tests/test_data/test_ling3_template.py	Adds focused unit tests for header registration and loss-mask spans, including a “zero supervised tokens” guard.
tests/test_data/test_references/ling-3.0_standard_ref.json	Reference snapshot for non-thinking Ling-3.0 rendering + loss mask.
tests/test_data/test_references/ling-3.0-thinking_reasoning-multi-turn_ref.json	Reference snapshot for thinking Ling-3.0 rendering + loss mask on multi-turn reasoning data.

Review details

• Files reviewed: 9/9 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[bighead-liat]

Two follow-ups.

Review nits addressed in the latest commit: LingParser is now exported in specforge/data/parse.py's __all__, and the thinking-flag test no longer pins the template's wording — it compares renderings instead (ours == an explicit enable_thinking=False rendering, and != the default rendering with no flag passed).

CI: both workflows are sitting at action_required, since this is my first PR here. Happy to rebase or split the change if that makes review easier once they run.

An offer. As noted in the description, the offline capture path cannot reach Ling-3.0 with the pinned sglang==0.5.18 — that release has no bailing_moe_v3 at all. Running scripts/prepare_hidden_states.py against SGLang main needed exactly four small adaptations:

1. ModelRunner asserts a published runtime context, so publish(server_args, role="scheduler") (sglang.srt.runtime_context) must be called before it is constructed;
2. ServerArgs.device stays None when ServerArgs is built directly, and initialize_dp_attention does torch.device(server_args.device);
3. with page_size=1 a hybrid target lands on an allocator without alloc_extend (NotImplementedError: alloc_extend is only for paged allocator);
4. sglang.srt.utils.require_mlp_sync() no longer takes server_args.

If SpecForge wants to track SGLang main — or is planning to bump the pin — I am happy to send those as a separate PR, guarded so that 0.5.18 keeps working. Let me know which shape you would prefer, and whether you would rather have this PR wait for that one.

[curnane-lab]

Using synthetic data alone is insufficient to demonstrate the draft model's effectiveness. Could you provide some training and evaluation metrics on real-world datasets?

[bighead-liat]

Real-dataset metrics, as requested.

I ran the recipe in this PR end to end on real data: 2,000 conversations from Ling-3.0-tiny's own regenerations over perfect-blend prompts (non-thinking), offline DSpark capture at max_length 2048 (64 GB of features), then 10 epochs = 19,920 optimizer steps on a single RTX 4090, about two hours end to end.

Training (real features): ce_loss 11.6 → 2.73, draft/target top-1 agreement acc 0.09 → 0.51, teacher_agreement 0.13 → 0.51 (means over the last 25 logged points). For contrast, the synthetic-feature run mentioned in the description never moves at all — acc stays 0.0 and ce_loss sits at ~13.2 — which is exactly why I labelled it a wiring check rather than evidence.

Evaluation: acceptance length measured by serving the exported draft in SGLang, greedy, n=60 per set.

eval set	NGRAM	this PR's recipe (2k convs, 10 ep)	reference draft (11.4k convs, 20 ep)
MT-Bench	1.42	1.95 (+37%)	3.22
HumanEval	1.68	2.11 (+26%)	4.27
GSM8K	1.64	2.61 (+59%)	4.74

The right-hand column is a draft of identical shape (same layers, hidden size, block size, target layers) that I trained earlier outside SpecForge on roughly 11x more data. So this run reaches about 60% of it with 1/11 of the token budget, and beats NGRAM on every set — the remaining gap is training budget, not the recipe.

Two things worth flagging from the run, both fixable and both independent of the template change:

1. specforge export --to hf nests the DSpark fields under dflash_config, but SGLang's Ling DSpark model reads markov_rank / mask_token_id / target_layer_ids / num_anchors at the top level, so serving an exported DSpark draft fails with DSPARK requires markov_rank > 0 in the draft config, got markov_rank=0 until they are flattened. Happy to fix that here or in a follow-up, whichever you prefer.
2. Capture only runs against SGLang main, as noted above — the pinned sglang==0.5.18 has no bailing_moe_v3. On my machine it also needed SGLANG_FORCE_FUSED_OP_BACKEND=torch, since it has a CUDA 11 toolchain and cannot build any JIT kernel; that part is local, not something the repo needs to care about. The four main adaptations are still on offer as a separate PR.

A larger run on 5,000 conversations of the same data is already captured and its training is queued behind a GPU outage on my side; I will post those numbers in this thread once it lands.

[bighead-liat]

Follow-up with the larger run, as promised.

Same pipeline, 5,000 conversations (2.5x the data), 10 epochs = 49,840 steps on one RTX 4090.

Training: draft/target top-1 agreement acc 0.00 → 0.70, ce_loss 8.14 → 1.21, teacher_agreement → 0.73.

eval set	NGRAM	2k convs	5k convs	5k vs 2k	reference (11.4k convs, 20 ep)
MT-Bench	1.42	1.95	2.28	+17%	3.22
HumanEval	1.68	2.11	2.57	+22%	4.27
GSM8K	1.64	2.61	3.16	+21%	4.74

2.5x the data buys a consistent ~20% across all three sets, and the draft now reaches 60–71% of the reference that saw 11.4k conversations for 20 epochs. So the gap really is training budget rather than the recipe — I can push it further if you would like a specific number before merging.

For reproducibility, the training data is open-perfectblend prompts regenerated by Ling-3.0-tiny itself in non-thinking mode, which is the same construction DeepSpec documents for its released DSpark checkpoints.

[bighead-liat]

The unit-test job failed at container start, before any test ran:

failed to fulfil mount request: open /run/nvidia-persistenced/socket: no such file or directory

on the self-hosted GPU runner. The same error hit the feature/robust-weight-load run on Sep 11, so it looks like nvidia-persistenced is down on that runner rather than anything in this PR. lint and cleanup-runner are green.

Could you re-run once the runner is back? Happy to rebase if needed.

[curnane-lab]

@bighead-liat 5k run looks good.

One blocker: serving. The raw export fails in SGLang (markov_rank=0), but no new code needed - scripts/gates/normalize_dflash_export.py --block-size 8 should fix it. The step is just buried in scripts/gates/README.md. Please run it, confirm serving works end to end.

Compat: separate PR is fine, but make them shims that keep 0.5.18 working, not straight patches.

[bighead-liat]

Serving gate done — the normalizer is all it takes, no new code.

On a fresh copy of the raw specforge export --to hf output of the 5k-conversation draft:

python scripts/gates/normalize_dflash_export.py --config <export>/config.json --block-size 8

exits 0 and rewrites the config to architectures=["Qwen3DSparkModel"] with markov_rank=256, markov_head_type=vanilla, enable_confidence_head=true, confidence_head_with_markov=true promoted to the top level (our export has model_type=qwen3, so the DSpark branch applies as-is).

Served end to end in SGLang (main @ de0fa171, one RTX 4090):

python -m sglang.launch_server --model-path inclusionAI/Ling-3.0-tiny --trust-remote-code \
  --speculative-algorithm DSPARK --speculative-draft-model-path <normalized export>

The draft loads under the Qwen3DSparkModel tag (SGLang's LingDSparkModel is the same class), and greedy acceptance length on 60 prompts each:

MT-Bench	HumanEval	GSM8K
2.278	2.571	3.156

identical to the numbers I posted above, which had used a hand-edited config — so the documented step reproduces the hand fix exactly. I'll add a one-line pointer to the normalizer in the Ling recipe doc in this PR so the next person does not miss it.

On compat: understood — the sglang-main drift will go in a separate PR as shims that keep 0.5.18 working, not patches. (Serving above is on sglang main because Ling-3.0's BailingMoeV3 is not in 0.5.18; the training side of this PR runs on the pinned 0.5.18 as before.)

[bighead-liat]

The sglang-main compat work is up as a separate PR, as shims rather than patches: #876. It removes the last hand-applied edits from the Ling-3.0 capture run described above; the pinned 0.5.18 path is unchanged.

[curnane-lab]

The CI failure is caused by an infra issue on GPU runner. It's unrelated to your code, will re-run once the runner is fixed.

[bighead-liat]

CI is green on 479adf7 — test_dspark_configs_are_qwen3_family ... ok, 1252 tests in 763 s: PR Test https://github.com/sgl-project/SpecForge/actions/runs/35054512247, Lint run 35054512300. Thanks for the click.

My 2026-09-12 nvidia-persistenced comment was wrong and is what sent you to the runner — the red job was my own guard test_dspark_configs_are_qwen3_gqa_only (tests/test_runtime/test_package_architecture.py:711), red since this PR opened and masked by the earlier job failures on this branch.

That guard asserted attention_mode == "gqa" for every checked-in DSpark config, which forbids what the library prescribes: specforge/modeling/draft/dspark.py:300 rejects gqa when num_key_value_heads >= num_attention_heads and says to set dflash_config.attention_mode='mha'; validate_dflash_attention_config (dflash.py:141) then requires kv == q there — the 12/12-head ling-3.0-tiny-dspark.json. 479adf7 branches the per-config assertions on attention_mode instead of dropping any (gqa keeps kv < q, mha requires equality, both keep model_type == "qwen3", mla still rejected) and renames the test; editing this allowlist alongside a new config has precedent in dca1120.

If you would rather not relax that guard here, say so and I will drop configs/ling-3.0-tiny-dspark.json and its offline YAML together — verified locally that the pair leaves the suite green.

target_layer_ids [1, 6, 13, 17, 20] is the set behind every number in this thread and will not move in this PR; a layer-set A/B is running and any change ships as a follow-up with its own gate metrics.

I also refreshed the body so the record lives there rather than only in this thread: the SpecForge-trained 5k acceptance numbers and the serving gate are in the Accuracy Test section, the capture note now says capture completes (the trap was needing a CUDA 13 toolkit for SGLang main's JIT, not SpecForge), #876 and #533 are linked, and the test guard above is listed under Modifications. Both Copilot review threads are answered and resolved — each was fixed in 2733af4. One correction while I was in there: the out-of-SpecForge reference draft I quoted on 2026-09-12 as 3.22 / 4.27 / 4.74 is 2.97 / 4.23 / 4.76 in my own ledger, and the body now carries the ledger values.

One ask: #876 has never run CI — all four runs on that branch have sat at action_required since 2026-09-12, so its rollup is empty, not failing. @jiapingW @curnane-lab could one of you approve them?

[jiapingW]

Can you address the comment above? I think Ling use the similar chat template as Qwen.

[bighead-liat]

Thanks for taking a look. You're right about the lineage: Bailing V3's Jinja shares Qwen's structure (same last_query_index reverse scan, same content.split('</think>') block, same enable_thinking kwarg). The surface tokens differ, though, and ChatTemplate anchors the loss mask on the literal assistant_header / end_of_turn_token. Here is a 2-turn conversation through the real inclusionAI/Ling-3.0-tiny tokenizer (add_generation_prompt=False, enable_thinking=False, transformers 5.12.1):

<role>SYSTEM</role>detailed thinking off<|role_end|><role>HUMAN</role>What is 2+2?<|role_end|><role>ASSISTANT</role>\n<think></think>It is 4.<|role_end|><role>HUMAN</role>And 3+3?<|role_end|><role>ASSISTANT</role>\n<think></think>That is 6.<|role_end|>

<|im_start|>assistant\n never appears, so the qwen and qwen3.5 entries match zero spans on it. I checked by running it through preprocess_conversations: 0 supervised tokens, with no exception or warning to flag it. The ling-3.0 entry supervises exactly It is 4.<|role_end|> and That is 6.<|role_end|>.

The one place Ling needs its own parser path is the system line. Bailing V3 writes detailed thinking on|off into it on every render and defaults to on when no flag is passed, while serving with thinking off renders off; Qwen's template renders the same bytes with the flag set, unset, or false. So the ling-3.0 entry passes enable_thinking=False explicitly to keep the training and serving prefixes identical. Folding the think tags into assistant_header follows the same shape as the qwen3.5 and qwen3-next-thinking entries, just with Ling's tokens.

If the comment you mean is one of Copilot's two, both should be resolved as of 2733af4: LingParser is exported in __all__, and the flag test now compares renderings against apply_chat_template(..., enable_thinking=False) instead of pinning the template's wording. If you had something else in mind, please point me to it and I'll take care of it. Happy to adjust the shape of this if you'd prefer it done differently.