[A-Gentle-Cat]

Motivation

Correct DSpine loss normalization and default alignment warmup after #965.

Modifications

• Average code loss over draft layers and normalize the total objective by sequence count.
• Normalize refinement CE and L1 separately, then scale them by the supervised-row count per microbatch.
• Default alignment warmup to the LR warmup while preserving explicit overrides.
• Add Qwen3-8B online training configs and update Supported Methods and News in the README.

Related Issues

Follow-up to #1004.

Accuracy Test

• CPU regression tests and configuration checks passed. The new four-trainer/global-batch-32 example also passed its launch-plan check.
• Completed a 20-step training run, with only minor differences from the previous run.

[Maxrio9]

Thanks for this. We're fine-tuning the released tencent/Qwen3.8-27B-DSpine-b16 with OnlineDSpineModel, on main and with this PR applied. Two things:

A pitfall unrelated to this PR: OnlineDSpineModel keeps its own copy of the draft's dspine_config, so replacing the config after construction is silently ignored. For us, that left predecessor replacement and the default alignment warm-up active when we'd turned both off. Details and a repro are in #1170.

A question about the alignment reduction: this PR averages the alignment loss over draft layers. As we read the paper's loss (eqs. 6–8), alignment is summed over layers, which makes it 5× stronger for the 5-layer b16 draft. An independent trainer we looked at also sums. Could you confirm that the mean is what the released checkpoints were trained with? On a small synthetic batch, this PR moves alignment from about 82% to 56% of the total loss and refinement from about 4% to 14%, so the choice matters.

We're running both objectives side by side from the same released checkpoint (all layers trainable, same data and schedule) and will post the dev acceptance-length comparison here when it's done.

[Maxrio9]

Update on the side-by-side run mentioned above. Both runs fine-tune the released tencent/Qwen3.8-27B-DSpine-b16 with OnlineDSpineModel, all layers trainable, using the #1170 workaround, the same data and the same schedule: AdamW at 3e-5 peak with a 100-step warm-up and cosine decay over 5,300 steps, batch 9 and 128 anchors. The teacher is a 4-bit Qwen3.8-27B target. We score dev tree tokens per round on 810 held-out prompts, paired against the released checkpoint, with 95% group-bootstrap intervals.

This PR vs main (change in tokens per round from the released checkpoint, with this PR's lead in the right-hand columns):

Prompts (released)	662: #1038 / main	662: lead	1324: #1038 / main	1324: lead
Agent (7.14)	+0.38 / +0.26	+0.11 (0.05–0.19)	+0.34 / +0.24	+0.10 (0.01–0.19)
Chat (2.93)	+0.012 / +0.004	+0.008 (n.s.)	+0.011 / +0.002	+0.009 (0.000–0.018)
Code (4.28)	+0.026 / +0.027	−0.001 (n.s.)	+0.021 / +0.014	+0.007 (n.s.)
Structured (5.01)	+0.033 / +0.023	+0.010 (n.s.)	+0.049 / +0.031	+0.018 (n.s.)
Overall	+0.020 / +0.012	+0.008 (0.002–0.014)	+0.017 / +0.008	+0.009 (0.003–0.016)

So for fine-tuning this checkpoint, this PR's objective beat main at both checkpoints, mostly on agent prompts. We stopped the main run at about step 1,850. This doesn't settle which reduction the released weights were trained with.

Later checkpoints with this PR's objective. The agent gain is largest at the first snapshot and falls after that: +0.38, +0.34, +0.30, +0.24 and +0.21 at steps 662 to 3,310. Structured rises over the same steps, from +0.033 to +0.055. We suspect overfitting: agent prompts take 4 of the 9 batch slots, and we have only 203 agent training conversations. A fork that cuts each block's loss after the first wrong position led this run on chat and code at step 1,986, but the lead was within noise by step 2,648, and its agent gain falls the same way.

We'll post the final checkpoints when they're scored.