[subhajitdchow]

Motivation

MuseParser supervises an assistant span only when it ends in <|eom|> or <|eot|>. A reply cut off
by the generation length cap has no closing token, so its last span gets no loss. When the reply is cut
inside its first span, the row has no trainable tokens and is filtered out.

Modifications

• Let the final span end at the end of the text: (?:<\|eom\|>|<\|eot\|>|$). No closing token is added,
  so the draft does not learn a stop the target never produced.
• Document the case in the MuseParser docstring.
• Add test_unterminated_final_span_is_supervised.

Complete spans are unchanged: the match still stops at the first closing token.

Related Issues

• feat(data): mask Muse Spark assistant spans #1000 (adds MuseParser)
• feat(eagle4): add stage-1 dense draft training #1024 (EAGLE4 branch with the same pattern)

Accuracy Test

tests/test_data/test_muse_parser.py: the new test fails on main and passes with this change; the
existing Muse parser tests pass unchanged.

Benchmark & Profiling

N/A. Loss-mask change only.

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://sgl-fru7574.slack.com/archives/C09784E3EN6 to discuss your PR.

[subhajitdchow]

@yudigege86 MuseParser on the EAGLE4 branch (#1024) has the same closing-token requirement; this fixes it on main.