diff --git a/.gitignore b/.gitignore index 8c3a30b..9f9ee83 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,7 @@ __pycache__/ .ruff_cache/ .venv/ site.local.json +config/*.local.json work/ checkpoints/ *.log @@ -15,3 +16,4 @@ checkpoints/ *.safetensors *.whl .env* +article/.backups/ diff --git a/README.md b/README.md index 28a35fc..34eb1e2 100644 --- a/README.md +++ b/README.md @@ -1,64 +1,63 @@ -# Multi-teacher on-policy distillation +# On-policy distillation -This repository preserves the scripts for three completed experiments with Miles, Prime-RL, and Slime. -Each experiment trains Qwen3.6-35B-A3B with frozen Countdown and Graph Coloring teachers. +On-policy distillation (OPD) experiments across five training frameworks: Miles, NeMo-RL, Prime-RL, Slime and verl. -**The original experiments completed. The portable package has passed local checks, but it has not been rerun on GPUs.** -The runtime image and converted model checkpoints are external prerequisites. This repository is not a self-contained container distribution. +The current experiment is multi-teacher OPD (MOPD): Qwen3.6-35B-A3B learns from frozen reasoning_gym GRPO teachers for `caesar_cipher` and `simple_geometry`. Thinking is on everywhere (training, teachers and evaluation), with a 32k context and up to 30,720 response tokens. -## Start here +## Status -1. Read [the setup guide](docs/SETUP.md), including the runtime and checkpoint requirements. -2. Copy `site.example.json` to `site.local.json` and supply your node names, addresses, and paths. -3. Run `python3 tools/prepare.py slime --site site.local.json --output /shared/opd-runs/slime-01` on the shared filesystem. -4. Complete the framework setup and inspect the command with `python3 tools/submit.py /shared/opd-runs/slime-01 --partition YOUR_PARTITION`. -5. Add `--submit` only when you intend to allocate 16 GPUs. The command remains attached and writes `allocation.out`. +- The single-teacher `caesar_cipher` variant (`--domains caesar_cipher`) has run on GPUs in Miles, Prime-RL, Slime and verl. NeMo-RL deadlocked before its first update. See the [benchmark report](docs/CAESAR-OPD-2026-09-27.md). +- The two-teacher MOPD experiment ran in Miles, Prime-RL and verl on 16 B200 GPUs (benchmarks and telemetry in [`docs/mopd-2026-09-29/data`](docs/mopd-2026-09-29/data)). Miles also ran it on 24 H100 80 GB GPUs with the `h100x3` layout: after 20 updates the student scores 0.773 avg@8 on caesar_cipher and 0.998 on simple_geometry (teachers 0.703 and 0.996). See the [H100 MOPD report](docs/MOPD-H100-2026-10-07.md); benchmarks and telemetry are in [`docs/mopd-h100-2026-10-07/data`](docs/mopd-h100-2026-10-07/data). +- A previous experiment (Countdown and Graph Coloring, thinking off) completed in Miles, Prime-RL and Slime on two nodes of eight B200 GPUs. See [provenance](docs/PROVENANCE.md). -Replace `slime` with `miles` or `prime-rl` to prepare another framework. Use a new campaign directory for each experiment. -Preparation clones source and writes files, but does not submit a GPU job or download model weights. -An existing campaign directory is never overwritten. +Results are single observations and do not rank the frameworks. The implementations differ in objective, scheduling, precision and checkpoint content, and maximum useful GPU utilization was not demonstrated. -## What is included +## Quick start -| Directory | Contents | -|---|---| -| `frameworks/miles/` | The fully async candidate-based objective, domain-balanced buffer, scoring retries, and instrumentation. | -| `frameworks/prime-rl/` | Native per-source OPD, configuration generation, task environment, and the frozen-vision compatibility patch. | -| `frameworks/slime/` | Native batched async OPD, teacher routing, baseline evaluation, timing, and runtime compatibility patches. | -| `data/` | The exact compressed puzzle files and hashes of their uncompressed contents. | -| `tools/` | Preparation, guarded Slurm submission, and source reconstruction checks. | +The runtime image and converted checkpoints are external prerequisites; the [setup guide](docs/SETUP.md) covers them. + +To rerun the whole `caesar_cipher` benchmark (every framework, checkpoint benchmarks and the telemetry report), fill in `config/site.local.json` and run this on a Slurm login node: -The files in `campaign/` are templates. Values such as `@BASE_MODEL@` are replaced during preparation. -Run the prepared copies rather than the templates. Each manifest records the upstream revision, patch digest, and original archive digest. -The [provenance document](docs/PROVENANCE.md) identifies packaging changes and scientific differences. +```bash +PARTITION=YOUR_PARTITION tools/reproduce.sh all +``` -The separate [Tinker campaign](frameworks/tinker/README.md) trains hosted puzzle -teachers and a 40-update student through the Tinker SDK. It retains remote -checkpoints and local evidence without downloading model weights. See its own -protocol for LoRA, objective, and teacher-comparability differences; it does not -use the Slurm preparation or submission tools below. +See [One command: `tools/reproduce.sh`](docs/SETUP.md#one-command-toolsreproducesh) for its settings. -## Fixed experiment +To run one framework by hand: -The experiments use 40 optimizer updates, 128 prompts per update, one response per prompt, learning rate 1e-6, and a 256-token response limit. -The GPU allocation contains eight trainer GPUs, six policy GPUs, and one GPU per frozen teacher. -Each experiment uses two nodes with eight NVIDIA B200 GPUs each. Evaluation uses 512 development prompts per domain with thinking disabled. +1. Read the [framework overview](frameworks/README.md) and the [setup guide](docs/SETUP.md). +2. Copy `config/site.example.json` to `config/site.local.json` and fill in your nodes, addresses and paths. +3. Prepare a campaign: + ```bash + python3 tools/prepare.py slime --site config/site.local.json --output /shared/opd-runs/slime-01 + ``` +4. Print the allocation, then add `--submit` to allocate 16 GPUs: + ```bash + python3 tools/submit.py /shared/opd-runs/slime-01 --partition YOUR_PARTITION + ``` -The implementations have different objectives, scheduling, precision choices, and checkpoint content. -These experiments are individual observations and do not establish a framework ranking. -The sampled utilization did not demonstrate maximum useful utilization. No GPU profiler was enabled. +Replace `slime` with `miles`, `prime-rl`, `verl` or `nemo-rl`. NeMo-RL needs three nodes and `config/site-nemo-rl.example.json`. Preparation never overwrites a directory, submits a job or downloads weights. + +## Layout + +| Path | Contents | +|---|---| +| `shared/` | Recipe, data checks, verifier, Slurm and container runtime, telemetry | +| `frameworks/` | Per-framework code, each with its pinned `upstream/` revision and patch, if any | +| `hosted/` | Tinker and Fireworks campaigns, each with its own protocol, and the hosted OPD/MOPD loop ([README](hosted/opd/README.md)) | +| `data/` | Compressed task rows and their hashes | +| `config/` | Example site and hosted-account files ([README](config/README.md)) | +| `tools/` | Preparation, teacher fusing, guarded submission, source verification, benchmarking, reporting, plots (`plot.py`), `reproduce.sh` | +| `docs/` | [Setup](docs/SETUP.md), [provenance](docs/PROVENANCE.md), [validation](docs/VALIDATION.md), [caesar_cipher benchmark](docs/CAESAR-OPD-2026-09-27.md), [H100 MOPD report](docs/MOPD-H100-2026-10-07.md), benchmark data under `docs/*/data` and `docs/mopd-h100-2026-10-07` | ## Checks -Python 3.12 is required for the preparation tools and archived campaign code. -The following checks need no GPU and do not contact the cluster. +Python 3.12 is required. These checks need no GPU: ```bash python3 -m unittest discover -s tests -v git diff --check ``` -Raw logs, model weights, generated checkpoints, local credentials, and site configuration are excluded from Git. -Keep run artifacts outside this checkout and retain their source snapshot, resolved configuration, exit status, and checksums. -The old repository contents remain in Git history at `a1bf83efd1a5cf11ef72b3ea18eb98132d83c92e`. -This change does not remove old data from Git history or change repository visibility. +Raw logs, weights, checkpoints, credentials and local config files are kept out of Git. Earlier repository contents remain in history at `a1bf83efd1a5cf11ef72b3ea18eb98132d83c92e`. diff --git a/config/README.md b/config/README.md new file mode 100644 index 0000000..a31f6f8 --- /dev/null +++ b/config/README.md @@ -0,0 +1,18 @@ +# Local configuration + +Copy an example to `*.local.json` and edit it. Local files are gitignored, so cluster addresses, account IDs and account-guard values stay out of the repository. + +| File | Used by | Required | +|---|---|---| +| `site.local.json` | `tools/prepare.py` | Yes, for Miles, Prime-RL, Slime and verl | +| `site-nemo-rl.local.json` | `tools/prepare.py nemo-rl` | Yes, for NeMo-RL. It is the site file plus `teacher_node`, `teacher_ip` and `nemo_rl_container` for the third node. | +| `tinker.local.json` | `hosted/tinker/launch.py` | No. Without it, `TINKER_API_KEY` is used with no account guard. | +| `fireworks.local.json` | `hosted/fireworks/campaign.py` | No. `--account` or `FIREWORKS_ACCOUNT_ID` also work. | + +## API keys + +API keys never go in these files. The hosted launchers read `TINKER_API_KEY` or `FIREWORKS_API_KEY` from the environment. To read a `LABEL=value` line from `~/.zprofile` instead, set `api_key_profile_label`. This helps when one machine holds keys for several accounts. + +## Tinker account guard + +Set `expected_email` and `expected_org` to make the Tinker launcher refuse any other account. On a shared organization account, keep these values in your local file. diff --git a/config/fireworks.example.json b/config/fireworks.example.json new file mode 100644 index 0000000..246b363 --- /dev/null +++ b/config/fireworks.example.json @@ -0,0 +1,4 @@ +{ + "account": "your-fireworks-account-id", + "api_key_profile_label": null +} diff --git a/config/site-nemo-rl.example.json b/config/site-nemo-rl.example.json new file mode 100644 index 0000000..cd41119 --- /dev/null +++ b/config/site-nemo-rl.example.json @@ -0,0 +1,16 @@ +{ + "assets": "/shared/opd-assets", + "base_model": "/shared/opd-assets/models/Qwen3.6-35B-A3B", + "megatron_model": "/shared/opd-assets/models/Qwen3.6-35B-A3B_torch_dist", + "teachers": "/shared/opd-assets/teachers", + "container_image": "/shared/opd-runtime/miles-amd64.sqsh", + "megatron_source": "/root/Megatron-LM", + "uv": "/usr/local/bin/uv", + "generation_node": "generation-node", + "trainer_node": "trainer-node", + "generation_ip": "192.0.2.20", + "trainer_ip": "192.0.2.10", + "teacher_node": "teacher-node", + "teacher_ip": "192.0.2.30", + "nemo_rl_container": "/shared/opd-runtime/nemo-rl-v0.7.0.sqsh" +} diff --git a/site.example.json b/config/site.example.json similarity index 85% rename from site.example.json rename to config/site.example.json index 39b69d7..17795d3 100644 --- a/site.example.json +++ b/config/site.example.json @@ -3,7 +3,7 @@ "base_model": "/shared/opd-assets/models/Qwen3.6-35B-A3B", "megatron_model": "/shared/opd-assets/models/Qwen3.6-35B-A3B_torch_dist", "teachers": "/shared/opd-assets/teachers", - "container_image": "/shared/opd-runtime/miles-dev-202609050049.sqsh", + "container_image": "/shared/opd-runtime/miles-amd64.sqsh", "megatron_source": "/root/Megatron-LM", "uv": "/usr/local/bin/uv", "generation_node": "generation-node", diff --git a/config/tinker.example.json b/config/tinker.example.json new file mode 100644 index 0000000..18168d4 --- /dev/null +++ b/config/tinker.example.json @@ -0,0 +1,5 @@ +{ + "api_key_profile_label": null, + "expected_email": null, + "expected_org": null +} diff --git a/data/caesar_cipher-dev.jsonl.gz b/data/caesar_cipher-dev.jsonl.gz new file mode 100644 index 0000000..40d8e08 Binary files /dev/null and b/data/caesar_cipher-dev.jsonl.gz differ diff --git a/data/caesar_cipher-train.jsonl.gz b/data/caesar_cipher-train.jsonl.gz new file mode 100644 index 0000000..3d2bfe4 Binary files /dev/null and b/data/caesar_cipher-train.jsonl.gz differ diff --git a/data/countdown4-dev.jsonl.gz b/data/countdown4-dev.jsonl.gz deleted file mode 100644 index 69b7c86..0000000 Binary files a/data/countdown4-dev.jsonl.gz and /dev/null differ diff --git a/data/countdown4-test.jsonl.gz b/data/countdown4-test.jsonl.gz deleted file mode 100644 index e87ec73..0000000 Binary files a/data/countdown4-test.jsonl.gz and /dev/null differ diff --git a/data/countdown4-train.jsonl.gz b/data/countdown4-train.jsonl.gz deleted file mode 100644 index e4bd9b1..0000000 Binary files a/data/countdown4-train.jsonl.gz and /dev/null differ diff --git a/data/graph12-dev.jsonl.gz b/data/graph12-dev.jsonl.gz deleted file mode 100644 index 1063f5a..0000000 Binary files a/data/graph12-dev.jsonl.gz and /dev/null differ diff --git a/data/graph12-test.jsonl.gz b/data/graph12-test.jsonl.gz deleted file mode 100644 index 3be34da..0000000 Binary files a/data/graph12-test.jsonl.gz and /dev/null differ diff --git a/data/graph12-train.jsonl.gz b/data/graph12-train.jsonl.gz deleted file mode 100644 index 628ce9e..0000000 Binary files a/data/graph12-train.jsonl.gz and /dev/null differ diff --git a/data/manifest.json b/data/manifest.json index 6dab0f7..dec56c3 100644 --- a/data/manifest.json +++ b/data/manifest.json @@ -1,37 +1,27 @@ { - "countdown4-dev.jsonl": { - "sha256": "ba7497391529339026f0db7bc90e1b26aa25751dad4a19a3ca90992b417bbda7", - "bytes": 319918, - "rows": 512 + "caesar_cipher-train.jsonl": { + "bytes": 11291100, + "rows": 7788, + "sha256": "41beb35347fd0774d702464466e3467b243535b594658161f2adaa2a7e4758c2" }, - "countdown4-test.jsonl": { - "sha256": "142bf2ff50c30afa81bd30dc36f63f1fd31de07c0e4b5b8134f957df6d4dcf7e", - "bytes": 641017, - "rows": 1024 + "caesar_cipher-dev.jsonl": { + "bytes": 183027, + "rows": 126, + "sha256": "c2f97f3f252640c9652faebac523b1f5669bd952ffb92f6890a74e5765297d41" }, - "countdown4-train.jsonl": { - "sha256": "2a48909ab0a876585fc007dcc021203dd2adc0bd5a6db5296b4ea09736702913", - "bytes": 6279290, - "rows": 10000 + "simple_geometry-train.jsonl": { + "bytes": 11946723, + "rows": 7788, + "sha256": "bb7fcfbda69a3b6ac7db9c5c545e0e84fa78d8eeb432d37608573b5464c8a02f" }, - "graph12-dev.jsonl": { - "sha256": "fe9cd47d24baf3fbc16e68b119adf6c55405bfaffa13e2539a79f224ebdcfff3", - "bytes": 557290, - "rows": 512 - }, - "graph12-test.jsonl": { - "sha256": "274531fe90a56a67acd7e2f4336bead075bd2e7d891bce70ac7323ace1b76246", - "bytes": 1115638, - "rows": 1024 - }, - "graph12-train.jsonl": { - "sha256": "7229bc6d4d7a37d8cf3aaab449ab35d9efb5bf63aaceac64fdd0334554f48066", - "bytes": 10908616, - "rows": 10000 + "simple_geometry-dev.jsonl": { + "bytes": 195111, + "rows": 128, + "sha256": "8dc5fc2aa9c3355d8ee3d4b521a997c7863f471723663a9889e9e4db5c5e6b8d" }, "mixed-train.jsonl": { - "sha256": "bafdaa11f181f55606eefa7227583d80783e7d781281a2b0136dcdf9d307a70e", - "bytes": 17187906, - "rows": 20000 + "bytes": 23237823, + "rows": 15576, + "sha256": "d8c46c8218174820779690e92535a16dea9244558315c4eb3d5c700812c5c278" } } diff --git a/data/mixed-train.jsonl.gz b/data/mixed-train.jsonl.gz index 902ef91..e88c75f 100644 Binary files a/data/mixed-train.jsonl.gz and b/data/mixed-train.jsonl.gz differ diff --git a/data/simple_geometry-dev.jsonl.gz b/data/simple_geometry-dev.jsonl.gz new file mode 100644 index 0000000..c69921d Binary files /dev/null and b/data/simple_geometry-dev.jsonl.gz differ diff --git a/data/simple_geometry-train.jsonl.gz b/data/simple_geometry-train.jsonl.gz new file mode 100644 index 0000000..890c64e Binary files /dev/null and b/data/simple_geometry-train.jsonl.gz differ diff --git a/docs/CAESAR-OPD-2026-09-27.md b/docs/CAESAR-OPD-2026-09-27.md new file mode 100644 index 0000000..0c28793 --- /dev/null +++ b/docs/CAESAR-OPD-2026-09-27.md @@ -0,0 +1,444 @@ +# Single-teacher OPD on caesar_cipher: Miles, Prime-RL, Slime and verl + +Runs of 27 September 2026 (UTC), plus the Prime-RL and Miles reruns of 28 September (jobs 1017 and +1021), on two nodes of eight B200 GPUs. Everything was driven by this repository's package (`shared/`, +`frameworks/`, `tools/`). All numbers come from the runs' own logs, telemetry and benchmark outputs, +stored in [`caesar-opd-2026-09-27/data/`](caesar-opd-2026-09-27/data/). + +## Summary + +We distilled a frozen `caesar_cipher` teacher into the base Qwen3.6-35B-A3B student with on-policy +distillation (OPD), running the same 20-update recipe in four RL frameworks. Nothing was tuned per +framework beyond what its own documentation and examples prescribe. + +Two frameworks were rerun. Prime-RL's first run (job 968) broke that rule: the package's config +overrode Prime-RL's defaults in seven places. We reran on upstream defaults (job 1017) and report the +rerun; the first run is compared in [Prime-RL: first run and rerun](#prime-rl-first-run-and-rerun). Miles +was rerun end to end with `tools/reproduce.sh miles` (job 1021) once the package logged its training +reward. It reproduced the first run (77.6% against 77.4%), and the Miles numbers below are from the rerun. +A fifth framework, NeMo-RL, never completed a training update (see [NeMo-RL](#nemo-rl)). + +| | Miles | Prime-RL | Slime | verl | +|---|---|---|---|---| +| caesar_cipher avg@8 after 20 updates (base 26.8%, teacher 72.1%) | 77.6% | 77.9% | 77.7% | 78.4% | +| simple_geometry avg@8, never trained (base 52.9%) | 65.9% | 70.4% | 71.8% | 68.7% | +| Median time per update | 119 s | 93 s | 197 s | 106 s | +| Median share of an update spent waiting for rollouts | 72% | 61% | 84% | 53% | +| Wall time for the whole allocation | 67.5 min | 39.6 min | 89.4 min | 71.3 min | +| Launches until a clean run | 3, then 1 for the rerun | 2, then 3 for the rerun | 1 | 5 | + +- **All four students beat the teacher.** Each ended 5–6 points above it on the held-out caesar set and + solved at least 125 of 126 prompts in eight samples. All four also gained 13–19 points on + `simple_geometry`, which no teacher taught. With one seed per framework, the 1-point spread between + frameworks is noise. +- **Our config, not Prime-RL, caused its poor first run.** Job 968 reached 58.0%. Its config forced bf16 + master weights (through a `model.vlm` block this text-only run does not need), capped concurrency and + engine sizes, put the teacher in eager mode and disabled FlashInfer autotune, none of it with a recorded + reason. On Prime-RL's defaults it reached 77.9% in 40 minutes, the fastest of the four. +- **Generation is the bottleneck everywhere.** With thinking on and up to 30,720 generated tokens, + trainers spent 53–84% of each update waiting for rollouts. Trainer GPUs averaged 16–25% utilization, + policy GPUs 62–87%. The frameworks differ mainly in how they handle the tail of 30k-token responses: + wait for it (Slime, slowest), drop it (Prime-RL, fastest), or resume it across weight updates (verl). + Quality was the same either way. +- **Most failures were environment problems.** Of the nine failed training, benchmark and conversion jobs + on 27 September, one was an upstream incompatibility (verl's legacy bridge against its own locked + megatron-core). The rest were package, environment or cluster issues, listed in + [Software user experience](#software-user-experience). +- **NeMo-RL did not train.** Eleven launches used 52 GPU-hours on three nodes. Seven failed before its + NGC container ran under its own `ray.sub` launcher. The first clean start was too slow in eager mode, + and the next three deadlocked in the first trainer-to-vLLM weight sync with all 24 GPUs idle. + Upstream's reference MOPD recipe with only the model changed to Qwen3.6-35B-A3B showed why: vLLM + rejected the fused MoE expert weights, and NeMo-RL's driver kept waiting instead of failing. + +![Final-model accuracy](caesar-opd-2026-09-27/figures/quality.png) + +## Setup + +### Task and recipe + +| | | +|---|---| +| Student | `Qwen/Qwen3.6-35B-A3B` at revision `995ad96e`, 35B total / 3B active MoE, hybrid linear + full attention | +| Teacher | `semianalysisai/Qwen3.6-35B-A3B-caesar-cipher-GRPO-20260924`, tag `step-125` (`dfedf677`), fused into the base layout with `tools/fuse_teacher.py` | +| Task | reasoning_gym 0.1.25 `caesar_cipher`: 7,788 training prompts, 126 held-out dev prompts | +| Objective | Sampled-token reverse KL to the teacher (each framework's native OPD path); no task reward | +| Updates | 20, 128 prompts × 1 sample each, learning rate 1e-6 constant, Adam β2 0.999, weight decay 0 | +| Lengths | Thinking on; 32,768-token context, up to 30,720 generated tokens | +| Policy lag | At most 1 update (each framework's own staleness control) | +| In-loop evaluation | Greedy on the caesar dev set before training and after updates 10 and 20 (included in every wall time) | + +Campaigns were prepared with `tools/prepare.py --domains caesar_cipher`. With both domains, the same +package runs the two-teacher MOPD recipe. + +### Hardware and layout + +Two exclusive nodes of 8 × B200 (183 GB), Slurm on Kubernetes (slinky), Pyxis/Enroot for containers, +Lustre for `/shared`. Each framework uses one node for the trainer (8 GPUs, expert parallel 8, no context +parallelism) and one for generation: the teacher on GPU 0, GPU 1 idle, and the student policy on GPUs +2–7 as three TP2 engines. (verl's telemetry shows its policy on GPUs 1–6 with GPU 7 idle.) + +| | Miles | Prime-RL | Slime | verl | +|---|---|---|---|---| +| Upstream revision | `8f8e4dff` + package patch | `550beb6f` + package patch | `4c193f1f` + package patch | `6093e007`, unmodified | +| Runtime | Container `radixark/miles@sha256:59a11219…` | uv venv from its `uv.lock` | Same container as Miles | uv venv from its `uv.lock`, extras `vllm`+`fsdp` | +| Trainer | Megatron (image: Transformer Engine 2.17, Megatron `8c1e0574`) | FSDP2, custom MoE impl, fp32 master weights, optimizer state offloaded to CPU (defaults) | Megatron (same image) | FSDP (`fully_async_ppo_trainer`) | +| Policy serving | SGLang 0.5.19.dev52, 3 × TP2 | vLLM 0.28.0, 3 × TP2 behind vllm-router | SGLang, 3 × TP2 | vLLM 0.29.0, 3 × TP2 | +| Teacher serving | SGLang prefill-only | vLLM | SGLang prefill-only | vLLM (verl's teacher pool) | +| Scheduling | Fully async, domain-balanced buffer, staleness 1 | Continuous async, `max_off_policy_steps` 1 | `train_async`, one rollout ahead | Fully async, staleness 1, partial rollout | +| Final weights | `--save-hf` export | DCP checkpoint → `tools/convert_dcp_to_bf16.py` | `--save-hf` export | FSDP shards → `verl.model_merger` | + +### Benchmark method + +`tools/benchmark.py` serves each final model with vLLM 0.29 (one TP1 engine per GPU, eight GPUs) and +samples every dev prompt 8 times at the training sampling settings (temperature 1, top-p 1, thinking on, +up to 30,720 tokens). `shared/scoring.py` scores only the text after the last ``, so an answer +cut off mid-thought scores 0. avg@8 is the mean score; pass@8 is the share of prompts solved at least +once. + +Telemetry comes from `shared/capture.py` (5-second `nvidia-smi`, `/proc` and Prometheus samples on both +nodes) and each framework's own per-update metrics; `tools/report.py` summarizes both. + +## Results + +### Final-model quality + +| Model | caesar avg@8 | caesar pass@8 | finished thinking | truncated | mean response | geometry avg@8 | geometry pass@8 | +|---|---|---|---|---|---|---|---| +| Base | 26.8% | 72.2% | 94.7% | 5.5% | 12.2k | 52.9% | 99.2% | +| Caesar teacher | 72.1% | 98.4% | 95.6% | 4.5% | 12.6k | 58.8% | 99.2% | +| Miles | 77.6% | 99.2% | 94.6% | 5.8% | 13.1k | 65.9% | 99.2% | +| Prime-RL | 77.9% | 99.2% | 94.8% | 5.7% | 13.3k | 70.4% | 100% | +| Slime | 77.7% | 99.2% | 95.8% | 4.6% | 12.7k | 71.8% | 100% | +| verl | 78.4% | 100% | 94.4% | 5.7% | 13.3k | 68.7% | 100% | + +Response length and truncation barely moved from the base, so the gains do not come from shorter or +cut-off answers. The teacher also beats the base on geometry (58.8% vs 52.9%), so some transfer from +caesar training is expected; the students went further than the teacher. + +### Training speed + +![Per-update metrics](caesar-opd-2026-09-27/figures/per_update.png) + +| | Miles | Prime-RL | Slime | verl | +|---|---|---|---|---| +| Median update | 119 s | 93 s | 197 s | 106 s | +| Sum of update times | 50.7 min | 32.5 min | 78.7 min | 34.2 min (19 of 20 logged) | +| Median trainer compute per update | 37 s | 24 s | 29 s | 40 s | +| Median wait for rollouts per update | 85 s | 57 s | 166 s | 56 s | +| Median weight sync | 2.9 s | 11.2 s | 2.9 s | 10.2 s | +| Trainer MFU (mean) | 3.5% | 6.4% (forward-backward only) | 12.2% (derived, see below) | 5.8% | +| Mean trained response | 8.9k | 5.5k | 13.1k | 12.8k | +| Samples dropped as stale | not logged | 862 | 0 (one rollout ahead) | 0 (partial rollouts resumed) | +| Reverse KL to teacher, update 1 → 20 | 0.0034 → 0.0012 | −0.0015 → 0.0004 | 0.0009 → 0.0002 | 0.0025 → 0.0011 | +| Training-rollout caesar score, first → last | 22% → 85% | 18% → 89% | 20% → 75% | 23% → 79% | + +Notes on comparability: + +- Update 1 includes the pre-training evaluation and warm-up, and update 11 (Miles, Slime) the second + in-loop evaluation, so compare medians rather than sums. +- verl logs no metrics line for its first update, so its sums cover 19 updates. +- Slime logs trainer TFLOPS but not MFU. Its MFU is `perf/actor_train_tflops` divided by the 2,250 TFLOPS + B200 peak that Miles records; Slime's longer trained sequences explain part of the gap. +- Each framework computes reverse KL on different quantities (Slime's uses rollout log-probabilities), so + compare trends, not values. +- Each framework times its phases differently (verl's "wait" is `timing_s/gen`, Prime-RL's is + `time/wait_for_batch`). `tools/report.py` records which metric feeds each field. + +### GPU utilization and energy + +![GPU utilization](caesar-opd-2026-09-27/figures/gpu_utilization.png) + +| | Miles | Prime-RL | Slime | verl | +|---|---|---|---|---| +| Policy GPUs, mean utilization | 79–86% | 86–87% | 79–84% | 62% | +| Trainer GPUs, mean utilization | 17% | 25% | 16% | 20% | +| Peak memory, trainer / policy GPU | 178 / 133 GB | 74 / 168 GB | 169 / 125 GB | 123 / 147 GB | +| Policy tokens generated (whole run) | 63.6M at 18.0k tok/s | 90.0M at 40.3k tok/s | 41.0M at 8.4k tok/s | not captured | +| Teacher prefill throughput | 14.4k tok/s | 8.9k tok/s | 6.5k tok/s | not captured | +| GPU energy, both nodes | 7.9 kWh | 5.2 kWh | 9.6 kWh | 7.6 kWh | + +- The trainer node idles for most of every update: the sawtooth is one training step followed by a + long wait for the next batch. +- Miles' trainer ran closest to the memory limit (178 of 183 GB), with no context parallelism at 32k + tokens per GPU. +- verl's first 22 minutes are empty: its vLLM teacher spent about 15 minutes compiling a FlashInfer MoE + kernel with every GPU idle (verl #5 below). +- Ray placed verl's pools opposite to the package's node roles, so its rollout ran on the node the site + file calls "trainer". The figure labels nodes by what they ran. +- The package's verl capture does not scrape vLLM's metrics endpoints, so verl has no token counters. + +### Where the time goes + +A caesar response averages 12–13k tokens, and the longest reach 30k at a few hundred tokens per second +per sequence. How each framework handles that tail sets its speed: + +- **Miles** blocks each update on a full batch of 128, so the slowest sample sets the pace (72% of each + update spent waiting). +- **Prime-RL** keeps generating, trains on what finishes first, and drops what goes stale: 862 samples + dropped, and trained responses average 5.5k tokens against the ~13k the model actually writes. Its + default concurrency (943 rollouts in flight) is sized for its default `max_off_policy_steps` of 8; with + the recipe's lag of 1 it logs "Discarded 911/1039 episodes (87.7%): stale=911 ... Review + max_off_policy_steps". Its policy engines still generated the most tokens: 90M at 40k tokens/s. +- **verl** resumes interrupted samples after each weight sync (partial rollout). Nothing is dropped; + 44% of samples at update 5 span two policy versions; trained responses average 12.8k. +- **Slime** generates exactly one rollout ahead and waits for all of it. Nothing is dropped and trained + responses are the longest (13.1k), but every update waits for its slowest sample (84% of each update, + 197 s median). Its engines generated 8.4k tokens/s combined, under half of Miles' or Prime-RL's, + because each batch ends with a few 30k-token sequences decoding alone (at the end of the first batch + the engines logged a single running request at about 280 tokens/s). + +At 20 updates, tail handling decided the time, not the result: Prime-RL matched the others while +training on its shortest, earliest-finishing samples. + +### Prime-RL: first run and rerun + +Prime-RL's first run (job 968) reached 58.0%, 20 points behind the others. An earlier version of this +report blamed Prime-RL; the cause was the package's config, which overrode Prime-RL's defaults in seven +places (listed under [Prime-RL](#prime-rl)). The rerun (job 1017) used upstream defaults with only the +shared recipe values set, keeping the recipe's policy lag of 1 and weight decay of 0. + +| | Job 968 (package overrides) | Job 1017 (upstream defaults) | +|---|---|---| +| caesar avg@8 / pass@8 | 58.0% / 96.0% | 77.9% / 99.2% | +| simple_geometry avg@8 | 58.2% | 70.4% | +| Master weights and gradient reductions | bf16 (forced by `model.vlm`) | fp32 | +| Policy engines | 128 sequences, 8,192 batched tokens, FlashInfer autotune off | vLLM defaults, autotune on | +| Rollouts in flight (max) | 256 | 1,024 (derived start: 943) | +| Policy tokens generated | 59.4M at 20.9k tok/s | 90.0M at 40.3k tok/s | +| Mean trained response / dropped as stale | 9.1k / 751 | 5.5k / 862 | +| Training-rollout caesar score, first → last | 27% → 64% | 18% → 89% | +| Median update / wall time | 105 s / 51.3 min | 93 s / 39.6 min | + +The rerun changed everything at once, so the cost of each override is not isolated. Precision is the +most likely cause: at a learning rate of 1e-6, an update smaller than half a bf16 step rounds away, and +every other trainer here keeps fp32 master weights. Sample selection is ruled out: the rerun trained on +even shorter samples and dropped more of them, yet matched the others. + +## Software user experience + +Every failure we hit, in order. "Package" means this repository's code, "upstream" the framework's own +code, and "environment" or "cluster" the machines it ran on. + +### Miles + +Miles was the most self-contained. The pinned upstream launcher plus a digest-pinned container ran the +whole recipe; its structured logs (`fn=train_actor phase=start/end`, `perf N: {...}` dicts) made every +phase easy to time; and `--save-hf` wrote a complete Hugging Face checkpoint, which no other framework +offered out of the box. + +| # | Symptom | Cause | Fix | Where | +|---|---|---|---|---| +| 1 | Placement check failed silently; the node process aborted 60 s later | The role process imported the image's own Miles (`1e3b7a08`), whose `_create_placement_group` returns a tuple, not the pinned revision's `PlacementGroupInfo` | Put the pinned source first on `sys.path` in the container role | Package | +| 2 | Crash at the first in-loop eval: `No module named 'pycosat'` | `reasoning-gym`'s `pycosat` has no cp312 wheel; installing `pydeps` from the host (Python 3.14) built a 3.14 extension | Install `pydeps` with the container's Python 3.12 | Package setup | +| 3 | Benchmark load failed: vision weights "not initialized" | The export holds only the language model but keeps the vision-language config | Benchmark with vLLM `language_model_only=True` | Benchmark tool | +| 4 | No training reward in the logs: `rollout/raw_reward` is 0 at every update (job 965) | In OPD, Miles' reward is the teacher's log-probabilities; the task score is computed only for evaluation | The package's reward hook scores each training sample with the shared verifier and stores it as `metadata["raw_reward"]`, which Miles logs instead of the constant 0 (the OPD loss does not read it). Rerun 1021 logs 22% → 85% | Package | + +Requests for upstream: a start-up version check (the image's Miles and the pinned source diverge +silently, which cost an allocation), and a stale-drop count from the fully async buffer (we could not +measure how much Miles discarded). + +### Prime-RL + +Prime-RL's configuration is the most explicit of the four: one validated pydantic schema, one resolved +JSON per component, and a `metrics.jsonl` with everything per step. Its documentation answered every +question we had (DCP export, native conversion, OPD configuration). + +| # | Symptom | Cause | Fix | Where | +|---|---|---|---|---| +| 1 | `uv sync` failed: `invalid type: boolean false, expected a timestamp string` in `uv.lock` | The lock needs uv ≥ 0.11.1 (`required-version`); we had 0.8.22 | uv 0.11.33 | Environment | +| 2 | Run died in 26 s writing its config | `make_config.py` serialized the recipe dict after `RLConfig.model_validate` had replaced nested dicts with config objects | Write `recipe-input.json` before validating | Package | +| 3 | Conversion job failed | Prime-RL nests `checkpoints/` under `ckpt.output_dir`; our chain pointed one level too high | Correct path | Package | +| 4 | Trainer died at its first step: `DSLRuntimeError` in `flash_attn/cute` (job 1015, the first run on upstream defaults) | Text-only training of this VLM checkpoint freezes the vision encoder but still runs it on dummy pixels every step. With fp32 master weights the encoder is fp32, and `attn='auto'` resolves to FlashAttention 4 on B200, which accepts only bf16/fp16. Upstream switches the encoder to SDPA only under Ulysses CP | Package patch: SDPA for a frozen Qwen3.5 vision encoder whenever text attention is FA4; text attention unchanged | Upstream | +| 5 | Teacher engine died at update 3: `CUDA out of memory. Tried to allocate 8.79 GiB` in `compute_logprobs` (job 1016) | OPD scores each sample with the teacher's prompt logprobs, computed in fp32 over the whole prefill (8.8 GiB for a 9.8k-token sample). vLLM's memory profiling does not reserve it, and at the default `gpu_memory_utilization=0.9` only 4 GiB was free | The teacher settings from upstream's OPD example (`configs/debug/algo/opd.toml`): `gpu_memory_utilization=0.5`, `enforce_eager` | Upstream default | + +The `.prime-v1` native conversion needs no manual step: the trainer writes it on first load (67 GB, a +few minutes), so the package's submit check that required it beforehand was removed. The only other +friction was uv: the version requirement surfaces as a TOML parse error, and the package's `--no-sync` +discipline is easy to break by running `uv run` without it. + +**Config audit.** The package's Prime-RL config, carried over from earlier work, overrode seven groups +of upstream defaults with no documented reason. Comparing every resolved setting of job 968 against a +config built from only the required keys found: + +| Override in job 968 | Upstream default | Finding | Now | +|---|---|---|---| +| `model.vlm` set, which Prime-RL's validator pairs with `optimization_dtype` and `reduce_dtype` = bf16 | No `model.vlm`; fp32 master weights and reductions | The `vlm` block is for multimodal data. Without it, upstream trains this checkpoint text-only and logs "Training a VLM checkpoint on text-only data; freezing the vision encoder" | Default: fp32 | +| Package source patch: SDPA for a frozen vision encoder under FlashAttention 4, only with `model.vlm` | Unmodified | Text-only training needs it too (job 1015, #4 above) | Extended to text-only training | +| Policy engines: `max_num_seqs=128`, `max_num_batched_tokens=8192`, `gpu_memory_utilization=0.8`, FlashInfer autotune off | vLLM defaults (256 sequences, 16,384 tokens, 0.9), autotune on | The package had disabled autotune after a crash it blamed on the 32k length. Re-tested on 2 GPUs: autotune crashes with an illegal memory access in `trtllm_bf16_moe` only with `max_num_seqs=128` (a known FlashInfer bug on B200, [flashinfer#4157](https://github.com/flashinfer-ai/flashinfer/issues/4157)). With the default 256 the engine is healthy in 150 s, also with `max_num_batched_tokens=8192` | Defaults, autotune on | +| Teacher engine: eager mode, `gpu_memory_utilization=0.8`, prefix caching off, 128 sequences, 4,096 batched tokens, `language_model_only` | vLLM defaults (CUDA graphs, 0.9, prefix caching on) | At the defaults the teacher ran out of memory (#5 above). Upstream's OPD example starts its teacher with `gpu_memory_utilization=0.5` and `enforce_eager`. The fused teacher has its vision weights | The example's two settings; the rest default | +| Router: `power_of_two`, circuit breaker off, 7,200 s timeout | Upstream's `start_router` arguments: `consistent_hash`, breaker on, 1,800 s | The breaker counts request errors, not slow requests, so the package's reason (long rollouts) did not apply | Upstream arguments | +| Trainer: optimizer CPU offload off, activation offload off, `impl`/`ep` pinned | Both offloads on; `impl`/`ep` auto (resolve to the same custom EP8) | No reason recorded | Defaults | +| Orchestrator concurrency 128 initial / 32 min / 256 max in flight | Derived from the KV cache (943 here), 1 min, 1,024 max | No reason recorded | Defaults | + +Kept as recipe values, the same for every framework: 20 updates, 128 prompts × 1 sample, 32,768-token +context, 30,720 response tokens, learning rate 1e-6, weight decay 0 (Prime-RL's default is 0.01), policy +lag 1 (`max_off_policy_steps`; Prime-RL's default is 8), thinking on, greedy evaluation every 10 updates. +The two-node layout (three TP2 policy engines, the teacher on one GPU, NCCL weight broadcast) is +unchanged. + +### verl + +verl has the broadest configuration surface, and most of our failures were configuration. Its examples +cover each piece we needed (multi-teacher OPD, the fully async trainer, FSDP for MoE) but not the +combination, and several defaults fail silently. + +| # | Symptom | Cause | Fix | Where | +|---|---|---|---|---| +| 1 | Node order rejected: `['gpu-nodes-3', 'gpu-nodes-1']` | Slurm lists nodes in its own record order, not alphabetically; the controller compared order | Compare the set of nodes; `srun -w` pins each role | Package | +| 2 | Driver failed at `ray.init` | Ray's `uv run` hook requires the working directory to hold `pyproject.toml` | `RAY_ENABLE_UV_RUN_RUNTIME_ENV=0`; workers already use the venv from `ray start` | Package | +| 3 | Trainer init: `Qwen3_5VLTransformerConfig.__init__() got an unexpected keyword argument` | `vanilla_mbridge=True` (the legacy bridge) is incompatible with the megatron-core in verl's own lock | Moved to FSDP (#4) | Upstream/package | +| 4 | Trainer init: `FLA is not installed` | flash-linear-attention is only in the `fsdp` extra, not `megatron` | FSDP with the `vllm`+`fsdp` extras, as in upstream `run_qwen3_8b_mopd_fsdp.sh` | Package | +| 5 | 15 minutes of idle GPUs at start-up | vLLM compiled FlashInfer's TRT-LLM MoE kernel for sm100 with `ninja`/`ptxas` on first use | Prebuilt `flashinfer-cubin` + `flashinfer-jit-cache` (cluster #5) | Environment | + +Configuration traps found by reading verl's source before launch: + +- **Multi-teacher entries load random weights.** Teachers added as `+distillation.teacher_models.` + do not inherit the `teacher_model` YAML entry and fall back to `RolloutConfig` defaults, including + `load_format: dummy`. Upstream's own MOPD example has this problem. With one teacher we used the + default `teacher_model` entry, as in the single-teacher example. +- **Turning partial rollout off trains truncated samples.** Every weight sync aborts in-flight requests; + with `partial_rollout=False` they are returned truncated rather than resumed or dropped. We left it on + (the default for this trainer). +- **The teacher's context must be one token longer** than prompt + response + (`max_model_len = prompt + response + 1`), as the examples compute. +- **Slurm's `ROCR_VISIBLE_DEVICES`**, set by the cluster's GPU plugin alongside `CUDA_VISIBLE_DEVICES`, + makes every verl worker refuse to start. The package now unsets it. + +verl logs the richest per-update metrics (staleness, partial ratio, per-phase timing), but only to its +console log (we found no TensorBoard files), and it prints no metrics line for update 1. + +### Slime + +Slime ran cleanly on its first launch, after one fix found by reading the package beforehand. Its hook +interface (custom reward, reward post-processing, sample stamping, train and eval log functions) let the +package record every training and eval sample and every driver phase to JSONL without touching the +trainer, and `--save-hf` wrote the final Hugging Face weights directly. It shares Miles' container, so it +inherited every container fix. + +| # | Symptom | Cause | Fix | Where | +|---|---|---|---|---| +| 1 | Would fail at import (found before launch) | The package's `source.patch` imported `record` from a module named `campaign`, since renamed to `opd_hooks` | Patch now imports `opd_hooks.record`; manifest hashes updated | Package | + +Minor friction: metric lines print some values as tensor reprs (`tensor([0.43], device='cuda:0')`) and +report TFLOPS but not MFU, so our parser unwraps them. The SGLang engines print harmless +connection-refused tracebacks during warm-up (`freeze_gc`) that look like failures in a log scan. +One-rollout-ahead mode is the simplest to reason about (lag exactly 1, nothing discarded) but the slowest +here, because each batch waits for its longest response. + +### NeMo-RL + +NeMo-RL (`NVIDIA-NeMo/RL` at `19244a09`, from the NGC container `nvcr.io/nvidia/nemo-rl:v0.7.0`) runs the +same objective as MOPD on its async GRPO trainer, with rollouts through NeMo Gym, which ships a +`reasoning_gym` server, so the task mapped directly. Its documentation keeps its two distillation paths +clearly apart, and every config key we set was checked against `grpo_math_1B.yaml` before any GPU time. + +The layout costs more than the others. MOPD teachers reserve whole nodes, and with more than one policy +node so does generation, so a 1-GPU teacher still takes a node and the smallest layout is 24 GPUs +(trainer, vLLM, teacher) against 16 elsewhere. There is no MOPD recipe for Qwen3.5 or 3.6; we took the +algorithm settings from the MOPD reference recipe (a Qwen3-1.7B self-distillation smoke test) and the +model settings from the Qwen3.5-35B-A3B GRPO recipes. + +| # | Symptom | Cause | Fix | Where | +|---|---|---|---|---| +| 1 | Driver failed at once: `uv` "Read-only file system" at `/root/.cache/uv` (job 985) | `ray.sub` starts every container without `--container-writable`, and upstream's flow runs `uv run` in the image | `PYXIS_CONTAINER_WRITABLE=1` (Pyxis' environment form of the flag). Earlier workarounds (`ray.sub`'s `UV_CACHE_DIR_OVERRIDE`, `--no-sync`, `PYTHONPATH`) were removed | Upstream | +| 2 | `uv run` tried to rebuild the editable `nemo_rl` in `/opt/nemo-rl`, also read-only (986) | The image's venv does not contain the project; `uv run` installs it on every launch | Same as #1 | Upstream | +| 3 | `No module named 'nemo_rl'` (987) | Our `--no-sync` workaround for #2 | Removed | Package | +| 4 | `cannot import name 'AutoProcessor' from 'transformers' (unknown location)` (988, 995) | The image's venv files are symlinks into its own uv cache; `ray.sub`'s documented `UV_CACHE_DIR_OVERRIDE` mounts over that cache and empties the venv | Do not use the override | Upstream | +| 5 | Ray head never started; 17 GB written to Lustre (994) | We set `HOME` for the whole allocation, and Enroot extracts containers under `$HOME` | Reverted | Package | +| 6 | Every vLLM worker failed: read-only `ray_executor.py.patch_lock` (996) | NeMo-RL patches vLLM's installed files at run time | Same as #1 | Upstream | +| 7 | About 21 minutes per rollout; trainer GPUs spin-waiting (1000) | All upstream Qwen3.5-35B-A3B recipes set `enforce_eager: true` (they generate at most 4k tokens); eager MoE decoding at 30k tokens is too slow | CUDA graphs on (the base config's default) | Upstream recipe | +| 8 | Deadlock in the first weight sync, all GPUs idle, no error in the driver log (1001–1003; TP2 and TP1 engines) | Job 1010 ran upstream's reference MOPD recipe with the model changed to Qwen3.6-35B-A3B and unbuffered output. At the same call, every vLLM worker logged `shard_dim=0 is not a valid data dimension for a 3D tensor (expected 1 or 2)` for the fused MoE experts. NeMo-RL catches the error and returns `False`, but the driver waits on the trainer's broadcast before checking vLLM's result | Unresolved | Upstream | + +Two more notes. The `v0.7.0` container is built from `19244a09`, one commit before the `v0.7.0` tag +(the tag only bumps docs and the version string), so the package pins that commit and refuses any other +`NEMO_RL_COMMIT`. NeMo Gym's `reasoning_gym` server installs an unpinned `reasoning-gym>=0.1.19` at +start-up; the package pins 0.1.25 with a uv constraint. + +The full record is in [`nemo-rl-log.md`](caesar-opd-2026-09-27/nemo-rl-log.md). +[`frameworks/nemo-rl/repro/`](../frameworks/nemo-rl/repro/README.md) reproduces #1, #4 and the +container's provenance in about two minutes each on one GPU, and `tools/reproduce.sh nemo-rl` reruns #8 +with the production configuration. + +### Cluster and environment + +| # | Issue | Effect | Resolution | +|---|---|---|---| +| 1 | `gpu-nodes-1`'s `/run/nvidia-persistenced/socket` is a deleted inode (link count 0) | Every Pyxis GPU container fails on that node | Container frameworks avoid the node; host-venv frameworks use it. Needs ops. | +| 2 | Slurm sets `NVIDIA_VISIBLE_DEVICES=void` for job steps | Containers start without GPUs or driver libraries | `NVIDIA_VISIBLE_DEVICES=all` (the package's container runtime already sets it) | +| 3 | Slurm node order is 2, 3, 0, 1, and Miles/Slime require the trainer IP to sort below the generation IP | Only a few node pairs can run the container frameworks | Pair gpu-nodes-0 (generation) with gpu-nodes-2 (trainer) | +| 4 | Host Python is 3.14; the runtimes are 3.12 | Packages with compiled extensions installed from the host are unusable | Install into each runtime with that runtime's Python | +| 5 | FlashInfer JIT-compiles on first use; compile caches live in per-job `/tmp` | Cold compiles cost idle GPU time on every new run | Prebuilt `flashinfer-cubin`/`flashinfer-jit-cache` at the venv's exact `flashinfer-python` version, and one persistent `/runtime-cache/` for Triton, Inductor, FlashInfer, vLLM and SGLang | +| 6 | The container image named in the setup guide is not on the cluster | Provenance gap | Used the digest-pinned `radixark/miles@sha256:59a11219…`; its Megatron revision matches the recorded one exactly | + +## Cost + +GPU-hours from Slurm accounting (elapsed time × GPUs held): + +| | GPU-hours | +|---|---| +| Miles first run (965) | 17.0 | +| Prime-RL first run (968) | 13.7 | +| verl training (966), of which ~4 idle in the FlashInfer compile | 19.0 | +| Slime training (979) | 23.8 | +| Failed training attempts (957–962, 967) | 4.4 | +| Final-model benchmarks (969–973, 977, 980) | 7.8 | +| Conversions and GPU/container debugging jobs | 1.3 | +| Prime-RL rerun: training (1017) | 10.6 | +| Miles rerun: training (1021) | 18.0 | +| Miles rerun: runtime set-up and benchmark (1020, 1022) | 1.4 | +| Prime-RL rerun: failed launches (1015, 1016), autotune tests (1011-1014), conversion and benchmark (1018, 1019) | 5.5 | +| NeMo-RL, 11 launches on 24 GPUs (985-1003), no training update | 52.3 | +| NeMo-RL reproduction jobs (1004-1010) | 11.2 | + +## Reproducing + +```bash +PARTITION= tools/reproduce.sh all +``` + +[`tools/reproduce.sh`](../tools/reproduce.sh) runs the whole benchmark with the production setup: + +1. Models, containers, conversions and runtimes. +2. For each framework: its campaign (`tools/prepare.py --domains caesar_cipher`), its Slurm + run, the final Hugging Face export and `tools/benchmark.py`. NeMo-RL's run is cancelled if its first + weight sync has not completed in 20 minutes. +3. The base and teacher benchmarks, then `tools/report.py`. + +Each step is skipped when its output exists. Settings and required site files are in +[setup](SETUP.md#one-command-toolsreproducesh). + +What has been verified: a resume over these runs (`NAME=caesar-01`) skipped every step and regenerated +`telemetry.json` identically, and the Miles rerun (1021) was `tools/reproduce.sh miles` into a new +campaign (`NAME=caesar-02`), covering preparation, runtime, the run, the export and the benchmark +unattended. The model downloads, container import and torch_dist conversion were already present, so +those steps have not been tested from scratch. + +Rebuild the charts with `uv run --with matplotlib python docs/caesar-opd-2026-09-27/build_charts.py`. + +## Limitations + +- One run per framework and one sampling seed per benchmark; differences of a few points are noise. +- Each framework ran in its own documented async mode, and "staleness 1" means something different in + each (see [Where the time goes](#where-the-time-goes)). Speed and quality reflect the mode as much as + the implementation. +- In-loop evaluations and start-up are inside every wall time; per-update medians exclude most of it. +- The package code was not frozen: fixes were copied into campaigns between attempts. Each run's + `launch-source.tar.gz` records the exact code it ran. +- Prime-RL's reported run (1017) ran a day after the others with an audited config. The other three + frameworks' configs have not had the same line-by-line audit against their defaults. + +## Artifacts + +On the cluster under `/shared/opd-runs/`: + +- Campaigns (results, logs, captures): `-caesar-01/` and `miles-caesar-02/`. NeMo-RL's runs + 985-1003 are in `nemo-rl-caesar-01/results/`. +- Final weights: + - Miles: `miles-caesar-02/checkpoints/miles-1021/hf/iter_19` (first run: `miles-caesar-01/checkpoints/miles-965/`) + - Prime-RL: `prime-rl-caesar-01/checkpoints/prime-rl-1017/checkpoints/step_20/weights` (first run: `prime-rl-968/`) + - Slime: `slime-caesar-01/checkpoints/slime-979/hf/iter_19` + - verl: `verl-caesar-01/checkpoints/verl-966/global_step_20/actor/huggingface-merged` +- `benchmarks/`: per-response outputs and summaries. + +In this repository: summaries and telemetry in [`caesar-opd-2026-09-27/data/`](caesar-opd-2026-09-27/data/), +figures and the scripts that draw them in [`caesar-opd-2026-09-27/`](caesar-opd-2026-09-27/). diff --git a/docs/MOPD-H100-2026-10-07.md b/docs/MOPD-H100-2026-10-07.md new file mode 100644 index 0000000..ca1c171 --- /dev/null +++ b/docs/MOPD-H100-2026-10-07.md @@ -0,0 +1,95 @@ +# MOPD on Miles, 24 H100 80 GB (CoreWeave) + +Two-teacher MOPD (`caesar_cipher`, `simple_geometry`) on Miles with the `h100x3` layout ([setup](SETUP.md#h100-layout-h100x3-miles-only)), CoreWeave yolo-new, 2026-10-07. The reference run is B200 Miles run 1042 (16 B200, 2026-09-29, data in [`mopd-2026-09-29/data`](mopd-2026-09-29/data)). These are single observations, and the hardware, layouts and Miles settings differ (see [the Miles notes](../frameworks/README.md#miles)). + +Data: [`mopd-h100-2026-10-07/data`](mopd-h100-2026-10-07/data) (benchmark `summary.json` files and `telemetry.json` for runs 533 and 535). Plots: [`mopd-h100-2026-10-07/plots`](mopd-h100-2026-10-07/plots). + +## Summary + +| | B200 run 1042 | H100 run 535 | +|---|---|---| +| GPUs | 16 B200 | 24 H100 80 GB | +| caesar_cipher avg@8, after 10 / 20 updates | 0.708 / 0.769 | 0.732 / 0.773 | +| simple_geometry avg@8, after 10 / 20 updates | 0.997 / 0.996 | 0.996 / 0.998 | +| Sum of the 20 update steps | 2,737 s | 2,924 s | +| Median update | 87 s | 123 s | +| Wall time, allocation start to end | 3,721 s | 4,184 s | +| Teacher prefill tokens per trained response token | 3.4 (47M / 14M) | 2.4 (37M / 15M) | + +The student matches or beats both teachers on their own task: 0.773 against 0.703 on caesar_cipher, and 0.998 against 0.996 on simple_geometry. It ends level with the B200 run. With 24 H100 the 20 update steps took 7% longer than with 16 B200. + +## Benchmarks + +`tools/benchmark.py`: 8 samples per dev prompt (126 caesar_cipher, 128 simple_geometry), temperature 1, thinking on, up to 30,720 response tokens, four TP2 vLLM engines on 8 H100. + +| Model | caesar avg@8 | caesar pass@8 | caesar truncated | simple_geometry avg@8 | simple_geometry pass@8 | +|---|---|---|---|---|---| +| Base Qwen3.6-35B-A3B | 0.270 | 0.643 | 4.3% | 0.553 | 1.000 | +| caesar_cipher teacher | 0.703 | 0.992 | 6.1% | 0.590 | 1.000 | +| simple_geometry teacher | 0.282 | 0.675 | 4.8% | 0.996 | 1.000 | +| Student, 10 updates (run 535) | 0.732 | 0.984 | 6.2% | 0.996 | 1.000 | +| Student, 20 updates (run 535) | **0.773** | 0.992 | 5.4% | **0.998** | 1.000 | +| B200 student, 20 updates (run 1042) | 0.769 | 1.000 | 6.4% | 0.996 | 1.000 | + +Mean response length for the final student: 13,044 tokens on caesar_cipher and 2,877 on simple_geometry. + +## Layout + +| | B200 run 1042 | H100 run 535 | +|---|---|---| +| Trainer | 8 B200, Megatron EP8 | 8 H100, Megatron TP2 + SP, CP2, EP8, CPU Adam (16 threads per rank) | +| Policy | 6 GPUs, 3 TP2 engines | 12 GPUs, 6 TP2 engines over 2 nodes | +| Teachers | 1 GPU each | 2 GPUs each (TP2) | +| In-flight samples | 256 | 160 | +| Domain balance | Evicts surplus finished samples | Submits the task with fewer pending samples; keeps surplus | +| Online eval | Steps 0, 10, 20 | Step 20 only | + +## Where the time goes (run 535) + +| | Median | Total | +|---|---|---| +| Update step | 123 s | 2,924 s | +| Train | 91 s | 2,086 s | +| Wait for rollouts | 12 s | 838 s | +| Weight sync | 5.4 s | 131 s | + +- **Trainer.** It is the larger cost. After update 1 (1,460 tokens/s) it trained 5,200-19,300 tokens/s on 8 GPUs, median about 10,000. Run 533, without context parallelism, managed 2,400-4,500. +- **Rollout waits.** Most updates wait 5-15 s. Updates 1, 5, 6, 9, 11, 14, 15 and 18 waited 50-200 s, when long caesar_cipher samples held the batch. Update 11's wait includes the step-10 checkpoint save. +- **Wall time outside the steps.** About 1,260 s: startup (Ray, engines and teachers, about 10 minutes), the final greedy evaluation and checkpoint saves. +- **Mean GPU utilization.** + - Trainer: 25-43%, with uneven data-parallel ranks on long sequences. + - Policy: 74-83%. + - Teachers: 4-12%. A teacher only prefills, and it needs two 80 GB GPUs for its weights alone. +- **Energy.** 7.6 kWh of GPU energy in total. + +## Runs before 535 + +| Run | Result | Change after it | +|---|---|---| +| 531 | The caesar_cipher teacher died at startup on `OSError: Stale file handle`. All three nodes shared one Triton cache on NFS | One kernel cache per role | +| 532 | About 60 s per update with every trainer GPU idle ([dashboard](mopd-h100-2026-10-07/plots/dashboard-miles-532.png)). Ray gives each actor one OpenMP thread, so CPU Adam ran on one core per rank. Stopped after 3 updates | 16 threads per trainer rank; the gap fell to 5-10 s | +| 533 | 14 updates, then `CUDA out of memory` in the backward pass of a full 32k-token sequence as responses grew (telemetry in `data/telemetry.json`) | Context parallelism 2, with 16,384 tokens per GPU and 32k per micro-batch | +| 535 | 20 updates, exit 0 | | + +## Train-batch verifier score (run 535) + +| Update | 1 | 5 | 10 | 15 | 20 | +|---|---|---|---|---|---| +| caesar_cipher | 0.28 | 0.42 | 0.78 | 0.72 | 0.92 | +| simple_geometry | 0.48 | 0.83 | 1.00 | 1.00 | 1.00 | + +The reverse KL to the teacher fell from 0.0038 to 0.0012 nats per token. Trained caesar_cipher responses average 8,000-12,000 tokens (p90 13,000-21,000), and under 1% of trained samples hit the response limit. Samples that take more than about two updates to generate are dropped as stale at policy lag 1, so the trained batches lean toward shorter responses than the benchmark's (13,000 tokens mean). + +![Run 535](mopd-h100-2026-10-07/plots/dashboard.png) + +`tools/plot.py RESULT_DIR OUT` draws this dashboard from files every run already writes. It shows per-update timing, score, response length and truncation, and reverse KL; GPU utilization, memory and power by role (trainer, policy, teacher); policy generation and teacher prefill throughput; and teacher latency per sample. + +## Reproducing + +On a Slurm login node with Pyxis, with a site file that names `generation_node_2` and `generation_ip_2`: + +```bash +PARTITION=h100 SITE=site.json RUNS=/mnt/data/opd/runs RUNTIME=/mnt/data/opd/runtime SHARED=/mnt/data \ + DOMAINS="caesar_cipher simple_geometry" NAME=mopd-h100 LAYOUT=h100x3 BENCH_TP=2 \ + tools/reproduce.sh miles references +``` diff --git a/docs/PROVENANCE.md b/docs/PROVENANCE.md index d1e17b7..8342701 100644 --- a/docs/PROVENANCE.md +++ b/docs/PROVENANCE.md @@ -1,51 +1,49 @@ # Provenance and interpretation +## Current experiment + +The recipe in `shared/recipe.py` runs 20 updates with policy lag 1, thinking on and a 32k context. See the [framework overview](../frameworks/README.md#shared-recipe-sharedrecipepy) for the full settings. + +This code has run the single-teacher `caesar_cipher` variant; see the [benchmark report](CAESAR-OPD-2026-09-27.md). It has not run the two-teacher experiment. + ## Source identity -| Framework | Upstream revision | Execution | +| Framework | Upstream revision | Patch | |---|---|---| -| Miles | `8f8e4dff55d80bd9d36c3490c333a119f38480ed` | Fully async candidate-based MOPD with a balanced consumer buffer. | -| Prime-RL | `550beb6f431b44390afde6b705919765bbe33289` | Native asynchronous per-source OPD through reference KL. | -| Slime | `4c193f1f37509cca70f0e88807a9305b70f63f4e` | Native batched asynchronous OPD through an advantage correction. | +| Miles | `8f8e4dff55d80bd9d36c3490c333a119f38480ed` | Yes | +| NeMo-RL | `19244a0949fb363326f71675223da75e09d14866` (NGC container `nvcr.io/nvidia/nemo-rl:v0.7.0`) | No | +| Prime-RL | `550beb6f431b44390afde6b705919765bbe33289` | Yes | +| Slime | `4c193f1f37509cca70f0e88807a9305b70f63f4e` | Yes | +| verl | `6093e007cc341973c9d9a6fb3867a85976c7c458` | No | + +verl and NeMo-RL were added after the previous experiment. + +For Miles, Prime-RL and Slime, `frameworks//upstream/manifest.json` also records the patch digest, the digest of the original successful-run source archive, the original run name, and `recorded_campaign_sha256` (hashes of the campaign files as the previous experiment ran them). The original archives and raw results were kept separately and are not in this repository. -Each `manifest.json` records the SHA-256 digest of the original successful-run source archive. -It also records hashes of the original campaign files before template substitutions and the exact framework patch digest. -Original archives and raw results were retained separately. They are not copied into this repository. +What the patches do: -Miles starts from the preserved revision associated with [PR 3116](https://github.com/radixark/miles/pull/3116). -Its patch requests and retains behavior-policy candidate scores in the class-based generator and exposes async launch settings. -The candidate objective and teacher routing remain those of the pinned revision. -The campaign supplies a bounded buffer that consumes 64 active samples per domain per update and rejects excessive staleness. -It also retries transient frozen-teacher transport errors with a bounded attempt count. +- **Miles** starts from the revision associated with [PR 3116](https://github.com/radixark/miles/pull/3116). Its patch requests and keeps behavior-policy candidate scores and exposes async launch settings. The current recipe uses sampled-token OPD, so it does not use the candidate scores. Its other changes are launcher arguments, not patch changes. +- **Prime-RL**'s patch selects SDPA for the frozen vision encoder and keeps FA4 for text attention. The aggregate DP3 configuration is only a resolver input. +- **Slime**'s patch adds baseline evaluation and timing, handles immutable SGLang arguments, opens and closes the required weight-update sessions, and uses native NCCL groups for the non-offloaded trainer. Its hooks route each task to its teacher. -Prime-RL retains its native OPD objective. Its patch selects SDPA for the frozen vision encoder while retaining FA4 for text attention. -The campaign launches three independent TP2 policy engines and a native router. -The aggregate DP3 configuration is a resolver input; it is not the executable engine configuration. +In every framework, task scores are diagnostic and do not enter the objective. -Slime retains its native OPD objective. Its patch adds baseline evaluation and timing, handles immutable SGLang arguments, -opens and closes required weight-update sessions, and uses native NCCL groups for the non-offloaded trainer. -Its reward hook routes each puzzle domain to the corresponding frozen teacher. -Task rewards remain diagnostic and do not enter the OPD objective. +## Packaging and restructuring -## Packaging changes +The package replaces the original machine paths, node names and addresses with values from the site file. It replaces the original submission wrappers with a dry-run-first wrapper, and keeps upstream source as pinned revisions plus patches. -This package replaces original machine paths, node names, and addresses with explicit site values. -It selects network addresses from that site file rather than inferring them from a public-route socket. -It passes the configured Megatron source path to the Miles launcher. -It replaces the original submission wrappers with a dry-run-first wrapper that requires `--submit`. -It compresses the unchanged datasets and checks their original byte hashes after decompression. -It keeps upstream source as pinned Git revisions plus patches rather than vendoring whole repositories. -Generated source archives exclude Git metadata and virtual environments. Dataset and training source contents remain included. +After the previous experiment, the code was restructured for readability: shared code moved to `shared/` and the preflight tests were removed. The previous experiment wrote `slime-opd-416`, `mopd-async-401` and `prime-opd-407`; new runs write `results/-`. -These changes have passed local package checks. They have not received a new GPU run. -Original runtime failures and compatibility repairs are documented in the framework patches and setup requirements. -The operator must not treat local syntax checks as a substitute for distributed runtime validation. +Most campaign files no longer match `recorded_campaign_sha256`. The pre-restructure package, including the previous data, is at commit `3e27ddb`. There, the files that held site values are templates and do not match, and neither does `teacher-provenance.json`. The other files match. -## Recorded outcomes +## Previous experiment -All experiments completed 40 updates with 128 consumed samples per update on 16 B200 GPUs. -The following scores use 512 development examples per domain and a 256-token response limit. -They are results from one run per framework and are not new held-out test results. +The previous experiment used Countdown and Graph Coloring puzzles with thinking off: 40 updates of 128 samples, learning rate 1e-6, a 256-token response limit and policy lag 2, on 16 B200 GPUs. Its teachers were: + +- `semianalysisai/Qwen3.6-35B-A3B-countdown-GRPO-20260909` at `237a7f0345e883705026acd7f0745a3637042f73` +- `semianalysisai/Qwen3.6-35B-A3B-graph-color-GRPO-20260909` at `ec87f87177a4ced7256928ce67c438fafa73c28e` + +Scores use 512 development examples per domain, from one run per framework. | Framework | Countdown baseline / final | Graph Coloring baseline / final | Successful allocation | |---|---|---|---| @@ -53,26 +51,20 @@ They are results from one run per framework and are not new held-out test result | Prime-RL | 10.94% / 39.65% | 25.78% / 64.65% | 24m 22s | | Slime | 10.94% / 44.14% | 26.37% / 89.26% | 13m 25s | -Allocation time includes startup, training, evaluations, checkpoint work, and cleanup in the successful attempt. -It excludes earlier failed attempts. It is not isolated trainer compute time or pure generation throughput. -The runs differ in objective, scheduler, source mixture, runtime, and checkpoint content. -Prime-RL intermediate evaluations can span changing weights; its baseline and final evaluations used fixed versions. -Slime pauses at weight broadcast boundaries, while Miles and Prime-RL use continuous async scheduling. -All runs bounded accepted policy lag at two versions, using each campaign's recorded version convention. -Maximum useful GPU utilization was not demonstrated. +How to read these numbers: + +- Miles used candidate-based OPD in this experiment. +- Allocation time covers startup, training, evaluations, checkpoint work and cleanup in the successful attempt. It excludes failed attempts and does not measure compute or generation throughput. +- Prime-RL's intermediate evaluations could span changing weights; its baseline and final evaluations used fixed versions. +- Slime paused at weight-broadcast boundaries; Miles and Prime-RL scheduled continuously. +- The runs differ in objective, scheduler, source mixture, runtime and checkpoint content. Maximum useful GPU utilization was not demonstrated. ## Data and licenses -The files under `data/` are the exact synthetic Countdown and Graph Coloring inputs used in the recorded experiments. -They contain 10,000 training examples per domain, 512 development examples per domain, and separate test files. -The mixed training file contains 20,000 rows. The development split was used for the reported scores. -The pinned Miles source contains the puzzle preparation and scoring implementation under `examples/mopd_puzzles/`. -Preserve `data/manifest.json` when copying the data so the exact inputs can be checked. +`data/` holds reasoning_gym 0.1.25 rows in the default task configuration, with reasoning_gym's default system prompt. Train rows use seed 20000 and dev rows seed 10000, so the problems are disjoint. There are 7,788 train rows per task, 126 `caesar_cipher` and 128 `simple_geometry` dev rows, and a 15,576-row mixed training file. Keep `data/manifest.json` with the data so the inputs can be checked. + +Upstream source keeps its original license; a copy sits next to each patch as `upstream/LICENSE`. Models keep their own licenses, and this package grants no rights to weights. -Upstream source remains subject to its original license. Copies of the top-level upstream license files accompany each patch. -Hugging Face models remain subject to their model licenses. This package does not grant additional rights to model weights. -No new blanket license has been assigned to this repository's campaign code. -The repository owner should select its distribution license before public release. +No license has been assigned to this repository's own code yet. The owner should choose one before public release. -The old experiments and logs were removed from the current layout, but remain recoverable in prior Git history. -This refresh does not constitute a credential audit of historical commits or a history rewrite. +Old experiments and logs remain in Git history. This refresh was not a credential audit or a history rewrite. diff --git a/docs/SETUP.md b/docs/SETUP.md index c4c94dd..7bf6e91 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -1,143 +1,213 @@ # Setup and execution -## Supported layout and prerequisites +## Prerequisites -The recorded layout uses Linux x86_64, Python 3.12, Slurm, a shared writable filesystem, and two exclusive eight-GPU B200 nodes. -Miles and Slime also require Slurm Pyxis/Enroot support for `--container-image`, root remapping, and writable ephemeral containers. -Their commands request 80 CPU cores per node and expose 64 cores to Ray. -Prime-RL runs from a virtual environment on the hosts and also requests 80 CPU cores per node. -This package does not install Slurm, configure Kubernetes, modify cluster services, or create a container image. +- Linux x86_64, Python 3.12, Slurm and a shared writable filesystem. +- Two exclusive eight-GPU B200 nodes. NeMo-RL needs a third, because its teacher reserves a whole node. Miles also runs on three eight-GPU H100 80 GB nodes (the `h100x3` layout, below). +- For Miles, Slime and NeMo-RL: Pyxis/Enroot with `--container-image`, root remapping and writable containers. Prime-RL and verl run from virtual environments on the hosts. +- Each node requests 80 CPU cores and exposes 64 to Ray. -The generation node must appear first in the expanded Slurm node list. -The trainer IPv4 address must sort numerically before the generation address because the recorded Ray placement uses that order. -The site example uses documentation-only addresses. Replace them with routable addresses between your nodes. -The preparation tool rejects unsupported ordering. The GPU preflight independently checks physical placement before loading the models. -The two framework roles use trusted cluster networking; do not expose Ray or model endpoints to the public internet. +This package does not install Slurm, change cluster services or build a container image. -The asset directories must be under the configured `assets` path so the containers can access them. -The prepared campaign directory and asset paths must be identical on both nodes. -Paths must not contain spaces, quotes, or shell metacharacters. +Site file rules, enforced by `tools/prepare.py`: -## Pinned models +- For Miles and Slime, the trainer IPv4 address must sort before the generation address. Both also check the Ray GPU placement before training. +- `base_model`, `megatron_model` and `teachers` must be under `assets`. +- Paths must be absolute and contain no spaces or shell metacharacters. The campaign and asset paths must be the same on every node. -The exact model identities are as follows. Model weights are not stored in this repository. -Use an authenticated Hugging Face client when required. Keep its token outside this repository and outside `site.local.json`. +Replace the documentation-only addresses in `config/site.example.json` with routable ones. Do not expose Ray or model endpoints publicly. + +### H100 layout (`h100x3`, Miles and verl) + +A 35B teacher does not fit on one 80 GB GPU, and Adam states for 35B do not fit next to 32k-token activations. Without context parallelism the trainer ran out of memory in the backward pass of a full-length sequence once responses grew (run 533, update 15). `tools/prepare.py miles --layout h100x3` or `tools/prepare.py verl --layout h100x3` (or `LAYOUT=h100x3` for `tools/reproduce.sh`) therefore uses three nodes: + +| Node | GPUs 0-1 | GPUs 2-7 | +|---|---|---| +| Trainer | Miles: Megatron TP2 with sequence parallelism, context parallelism 2, EP8, Adam offloaded to the CPU. verl: FSDP, Ulysses sequence parallelism 2, optimizer offloaded to the CPU | (same) | +| Generation 1 | caesar_cipher teacher (TP2) | Policy, three TP2 engines | +| Generation 2 | simple_geometry teacher (TP2) | Policy, three TP2 engines | + +The policy gets 12 GPUs, twice the six of `b200x2`. verl places its own trainer, teacher and policy pools on the three-node Ray cluster, so which node and GPU indices each pool gets is Ray's choice; the pool sizes are the same. The site file adds `generation_node_2` and `generation_ip_2`; the trainer IP must sort below both generation IPs. Each node requests all 128 CPU cores and its whole memory (`--mem=0`), and each trainer rank gets 16 OpenMP threads for the CPU Adam step. For the benchmarks set `BENCH_TP=2`, since one model and its 32k cache do not fit on one GPU. If the shared filesystem is not `/shared`, set `SHARED` to its mount point. + +## One command: `tools/reproduce.sh` + +`tools/reproduce.sh` runs every step in this guide for the single-teacher `caesar_cipher` benchmark in the [report](CAESAR-OPD-2026-09-27.md), then benchmarks each saved checkpoint and writes the telemetry report. + +```bash +PARTITION=YOUR_PARTITION tools/reproduce.sh all # references, every framework, then the report +PARTITION=YOUR_PARTITION tools/reproduce.sh verl slime # chosen frameworks +PARTITION=YOUR_PARTITION tools/reproduce.sh references report # base and teacher benchmarks; telemetry summary +``` + +Run it on a Slurm login node with Pyxis/Enroot, after filling in `config/site.local.json`. For NeMo-RL, also copy `config/site-nemo-rl.example.json` to `config/site-nemo-rl.local.json` and fill it in. + +For each framework the script: + +1. downloads the models and fuses the teacher; +2. prepares `$RUNS/-$NAME`; +3. imports the pinned container, converts the Megatron checkpoint (Miles and Slime) and builds the runtime: pydeps with the runtime's own Python, Slime's wheels, the uv venvs and prebuilt FlashInfer; +4. submits the run and blocks until the allocation ends; +5. exports Hugging Face weights of each saved checkpoint and benchmarks them. + +A step is skipped when its output exists, so rerunning resumes. Settings (`RUNS`, `RUNTIME`, `NAME`, `GPU_NODE`, `BENCH_CAMPAIGN`, `DEADLOCK_MIN` and the site files) are listed at the top of the script. + +For NeMo-RL the script also runs a watchdog. If the first trainer-to-vLLM weight sync has not completed `DEADLOCK_MIN` minutes (default 20) after it starts, the watchdog writes the vLLM workers' weight-update errors to the run's `DEADLOCK` file and cancels the job. Otherwise the job would hold 24 GPUs until its time limit. See [the NeMo-RL reproduction](../frameworks/nemo-rl/repro/README.md). + +What has been tested: + +- A resume over this repository's finished runs (`NAME=caesar-01`): every step was skipped and the report was regenerated identically. +- `tools/reproduce.sh miles` into a new campaign (`NAME=caesar-02`, job 1021): it prepared the campaign, built its pydeps, ran 20 updates, exported the Hugging Face weights and benchmarked them unattended. `BENCH_CAMPAIGN` pointed it at an existing verl venv. + +The downloads, container imports, `torch_dist` conversion and venv builds use the commands in this guide, but the script has not yet run them from an empty `RUNS`. + +## Models + +Revisions are pinned in `shared/recipe.py`. Weights are not stored here. Keep any Hugging Face token outside the repository and the site file. | Role | Hugging Face repository | Revision | |---|---|---| | Base | `Qwen/Qwen3.6-35B-A3B` | `995ad96eacd98c81ed38be0c5b274b04031597b0` | -| Countdown teacher | `semianalysisai/Qwen3.6-35B-A3B-countdown-GRPO-20260909` | `237a7f0345e883705026acd7f0745a3637042f73` | -| Graph Coloring teacher | `semianalysisai/Qwen3.6-35B-A3B-graph-color-GRPO-20260909` | `ec87f87177a4ced7256928ce67c438fafa73c28e` | - -The following commands download the pinned snapshots. They can consume substantial storage and bandwidth. They do not launch training. -Use paths that match your site configuration. +| `caesar_cipher` teacher | `semianalysisai/Qwen3.6-35B-A3B-caesar-cipher-GRPO-20260924` | `dfedf677a5bf08a451733ec48c845c2661a6ab14` (tag `step-125`) | +| `simple_geometry` teacher | `semianalysisai/Qwen3.6-35B-A3B-simple-geometry-GRPO-20260925` | `679d4b66018c03249a3bfd0ecca3195ed9cd0e0d` (tag `step-50`) | ```bash hf download Qwen/Qwen3.6-35B-A3B --revision 995ad96eacd98c81ed38be0c5b274b04031597b0 --local-dir /shared/opd-assets/models/Qwen3.6-35B-A3B -hf download semianalysisai/Qwen3.6-35B-A3B-countdown-GRPO-20260909 --revision 237a7f0345e883705026acd7f0745a3637042f73 --local-dir /shared/opd-assets/teachers/countdown -hf download semianalysisai/Qwen3.6-35B-A3B-graph-color-GRPO-20260909 --revision ec87f87177a4ced7256928ce67c438fafa73c28e --local-dir /shared/opd-assets/teachers/graph_color +hf download semianalysisai/Qwen3.6-35B-A3B-caesar-cipher-GRPO-20260924 --revision dfedf677a5bf08a451733ec48c845c2661a6ab14 --local-dir /shared/opd-assets/hub/caesar_cipher +hf download semianalysisai/Qwen3.6-35B-A3B-simple-geometry-GRPO-20260925 --revision 679d4b66018c03249a3bfd0ecca3195ed9cd0e0d --local-dir /shared/opd-assets/hub/simple_geometry +``` + +The Hub teachers use Prime-RL's per-expert layout. Every framework serves copies fused into the base model's layout, with the base model's MTP tensors. The site's `teachers` directory must hold one fused copy per trained task (`caesar_cipher/`, `simple_geometry/`). `tools/fuse_teacher.py` needs torch and safetensors: + +```bash +python3 tools/fuse_teacher.py /shared/opd-assets/hub/caesar_cipher /shared/opd-assets/teachers/caesar_cipher /shared/opd-assets/models/Qwen3.6-35B-A3B +python3 tools/fuse_teacher.py /shared/opd-assets/hub/simple_geometry /shared/opd-assets/teachers/simple_geometry /shared/opd-assets/models/Qwen3.6-35B-A3B ``` -Download the teachers as frozen scoring models. MOPD trains the student, not these teachers. -The data files are included, so do not regenerate a different dataset when reproducing the recorded experiment. +The teachers are frozen. Use the included data; do not regenerate it. ## Prepare a campaign ```bash -cp site.example.json site.local.json -# Edit site.local.json before continuing. -python3 tools/prepare.py slime --site site.local.json --output /shared/opd-runs/slime-01 +cp config/site.example.json config/site.local.json # Edit before continuing. +python3 tools/prepare.py slime --site config/site.local.json --output /shared/opd-runs/slime-01 +``` + +Add `--domains caesar_cipher` for single-teacher OPD on one task. This writes the campaign's `experiment.json`. + +Preparation: + +- checks out the pinned upstream revision and applies the checked patch, if any; +- verifies every dataset hash; +- copies `shared/` and the framework's files, and links the models; +- writes `site.json`, which campaign processes read at runtime, and `package-provenance.json`. + +It does no model conversion, environment installation or submission. If it fails, use a fresh output path. + +Every framework except NeMo-RL scores with `reasoning-gym==0.1.25` from the campaign's `pydeps/`. Install it with a Python 3.12 Linux x86_64 interpreter that matches the runtime (the container's Python for Miles and Slime, the venv's for Prime-RL and verl): + +```bash +python3 -m pip install --target /shared/opd-runs/slime-01/pydeps reasoning-gym==0.1.25 ``` -The tool checks out the recorded revision, applies the checked patch, expands the site values, and verifies every dataset hash. -It creates model symlinks and retains `site.json` plus `package-provenance.json` in the new campaign. -The tool performs no model conversion, environment installation, scheduler submission, or automatic retry. -If preparation fails, inspect the partial directory and select a fresh output path for the next attempt. +`$CAMPAIGN_PYDEPS` appends this directory to `sys.path`, so runtime packages take precedence. ## Miles and Slime runtime -Both experiments reused a container identified as `miles-dev-202609050049.sqsh`. -The image bytes are not distributed here, and its name alone is not a content digest. -The recorded runtime contained PyTorch 2.13.0+cu130, SGLang 0.5.19.dev56+ga359142, Ray 2.58.0, Transformers 5.12.1, -Transformer Engine 2.17.0, Flash Linear Attention 0.5.2, and SGLang Router 0.3.2. -SGLang's recorded source revision was `a3591427c20431d33570f0c766d0cee6711f5ee8`. -The Megatron source revision was `8c1e05747eb612b382df2632783df5c83a853646`. -Set `megatron_source` to its path inside the container. - -Use the original compatible image or provision a compatible runtime separately. -The package does not claim that installing these version strings recreates the original CUDA binaries. -The original image used several local compiled wheels, so its package inventory is evidence rather than a universal installation lock. -The Slime package retains the inventory observed during the shared-image preflight in `recorded-runtime.json`. -The framework-specific preflight must pass in the actual runtime before training. - -Both frameworks require a converted Megatron `torch_dist` base checkpoint at `megatron_model`. -The recorded runs reused an existing conversion; no fresh conversion or restore test is claimed for this package. -Use the conversion implementation in the pinned upstream source and validate its configuration against the base model. -Preparation links the configured conversion to `models/Qwen3.6-35B-A3B_torch_dist`. -Model conversion can require GPUs and substantial storage, so it is not silently performed by preparation. - -Slime additionally installs NumPy 1.26.4 and SciPy 1.15.3 inside each ephemeral container. -Download the exact CPython 3.12 Linux wheels into the prepared campaign, then verify their recorded digests. +`tools/reproduce.sh` imports the digest-pinned image `radixark/miles@sha256:59a11219eae0defc6594ec678fafe4e897c16904263223f79968cd3e0209a502` to the site's `container_image`. The September 27 runs used it; its Megatron revision matches the recorded one below. Set `megatron_source` to Megatron's path inside the container. + +The previous experiment used an image named `miles-dev-202609050049.sqsh`. Its bytes are not distributed, and the name is not a digest. It contained: + +- PyTorch 2.13.0+cu130, SGLang 0.5.19.dev56+ga359142 (source `a3591427c20431d33570f0c766d0cee6711f5ee8`), Ray 2.58.0, Transformers 5.12.1 +- Transformer Engine 2.17.0, Flash Linear Attention 0.5.2, SGLang Router 0.3.2 +- Megatron `8c1e05747eb612b382df2632783df5c83a853646` + +Installing these versions does not necessarily recreate its CUDA binaries, because the image used locally compiled wheels. Slime's observed inventory is in `frameworks/slime/upstream/recorded-runtime.json`. + +Both frameworks need a Megatron `torch_dist` base checkpoint at `megatron_model`, linked into the campaign as `models/Qwen3.6-35B-A3B_torch_dist`. Use the pinned Miles converter (`tools/convert_hf_to_torch_dist.py`), as `tools/reproduce.sh` does. The recorded runs reused an existing conversion. + +Slime installs NumPy 1.26.4 and SciPy 1.15.3 in each container from checked wheels. Download them into the campaign first: ```bash python3 -m pip download --only-binary=:all: --no-deps --platform manylinux2014_x86_64 --python-version 312 --implementation cp --abi cp312 --dest /shared/opd-runs/slime-01/wheels numpy==1.26.4 scipy==1.15.3 -cd /shared/opd-runs/slime-01 -sha256sum -c wheels.sha256 +cd /shared/opd-runs/slime-01 && sha256sum -c wheels.sha256 ``` -Slime checks immutable SGLang arguments, weight-update session ordering, native NCCL collectives, asynchronous waits, and the campaign objective before training. -It uses native NCCL groups only with trainer offload and release disabled. -Miles checks candidate scoring, the domain-balanced async buffer, bounded scoring retries, evaluation, and TensorBoard initialization. +Slime uses native NCCL groups. This is safe only because trainer offload and release are disabled. ## Prime-RL runtime -Prime-RL requires a working environment at the prepared campaign's `source/.venv`. -The original environment used PyTorch 2.13.0+cu130 and vLLM 0.28.0. -Its recorded package inventory is `frameworks/prime-rl/recorded-packages.txt`. -The pinned source includes `uv.lock`, and preparation initializes its submodules at their recorded commits over HTTPS. -Follow the installation instructions in that pinned source. Ensure that the generated task environment is installed into the same environment. +Prime-RL needs a working environment at the campaign's `source/.venv`. Preparation initializes the pinned submodules over HTTPS. Build the environment from the pinned lock, then install the task environment: ```bash -# Run this only after provisioning source/.venv with the pinned framework dependencies. -/usr/local/bin/uv pip install --python /shared/opd-runs/prime-01/source/.venv/bin/python --no-deps -e /shared/opd-runs/prime-01/environments/mopd_puzzles +cd /shared/opd-runs/prime-01/source && uv sync --frozen --python 3.12 --all-extras +uv pip install --python /shared/opd-runs/prime-01/source/.venv/bin/python --no-deps -e /shared/opd-runs/prime-01/environments/rg_tasks ``` -Use the `uv` executable configured in your site file if its path differs. -The controller deliberately uses `--no-sync`, so submission does not silently replace the runtime. -Prime-RL requires a completed native conversion at `base_model/prime`, including `.prime-v1` and the safetensors index. -The preflight checks all indexed shard headers, lengths, tokenizer agreement, scoring cases, and the frozen-vision attention configuration. -A partial conversion is rejected. Provision and validate this conversion through the pinned Prime-RL implementation before submission. +Use the `uv` path from your site file. Runs use `--no-sync`, so submission never replaces the environment. The previous experiment used PyTorch 2.13.0+cu130 and vLLM 0.28.0; its inventory is `frameworks/prime-rl/upstream/recorded-packages.txt`. + +On its first run, Prime-RL's trainer converts the base model to its native layout in `base_model/prime`, so the base model directory must be writable. -## Inspect, submit, and monitor +## verl runtime -The following command prints the allocation request without executing it. +verl runs unmodified from the campaign's `source/.venv`. It is built from the pinned `uv.lock` with the extras of upstream `examples/on_policy_distillation_trainer/run_qwen3_8b_mopd_fsdp.sh`. It trains with FSDP and needs no Megatron conversion. ```bash -python3 tools/submit.py /shared/opd-runs/slime-01 --partition YOUR_PARTITION +cd /shared/opd-runs/verl-01/source && uv sync --frozen --python 3.12 --all-packages --extra vllm --extra fsdp ``` -Only the following form submits work. Run it on the Slurm controller or from a suitable Slurm pod. -For Kubernetes, enter the appropriate Slurm pod using your own context and namespace before running the command. -No particular Kubernetes context, credentials, or pod name is built into this repository. +## NeMo-RL runtime + +NeMo-RL runs unmodified from `nvcr.io/nvidia/nemo-rl:v0.7.0`, imported once with Enroot (43 GB) to the site's `nemo_rl_container`: ```bash -python3 tools/submit.py /shared/opd-runs/slime-01 --partition YOUR_PARTITION --submit +enroot import -o /shared/opd-runtime/nemo-rl-v0.7.0.sqsh 'docker://nvcr.io#nvidia/nemo-rl:v0.7.0' ``` -The submission process remains attached. Use your normal persistent terminal if the login connection can disconnect. -It holds a per-campaign lock and refuses another run when a previous terminal record is absent. -The controller stops its owned processes when training finishes or fails. It does not repair failures or resubmit automatically. -Read the job number from `active-run.json`, use `squeue` and `sacct` for scheduler state, and follow `results/*/learner-train.out`. -Use `scancel YOUR_JOB_ID` to cancel only the job you intend to stop. +The image is built from `19244a09` on `r0.7.0`, one commit before the `v0.7.0` tag. The controller refuses to start if the container's `NEMO_RL_COMMIT` differs. NeMo-RL needs no pydeps or conversion: NeMo Gym scores the rollouts, and the package pins its `reasoning-gym` to 0.1.25 with a uv constraint. + +The controller launches upstream's `ray.sub` inside the allocation with `PYXIS_CONTAINER_WRITABLE=1`, because NeMo-RL writes into its image at runtime. Do not set `ray.sub`'s `UV_CACHE_DIR_OVERRIDE`; it hides the image's venv. + +No NeMo-RL run has completed a training update. Every run deadlocked at the first weight sync. + +## Prebuilt FlashInfer kernels (Prime-RL and verl) + +Both locks install only `flashinfer-python`, which compiles or downloads kernels on first use. On B200 the MoE kernel alone leaves the GPUs idle for about 15 minutes. Install FlashInfer's prebuilt packages at exactly the venv's `flashinfer-python` version (0.6.16.post3 for Prime-RL, 0.6.18 for verl), then check `flashinfer show-config`: + +```bash +V=0.6.18; PY=/shared/opd-runs/verl-01/source/.venv/bin/python +uv pip install --python $PY --no-deps --index-url https://flashinfer.ai/whl/ flashinfer-cubin==$V +uv pip install --python $PY --no-deps --index-url https://flashinfer.ai/whl/cu130/ flashinfer-jit-cache==$V+cu130 +``` + +Every framework keeps its other compiled kernels (Triton, Inductor, vLLM, SGLang) in the campaign's `runtime-cache/`, so only a campaign's first run compiles them. + +## Submit and monitor + +```bash +python3 tools/submit.py /shared/opd-runs/slime-01 --partition YOUR_PARTITION # Prints the request. +python3 tools/submit.py /shared/opd-runs/slime-01 --partition YOUR_PARTITION --submit # Allocates 16 GPUs. +``` + +Run it on a Slurm login host or pod; there is no built-in Kubernetes support. It runs `python3 /node.py` under `salloc`. `node.py` is the controller: it starts `node.py learner` and `node.py generation` on the two nodes. + +- The command stays attached and holds a per-campaign lock. It refuses a new run while the previous one has no terminal record. +- The default time limit is 4 hours (`--time 04:00:00`). +- The controller stops its processes when training ends or fails. It never retries or resubmits. + +There is no automated preflight. `tools/submit.py` only checks that required files exist (models, teachers, pydeps, venv or container, and Slime's wheels). Validate the runtime, conversions and teacher endpoints yourself, for example with a short run, before a full experiment. + +The job number is in `active-run.json`. Each run writes `results/-/`, including `start.json`, `end.json`, `exit-code.txt`, `launch-source.tar.gz` and `-capture.jsonl.gz` (GPU, host and Prometheus samples). + +Driver logs: + +| Framework | Log | +|---|---| +| Miles, Slime, verl | `results//learner-train.out` | +| Prime-RL | Component logs in the directory named by `resolved-paths.json` | +| NeMo-RL | `results//*-logs/ray-driver.log` | -Each run retains source, launch commands, metrics, GPU samples, node counters, teacher timing, and component logs. -Slime also retains samples and rollout tensors. Prime-RL retains native compressed traces. -These files can contain full prompts and responses. Review them before sharing. -Raw logs and model checkpoints are excluded from this repository. +Results can contain full prompts and responses; review them before sharing. Cancel a run with `scancel YOUR_JOB_ID`. ## Known limits -The original runs used 16 B200 GPUs. This package does not establish support for a different GPU or node count. -The runtime and conversion requirements above prevent a claim of fresh-machine, one-command reproduction. -The package makes these requirements explicit and retains the scientific recipe rather than silently substituting another environment. -Perform a separately authorized GPU validation before advertising a new image or hardware configuration as equivalent. +Only 16 B200 GPUs (24 for NeMo-RL) and, for Miles, 24 H100 80 GB GPUs (`h100x3`) have been tested; no other GPU type or node count is supported. `tools/reproduce.sh` has not yet run on a fresh machine. Validate on GPUs before treating a new image or hardware configuration as equivalent. diff --git a/docs/VALIDATION.md b/docs/VALIDATION.md index f1f4929..5463d7a 100644 --- a/docs/VALIDATION.md +++ b/docs/VALIDATION.md @@ -1,32 +1,49 @@ -# Package validation +# Validation -The package was checked locally on September 15, 2026. No cluster job was submitted during packaging. +## What has run on GPUs -The following six tests passed under Python 3.12. +On September 27, 2026 the package ran the single-teacher `caesar_cipher` variant in Miles, Prime-RL, Slime and verl on 16 B200 GPUs. Every run completed 20 updates and was benchmarked; see the [report](CAESAR-OPD-2026-09-27.md). NeMo-RL deadlocked before its first update. On October 7, 2026 Miles ran the two-teacher MOPD recipe on 24 H100 80 GB GPUs (`h100x3`) for 20 updates, and its checkpoints were benchmarked; see the [H100 report](MOPD-H100-2026-10-07.md). + +## Package tests + +These need no GPU: ```bash python3 -m unittest discover -s tests -v ``` -The tests render and parse every campaign template, check every dataset digest and row count, -verify patch digests, reject invalid site values, preserve existing output directories, and confirm that submission defaults to a dry run. +They check that: + +- campaign files compile and contain no machine paths; +- dataset digests and row counts match; +- patch digests match; +- invalid site files are rejected; +- existing output directories are preserved; +- submission defaults to a dry run; +- telemetry finds the announced engine ports; +- hosted credentials are read as configured; +- Miles, Prime-RL, Slime and verl launch configurations enable thinking; +- the scorer requires the answer after `` (skipped unless `reasoning-gym==0.1.25` is installed). + +The hosted campaigns have their own offline tests; see their READMEs. -All framework patches were applied to clean archives of their pinned upstream revisions. -Every changed source file matched its counterpart from the original successful-run source archive by SHA-256. -Miles had four changed files, Prime-RL had one, and Slime had five. -The check can be repeated with local upstream Git clones as follows. +## Source patches + +`tools/verify_sources.py` applies each patch to a clean archive of its pinned revision and checks every changed file against the SHA-256 recorded in the manifest. verl and NeMo-RL run unmodified and have nothing to check. ```bash python3 tools/verify_sources.py --miles /path/to/miles --prime-rl /path/to/prime-rl --slime /path/to/slime ``` -A complete Slime campaign was prepared in a temporary directory from a local clone. -It checked out the correct revision, applied the patch, expanded site values, and materialized the datasets. -Its submission command printed the expected two-node, 16-GPU allocation request and did not invoke Slurm. +## Not covered + +There is no automated preflight. Distributed training, checkpoint conversion, teacher fusing on real weights, image reconstruction and other GPU types are not covered by these tests. + +## September 15, 2026 packaging checks -The new package was scanned for common Hugging Face and GitHub token patterns, private-key headers, -original private network addresses, and original personal or cluster paths. No such values were found in the new package. -This targeted scan is not a security audit of the repository's historical commits. +These checks covered the previous experiment's package, before the restructure. No cluster job was submitted. -Distributed training, new checkpoint conversion, runtime image reconstruction, and execution on a different GPU remain untested for this package. -The original experiment outcomes do not remove these validation gaps. +- Six package tests passed under Python 3.12. +- Every patch applied to a clean archive of its pinned revision, and each changed file matched the original successful-run archive by SHA-256. Miles had four changed files, Prime-RL one and Slime five. +- A Slime campaign was prepared from a local clone. Its submission command printed the expected two-node, 16-GPU request without invoking Slurm. +- A scan for token patterns, private keys, and the original private addresses and paths found none. It did not cover historical commits. diff --git a/docs/caesar-opd-2026-09-27/build_charts.py b/docs/caesar-opd-2026-09-27/build_charts.py new file mode 100644 index 0000000..487068f --- /dev/null +++ b/docs/caesar-opd-2026-09-27/build_charts.py @@ -0,0 +1,162 @@ +"""Render the report's figures from `data/` (benchmark summaries, telemetry.json, gpu-series.csv). + + uv run --with matplotlib python docs/caesar-opd-2026-09-27/build_charts.py + +Each framework keeps one color in every figure; base and teacher are grey. Two of the colors +fall below 3:1 contrast on the light background, so every series is also labeled directly and +every value appears in the report's tables. +""" +import csv +import json +from pathlib import Path + +import matplotlib + +matplotlib.use('Agg') +import matplotlib.pyplot as plt # noqa: E402 + +HERE = Path(__file__).resolve().parent +DATA, OUT = HERE / 'data', HERE / 'figures' +SURFACE, INK, INK_2, MUTED, GRID = '#fcfcfb', '#0b0b0b', '#52514e', '#8a8984', '#e6e5e1' +FRAMEWORKS = { # Fixed order; color follows the framework, not its rank. + 'miles-1021': ('Miles', '#2a78d6'), 'prime-rl-1017': ('Prime-RL', '#eb6834'), + 'slime-979': ('Slime', '#1baf7a'), 'verl-966': ('verl', '#eda100')} +REFERENCES = {'base': ('Base', '#b7b6b1'), 'teacher-caesar': ('Caesar teacher', '#6f6e69')} + +plt.rcParams.update({ + 'figure.facecolor': SURFACE, 'axes.facecolor': SURFACE, 'savefig.facecolor': SURFACE, + 'font.size': 10, 'axes.edgecolor': GRID, 'axes.labelcolor': INK_2, 'xtick.color': INK_2, 'ytick.color': INK_2, + 'axes.grid': True, 'grid.color': GRID, 'grid.linewidth': 0.8, 'axes.axisbelow': True, + 'axes.spines.top': False, 'axes.spines.right': False, 'axes.titlelocation': 'left', + 'axes.titlesize': 11, 'axes.titlecolor': INK, 'lines.linewidth': 2, 'lines.solid_capstyle': 'round'}) + + +def end_labels(ax, labels, min_gap=0.06): + """Label each line at its end, nudging labels apart vertically so they do not overlap.""" + low, high = ax.get_ylim() + span = high - low + placed = [] + for x, y, text in sorted(labels, key=lambda item: item[1]): + y = max(y, placed[-1] + min_gap * span) if placed else y + placed.append(y) + ax.annotate(text, (x, y), xytext=(6, 0), textcoords='offset points', va='center', color=INK_2, fontsize=8.5) + + +def available(names): + return [n for n in names if (DATA / n / 'summary.json').exists()] + + +def save(fig, name): + OUT.mkdir(exist_ok=True) + fig.savefig(OUT / name, dpi=180, bbox_inches='tight') + plt.close(fig) + + +def quality(): + """Final-model avg@8 on both dev sets, one panel per task on the same 0-100 scale.""" + models = available([*REFERENCES, *FRAMEWORKS]) + summaries = {m: json.loads((DATA / m / 'summary.json').read_text()) for m in models} + style = {**REFERENCES, **FRAMEWORKS} + fig, axes = plt.subplots(1, 2, figsize=(10, 3.6), sharey=True) + for ax, (domain, title) in zip(axes, (('caesar_cipher', 'caesar_cipher (trained task)'), + ('simple_geometry', 'simple_geometry (not trained)'))): + values = [100 * summaries[m]['domains'][domain]['avg@8'] for m in models] + bars = ax.bar(range(len(models)), values, width=0.62, color=[style[m][1] for m in models], + edgecolor=SURFACE, linewidth=2) + for bar, value in zip(bars, values): + ax.text(bar.get_x() + bar.get_width() / 2, value + 1.5, f'{value:.1f}', ha='center', va='bottom', + color=INK, fontsize=9) + ax.set_xticks(range(len(models)), [style[m][0] for m in models], rotation=20, ha='right') + ax.set_title(title) + ax.set_ylim(0, 100) + ax.grid(axis='x', visible=False) + axes[0].set_ylabel('avg@8 (%)') + fig.suptitle('Final-model accuracy, dev set, 8 samples per prompt, temperature 1, thinking on', + x=0.01, ha='left', color=INK, fontsize=12) + fig.tight_layout(rect=(0, 0, 1, 0.93)) + save(fig, 'quality.png') + + +def updates(): + runs = {r['run']: r for r in json.loads((DATA / 'telemetry.json').read_text()) if r.get('updates')} + return {run: runs[run]['updates']['per_update'] for run in FRAMEWORKS if run in runs} + + +def per_update_panels(): + """One panel per per-update metric, all frameworks overlaid.""" + series = updates() + panels = [('step_s', 'Update time (s)', 1), + ('wait_share', 'Share of update waiting for rollouts (%)', 100), + ('response_tokens_mean', 'Mean trained response (k tokens)', 1e-3), + ('reverse_kl', 'Reverse KL to teacher (sampled tokens)', 1)] + fig, axes = plt.subplots(2, 2, figsize=(10, 6.6)) + for ax, (field, title, scale) in zip(axes.flat, panels): + labels = [] + for run, rows in series.items(): + name, color = FRAMEWORKS[run] + xs, ys = [], [] + for i, row in enumerate(rows, start=1): + value = (row['wait_for_rollouts_s'] / row['step_s'] + if field == 'wait_share' and row['wait_for_rollouts_s'] and row['step_s'] else row.get(field)) + if isinstance(value, (int, float)): + xs.append(i if run != 'verl-966' else i + 1) # verl logs no line for its first update. + ys.append(value * scale) + ax.plot(xs, ys, color=color, marker='o', markersize=3.5) + labels.append((xs[-1], ys[-1], name)) + ax.set_title(title) + ax.set_xlim(0.5, 23) + ax.set_xticks([1, 5, 10, 15, 20]) + ax.set_xlabel('Update') + end_labels(ax, labels) + fig.suptitle('Per-update training metrics from each framework\'s own logs (updates 1 and 11 include in-loop evals)', + x=0.01, ha='left', color=INK, fontsize=12) + fig.legend(handles=[plt.Line2D([], [], color=FRAMEWORKS[r][1], marker='o', label=FRAMEWORKS[r][0]) + for r in series], loc='upper left', bbox_to_anchor=(0.01, 0.955), ncol=len(series), + frameon=False, fontsize=9) + fig.tight_layout(rect=(0, 0, 1, 0.92)) + save(fig, 'per_update.png') + + +# Capture roles follow the site file, but Ray put verl's rollout on the "learner" node. +ROLLOUT_NODE_ROLE = {'verl-966': 'learner'} + + +def rolling(values, width=12): + """Trailing mean over `width` samples (12 x 5 s = 1 minute).""" + return [sum(values[max(0, i - width + 1):i + 1]) / len(values[max(0, i - width + 1):i + 1]) + for i in range(len(values))] + + +def gpu_utilization(): + """Mean utilization of each node's used GPUs over the run, one panel per framework.""" + points = {} + with (DATA / 'gpu-series.csv').open() as stream: + for row in csv.DictReader(stream): + points.setdefault(row['run'], {}).setdefault(row['role'], []).append( + (float(row['seconds']) / 60, float(row['gpu_util_mean']))) + runs = [r for r in FRAMEWORKS if r in points] + fig, axes = plt.subplots(len(runs), 1, figsize=(10, 1.9 * len(runs)), sharex=True, sharey=True) + for ax, run in zip(axes, runs): + name, color = FRAMEWORKS[run] + rollout_role = ROLLOUT_NODE_ROLE.get(run, 'generation') + for role in sorted(points[run], key=lambda r: r != rollout_role): # Rollout node first. + xs, ys = zip(*points[run][role]) + rollout = role == rollout_role + ax.plot(xs, rolling(list(ys)), color=color if rollout else MUTED, linewidth=1.6, + label='rollout + teacher node' if rollout else 'trainer node') + ax.set_title(name, fontsize=10) + ax.set_ylim(0, 105) + ax.set_ylabel('GPU util (%)') + ax.legend(loc='lower right', bbox_to_anchor=(1, 1.0), frameon=False, fontsize=8, ncol=2) + axes[-1].set_xlabel('Minutes since the allocation started') + fig.suptitle('GPU utilization per node over the whole run (mean of the GPUs the run used; 1-minute rolling mean)', + x=0.01, ha='left', color=INK, fontsize=12) + fig.tight_layout(rect=(0, 0, 1, 0.97)) + save(fig, 'gpu_utilization.png') + + +if __name__ == '__main__': + quality() + per_update_panels() + gpu_utilization() + print('figures written to', OUT) diff --git a/docs/caesar-opd-2026-09-27/compare_miles_prime_rl.py b/docs/caesar-opd-2026-09-27/compare_miles_prime_rl.py new file mode 100644 index 0000000..a63af88 --- /dev/null +++ b/docs/caesar-opd-2026-09-27/compare_miles_prime_rl.py @@ -0,0 +1,104 @@ +"""Compare Miles' first run (965) with Prime-RL on upstream defaults (1017), plus Prime-RL's first run (968) dashed. + + uv run --with matplotlib python docs/caesar-opd-2026-09-27/compare_miles_prime_rl.py + +Reads `data/` and writes `figures/miles_vs_prime_rl.png`. +""" +import json + +import matplotlib.pyplot as plt + +from build_charts import DATA, INK, INK_2, MUTED, SURFACE, end_labels, save + +MILES, PRIME, PRIME_OLD = 'miles-965', 'prime-rl-1017', 'prime-rl-968' +STYLE = { # Same colors as build_charts.py; Prime-RL's first run is grey and dashed. + MILES: dict(label='Miles', color='#2a78d6', linestyle='-'), + PRIME: dict(label='Prime-RL (defaults, 1017)', color='#eb6834', linestyle='-'), + PRIME_OLD: dict(label='Prime-RL (first run, 968)', color='#b7b6b1', linestyle='--'), +} +TEACHER = 'teacher-caesar' +# Whole-run policy-engine throughput (k tokens/s) from the engine counters in data/telemetry*.json. +# Miles is run 965 here (18.3k); the report's GPU table shows the 1021 rerun (18.0k). +POLICY_TOKENS_PER_S = {MILES: 18.3, PRIME: 40.3, PRIME_OLD: 20.9} + + +def telemetry(): + runs = {r['run']: r for r in json.loads((DATA / 'telemetry.json').read_text())} + runs.update({r['run']: r for r in json.loads((DATA / 'telemetry-prime-rl-968.json').read_text())}) + return {run: runs[run]['updates']['per_update'] for run in STYLE} + + +def bars(ax, runs, values, fmt, reference=None): + xs = range(len(runs)) + drawn = ax.bar(xs, values, width=0.6, color=[STYLE[r]['color'] for r in runs], edgecolor=SURFACE, linewidth=2) + for bar, run in zip(drawn, runs): + if STYLE[run]['linestyle'] == '--': + bar.set_hatch('///') + bar.set_edgecolor(SURFACE) + top = max(values + ([reference] if reference else [])) + for x, value in zip(xs, values): + ax.text(x, value + 0.02 * top, fmt.format(value), ha='center', va='bottom', color=INK, fontsize=9) + if reference: # The panel title names the reference line, so it gets no label. + ax.axhline(reference, color=MUTED, linewidth=1, linestyle=':') + ax.set_xticks(list(xs), [STYLE[r]['label'].replace(' (', '\n(') for r in runs], fontsize=8.5) + ax.set_ylim(0, top * 1.18) + ax.grid(axis='x', visible=False) + + +def lines(ax, series, field, scale=1.0, cumulative=False): + labels = [] + for run, rows in series.items(): + ys, total = [], 0.0 + for row in rows: + value = row[field] * scale + total += value + ys.append(total if cumulative else value) + xs = list(range(1, len(ys) + 1)) + style = STYLE[run] + ax.plot(xs, ys, color=style['color'], linestyle=style['linestyle'], marker='o', markersize=3.5, + linewidth=2 if style['linestyle'] == '-' else 1.6) + labels.append((xs[-1], ys[-1], style['label'].split(' (')[0] + (' 968' if run == PRIME_OLD else ''))) + ax.set_xlim(0.5, 23.5) + ax.set_xticks([1, 5, 10, 15, 20]) + ax.set_xlabel('Update') + end_labels(ax, labels) + + +def main(): + summaries = {m: json.loads((DATA / m / 'summary.json').read_text()) for m in (*STYLE, TEACHER)} + series = telemetry() + order = [MILES, PRIME, PRIME_OLD] + fig, axes = plt.subplots(3, 2, figsize=(10, 10)) + + for ax, (domain, title) in zip(axes[0], (('caesar_cipher', 'caesar_cipher avg@8, trained task (%)'), + ('simple_geometry', 'simple_geometry avg@8, not trained (%)'))): + values = [100 * summaries[r]['domains'][domain]['avg@8'] for r in order] + teacher = 100 * summaries[TEACHER]['domains'][domain]['avg@8'] + bars(ax, order, values, '{:.1f}', reference=teacher) + ax.set_title(f'{title}\ndotted: caesar teacher {teacher:.1f}', fontsize=10.5) + ax.set_ylim(0, 100) + + axes[1, 0].set_title('Cumulative training time (min)') + lines(axes[1, 0], series, 'step_s', scale=1 / 60, cumulative=True) + axes[1, 1].set_title('Time per update (s)') + lines(axes[1, 1], series, 'step_s') + + axes[2, 0].set_title('Mean trained response (k tokens)') + lines(axes[2, 0], series, 'response_tokens_mean', scale=1e-3) + axes[2, 1].set_title('Policy generation throughput, whole run (k tokens/s)') + bars(axes[2, 1], order, [POLICY_TOKENS_PER_S[r] for r in order], '{:.1f}') + + fig.suptitle('Miles vs Prime-RL: single-teacher OPD on caesar_cipher, same recipe, 20 updates, 2 x 8 B200', + x=0.01, ha='left', color=INK, fontsize=12) + fig.legend(handles=[plt.Line2D([], [], color=STYLE[r]['color'], linestyle=STYLE[r]['linestyle'], marker='o', + label=STYLE[r]['label']) for r in order], + loc='upper left', bbox_to_anchor=(0.01, 0.97), ncol=3, frameon=False, fontsize=9) + fig.text(0.01, 0.005, "Update 1 includes each run's start-up; Miles' update 11 its second in-loop eval (Prime-RL's evals overlap training). " + 'Whole allocations: Miles 63.6 min, Prime-RL 39.6 min (968: 51.3 min).\n' + 'Benchmark: 126 dev prompts x 8 samples, temperature 1, thinking on.', color=MUTED, fontsize=8) + fig.tight_layout(rect=(0, 0.035, 1, 0.94)) + save(fig, 'miles_vs_prime_rl.png') + + +if __name__ == '__main__': + main() diff --git a/docs/caesar-opd-2026-09-27/data/base/summary.json b/docs/caesar-opd-2026-09-27/data/base/summary.json new file mode 100644 index 0000000..652a9e7 --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/base/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-assets/models/Qwen3.6-35B-A3B", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 374.07481622695923, + "output_tokens_per_second": 42745.65222348055, + "domains": { + "caesar_cipher": { + "avg@8": 0.26785714285714285, + "pass@8": 0.7222222222222222, + "finished_thinking": 0.9474206349206349, + "truncated": 0.054563492063492064, + "response_tokens_mean": 12181.030753968254, + "response_tokens_median": 9989.0 + }, + "simple_geometry": { + "avg@8": 0.529296875, + "pass@8": 0.9921875, + "finished_thinking": 0.99609375, + "truncated": 0.00390625, + "response_tokens_mean": 3624.6025390625, + "response_tokens_median": 3177.5 + } + } +} diff --git a/docs/caesar-opd-2026-09-27/data/gpu-series.csv b/docs/caesar-opd-2026-09-27/data/gpu-series.csv new file mode 100644 index 0000000..ae05ee9 --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/gpu-series.csv @@ -0,0 +1,6337 @@ +run,role,seconds,gpu_util_mean,gpus +miles-1021,generation,31.3,0.0,7 +miles-1021,generation,36.3,0.0,7 +miles-1021,generation,41.3,0.0,7 +miles-1021,generation,46.3,0.0,7 +miles-1021,generation,51.3,0.0,7 +miles-1021,generation,56.3,0.0,7 +miles-1021,generation,61.3,0.0,7 +miles-1021,generation,66.3,0.0,7 +miles-1021,generation,71.3,0.0,7 +miles-1021,generation,76.3,0.0,7 +miles-1021,generation,81.3,0.0,7 +miles-1021,generation,86.3,0.1,7 +miles-1021,generation,91.3,0.9,7 +miles-1021,generation,96.3,0.4,7 +miles-1021,generation,101.3,0.1,7 +miles-1021,generation,106.3,0.1,7 +miles-1021,generation,111.3,0.6,7 +miles-1021,generation,116.3,0.1,7 +miles-1021,generation,121.3,0.1,7 +miles-1021,generation,126.3,1.0,7 +miles-1021,generation,131.3,0.6,7 +miles-1021,generation,136.3,0.1,7 +miles-1021,generation,141.3,1.0,7 +miles-1021,generation,146.3,0.1,7 +miles-1021,generation,151.3,0.1,7 +miles-1021,generation,156.3,0.0,7 +miles-1021,generation,161.3,0.0,7 +miles-1021,generation,166.3,0.0,7 +miles-1021,generation,171.3,0.0,7 +miles-1021,generation,176.3,0.0,7 +miles-1021,generation,181.3,0.0,7 +miles-1021,generation,186.3,0.0,7 +miles-1021,generation,191.4,0.0,7 +miles-1021,generation,196.4,0.0,7 +miles-1021,generation,201.4,0.0,7 +miles-1021,generation,206.4,0.0,7 +miles-1021,generation,211.4,0.0,7 +miles-1021,generation,216.4,0.0,7 +miles-1021,generation,221.4,0.0,7 +miles-1021,generation,226.4,0.0,7 +miles-1021,generation,231.4,0.0,7 +miles-1021,generation,236.4,0.0,7 +miles-1021,generation,241.4,0.0,7 +miles-1021,generation,246.4,0.0,7 +miles-1021,generation,251.4,0.0,7 +miles-1021,generation,256.4,0.0,7 +miles-1021,generation,261.4,0.0,7 +miles-1021,generation,266.4,0.3,7 +miles-1021,generation,271.4,0.0,7 +miles-1021,generation,276.4,0.0,7 +miles-1021,generation,281.4,0.0,7 +miles-1021,generation,286.4,0.0,7 +miles-1021,generation,291.4,0.9,7 +miles-1021,generation,296.4,0.4,7 +miles-1021,generation,301.4,0.4,7 +miles-1021,generation,306.4,0.6,7 +miles-1021,generation,311.4,0.9,7 +miles-1021,generation,316.4,0.0,7 +miles-1021,generation,321.4,0.0,7 +miles-1021,generation,326.4,0.4,7 +miles-1021,generation,331.4,0.6,7 +miles-1021,generation,336.4,0.0,7 +miles-1021,generation,341.4,0.4,7 +miles-1021,generation,346.4,0.0,7 +miles-1021,generation,351.4,0.3,7 +miles-1021,generation,356.4,0.7,7 +miles-1021,generation,361.4,0.0,7 +miles-1021,generation,366.4,0.0,7 +miles-1021,generation,371.4,0.7,7 +miles-1021,generation,376.4,0.6,7 +miles-1021,generation,381.4,0.4,7 +miles-1021,generation,386.4,0.6,7 +miles-1021,generation,391.4,0.4,7 +miles-1021,generation,396.4,0.3,7 +miles-1021,generation,401.4,0.0,7 +miles-1021,generation,406.4,0.6,7 +miles-1021,generation,411.4,0.4,7 +miles-1021,generation,416.4,29.9,7 +miles-1021,generation,421.4,0.1,7 +miles-1021,generation,426.4,0.3,7 +miles-1021,generation,431.4,74.7,7 +miles-1021,generation,436.4,26.7,7 +miles-1021,generation,441.4,0.0,7 +miles-1021,generation,446.4,0.0,7 +miles-1021,generation,451.4,0.0,7 +miles-1021,generation,456.4,30.1,7 +miles-1021,generation,461.4,0.0,7 +miles-1021,generation,466.4,10.0,7 +miles-1021,generation,471.4,0.0,7 +miles-1021,generation,476.4,0.0,7 +miles-1021,generation,481.4,0.0,7 +miles-1021,generation,486.4,0.0,7 +miles-1021,generation,491.4,0.0,7 +miles-1021,generation,496.4,0.0,7 +miles-1021,generation,501.4,0.0,7 +miles-1021,generation,506.4,0.0,7 +miles-1021,generation,511.4,0.0,7 +miles-1021,generation,516.4,0.0,7 +miles-1021,generation,521.4,0.0,7 +miles-1021,generation,526.4,0.0,7 +miles-1021,generation,531.4,0.0,7 +miles-1021,generation,536.4,0.0,7 +miles-1021,generation,541.4,33.7,7 +miles-1021,generation,546.4,0.0,7 +miles-1021,generation,551.4,85.7,7 +miles-1021,generation,556.4,85.7,7 +miles-1021,generation,561.4,85.7,7 +miles-1021,generation,566.4,85.7,7 +miles-1021,generation,571.4,85.7,7 +miles-1021,generation,576.4,85.7,7 +miles-1021,generation,581.4,85.7,7 +miles-1021,generation,586.4,85.7,7 +miles-1021,generation,591.4,85.7,7 +miles-1021,generation,596.4,85.7,7 +miles-1021,generation,601.4,85.7,7 +miles-1021,generation,606.4,85.7,7 +miles-1021,generation,611.4,85.7,7 +miles-1021,generation,616.4,85.7,7 +miles-1021,generation,621.4,85.7,7 +miles-1021,generation,626.4,85.7,7 +miles-1021,generation,631.4,85.7,7 +miles-1021,generation,636.4,85.7,7 +miles-1021,generation,641.4,85.7,7 +miles-1021,generation,646.4,85.7,7 +miles-1021,generation,651.4,85.7,7 +miles-1021,generation,656.4,85.7,7 +miles-1021,generation,661.4,85.7,7 +miles-1021,generation,666.4,85.7,7 +miles-1021,generation,671.4,85.7,7 +miles-1021,generation,676.4,85.7,7 +miles-1021,generation,681.4,85.7,7 +miles-1021,generation,686.4,85.7,7 +miles-1021,generation,691.4,85.7,7 +miles-1021,generation,696.4,85.7,7 +miles-1021,generation,701.4,85.7,7 +miles-1021,generation,706.4,85.7,7 +miles-1021,generation,711.4,85.7,7 +miles-1021,generation,716.4,85.7,7 +miles-1021,generation,721.4,85.7,7 +miles-1021,generation,726.4,57.1,7 +miles-1021,generation,731.4,28.6,7 +miles-1021,generation,736.4,28.6,7 +miles-1021,generation,741.4,28.6,7 +miles-1021,generation,746.4,28.6,7 +miles-1021,generation,751.4,28.6,7 +miles-1021,generation,756.4,28.6,7 +miles-1021,generation,761.4,28.6,7 +miles-1021,generation,766.4,28.6,7 +miles-1021,generation,771.4,28.6,7 +miles-1021,generation,776.4,28.6,7 +miles-1021,generation,781.4,28.6,7 +miles-1021,generation,786.4,28.6,7 +miles-1021,generation,791.4,28.6,7 +miles-1021,generation,796.4,28.6,7 +miles-1021,generation,801.4,28.6,7 +miles-1021,generation,806.4,28.6,7 +miles-1021,generation,811.4,28.6,7 +miles-1021,generation,816.4,28.6,7 +miles-1021,generation,821.4,28.6,7 +miles-1021,generation,826.4,28.6,7 +miles-1021,generation,831.4,28.6,7 +miles-1021,generation,836.4,28.6,7 +miles-1021,generation,841.4,28.6,7 +miles-1021,generation,846.4,28.6,7 +miles-1021,generation,851.4,85.7,7 +miles-1021,generation,856.4,85.7,7 +miles-1021,generation,861.4,85.7,7 +miles-1021,generation,866.4,85.7,7 +miles-1021,generation,871.4,85.7,7 +miles-1021,generation,876.4,85.7,7 +miles-1021,generation,881.4,85.7,7 +miles-1021,generation,886.4,86.4,7 +miles-1021,generation,891.4,87.0,7 +miles-1021,generation,896.4,85.7,7 +miles-1021,generation,901.4,85.0,7 +miles-1021,generation,906.4,98.9,7 +miles-1021,generation,911.4,85.4,7 +miles-1021,generation,916.4,85.7,7 +miles-1021,generation,921.4,85.7,7 +miles-1021,generation,926.4,98.1,7 +miles-1021,generation,931.4,85.7,7 +miles-1021,generation,936.4,91.6,7 +miles-1021,generation,941.4,85.7,7 +miles-1021,generation,946.4,99.1,7 +miles-1021,generation,951.4,85.7,7 +miles-1021,generation,956.4,85.7,7 +miles-1021,generation,961.4,85.7,7 +miles-1021,generation,966.4,85.7,7 +miles-1021,generation,971.4,85.7,7 +miles-1021,generation,976.4,85.7,7 +miles-1021,generation,981.4,86.6,7 +miles-1021,generation,986.4,96.6,7 +miles-1021,generation,991.4,85.7,7 +miles-1021,generation,996.4,98.1,7 +miles-1021,generation,1001.4,85.7,7 +miles-1021,generation,1006.4,88.3,7 +miles-1021,generation,1011.4,99.7,7 +miles-1021,generation,1016.4,85.7,7 +miles-1021,generation,1021.4,85.7,7 +miles-1021,generation,1026.4,99.1,7 +miles-1021,generation,1031.4,99.4,7 +miles-1021,generation,1036.4,97.4,7 +miles-1021,generation,1041.4,88.9,7 +miles-1021,generation,1046.4,86.6,7 +miles-1021,generation,1051.4,99.3,7 +miles-1021,generation,1056.4,85.7,7 +miles-1021,generation,1061.4,85.7,7 +miles-1021,generation,1066.4,90.3,7 +miles-1021,generation,1071.4,85.7,7 +miles-1021,generation,1076.4,85.7,7 +miles-1021,generation,1081.4,88.4,7 +miles-1021,generation,1086.4,100.0,7 +miles-1021,generation,1091.4,99.4,7 +miles-1021,generation,1096.4,86.0,7 +miles-1021,generation,1101.4,87.9,7 +miles-1021,generation,1106.4,85.7,7 +miles-1021,generation,1111.5,100.0,7 +miles-1021,generation,1116.5,85.7,7 +miles-1021,generation,1121.5,85.7,7 +miles-1021,generation,1126.5,93.4,7 +miles-1021,generation,1131.5,99.0,7 +miles-1021,generation,1136.5,90.6,7 +miles-1021,generation,1141.5,67.6,7 +miles-1021,generation,1146.5,85.1,7 +miles-1021,generation,1151.5,85.1,7 +miles-1021,generation,1156.5,85.7,7 +miles-1021,generation,1161.5,87.0,7 +miles-1021,generation,1166.5,85.7,7 +miles-1021,generation,1171.5,85.7,7 +miles-1021,generation,1176.5,85.7,7 +miles-1021,generation,1181.5,85.7,7 +miles-1021,generation,1186.5,88.9,7 +miles-1021,generation,1191.5,100.0,7 +miles-1021,generation,1196.5,85.7,7 +miles-1021,generation,1201.5,85.7,7 +miles-1021,generation,1206.5,85.7,7 +miles-1021,generation,1211.5,93.6,7 +miles-1021,generation,1216.5,88.1,7 +miles-1021,generation,1221.5,95.4,7 +miles-1021,generation,1226.5,61.1,7 +miles-1021,generation,1231.5,82.4,7 +miles-1021,generation,1236.5,85.1,7 +miles-1021,generation,1241.5,85.7,7 +miles-1021,generation,1246.5,85.7,7 +miles-1021,generation,1251.5,100.0,7 +miles-1021,generation,1256.5,85.7,7 +miles-1021,generation,1261.5,86.9,7 +miles-1021,generation,1266.5,99.4,7 +miles-1021,generation,1271.5,85.7,7 +miles-1021,generation,1276.5,85.7,7 +miles-1021,generation,1281.5,85.7,7 +miles-1021,generation,1286.5,99.4,7 +miles-1021,generation,1291.5,85.7,7 +miles-1021,generation,1296.5,85.3,7 +miles-1021,generation,1301.5,85.1,7 +miles-1021,generation,1306.5,83.1,7 +miles-1021,generation,1311.5,81.7,7 +miles-1021,generation,1316.5,84.9,7 +miles-1021,generation,1321.5,85.7,7 +miles-1021,generation,1326.5,85.7,7 +miles-1021,generation,1331.5,85.7,7 +miles-1021,generation,1336.5,85.0,7 +miles-1021,generation,1341.5,85.3,7 +miles-1021,generation,1346.5,85.7,7 +miles-1021,generation,1351.5,85.7,7 +miles-1021,generation,1356.5,88.7,7 +miles-1021,generation,1361.5,85.7,7 +miles-1021,generation,1366.5,85.7,7 +miles-1021,generation,1371.5,86.0,7 +miles-1021,generation,1376.5,98.0,7 +miles-1021,generation,1381.5,85.3,7 +miles-1021,generation,1386.5,85.7,7 +miles-1021,generation,1391.5,85.7,7 +miles-1021,generation,1396.5,85.7,7 +miles-1021,generation,1401.5,90.9,7 +miles-1021,generation,1406.5,85.7,7 +miles-1021,generation,1411.5,85.7,7 +miles-1021,generation,1416.5,85.7,7 +miles-1021,generation,1421.5,66.0,7 +miles-1021,generation,1426.5,82.9,7 +miles-1021,generation,1431.5,85.0,7 +miles-1021,generation,1436.5,98.4,7 +miles-1021,generation,1441.5,85.7,7 +miles-1021,generation,1446.5,85.7,7 +miles-1021,generation,1451.5,86.3,7 +miles-1021,generation,1456.5,85.7,7 +miles-1021,generation,1461.5,85.7,7 +miles-1021,generation,1466.5,85.7,7 +miles-1021,generation,1471.5,85.7,7 +miles-1021,generation,1476.5,85.7,7 +miles-1021,generation,1481.5,85.7,7 +miles-1021,generation,1486.5,85.7,7 +miles-1021,generation,1491.5,85.7,7 +miles-1021,generation,1496.5,98.1,7 +miles-1021,generation,1501.5,85.7,7 +miles-1021,generation,1506.5,85.7,7 +miles-1021,generation,1511.5,99.4,7 +miles-1021,generation,1516.5,99.0,7 +miles-1021,generation,1521.5,99.7,7 +miles-1021,generation,1526.5,99.6,7 +miles-1021,generation,1531.5,85.7,7 +miles-1021,generation,1536.5,85.7,7 +miles-1021,generation,1541.5,85.7,7 +miles-1021,generation,1546.5,85.7,7 +miles-1021,generation,1551.5,85.7,7 +miles-1021,generation,1556.5,66.1,7 +miles-1021,generation,1561.5,83.9,7 +miles-1021,generation,1566.5,85.0,7 +miles-1021,generation,1571.5,85.0,7 +miles-1021,generation,1576.5,97.3,7 +miles-1021,generation,1581.5,85.7,7 +miles-1021,generation,1586.5,85.7,7 +miles-1021,generation,1591.5,85.7,7 +miles-1021,generation,1596.5,85.7,7 +miles-1021,generation,1601.5,85.7,7 +miles-1021,generation,1606.5,85.7,7 +miles-1021,generation,1611.5,100.0,7 +miles-1021,generation,1616.5,85.7,7 +miles-1021,generation,1621.5,85.7,7 +miles-1021,generation,1626.5,100.0,7 +miles-1021,generation,1631.5,100.0,7 +miles-1021,generation,1636.5,85.0,7 +miles-1021,generation,1641.5,100.0,7 +miles-1021,generation,1646.5,99.4,7 +miles-1021,generation,1651.5,100.0,7 +miles-1021,generation,1656.5,93.0,7 +miles-1021,generation,1661.5,85.7,7 +miles-1021,generation,1666.5,99.9,7 +miles-1021,generation,1671.5,85.7,7 +miles-1021,generation,1676.5,85.7,7 +miles-1021,generation,1681.5,85.7,7 +miles-1021,generation,1686.5,83.9,7 +miles-1021,generation,1691.5,82.4,7 +miles-1021,generation,1696.5,97.3,7 +miles-1021,generation,1701.5,85.7,7 +miles-1021,generation,1706.5,85.7,7 +miles-1021,generation,1711.5,98.0,7 +miles-1021,generation,1716.5,88.3,7 +miles-1021,generation,1721.5,99.7,7 +miles-1021,generation,1726.5,99.7,7 +miles-1021,generation,1731.5,93.9,7 +miles-1021,generation,1736.5,85.7,7 +miles-1021,generation,1741.5,98.9,7 +miles-1021,generation,1746.5,85.7,7 +miles-1021,generation,1751.5,91.1,7 +miles-1021,generation,1756.5,97.1,7 +miles-1021,generation,1761.5,92.9,7 +miles-1021,generation,1766.5,88.7,7 +miles-1021,generation,1771.5,85.3,7 +miles-1021,generation,1776.5,100.0,7 +miles-1021,generation,1781.5,85.7,7 +miles-1021,generation,1786.5,94.3,7 +miles-1021,generation,1791.5,85.7,7 +miles-1021,generation,1796.5,85.7,7 +miles-1021,generation,1801.5,85.7,7 +miles-1021,generation,1806.5,65.6,7 +miles-1021,generation,1811.5,83.6,7 +miles-1021,generation,1816.5,85.1,7 +miles-1021,generation,1821.5,91.7,7 +miles-1021,generation,1826.5,90.7,7 +miles-1021,generation,1831.5,85.7,7 +miles-1021,generation,1836.5,85.7,7 +miles-1021,generation,1841.5,85.7,7 +miles-1021,generation,1846.5,85.7,7 +miles-1021,generation,1851.5,99.4,7 +miles-1021,generation,1856.5,88.0,7 +miles-1021,generation,1861.5,85.7,7 +miles-1021,generation,1866.5,92.1,7 +miles-1021,generation,1871.5,86.1,7 +miles-1021,generation,1876.5,91.3,7 +miles-1021,generation,1881.5,95.1,7 +miles-1021,generation,1886.5,85.7,7 +miles-1021,generation,1891.5,85.7,7 +miles-1021,generation,1896.5,85.7,7 +miles-1021,generation,1901.5,85.7,7 +miles-1021,generation,1906.5,90.0,7 +miles-1021,generation,1911.5,85.4,7 +miles-1021,generation,1916.5,85.7,7 +miles-1021,generation,1921.5,99.7,7 +miles-1021,generation,1926.5,88.7,7 +miles-1021,generation,1931.5,84.1,7 +miles-1021,generation,1936.5,81.1,7 +miles-1021,generation,1941.5,90.7,7 +miles-1021,generation,1946.5,85.7,7 +miles-1021,generation,1951.5,97.1,7 +miles-1021,generation,1956.5,85.7,7 +miles-1021,generation,1961.5,95.3,7 +miles-1021,generation,1966.5,85.7,7 +miles-1021,generation,1971.5,85.7,7 +miles-1021,generation,1976.5,85.7,7 +miles-1021,generation,1981.5,85.7,7 +miles-1021,generation,1986.5,85.7,7 +miles-1021,generation,1991.5,85.7,7 +miles-1021,generation,1996.5,85.0,7 +miles-1021,generation,2001.5,85.7,7 +miles-1021,generation,2006.5,86.3,7 +miles-1021,generation,2011.5,85.7,7 +miles-1021,generation,2016.5,100.0,7 +miles-1021,generation,2021.5,85.7,7 +miles-1021,generation,2026.5,85.7,7 +miles-1021,generation,2031.6,95.3,7 +miles-1021,generation,2036.6,99.6,7 +miles-1021,generation,2041.6,100.0,7 +miles-1021,generation,2046.6,88.0,7 +miles-1021,generation,2051.6,75.4,7 +miles-1021,generation,2056.6,83.7,7 +miles-1021,generation,2061.6,84.4,7 +miles-1021,generation,2066.6,90.3,7 +miles-1021,generation,2071.6,97.7,7 +miles-1021,generation,2076.6,85.7,7 +miles-1021,generation,2081.6,100.0,7 +miles-1021,generation,2086.6,85.7,7 +miles-1021,generation,2091.6,85.7,7 +miles-1021,generation,2096.6,99.1,7 +miles-1021,generation,2101.6,85.7,7 +miles-1021,generation,2106.6,85.7,7 +miles-1021,generation,2111.6,100.0,7 +miles-1021,generation,2116.6,85.7,7 +miles-1021,generation,2121.6,85.7,7 +miles-1021,generation,2126.6,85.3,7 +miles-1021,generation,2131.6,85.7,7 +miles-1021,generation,2136.6,85.7,7 +miles-1021,generation,2141.6,94.6,7 +miles-1021,generation,2146.6,85.7,7 +miles-1021,generation,2151.6,85.7,7 +miles-1021,generation,2156.6,100.0,7 +miles-1021,generation,2161.6,100.0,7 +miles-1021,generation,2166.6,96.4,7 +miles-1021,generation,2171.6,85.7,7 +miles-1021,generation,2176.6,63.7,7 +miles-1021,generation,2181.6,84.3,7 +miles-1021,generation,2186.6,85.7,7 +miles-1021,generation,2191.6,85.7,7 +miles-1021,generation,2196.6,85.7,7 +miles-1021,generation,2201.6,85.7,7 +miles-1021,generation,2206.6,85.7,7 +miles-1021,generation,2211.6,100.0,7 +miles-1021,generation,2216.6,85.7,7 +miles-1021,generation,2221.6,85.7,7 +miles-1021,generation,2226.6,85.7,7 +miles-1021,generation,2231.6,85.7,7 +miles-1021,generation,2236.6,100.0,7 +miles-1021,generation,2241.6,100.0,7 +miles-1021,generation,2246.6,85.7,7 +miles-1021,generation,2251.6,100.0,7 +miles-1021,generation,2256.6,85.7,7 +miles-1021,generation,2261.6,100.0,7 +miles-1021,generation,2266.6,99.3,7 +miles-1021,generation,2271.6,100.0,7 +miles-1021,generation,2276.6,85.7,7 +miles-1021,generation,2281.6,86.1,7 +miles-1021,generation,2286.6,100.0,7 +miles-1021,generation,2291.6,85.7,7 +miles-1021,generation,2296.6,97.9,7 +miles-1021,generation,2301.6,85.7,7 +miles-1021,generation,2306.6,85.7,7 +miles-1021,generation,2311.6,100.0,7 +miles-1021,generation,2316.6,85.7,7 +miles-1021,generation,2321.6,85.7,7 +miles-1021,generation,2326.6,85.7,7 +miles-1021,generation,2331.6,85.7,7 +miles-1021,generation,2336.6,95.3,7 +miles-1021,generation,2341.6,85.7,7 +miles-1021,generation,2346.6,85.7,7 +miles-1021,generation,2351.6,85.7,7 +miles-1021,generation,2356.6,85.7,7 +miles-1021,generation,2361.6,85.7,7 +miles-1021,generation,2366.6,100.0,7 +miles-1021,generation,2371.6,85.7,7 +miles-1021,generation,2376.6,100.0,7 +miles-1021,generation,2381.6,88.3,7 +miles-1021,generation,2386.6,85.7,7 +miles-1021,generation,2391.6,85.7,7 +miles-1021,generation,2396.6,94.9,7 +miles-1021,generation,2401.6,100.0,7 +miles-1021,generation,2406.6,85.7,7 +miles-1021,generation,2411.6,85.7,7 +miles-1021,generation,2416.6,85.7,7 +miles-1021,generation,2421.6,85.7,7 +miles-1021,generation,2426.6,85.7,7 +miles-1021,generation,2431.6,85.7,7 +miles-1021,generation,2436.6,100.0,7 +miles-1021,generation,2441.6,85.7,7 +miles-1021,generation,2446.6,85.7,7 +miles-1021,generation,2451.6,85.7,7 +miles-1021,generation,2456.6,85.7,7 +miles-1021,generation,2461.6,85.7,7 +miles-1021,generation,2466.6,57.1,7 +miles-1021,generation,2471.6,57.1,7 +miles-1021,generation,2476.6,28.6,7 +miles-1021,generation,2481.6,28.6,7 +miles-1021,generation,2486.6,28.6,7 +miles-1021,generation,2491.6,28.6,7 +miles-1021,generation,2496.6,28.6,7 +miles-1021,generation,2501.6,28.6,7 +miles-1021,generation,2506.6,28.6,7 +miles-1021,generation,2511.6,28.6,7 +miles-1021,generation,2516.6,28.6,7 +miles-1021,generation,2521.6,28.6,7 +miles-1021,generation,2526.6,28.6,7 +miles-1021,generation,2531.6,85.7,7 +miles-1021,generation,2536.6,85.7,7 +miles-1021,generation,2541.6,85.7,7 +miles-1021,generation,2546.6,85.7,7 +miles-1021,generation,2551.6,85.7,7 +miles-1021,generation,2556.6,85.3,7 +miles-1021,generation,2561.6,85.7,7 +miles-1021,generation,2566.6,85.7,7 +miles-1021,generation,2571.6,65.3,7 +miles-1021,generation,2576.6,84.9,7 +miles-1021,generation,2581.6,85.7,7 +miles-1021,generation,2586.6,85.1,7 +miles-1021,generation,2591.6,99.0,7 +miles-1021,generation,2596.6,85.3,7 +miles-1021,generation,2601.6,85.7,7 +miles-1021,generation,2606.6,89.1,7 +miles-1021,generation,2611.6,97.3,7 +miles-1021,generation,2616.6,95.4,7 +miles-1021,generation,2621.6,85.3,7 +miles-1021,generation,2626.6,85.1,7 +miles-1021,generation,2631.6,98.0,7 +miles-1021,generation,2636.6,85.7,7 +miles-1021,generation,2641.6,84.7,7 +miles-1021,generation,2646.6,97.3,7 +miles-1021,generation,2651.6,81.3,7 +miles-1021,generation,2656.6,84.0,7 +miles-1021,generation,2661.6,85.7,7 +miles-1021,generation,2666.6,85.7,7 +miles-1021,generation,2671.6,97.0,7 +miles-1021,generation,2676.6,88.4,7 +miles-1021,generation,2681.6,85.7,7 +miles-1021,generation,2686.6,93.7,7 +miles-1021,generation,2691.6,85.7,7 +miles-1021,generation,2696.6,85.7,7 +miles-1021,generation,2701.6,85.7,7 +miles-1021,generation,2706.6,99.6,7 +miles-1021,generation,2711.6,95.7,7 +miles-1021,generation,2716.6,87.9,7 +miles-1021,generation,2721.6,88.1,7 +miles-1021,generation,2726.6,89.4,7 +miles-1021,generation,2731.6,89.9,7 +miles-1021,generation,2736.6,99.9,7 +miles-1021,generation,2741.6,85.7,7 +miles-1021,generation,2746.6,86.1,7 +miles-1021,generation,2751.6,87.7,7 +miles-1021,generation,2756.6,99.7,7 +miles-1021,generation,2761.6,85.7,7 +miles-1021,generation,2766.6,85.7,7 +miles-1021,generation,2771.6,100.0,7 +miles-1021,generation,2776.6,85.7,7 +miles-1021,generation,2781.6,85.7,7 +miles-1021,generation,2786.6,98.6,7 +miles-1021,generation,2791.6,85.7,7 +miles-1021,generation,2796.6,64.6,7 +miles-1021,generation,2801.6,81.7,7 +miles-1021,generation,2806.6,85.7,7 +miles-1021,generation,2811.6,85.7,7 +miles-1021,generation,2816.6,85.7,7 +miles-1021,generation,2821.6,85.7,7 +miles-1021,generation,2826.6,87.4,7 +miles-1021,generation,2831.6,85.7,7 +miles-1021,generation,2836.6,91.3,7 +miles-1021,generation,2841.6,85.7,7 +miles-1021,generation,2846.6,85.4,7 +miles-1021,generation,2851.6,85.7,7 +miles-1021,generation,2856.6,91.7,7 +miles-1021,generation,2861.6,85.7,7 +miles-1021,generation,2866.6,92.4,7 +miles-1021,generation,2871.6,99.4,7 +miles-1021,generation,2876.6,86.9,7 +miles-1021,generation,2881.6,99.4,7 +miles-1021,generation,2886.6,100.0,7 +miles-1021,generation,2891.6,90.7,7 +miles-1021,generation,2896.6,85.7,7 +miles-1021,generation,2901.6,63.9,7 +miles-1021,generation,2906.6,83.6,7 +miles-1021,generation,2911.6,99.4,7 +miles-1021,generation,2916.6,85.7,7 +miles-1021,generation,2921.6,97.0,7 +miles-1021,generation,2926.6,85.7,7 +miles-1021,generation,2931.6,85.7,7 +miles-1021,generation,2936.6,85.7,7 +miles-1021,generation,2941.6,99.1,7 +miles-1021,generation,2946.6,85.7,7 +miles-1021,generation,2951.6,100.0,7 +miles-1021,generation,2956.6,85.1,7 +miles-1021,generation,2961.6,100.0,7 +miles-1021,generation,2966.6,85.7,7 +miles-1021,generation,2971.7,85.7,7 +miles-1021,generation,2976.7,99.4,7 +miles-1021,generation,2981.7,85.7,7 +miles-1021,generation,2986.7,98.7,7 +miles-1021,generation,2991.7,85.7,7 +miles-1021,generation,2996.7,96.9,7 +miles-1021,generation,3001.7,85.7,7 +miles-1021,generation,3006.7,87.9,7 +miles-1021,generation,3011.7,85.7,7 +miles-1021,generation,3016.7,92.1,7 +miles-1021,generation,3021.7,85.7,7 +miles-1021,generation,3026.7,85.7,7 +miles-1021,generation,3031.7,90.0,7 +miles-1021,generation,3036.7,66.9,7 +miles-1021,generation,3041.7,83.9,7 +miles-1021,generation,3046.7,84.6,7 +miles-1021,generation,3051.7,88.7,7 +miles-1021,generation,3056.7,89.6,7 +miles-1021,generation,3061.7,85.7,7 +miles-1021,generation,3066.7,85.7,7 +miles-1021,generation,3071.7,85.7,7 +miles-1021,generation,3076.7,85.7,7 +miles-1021,generation,3081.7,85.7,7 +miles-1021,generation,3086.7,85.7,7 +miles-1021,generation,3091.7,85.7,7 +miles-1021,generation,3096.7,100.0,7 +miles-1021,generation,3101.7,85.7,7 +miles-1021,generation,3106.7,99.6,7 +miles-1021,generation,3111.7,85.7,7 +miles-1021,generation,3116.7,99.3,7 +miles-1021,generation,3121.7,85.7,7 +miles-1021,generation,3126.7,85.6,7 +miles-1021,generation,3131.7,85.7,7 +miles-1021,generation,3136.7,99.6,7 +miles-1021,generation,3141.7,96.0,7 +miles-1021,generation,3146.7,85.7,7 +miles-1021,generation,3151.7,85.7,7 +miles-1021,generation,3156.7,65.9,7 +miles-1021,generation,3161.7,82.6,7 +miles-1021,generation,3166.7,85.0,7 +miles-1021,generation,3171.7,85.0,7 +miles-1021,generation,3176.7,95.6,7 +miles-1021,generation,3181.7,88.0,7 +miles-1021,generation,3186.7,85.7,7 +miles-1021,generation,3191.7,94.3,7 +miles-1021,generation,3196.7,85.3,7 +miles-1021,generation,3201.7,85.7,7 +miles-1021,generation,3206.7,99.7,7 +miles-1021,generation,3211.7,99.4,7 +miles-1021,generation,3216.7,92.1,7 +miles-1021,generation,3221.7,86.7,7 +miles-1021,generation,3226.7,85.7,7 +miles-1021,generation,3231.7,85.7,7 +miles-1021,generation,3236.7,85.7,7 +miles-1021,generation,3241.7,85.7,7 +miles-1021,generation,3246.7,93.9,7 +miles-1021,generation,3251.7,85.7,7 +miles-1021,generation,3256.7,88.9,7 +miles-1021,generation,3261.7,99.0,7 +miles-1021,generation,3266.7,85.7,7 +miles-1021,generation,3271.7,97.0,7 +miles-1021,generation,3276.7,100.0,7 +miles-1021,generation,3281.7,40.1,7 +miles-1021,generation,3286.7,84.7,7 +miles-1021,generation,3291.7,87.3,7 +miles-1021,generation,3296.7,99.3,7 +miles-1021,generation,3301.7,85.7,7 +miles-1021,generation,3306.7,86.3,7 +miles-1021,generation,3311.7,85.7,7 +miles-1021,generation,3316.7,85.7,7 +miles-1021,generation,3321.7,99.6,7 +miles-1021,generation,3326.7,85.7,7 +miles-1021,generation,3331.7,85.7,7 +miles-1021,generation,3336.7,85.7,7 +miles-1021,generation,3341.7,85.7,7 +miles-1021,generation,3346.7,86.7,7 +miles-1021,generation,3351.7,85.7,7 +miles-1021,generation,3356.7,95.3,7 +miles-1021,generation,3361.7,85.3,7 +miles-1021,generation,3366.7,85.7,7 +miles-1021,generation,3371.7,93.7,7 +miles-1021,generation,3376.7,85.7,7 +miles-1021,generation,3381.7,85.7,7 +miles-1021,generation,3386.7,85.3,7 +miles-1021,generation,3391.7,85.7,7 +miles-1021,generation,3396.7,100.0,7 +miles-1021,generation,3401.7,98.7,7 +miles-1021,generation,3406.7,99.9,7 +miles-1021,generation,3411.7,56.4,7 +miles-1021,generation,3416.7,84.6,7 +miles-1021,generation,3421.7,88.6,7 +miles-1021,generation,3426.7,85.7,7 +miles-1021,generation,3431.7,85.7,7 +miles-1021,generation,3436.7,85.7,7 +miles-1021,generation,3441.7,85.7,7 +miles-1021,generation,3446.7,85.7,7 +miles-1021,generation,3451.7,85.7,7 +miles-1021,generation,3456.7,91.3,7 +miles-1021,generation,3461.7,100.0,7 +miles-1021,generation,3466.7,100.0,7 +miles-1021,generation,3471.7,85.7,7 +miles-1021,generation,3476.7,85.7,7 +miles-1021,generation,3481.7,100.0,7 +miles-1021,generation,3486.7,85.7,7 +miles-1021,generation,3491.7,93.3,7 +miles-1021,generation,3496.7,85.7,7 +miles-1021,generation,3501.7,85.7,7 +miles-1021,generation,3506.7,98.9,7 +miles-1021,generation,3511.7,99.3,7 +miles-1021,generation,3516.7,99.3,7 +miles-1021,generation,3521.7,85.7,7 +miles-1021,generation,3526.7,85.7,7 +miles-1021,generation,3531.7,85.7,7 +miles-1021,generation,3536.7,91.6,7 +miles-1021,generation,3541.7,60.6,7 +miles-1021,generation,3546.7,83.9,7 +miles-1021,generation,3551.7,96.6,7 +miles-1021,generation,3556.7,88.0,7 +miles-1021,generation,3561.7,85.7,7 +miles-1021,generation,3566.7,99.4,7 +miles-1021,generation,3571.7,91.9,7 +miles-1021,generation,3576.7,90.7,7 +miles-1021,generation,3581.7,85.7,7 +miles-1021,generation,3586.7,85.7,7 +miles-1021,generation,3591.7,100.0,7 +miles-1021,generation,3596.7,85.7,7 +miles-1021,generation,3601.7,84.1,7 +miles-1021,generation,3606.7,99.1,7 +miles-1021,generation,3611.7,85.7,7 +miles-1021,generation,3616.7,85.7,7 +miles-1021,generation,3621.7,85.7,7 +miles-1021,generation,3626.7,91.7,7 +miles-1021,generation,3631.7,100.0,7 +miles-1021,generation,3636.7,85.7,7 +miles-1021,generation,3641.7,85.7,7 +miles-1021,generation,3646.7,85.7,7 +miles-1021,generation,3651.7,100.0,7 +miles-1021,generation,3656.7,76.7,7 +miles-1021,generation,3661.7,83.9,7 +miles-1021,generation,3666.7,90.3,7 +miles-1021,generation,3671.7,85.7,7 +miles-1021,generation,3676.7,97.1,7 +miles-1021,generation,3681.7,100.0,7 +miles-1021,generation,3686.7,85.7,7 +miles-1021,generation,3691.7,85.7,7 +miles-1021,generation,3696.7,85.7,7 +miles-1021,generation,3701.7,99.4,7 +miles-1021,generation,3706.7,85.7,7 +miles-1021,generation,3711.7,85.7,7 +miles-1021,generation,3716.7,96.6,7 +miles-1021,generation,3721.7,100.0,7 +miles-1021,generation,3726.7,85.7,7 +miles-1021,generation,3731.7,85.7,7 +miles-1021,generation,3736.7,85.7,7 +miles-1021,generation,3741.7,85.7,7 +miles-1021,generation,3746.7,85.7,7 +miles-1021,generation,3751.7,85.7,7 +miles-1021,generation,3756.7,85.7,7 +miles-1021,generation,3761.7,89.3,7 +miles-1021,generation,3766.7,85.7,7 +miles-1021,generation,3771.7,85.7,7 +miles-1021,generation,3776.7,99.9,7 +miles-1021,generation,3781.7,99.3,7 +miles-1021,generation,3786.7,85.7,7 +miles-1021,generation,3791.7,85.7,7 +miles-1021,generation,3796.7,85.7,7 +miles-1021,generation,3801.7,85.7,7 +miles-1021,generation,3806.7,85.7,7 +miles-1021,generation,3811.7,85.7,7 +miles-1021,generation,3816.7,85.7,7 +miles-1021,generation,3821.7,85.7,7 +miles-1021,generation,3826.7,85.7,7 +miles-1021,generation,3831.7,100.0,7 +miles-1021,generation,3836.7,100.0,7 +miles-1021,generation,3841.7,85.7,7 +miles-1021,generation,3846.7,85.7,7 +miles-1021,generation,3851.7,85.7,7 +miles-1021,generation,3856.7,85.7,7 +miles-1021,generation,3861.7,85.7,7 +miles-1021,generation,3866.7,85.7,7 +miles-1021,generation,3871.7,100.0,7 +miles-1021,generation,3876.7,85.7,7 +miles-1021,generation,3881.7,85.7,7 +miles-1021,generation,3886.7,85.7,7 +miles-1021,generation,3891.7,100.0,7 +miles-1021,generation,3896.7,85.7,7 +miles-1021,generation,3901.8,85.7,7 +miles-1021,generation,3906.8,85.7,7 +miles-1021,generation,3911.8,85.7,7 +miles-1021,generation,3916.8,90.0,7 +miles-1021,generation,3921.8,85.7,7 +miles-1021,generation,3926.8,85.7,7 +miles-1021,generation,3931.8,28.6,7 +miles-1021,generation,3936.8,28.6,7 +miles-1021,generation,3941.8,28.6,7 +miles-1021,generation,3946.8,28.6,7 +miles-1021,generation,3951.8,28.6,7 +miles-1021,generation,3956.8,28.6,7 +miles-1021,generation,3961.8,28.6,7 +miles-1021,generation,3966.8,28.6,7 +miles-1021,generation,3971.8,28.6,7 +miles-1021,generation,3976.8,28.6,7 +miles-1021,generation,3981.8,28.6,7 +miles-1021,generation,3986.8,28.6,7 +miles-1021,generation,3991.8,0.0,7 +miles-1021,generation,3996.8,0.0,7 +miles-1021,generation,4001.8,0.0,7 +miles-1021,generation,4006.8,0.0,7 +miles-1021,generation,4011.8,0.0,7 +miles-1021,generation,4016.8,0.0,7 +miles-1021,generation,4021.8,0.0,7 +miles-1021,generation,4026.8,0.0,7 +miles-1021,generation,4031.8,0.0,7 +miles-1021,learner,31.5,0.0,8 +miles-1021,learner,36.5,0.0,8 +miles-1021,learner,41.5,0.0,8 +miles-1021,learner,46.5,0.0,8 +miles-1021,learner,51.5,0.0,8 +miles-1021,learner,56.5,0.0,8 +miles-1021,learner,61.5,0.0,8 +miles-1021,learner,66.5,0.0,8 +miles-1021,learner,71.5,0.0,8 +miles-1021,learner,76.5,0.0,8 +miles-1021,learner,81.5,0.0,8 +miles-1021,learner,86.5,0.0,8 +miles-1021,learner,91.5,0.0,8 +miles-1021,learner,96.5,0.0,8 +miles-1021,learner,101.5,0.0,8 +miles-1021,learner,106.5,0.0,8 +miles-1021,learner,111.5,0.0,8 +miles-1021,learner,116.5,0.0,8 +miles-1021,learner,121.5,0.0,8 +miles-1021,learner,126.5,0.0,8 +miles-1021,learner,131.5,0.0,8 +miles-1021,learner,136.5,0.0,8 +miles-1021,learner,141.5,0.0,8 +miles-1021,learner,146.5,0.0,8 +miles-1021,learner,151.5,0.0,8 +miles-1021,learner,156.5,0.0,8 +miles-1021,learner,161.5,0.0,8 +miles-1021,learner,166.5,0.0,8 +miles-1021,learner,171.5,0.0,8 +miles-1021,learner,176.5,0.0,8 +miles-1021,learner,181.5,0.0,8 +miles-1021,learner,186.5,0.0,8 +miles-1021,learner,191.5,0.0,8 +miles-1021,learner,196.5,0.0,8 +miles-1021,learner,201.5,0.0,8 +miles-1021,learner,206.5,0.0,8 +miles-1021,learner,211.5,0.0,8 +miles-1021,learner,216.5,0.0,8 +miles-1021,learner,221.5,0.0,8 +miles-1021,learner,226.5,0.0,8 +miles-1021,learner,231.5,0.0,8 +miles-1021,learner,236.5,0.0,8 +miles-1021,learner,241.5,0.0,8 +miles-1021,learner,246.5,0.0,8 +miles-1021,learner,251.5,0.0,8 +miles-1021,learner,256.5,0.0,8 +miles-1021,learner,261.5,0.0,8 +miles-1021,learner,266.5,0.0,8 +miles-1021,learner,271.5,0.0,8 +miles-1021,learner,276.5,0.0,8 +miles-1021,learner,281.5,0.0,8 +miles-1021,learner,286.5,0.0,8 +miles-1021,learner,291.5,0.0,8 +miles-1021,learner,296.5,0.0,8 +miles-1021,learner,301.5,0.0,8 +miles-1021,learner,306.5,0.0,8 +miles-1021,learner,311.5,0.0,8 +miles-1021,learner,316.5,0.0,8 +miles-1021,learner,321.5,0.0,8 +miles-1021,learner,326.5,0.0,8 +miles-1021,learner,331.5,0.0,8 +miles-1021,learner,336.5,0.0,8 +miles-1021,learner,341.5,0.0,8 +miles-1021,learner,346.5,0.0,8 +miles-1021,learner,351.5,0.0,8 +miles-1021,learner,356.5,0.0,8 +miles-1021,learner,361.5,0.0,8 +miles-1021,learner,366.5,0.0,8 +miles-1021,learner,371.5,0.0,8 +miles-1021,learner,376.5,0.0,8 +miles-1021,learner,381.5,0.0,8 +miles-1021,learner,386.5,0.0,8 +miles-1021,learner,391.5,0.0,8 +miles-1021,learner,396.5,0.0,8 +miles-1021,learner,401.5,0.0,8 +miles-1021,learner,406.5,0.0,8 +miles-1021,learner,411.5,0.0,8 +miles-1021,learner,416.5,0.0,8 +miles-1021,learner,421.5,0.0,8 +miles-1021,learner,426.5,0.0,8 +miles-1021,learner,431.5,0.0,8 +miles-1021,learner,436.5,0.0,8 +miles-1021,learner,441.5,0.0,8 +miles-1021,learner,446.5,0.0,8 +miles-1021,learner,451.5,0.0,8 +miles-1021,learner,456.5,0.0,8 +miles-1021,learner,461.5,0.0,8 +miles-1021,learner,466.5,0.0,8 +miles-1021,learner,471.5,0.0,8 +miles-1021,learner,476.5,0.0,8 +miles-1021,learner,481.5,0.0,8 +miles-1021,learner,486.5,0.0,8 +miles-1021,learner,491.5,0.0,8 +miles-1021,learner,496.5,0.9,8 +miles-1021,learner,501.5,0.2,8 +miles-1021,learner,506.5,27.6,8 +miles-1021,learner,511.5,0.9,8 +miles-1021,learner,516.5,1.0,8 +miles-1021,learner,521.5,1.0,8 +miles-1021,learner,526.5,1.0,8 +miles-1021,learner,531.5,1.0,8 +miles-1021,learner,536.5,1.0,8 +miles-1021,learner,541.5,0.0,8 +miles-1021,learner,546.5,0.0,8 +miles-1021,learner,551.5,0.0,8 +miles-1021,learner,556.5,0.0,8 +miles-1021,learner,561.5,0.0,8 +miles-1021,learner,566.5,0.0,8 +miles-1021,learner,571.5,0.0,8 +miles-1021,learner,576.5,0.0,8 +miles-1021,learner,581.5,0.0,8 +miles-1021,learner,586.5,0.0,8 +miles-1021,learner,591.5,0.0,8 +miles-1021,learner,596.5,0.0,8 +miles-1021,learner,601.5,0.0,8 +miles-1021,learner,606.5,0.0,8 +miles-1021,learner,611.5,0.0,8 +miles-1021,learner,616.5,0.0,8 +miles-1021,learner,621.5,0.0,8 +miles-1021,learner,626.5,0.0,8 +miles-1021,learner,631.5,0.0,8 +miles-1021,learner,636.5,0.0,8 +miles-1021,learner,641.5,0.0,8 +miles-1021,learner,646.5,0.0,8 +miles-1021,learner,651.5,0.0,8 +miles-1021,learner,656.5,0.0,8 +miles-1021,learner,661.5,0.0,8 +miles-1021,learner,666.5,0.0,8 +miles-1021,learner,671.5,0.0,8 +miles-1021,learner,676.5,0.0,8 +miles-1021,learner,681.5,0.0,8 +miles-1021,learner,686.5,0.0,8 +miles-1021,learner,691.6,0.0,8 +miles-1021,learner,696.5,0.0,8 +miles-1021,learner,701.6,0.0,8 +miles-1021,learner,706.6,0.0,8 +miles-1021,learner,711.6,0.0,8 +miles-1021,learner,716.6,0.0,8 +miles-1021,learner,721.6,0.0,8 +miles-1021,learner,726.6,0.0,8 +miles-1021,learner,731.6,0.0,8 +miles-1021,learner,736.6,0.0,8 +miles-1021,learner,741.6,0.0,8 +miles-1021,learner,746.6,0.0,8 +miles-1021,learner,751.6,0.0,8 +miles-1021,learner,756.6,0.0,8 +miles-1021,learner,761.6,0.0,8 +miles-1021,learner,766.6,0.0,8 +miles-1021,learner,771.6,0.0,8 +miles-1021,learner,776.6,0.0,8 +miles-1021,learner,781.6,0.0,8 +miles-1021,learner,786.6,0.0,8 +miles-1021,learner,791.6,0.0,8 +miles-1021,learner,796.6,0.0,8 +miles-1021,learner,801.6,0.0,8 +miles-1021,learner,806.6,0.0,8 +miles-1021,learner,811.6,0.0,8 +miles-1021,learner,816.6,0.0,8 +miles-1021,learner,821.6,0.0,8 +miles-1021,learner,826.6,0.0,8 +miles-1021,learner,831.6,0.0,8 +miles-1021,learner,836.6,0.0,8 +miles-1021,learner,841.6,0.0,8 +miles-1021,learner,846.6,0.0,8 +miles-1021,learner,851.6,0.0,8 +miles-1021,learner,856.6,0.0,8 +miles-1021,learner,861.6,0.0,8 +miles-1021,learner,866.6,0.0,8 +miles-1021,learner,871.6,0.0,8 +miles-1021,learner,876.6,0.0,8 +miles-1021,learner,881.6,0.0,8 +miles-1021,learner,886.6,0.0,8 +miles-1021,learner,891.6,0.0,8 +miles-1021,learner,896.6,0.0,8 +miles-1021,learner,901.6,0.0,8 +miles-1021,learner,906.6,0.0,8 +miles-1021,learner,911.6,0.0,8 +miles-1021,learner,916.6,0.0,8 +miles-1021,learner,921.6,0.0,8 +miles-1021,learner,926.6,0.0,8 +miles-1021,learner,931.6,0.0,8 +miles-1021,learner,936.6,51.4,8 +miles-1021,learner,941.6,18.1,8 +miles-1021,learner,946.6,0.0,8 +miles-1021,learner,951.6,0.0,8 +miles-1021,learner,956.6,0.0,8 +miles-1021,learner,961.6,29.9,8 +miles-1021,learner,966.6,0.0,8 +miles-1021,learner,971.6,1.2,8 +miles-1021,learner,976.6,0.0,8 +miles-1021,learner,981.6,42.9,8 +miles-1021,learner,986.6,0.0,8 +miles-1021,learner,991.6,0.0,8 +miles-1021,learner,996.6,1.6,8 +miles-1021,learner,1001.6,99.9,8 +miles-1021,learner,1006.6,23.0,8 +miles-1021,learner,1011.6,75.4,8 +miles-1021,learner,1016.6,2.0,8 +miles-1021,learner,1021.6,11.6,8 +miles-1021,learner,1026.6,18.1,8 +miles-1021,learner,1031.6,2.4,8 +miles-1021,learner,1036.6,0.0,8 +miles-1021,learner,1041.6,0.0,8 +miles-1021,learner,1046.6,0.0,8 +miles-1021,learner,1051.6,0.0,8 +miles-1021,learner,1056.6,0.0,8 +miles-1021,learner,1061.6,0.0,8 +miles-1021,learner,1066.6,0.4,8 +miles-1021,learner,1071.6,19.4,8 +miles-1021,learner,1076.6,4.2,8 +miles-1021,learner,1081.6,87.5,8 +miles-1021,learner,1086.6,87.5,8 +miles-1021,learner,1091.6,100.0,8 +miles-1021,learner,1096.6,87.5,8 +miles-1021,learner,1101.6,33.0,8 +miles-1021,learner,1106.6,90.8,8 +miles-1021,learner,1111.6,79.6,8 +miles-1021,learner,1116.6,98.2,8 +miles-1021,learner,1121.6,62.5,8 +miles-1021,learner,1126.6,76.5,8 +miles-1021,learner,1131.6,0.0,8 +miles-1021,learner,1136.6,0.0,8 +miles-1021,learner,1141.6,84.0,8 +miles-1021,learner,1146.6,74.8,8 +miles-1021,learner,1151.6,87.5,8 +miles-1021,learner,1156.6,87.5,8 +miles-1021,learner,1161.6,100.0,8 +miles-1021,learner,1166.6,87.5,8 +miles-1021,learner,1171.6,78.1,8 +miles-1021,learner,1176.6,93.0,8 +miles-1021,learner,1181.6,79.8,8 +miles-1021,learner,1186.6,87.5,8 +miles-1021,learner,1191.6,91.0,8 +miles-1021,learner,1196.6,57.4,8 +miles-1021,learner,1201.6,87.5,8 +miles-1021,learner,1206.6,63.0,8 +miles-1021,learner,1211.6,72.2,8 +miles-1021,learner,1216.6,100.0,8 +miles-1021,learner,1221.6,38.1,8 +miles-1021,learner,1226.6,11.2,8 +miles-1021,learner,1231.6,12.5,8 +miles-1021,learner,1236.6,100.0,8 +miles-1021,learner,1241.6,99.6,8 +miles-1021,learner,1246.6,99.4,8 +miles-1021,learner,1251.6,97.8,8 +miles-1021,learner,1256.6,98.8,8 +miles-1021,learner,1261.6,89.0,8 +miles-1021,learner,1266.6,100.0,8 +miles-1021,learner,1271.6,50.4,8 +miles-1021,learner,1276.6,0.0,8 +miles-1021,learner,1281.6,0.0,8 +miles-1021,learner,1286.6,0.0,8 +miles-1021,learner,1291.6,0.0,8 +miles-1021,learner,1296.6,0.0,8 +miles-1021,learner,1301.6,0.0,8 +miles-1021,learner,1306.6,37.5,8 +miles-1021,learner,1311.6,99.8,8 +miles-1021,learner,1316.6,99.4,8 +miles-1021,learner,1321.6,76.6,8 +miles-1021,learner,1326.6,0.0,8 +miles-1021,learner,1331.6,0.0,8 +miles-1021,learner,1336.6,0.0,8 +miles-1021,learner,1341.6,0.0,8 +miles-1021,learner,1346.6,0.0,8 +miles-1021,learner,1351.6,0.0,8 +miles-1021,learner,1356.6,0.0,8 +miles-1021,learner,1361.6,0.0,8 +miles-1021,learner,1366.6,0.0,8 +miles-1021,learner,1371.6,0.0,8 +miles-1021,learner,1376.6,0.0,8 +miles-1021,learner,1381.6,0.0,8 +miles-1021,learner,1386.6,0.0,8 +miles-1021,learner,1391.6,0.0,8 +miles-1021,learner,1396.6,0.0,8 +miles-1021,learner,1401.6,0.0,8 +miles-1021,learner,1406.6,0.0,8 +miles-1021,learner,1411.6,0.0,8 +miles-1021,learner,1416.6,0.0,8 +miles-1021,learner,1421.6,95.4,8 +miles-1021,learner,1426.6,87.5,8 +miles-1021,learner,1431.6,12.5,8 +miles-1021,learner,1436.6,96.2,8 +miles-1021,learner,1441.6,87.5,8 +miles-1021,learner,1446.6,87.5,8 +miles-1021,learner,1451.6,75.0,8 +miles-1021,learner,1456.6,88.4,8 +miles-1021,learner,1461.6,75.0,8 +miles-1021,learner,1466.6,87.5,8 +miles-1021,learner,1471.6,87.5,8 +miles-1021,learner,1476.6,91.5,8 +miles-1021,learner,1481.6,87.5,8 +miles-1021,learner,1486.6,94.9,8 +miles-1021,learner,1491.6,14.9,8 +miles-1021,learner,1496.6,12.8,8 +miles-1021,learner,1501.6,0.0,8 +miles-1021,learner,1506.6,0.0,8 +miles-1021,learner,1511.6,0.0,8 +miles-1021,learner,1516.6,0.0,8 +miles-1021,learner,1521.6,0.0,8 +miles-1021,learner,1526.6,0.0,8 +miles-1021,learner,1531.6,0.0,8 +miles-1021,learner,1536.6,0.0,8 +miles-1021,learner,1541.6,0.0,8 +miles-1021,learner,1546.6,0.0,8 +miles-1021,learner,1551.6,0.0,8 +miles-1021,learner,1556.6,96.1,8 +miles-1021,learner,1561.6,25.9,8 +miles-1021,learner,1566.6,100.0,8 +miles-1021,learner,1571.6,87.5,8 +miles-1021,learner,1576.6,74.9,8 +miles-1021,learner,1581.6,100.0,8 +miles-1021,learner,1586.6,0.1,8 +miles-1021,learner,1591.6,0.0,8 +miles-1021,learner,1596.6,0.0,8 +miles-1021,learner,1601.6,0.0,8 +miles-1021,learner,1606.6,0.0,8 +miles-1021,learner,1611.6,0.0,8 +miles-1021,learner,1616.6,0.0,8 +miles-1021,learner,1621.7,0.0,8 +miles-1021,learner,1626.7,0.0,8 +miles-1021,learner,1631.7,0.0,8 +miles-1021,learner,1636.7,0.0,8 +miles-1021,learner,1641.7,0.0,8 +miles-1021,learner,1646.7,0.0,8 +miles-1021,learner,1651.7,0.0,8 +miles-1021,learner,1656.7,0.0,8 +miles-1021,learner,1661.7,0.0,8 +miles-1021,learner,1666.7,0.0,8 +miles-1021,learner,1671.7,0.0,8 +miles-1021,learner,1676.7,0.0,8 +miles-1021,learner,1681.7,37.9,8 +miles-1021,learner,1686.7,85.5,8 +miles-1021,learner,1691.7,100.0,8 +miles-1021,learner,1696.7,99.8,8 +miles-1021,learner,1701.7,100.0,8 +miles-1021,learner,1706.7,0.0,8 +miles-1021,learner,1711.7,12.5,8 +miles-1021,learner,1716.7,4.0,8 +miles-1021,learner,1721.7,0.0,8 +miles-1021,learner,1726.7,0.0,8 +miles-1021,learner,1731.7,0.0,8 +miles-1021,learner,1736.7,0.0,8 +miles-1021,learner,1741.7,0.0,8 +miles-1021,learner,1746.7,0.0,8 +miles-1021,learner,1751.7,0.0,8 +miles-1021,learner,1756.7,0.0,8 +miles-1021,learner,1761.7,0.0,8 +miles-1021,learner,1766.7,0.0,8 +miles-1021,learner,1771.7,0.0,8 +miles-1021,learner,1776.7,0.0,8 +miles-1021,learner,1781.7,0.0,8 +miles-1021,learner,1786.7,0.0,8 +miles-1021,learner,1791.7,0.0,8 +miles-1021,learner,1796.7,0.0,8 +miles-1021,learner,1801.7,0.0,8 +miles-1021,learner,1806.7,24.2,8 +miles-1021,learner,1811.7,100.0,8 +miles-1021,learner,1816.7,64.2,8 +miles-1021,learner,1821.7,100.0,8 +miles-1021,learner,1826.7,88.2,8 +miles-1021,learner,1831.7,89.6,8 +miles-1021,learner,1836.7,93.2,8 +miles-1021,learner,1841.7,94.1,8 +miles-1021,learner,1846.7,0.2,8 +miles-1021,learner,1851.7,75.0,8 +miles-1021,learner,1856.7,0.0,8 +miles-1021,learner,1861.7,0.0,8 +miles-1021,learner,1866.7,0.0,8 +miles-1021,learner,1871.7,0.0,8 +miles-1021,learner,1876.7,0.0,8 +miles-1021,learner,1881.7,0.0,8 +miles-1021,learner,1886.7,0.0,8 +miles-1021,learner,1891.7,0.0,8 +miles-1021,learner,1896.7,0.0,8 +miles-1021,learner,1901.7,0.0,8 +miles-1021,learner,1906.7,0.0,8 +miles-1021,learner,1911.7,0.0,8 +miles-1021,learner,1916.7,0.0,8 +miles-1021,learner,1921.7,0.0,8 +miles-1021,learner,1926.7,4.2,8 +miles-1021,learner,1931.7,85.5,8 +miles-1021,learner,1936.7,90.9,8 +miles-1021,learner,1941.7,62.5,8 +miles-1021,learner,1946.7,100.0,8 +miles-1021,learner,1951.7,62.5,8 +miles-1021,learner,1956.7,0.0,8 +miles-1021,learner,1961.7,0.0,8 +miles-1021,learner,1966.7,0.0,8 +miles-1021,learner,1971.7,0.0,8 +miles-1021,learner,1976.7,0.0,8 +miles-1021,learner,1981.7,0.0,8 +miles-1021,learner,1986.7,0.0,8 +miles-1021,learner,1991.7,0.0,8 +miles-1021,learner,1996.7,0.0,8 +miles-1021,learner,2001.7,0.0,8 +miles-1021,learner,2006.7,0.0,8 +miles-1021,learner,2011.7,0.0,8 +miles-1021,learner,2016.7,0.0,8 +miles-1021,learner,2021.7,0.0,8 +miles-1021,learner,2026.7,0.0,8 +miles-1021,learner,2031.7,0.0,8 +miles-1021,learner,2036.7,0.0,8 +miles-1021,learner,2041.7,0.0,8 +miles-1021,learner,2046.7,0.0,8 +miles-1021,learner,2051.7,57.1,8 +miles-1021,learner,2056.7,100.0,8 +miles-1021,learner,2061.7,87.4,8 +miles-1021,learner,2066.7,87.5,8 +miles-1021,learner,2071.7,99.6,8 +miles-1021,learner,2076.7,12.6,8 +miles-1021,learner,2081.7,0.0,8 +miles-1021,learner,2086.7,0.0,8 +miles-1021,learner,2091.7,0.0,8 +miles-1021,learner,2096.7,0.0,8 +miles-1021,learner,2101.7,0.0,8 +miles-1021,learner,2106.7,0.0,8 +miles-1021,learner,2111.7,0.0,8 +miles-1021,learner,2116.7,0.0,8 +miles-1021,learner,2121.7,0.0,8 +miles-1021,learner,2126.7,0.0,8 +miles-1021,learner,2131.7,0.0,8 +miles-1021,learner,2136.7,0.0,8 +miles-1021,learner,2141.7,0.0,8 +miles-1021,learner,2146.7,0.0,8 +miles-1021,learner,2151.7,0.0,8 +miles-1021,learner,2156.7,0.0,8 +miles-1021,learner,2161.7,0.0,8 +miles-1021,learner,2166.7,0.0,8 +miles-1021,learner,2171.7,0.0,8 +miles-1021,learner,2176.7,0.0,8 +miles-1021,learner,2181.7,0.0,8 +miles-1021,learner,2186.7,0.0,8 +miles-1021,learner,2191.7,0.0,8 +miles-1021,learner,2196.7,0.0,8 +miles-1021,learner,2201.7,0.0,8 +miles-1021,learner,2206.7,0.0,8 +miles-1021,learner,2211.7,0.0,8 +miles-1021,learner,2216.7,0.0,8 +miles-1021,learner,2221.7,0.0,8 +miles-1021,learner,2226.7,0.0,8 +miles-1021,learner,2231.7,0.0,8 +miles-1021,learner,2236.7,0.0,8 +miles-1021,learner,2241.7,0.0,8 +miles-1021,learner,2246.7,0.0,8 +miles-1021,learner,2251.7,0.0,8 +miles-1021,learner,2256.7,0.0,8 +miles-1021,learner,2261.7,0.0,8 +miles-1021,learner,2266.7,0.0,8 +miles-1021,learner,2271.7,0.0,8 +miles-1021,learner,2276.7,0.0,8 +miles-1021,learner,2281.7,0.0,8 +miles-1021,learner,2286.7,0.0,8 +miles-1021,learner,2291.7,0.0,8 +miles-1021,learner,2296.7,0.0,8 +miles-1021,learner,2301.7,0.0,8 +miles-1021,learner,2306.7,0.0,8 +miles-1021,learner,2311.7,0.0,8 +miles-1021,learner,2316.7,0.0,8 +miles-1021,learner,2321.7,0.0,8 +miles-1021,learner,2326.7,0.0,8 +miles-1021,learner,2331.7,0.0,8 +miles-1021,learner,2336.7,0.0,8 +miles-1021,learner,2341.7,0.0,8 +miles-1021,learner,2346.7,0.0,8 +miles-1021,learner,2351.8,0.0,8 +miles-1021,learner,2356.8,0.0,8 +miles-1021,learner,2361.8,0.0,8 +miles-1021,learner,2366.8,0.0,8 +miles-1021,learner,2371.8,0.0,8 +miles-1021,learner,2376.8,0.0,8 +miles-1021,learner,2381.8,0.0,8 +miles-1021,learner,2386.8,0.0,8 +miles-1021,learner,2391.8,0.0,8 +miles-1021,learner,2396.8,0.0,8 +miles-1021,learner,2401.8,0.0,8 +miles-1021,learner,2406.8,0.0,8 +miles-1021,learner,2411.8,0.0,8 +miles-1021,learner,2416.8,0.0,8 +miles-1021,learner,2421.8,0.0,8 +miles-1021,learner,2426.8,0.0,8 +miles-1021,learner,2431.8,0.0,8 +miles-1021,learner,2436.8,0.0,8 +miles-1021,learner,2441.8,0.0,8 +miles-1021,learner,2446.8,0.0,8 +miles-1021,learner,2451.8,0.0,8 +miles-1021,learner,2456.8,0.0,8 +miles-1021,learner,2461.8,0.0,8 +miles-1021,learner,2466.8,0.0,8 +miles-1021,learner,2471.8,0.0,8 +miles-1021,learner,2476.8,0.0,8 +miles-1021,learner,2481.8,0.0,8 +miles-1021,learner,2486.8,0.0,8 +miles-1021,learner,2491.8,0.0,8 +miles-1021,learner,2496.8,0.0,8 +miles-1021,learner,2501.8,0.0,8 +miles-1021,learner,2506.8,0.0,8 +miles-1021,learner,2511.8,0.0,8 +miles-1021,learner,2516.8,0.0,8 +miles-1021,learner,2521.8,0.0,8 +miles-1021,learner,2526.8,0.0,8 +miles-1021,learner,2531.8,12.5,8 +miles-1021,learner,2536.8,99.6,8 +miles-1021,learner,2541.8,100.0,8 +miles-1021,learner,2546.8,87.4,8 +miles-1021,learner,2551.8,87.5,8 +miles-1021,learner,2556.8,87.5,8 +miles-1021,learner,2561.8,100.0,8 +miles-1021,learner,2566.8,13.0,8 +miles-1021,learner,2571.8,58.2,8 +miles-1021,learner,2576.8,0.0,8 +miles-1021,learner,2581.8,25.0,8 +miles-1021,learner,2586.8,100.0,8 +miles-1021,learner,2591.8,100.0,8 +miles-1021,learner,2596.8,87.5,8 +miles-1021,learner,2601.8,91.1,8 +miles-1021,learner,2606.8,96.9,8 +miles-1021,learner,2611.8,87.5,8 +miles-1021,learner,2616.8,100.0,8 +miles-1021,learner,2621.8,25.0,8 +miles-1021,learner,2626.8,25.0,8 +miles-1021,learner,2631.8,28.5,8 +miles-1021,learner,2636.8,0.0,8 +miles-1021,learner,2641.8,0.0,8 +miles-1021,learner,2646.8,0.0,8 +miles-1021,learner,2651.8,35.0,8 +miles-1021,learner,2656.8,69.9,8 +miles-1021,learner,2661.8,99.0,8 +miles-1021,learner,2666.8,0.2,8 +miles-1021,learner,2671.8,13.1,8 +miles-1021,learner,2676.8,0.0,8 +miles-1021,learner,2681.8,0.0,8 +miles-1021,learner,2686.8,0.0,8 +miles-1021,learner,2691.8,0.0,8 +miles-1021,learner,2696.8,0.0,8 +miles-1021,learner,2701.8,0.0,8 +miles-1021,learner,2706.8,0.0,8 +miles-1021,learner,2711.8,0.0,8 +miles-1021,learner,2716.8,0.0,8 +miles-1021,learner,2721.8,0.0,8 +miles-1021,learner,2726.8,0.0,8 +miles-1021,learner,2731.8,0.0,8 +miles-1021,learner,2736.8,0.0,8 +miles-1021,learner,2741.8,0.0,8 +miles-1021,learner,2746.8,0.0,8 +miles-1021,learner,2751.8,0.0,8 +miles-1021,learner,2756.8,0.0,8 +miles-1021,learner,2761.8,0.0,8 +miles-1021,learner,2766.8,0.0,8 +miles-1021,learner,2771.8,0.0,8 +miles-1021,learner,2776.8,0.0,8 +miles-1021,learner,2781.8,0.0,8 +miles-1021,learner,2786.8,0.0,8 +miles-1021,learner,2791.8,0.0,8 +miles-1021,learner,2796.8,37.6,8 +miles-1021,learner,2801.8,100.0,8 +miles-1021,learner,2806.8,37.5,8 +miles-1021,learner,2811.8,100.0,8 +miles-1021,learner,2816.8,0.1,8 +miles-1021,learner,2821.8,25.8,8 +miles-1021,learner,2826.8,0.0,8 +miles-1021,learner,2831.8,0.0,8 +miles-1021,learner,2836.8,0.0,8 +miles-1021,learner,2841.8,0.0,8 +miles-1021,learner,2846.8,0.0,8 +miles-1021,learner,2851.8,0.0,8 +miles-1021,learner,2856.8,0.0,8 +miles-1021,learner,2861.8,0.0,8 +miles-1021,learner,2866.8,0.0,8 +miles-1021,learner,2871.8,0.0,8 +miles-1021,learner,2876.8,0.0,8 +miles-1021,learner,2881.8,0.0,8 +miles-1021,learner,2886.8,0.0,8 +miles-1021,learner,2891.8,0.0,8 +miles-1021,learner,2896.8,0.0,8 +miles-1021,learner,2901.8,28.9,8 +miles-1021,learner,2906.8,0.0,8 +miles-1021,learner,2911.8,50.0,8 +miles-1021,learner,2916.8,87.6,8 +miles-1021,learner,2921.8,99.8,8 +miles-1021,learner,2926.8,99.6,8 +miles-1021,learner,2931.8,75.0,8 +miles-1021,learner,2936.8,0.0,8 +miles-1021,learner,2941.8,0.0,8 +miles-1021,learner,2946.8,0.0,8 +miles-1021,learner,2951.8,0.0,8 +miles-1021,learner,2956.8,0.0,8 +miles-1021,learner,2961.8,0.0,8 +miles-1021,learner,2966.8,0.0,8 +miles-1021,learner,2971.8,0.0,8 +miles-1021,learner,2976.8,0.0,8 +miles-1021,learner,2981.8,0.0,8 +miles-1021,learner,2986.8,0.0,8 +miles-1021,learner,2991.8,0.0,8 +miles-1021,learner,2996.8,0.0,8 +miles-1021,learner,3001.8,0.0,8 +miles-1021,learner,3006.8,0.0,8 +miles-1021,learner,3011.8,0.0,8 +miles-1021,learner,3016.8,0.0,8 +miles-1021,learner,3021.8,0.0,8 +miles-1021,learner,3026.8,0.0,8 +miles-1021,learner,3031.8,0.0,8 +miles-1021,learner,3036.8,96.4,8 +miles-1021,learner,3041.8,0.0,8 +miles-1021,learner,3046.8,76.0,8 +miles-1021,learner,3051.8,75.2,8 +miles-1021,learner,3056.8,87.5,8 +miles-1021,learner,3061.8,50.0,8 +miles-1021,learner,3066.8,90.2,8 +miles-1021,learner,3071.8,92.0,8 +miles-1021,learner,3076.8,87.5,8 +miles-1021,learner,3081.8,100.0,8 +miles-1021,learner,3086.8,87.5,8 +miles-1021,learner,3091.8,0.0,8 +miles-1021,learner,3096.8,0.0,8 +miles-1021,learner,3101.9,0.0,8 +miles-1021,learner,3106.9,0.0,8 +miles-1021,learner,3111.9,0.0,8 +miles-1021,learner,3116.9,0.0,8 +miles-1021,learner,3121.9,0.0,8 +miles-1021,learner,3126.9,0.0,8 +miles-1021,learner,3131.9,0.0,8 +miles-1021,learner,3136.9,0.0,8 +miles-1021,learner,3141.9,0.0,8 +miles-1021,learner,3146.9,0.0,8 +miles-1021,learner,3151.9,0.0,8 +miles-1021,learner,3156.9,96.4,8 +miles-1021,learner,3161.9,0.9,8 +miles-1021,learner,3166.9,50.9,8 +miles-1021,learner,3171.9,62.5,8 +miles-1021,learner,3176.9,100.0,8 +miles-1021,learner,3181.9,87.5,8 +miles-1021,learner,3186.9,86.9,8 +miles-1021,learner,3191.9,92.4,8 +miles-1021,learner,3196.9,87.5,8 +miles-1021,learner,3201.9,100.0,8 +miles-1021,learner,3206.9,0.5,8 +miles-1021,learner,3211.9,0.0,8 +miles-1021,learner,3216.9,0.0,8 +miles-1021,learner,3221.9,0.0,8 +miles-1021,learner,3226.9,0.0,8 +miles-1021,learner,3231.9,0.0,8 +miles-1021,learner,3236.9,0.0,8 +miles-1021,learner,3241.9,0.0,8 +miles-1021,learner,3246.9,0.0,8 +miles-1021,learner,3251.9,0.0,8 +miles-1021,learner,3256.9,0.0,8 +miles-1021,learner,3261.9,0.0,8 +miles-1021,learner,3266.9,0.0,8 +miles-1021,learner,3271.9,0.0,8 +miles-1021,learner,3276.9,0.0,8 +miles-1021,learner,3281.9,95.4,8 +miles-1021,learner,3286.9,57.1,8 +miles-1021,learner,3291.9,62.2,8 +miles-1021,learner,3296.9,89.2,8 +miles-1021,learner,3301.9,94.4,8 +miles-1021,learner,3306.9,0.2,8 +miles-1021,learner,3311.9,0.0,8 +miles-1021,learner,3316.9,0.0,8 +miles-1021,learner,3321.9,0.0,8 +miles-1021,learner,3326.9,0.0,8 +miles-1021,learner,3331.9,0.0,8 +miles-1021,learner,3336.9,0.0,8 +miles-1021,learner,3341.9,0.0,8 +miles-1021,learner,3346.9,0.0,8 +miles-1021,learner,3351.9,0.0,8 +miles-1021,learner,3356.9,0.0,8 +miles-1021,learner,3361.9,0.0,8 +miles-1021,learner,3366.9,0.0,8 +miles-1021,learner,3371.9,0.0,8 +miles-1021,learner,3376.9,0.0,8 +miles-1021,learner,3381.9,0.0,8 +miles-1021,learner,3386.9,0.0,8 +miles-1021,learner,3391.9,0.0,8 +miles-1021,learner,3396.9,0.0,8 +miles-1021,learner,3401.9,0.0,8 +miles-1021,learner,3406.9,0.0,8 +miles-1021,learner,3411.9,0.1,8 +miles-1021,learner,3416.9,0.0,8 +miles-1021,learner,3421.9,51.4,8 +miles-1021,learner,3426.9,99.9,8 +miles-1021,learner,3431.9,100.0,8 +miles-1021,learner,3436.9,0.2,8 +miles-1021,learner,3441.9,0.0,8 +miles-1021,learner,3446.9,0.0,8 +miles-1021,learner,3451.9,0.0,8 +miles-1021,learner,3456.9,0.0,8 +miles-1021,learner,3461.9,0.0,8 +miles-1021,learner,3466.9,0.0,8 +miles-1021,learner,3471.9,0.0,8 +miles-1021,learner,3476.9,0.0,8 +miles-1021,learner,3481.9,0.0,8 +miles-1021,learner,3486.9,0.0,8 +miles-1021,learner,3491.9,0.0,8 +miles-1021,learner,3496.9,0.0,8 +miles-1021,learner,3501.9,0.0,8 +miles-1021,learner,3506.9,0.0,8 +miles-1021,learner,3511.9,0.0,8 +miles-1021,learner,3516.9,0.0,8 +miles-1021,learner,3521.9,0.0,8 +miles-1021,learner,3526.9,0.0,8 +miles-1021,learner,3531.9,0.0,8 +miles-1021,learner,3536.9,0.0,8 +miles-1021,learner,3541.9,96.1,8 +miles-1021,learner,3546.9,0.2,8 +miles-1021,learner,3551.9,75.0,8 +miles-1021,learner,3556.9,100.0,8 +miles-1021,learner,3561.9,91.0,8 +miles-1021,learner,3566.9,99.6,8 +miles-1021,learner,3571.9,97.0,8 +miles-1021,learner,3576.9,100.0,8 +miles-1021,learner,3581.9,0.2,8 +miles-1021,learner,3586.9,1.1,8 +miles-1021,learner,3591.9,0.0,8 +miles-1021,learner,3596.9,25.2,8 +miles-1021,learner,3601.9,87.5,8 +miles-1021,learner,3606.9,87.0,8 +miles-1021,learner,3611.9,86.8,8 +miles-1021,learner,3616.9,87.5,8 +miles-1021,learner,3621.9,87.2,8 +miles-1021,learner,3626.9,87.1,8 +miles-1021,learner,3631.9,86.5,8 +miles-1021,learner,3636.9,87.5,8 +miles-1021,learner,3641.9,87.0,8 +miles-1021,learner,3646.9,87.4,8 +miles-1021,learner,3651.9,87.4,8 +miles-1021,learner,3656.9,0.0,8 +miles-1021,learner,3661.9,0.0,8 +miles-1021,learner,3666.9,0.0,8 +miles-1021,learner,3671.9,0.0,8 +miles-1021,learner,3676.9,0.0,8 +miles-1021,learner,3681.9,0.0,8 +miles-1021,learner,3686.9,0.0,8 +miles-1021,learner,3691.9,0.0,8 +miles-1021,learner,3696.9,0.0,8 +miles-1021,learner,3701.9,0.0,8 +miles-1021,learner,3706.9,0.0,8 +miles-1021,learner,3711.9,0.0,8 +miles-1021,learner,3716.9,0.0,8 +miles-1021,learner,3721.9,0.0,8 +miles-1021,learner,3726.9,0.0,8 +miles-1021,learner,3731.9,0.0,8 +miles-1021,learner,3736.9,0.0,8 +miles-1021,learner,3741.9,0.0,8 +miles-1021,learner,3746.9,0.0,8 +miles-1021,learner,3751.9,0.0,8 +miles-1021,learner,3756.9,0.0,8 +miles-1021,learner,3761.9,0.0,8 +miles-1021,learner,3766.9,0.0,8 +miles-1021,learner,3771.9,0.0,8 +miles-1021,learner,3776.9,0.0,8 +miles-1021,learner,3781.9,0.0,8 +miles-1021,learner,3786.9,0.0,8 +miles-1021,learner,3791.9,0.0,8 +miles-1021,learner,3796.9,0.0,8 +miles-1021,learner,3801.9,0.0,8 +miles-1021,learner,3806.9,0.0,8 +miles-1021,learner,3811.9,0.0,8 +miles-1021,learner,3816.9,0.0,8 +miles-1021,learner,3821.9,0.0,8 +miles-1021,learner,3826.9,0.0,8 +miles-1021,learner,3831.9,0.0,8 +miles-1021,learner,3836.9,0.0,8 +miles-1021,learner,3841.9,0.0,8 +miles-1021,learner,3846.9,0.0,8 +miles-1021,learner,3851.9,0.0,8 +miles-1021,learner,3856.9,0.0,8 +miles-1021,learner,3861.9,0.0,8 +miles-1021,learner,3867.0,0.0,8 +miles-1021,learner,3872.0,0.0,8 +miles-1021,learner,3877.0,0.0,8 +miles-1021,learner,3882.0,0.0,8 +miles-1021,learner,3887.0,0.0,8 +miles-1021,learner,3892.0,0.0,8 +miles-1021,learner,3897.0,0.0,8 +miles-1021,learner,3902.0,0.0,8 +miles-1021,learner,3907.0,0.0,8 +miles-1021,learner,3912.0,0.0,8 +miles-1021,learner,3917.0,0.0,8 +miles-1021,learner,3922.0,0.0,8 +miles-1021,learner,3927.0,0.0,8 +miles-1021,learner,3932.0,0.0,8 +miles-1021,learner,3937.0,0.0,8 +miles-1021,learner,3942.0,0.0,8 +miles-1021,learner,3947.0,0.0,8 +miles-1021,learner,3952.0,0.0,8 +miles-1021,learner,3957.0,0.0,8 +miles-1021,learner,3962.0,0.0,8 +miles-1021,learner,3967.0,0.0,8 +miles-1021,learner,3972.0,0.0,8 +miles-1021,learner,3977.0,0.0,8 +miles-1021,learner,3982.0,0.0,8 +miles-1021,learner,3987.0,0.0,8 +miles-1021,learner,3997.0,0.0,8 +miles-1021,learner,4007.0,0.0,8 +verl-966,generation,8.1,0.0,8 +verl-966,generation,13.1,0.0,8 +verl-966,generation,18.1,0.0,8 +verl-966,generation,23.1,0.0,8 +verl-966,generation,28.1,0.0,8 +verl-966,generation,33.1,0.0,8 +verl-966,generation,38.1,0.0,8 +verl-966,generation,43.1,0.0,8 +verl-966,generation,48.1,0.0,8 +verl-966,generation,53.1,0.0,8 +verl-966,generation,58.1,0.0,8 +verl-966,generation,63.1,0.0,8 +verl-966,generation,68.1,0.0,8 +verl-966,generation,73.1,0.0,8 +verl-966,generation,78.1,0.0,8 +verl-966,generation,83.1,0.0,8 +verl-966,generation,88.1,0.0,8 +verl-966,generation,93.1,0.0,8 +verl-966,generation,98.1,0.0,8 +verl-966,generation,103.1,0.0,8 +verl-966,generation,108.1,0.0,8 +verl-966,generation,113.1,0.0,8 +verl-966,generation,118.1,0.0,8 +verl-966,generation,123.1,0.0,8 +verl-966,generation,128.1,0.0,8 +verl-966,generation,133.1,0.0,8 +verl-966,generation,138.1,0.0,8 +verl-966,generation,143.1,0.0,8 +verl-966,generation,148.1,0.0,8 +verl-966,generation,153.1,0.0,8 +verl-966,generation,158.1,0.0,8 +verl-966,generation,163.1,0.0,8 +verl-966,generation,168.1,0.0,8 +verl-966,generation,173.1,0.0,8 +verl-966,generation,178.1,0.0,8 +verl-966,generation,183.1,0.0,8 +verl-966,generation,188.1,0.0,8 +verl-966,generation,193.1,0.0,8 +verl-966,generation,198.1,0.0,8 +verl-966,generation,203.1,0.0,8 +verl-966,generation,208.1,0.0,8 +verl-966,generation,213.1,0.0,8 +verl-966,generation,218.1,0.0,8 +verl-966,generation,223.1,0.0,8 +verl-966,generation,228.1,0.0,8 +verl-966,generation,233.1,0.0,8 +verl-966,generation,238.1,0.0,8 +verl-966,generation,243.1,0.0,8 +verl-966,generation,248.2,0.0,8 +verl-966,generation,253.2,0.0,8 +verl-966,generation,258.2,0.0,8 +verl-966,generation,263.2,0.0,8 +verl-966,generation,268.2,0.0,8 +verl-966,generation,273.2,0.0,8 +verl-966,generation,278.2,0.0,8 +verl-966,generation,283.2,0.0,8 +verl-966,generation,288.2,0.0,8 +verl-966,generation,293.2,0.0,8 +verl-966,generation,298.2,0.0,8 +verl-966,generation,303.2,0.0,8 +verl-966,generation,308.2,0.0,8 +verl-966,generation,313.2,0.0,8 +verl-966,generation,318.2,0.0,8 +verl-966,generation,323.2,0.0,8 +verl-966,generation,328.2,17.0,8 +verl-966,generation,333.2,25.4,8 +verl-966,generation,338.2,21.8,8 +verl-966,generation,343.2,17.0,8 +verl-966,generation,348.2,46.8,8 +verl-966,generation,353.2,0.0,8 +verl-966,generation,358.2,0.0,8 +verl-966,generation,363.2,0.0,8 +verl-966,generation,368.2,0.0,8 +verl-966,generation,373.2,0.0,8 +verl-966,generation,378.2,0.0,8 +verl-966,generation,383.2,0.0,8 +verl-966,generation,388.2,0.0,8 +verl-966,generation,393.2,0.0,8 +verl-966,generation,398.2,0.0,8 +verl-966,generation,403.2,0.0,8 +verl-966,generation,408.2,0.0,8 +verl-966,generation,413.2,0.0,8 +verl-966,generation,418.2,0.0,8 +verl-966,generation,423.2,0.0,8 +verl-966,generation,428.2,0.0,8 +verl-966,generation,433.2,0.0,8 +verl-966,generation,438.2,0.0,8 +verl-966,generation,443.2,0.0,8 +verl-966,generation,448.2,0.0,8 +verl-966,generation,453.2,0.0,8 +verl-966,generation,458.2,0.0,8 +verl-966,generation,463.2,0.0,8 +verl-966,generation,468.2,0.0,8 +verl-966,generation,473.2,0.0,8 +verl-966,generation,478.2,0.0,8 +verl-966,generation,483.2,0.0,8 +verl-966,generation,488.2,0.0,8 +verl-966,generation,493.2,0.0,8 +verl-966,generation,498.2,0.0,8 +verl-966,generation,503.2,0.0,8 +verl-966,generation,508.2,0.0,8 +verl-966,generation,513.2,0.0,8 +verl-966,generation,518.2,0.0,8 +verl-966,generation,523.2,0.0,8 +verl-966,generation,528.2,0.0,8 +verl-966,generation,533.2,0.0,8 +verl-966,generation,538.2,0.0,8 +verl-966,generation,543.2,0.0,8 +verl-966,generation,548.2,0.0,8 +verl-966,generation,553.2,0.0,8 +verl-966,generation,558.2,0.0,8 +verl-966,generation,563.2,0.0,8 +verl-966,generation,568.2,0.0,8 +verl-966,generation,573.2,0.0,8 +verl-966,generation,578.2,0.0,8 +verl-966,generation,583.2,0.0,8 +verl-966,generation,588.2,0.0,8 +verl-966,generation,593.2,0.0,8 +verl-966,generation,598.2,0.0,8 +verl-966,generation,603.2,0.0,8 +verl-966,generation,608.2,0.0,8 +verl-966,generation,613.2,0.0,8 +verl-966,generation,618.2,0.0,8 +verl-966,generation,623.2,0.0,8 +verl-966,generation,628.2,0.0,8 +verl-966,generation,633.2,0.0,8 +verl-966,generation,638.2,0.0,8 +verl-966,generation,643.2,0.0,8 +verl-966,generation,648.2,0.0,8 +verl-966,generation,653.2,0.0,8 +verl-966,generation,658.2,0.0,8 +verl-966,generation,663.2,0.0,8 +verl-966,generation,668.2,0.0,8 +verl-966,generation,673.2,0.0,8 +verl-966,generation,678.2,0.0,8 +verl-966,generation,683.2,0.0,8 +verl-966,generation,688.2,0.0,8 +verl-966,generation,693.2,0.0,8 +verl-966,generation,698.2,0.0,8 +verl-966,generation,703.2,0.0,8 +verl-966,generation,708.2,0.0,8 +verl-966,generation,713.2,0.0,8 +verl-966,generation,718.2,0.0,8 +verl-966,generation,723.2,0.0,8 +verl-966,generation,728.2,0.0,8 +verl-966,generation,733.2,0.0,8 +verl-966,generation,738.2,0.0,8 +verl-966,generation,743.2,0.0,8 +verl-966,generation,748.2,0.0,8 +verl-966,generation,753.2,0.0,8 +verl-966,generation,758.2,0.0,8 +verl-966,generation,763.2,0.0,8 +verl-966,generation,768.2,0.0,8 +verl-966,generation,773.2,0.0,8 +verl-966,generation,778.2,0.0,8 +verl-966,generation,783.2,0.0,8 +verl-966,generation,788.2,0.0,8 +verl-966,generation,793.2,0.0,8 +verl-966,generation,798.2,0.0,8 +verl-966,generation,803.2,0.0,8 +verl-966,generation,808.2,0.0,8 +verl-966,generation,813.2,0.0,8 +verl-966,generation,818.2,0.0,8 +verl-966,generation,823.2,0.0,8 +verl-966,generation,828.2,0.0,8 +verl-966,generation,833.2,0.0,8 +verl-966,generation,838.2,0.0,8 +verl-966,generation,843.2,0.0,8 +verl-966,generation,848.2,0.0,8 +verl-966,generation,853.2,0.0,8 +verl-966,generation,858.2,0.0,8 +verl-966,generation,863.2,0.0,8 +verl-966,generation,868.2,0.0,8 +verl-966,generation,873.2,0.0,8 +verl-966,generation,878.2,0.0,8 +verl-966,generation,883.2,0.0,8 +verl-966,generation,888.2,0.0,8 +verl-966,generation,893.2,0.0,8 +verl-966,generation,898.2,0.0,8 +verl-966,generation,903.2,0.0,8 +verl-966,generation,908.2,0.0,8 +verl-966,generation,913.2,0.0,8 +verl-966,generation,918.2,0.0,8 +verl-966,generation,923.2,0.0,8 +verl-966,generation,928.2,0.0,8 +verl-966,generation,933.2,0.0,8 +verl-966,generation,938.2,0.0,8 +verl-966,generation,943.2,0.0,8 +verl-966,generation,948.2,0.0,8 +verl-966,generation,953.2,0.0,8 +verl-966,generation,958.2,0.0,8 +verl-966,generation,963.2,0.0,8 +verl-966,generation,968.2,0.0,8 +verl-966,generation,973.2,0.0,8 +verl-966,generation,978.2,0.0,8 +verl-966,generation,983.2,0.0,8 +verl-966,generation,988.2,0.0,8 +verl-966,generation,993.2,0.0,8 +verl-966,generation,998.2,0.0,8 +verl-966,generation,1003.2,0.0,8 +verl-966,generation,1008.3,0.0,8 +verl-966,generation,1013.3,0.0,8 +verl-966,generation,1018.3,0.0,8 +verl-966,generation,1023.3,0.0,8 +verl-966,generation,1028.3,0.0,8 +verl-966,generation,1033.3,0.0,8 +verl-966,generation,1038.3,0.0,8 +verl-966,generation,1043.3,0.0,8 +verl-966,generation,1048.3,0.0,8 +verl-966,generation,1053.3,0.0,8 +verl-966,generation,1058.3,0.0,8 +verl-966,generation,1063.3,0.0,8 +verl-966,generation,1068.3,0.0,8 +verl-966,generation,1073.3,0.0,8 +verl-966,generation,1078.3,0.0,8 +verl-966,generation,1083.3,0.0,8 +verl-966,generation,1088.3,0.0,8 +verl-966,generation,1093.3,0.0,8 +verl-966,generation,1098.3,0.0,8 +verl-966,generation,1103.3,0.0,8 +verl-966,generation,1108.3,0.0,8 +verl-966,generation,1113.3,0.0,8 +verl-966,generation,1118.3,0.0,8 +verl-966,generation,1123.3,0.0,8 +verl-966,generation,1128.3,0.0,8 +verl-966,generation,1133.3,0.0,8 +verl-966,generation,1138.3,0.0,8 +verl-966,generation,1143.3,0.0,8 +verl-966,generation,1148.3,0.0,8 +verl-966,generation,1153.3,0.0,8 +verl-966,generation,1158.3,0.0,8 +verl-966,generation,1163.3,0.0,8 +verl-966,generation,1168.3,0.0,8 +verl-966,generation,1173.3,0.0,8 +verl-966,generation,1178.3,0.0,8 +verl-966,generation,1183.3,0.0,8 +verl-966,generation,1188.3,0.0,8 +verl-966,generation,1193.3,0.0,8 +verl-966,generation,1198.3,0.0,8 +verl-966,generation,1203.3,0.0,8 +verl-966,generation,1208.3,0.0,8 +verl-966,generation,1213.3,0.0,8 +verl-966,generation,1218.3,0.0,8 +verl-966,generation,1223.3,0.0,8 +verl-966,generation,1228.3,0.0,8 +verl-966,generation,1233.3,0.0,8 +verl-966,generation,1238.3,0.0,8 +verl-966,generation,1243.3,0.0,8 +verl-966,generation,1248.3,0.0,8 +verl-966,generation,1253.3,0.0,8 +verl-966,generation,1258.3,0.0,8 +verl-966,generation,1263.3,0.0,8 +verl-966,generation,1268.3,0.0,8 +verl-966,generation,1273.3,0.0,8 +verl-966,generation,1278.3,0.0,8 +verl-966,generation,1283.3,0.0,8 +verl-966,generation,1288.3,0.0,8 +verl-966,generation,1293.3,0.0,8 +verl-966,generation,1298.3,0.0,8 +verl-966,generation,1303.3,0.0,8 +verl-966,generation,1308.3,0.0,8 +verl-966,generation,1313.3,0.0,8 +verl-966,generation,1318.3,0.0,8 +verl-966,generation,1323.3,0.0,8 +verl-966,generation,1328.3,0.0,8 +verl-966,generation,1333.3,0.0,8 +verl-966,generation,1338.3,99.8,8 +verl-966,generation,1343.3,0.0,8 +verl-966,generation,1348.3,0.0,8 +verl-966,generation,1353.3,0.0,8 +verl-966,generation,1358.3,0.0,8 +verl-966,generation,1363.3,0.0,8 +verl-966,generation,1368.3,0.0,8 +verl-966,generation,1373.3,0.0,8 +verl-966,generation,1378.3,0.0,8 +verl-966,generation,1383.3,0.0,8 +verl-966,generation,1388.3,0.0,8 +verl-966,generation,1393.3,0.0,8 +verl-966,generation,1398.3,0.0,8 +verl-966,generation,1403.3,0.0,8 +verl-966,generation,1408.3,0.0,8 +verl-966,generation,1413.3,0.0,8 +verl-966,generation,1418.3,0.0,8 +verl-966,generation,1423.3,0.0,8 +verl-966,generation,1428.3,0.0,8 +verl-966,generation,1433.3,0.0,8 +verl-966,generation,1438.3,0.0,8 +verl-966,generation,1443.3,0.0,8 +verl-966,generation,1448.3,0.0,8 +verl-966,generation,1453.3,0.0,8 +verl-966,generation,1458.3,0.0,8 +verl-966,generation,1463.3,0.0,8 +verl-966,generation,1468.3,0.0,8 +verl-966,generation,1473.3,0.0,8 +verl-966,generation,1478.3,0.0,8 +verl-966,generation,1483.3,0.0,8 +verl-966,generation,1488.3,0.0,8 +verl-966,generation,1493.3,0.0,8 +verl-966,generation,1498.3,0.0,8 +verl-966,generation,1503.3,0.0,8 +verl-966,generation,1508.3,0.0,8 +verl-966,generation,1513.3,0.0,8 +verl-966,generation,1518.3,0.0,8 +verl-966,generation,1523.3,0.0,8 +verl-966,generation,1528.3,0.0,8 +verl-966,generation,1533.3,0.0,8 +verl-966,generation,1538.3,0.0,8 +verl-966,generation,1543.3,0.0,8 +verl-966,generation,1548.3,0.0,8 +verl-966,generation,1553.3,0.0,8 +verl-966,generation,1558.3,0.0,8 +verl-966,generation,1563.3,0.0,8 +verl-966,generation,1568.3,0.0,8 +verl-966,generation,1573.3,0.0,8 +verl-966,generation,1578.3,0.0,8 +verl-966,generation,1583.3,0.0,8 +verl-966,generation,1588.3,0.0,8 +verl-966,generation,1593.3,0.0,8 +verl-966,generation,1598.3,0.0,8 +verl-966,generation,1603.3,0.0,8 +verl-966,generation,1608.3,0.0,8 +verl-966,generation,1613.3,0.0,8 +verl-966,generation,1618.3,0.0,8 +verl-966,generation,1623.3,0.0,8 +verl-966,generation,1628.3,0.0,8 +verl-966,generation,1633.3,0.0,8 +verl-966,generation,1638.3,0.0,8 +verl-966,generation,1643.3,22.9,8 +verl-966,generation,1648.3,60.2,8 +verl-966,generation,1653.3,44.4,8 +verl-966,generation,1658.3,0.0,8 +verl-966,generation,1663.3,27.0,8 +verl-966,generation,1668.3,41.0,8 +verl-966,generation,1673.3,44.4,8 +verl-966,generation,1678.3,11.8,8 +verl-966,generation,1683.3,0.0,8 +verl-966,generation,1688.3,0.0,8 +verl-966,generation,1693.3,50.0,8 +verl-966,generation,1698.3,0.0,8 +verl-966,generation,1703.3,10.2,8 +verl-966,generation,1708.3,56.1,8 +verl-966,generation,1713.3,99.8,8 +verl-966,generation,1718.3,99.9,8 +verl-966,generation,1723.3,65.9,8 +verl-966,generation,1728.3,0.5,8 +verl-966,generation,1733.3,0.0,8 +verl-966,generation,1738.4,0.0,8 +verl-966,generation,1743.4,0.0,8 +verl-966,generation,1748.4,0.0,8 +verl-966,generation,1753.4,0.0,8 +verl-966,generation,1758.4,0.0,8 +verl-966,generation,1763.4,0.0,8 +verl-966,generation,1768.4,51.8,8 +verl-966,generation,1773.4,42.8,8 +verl-966,generation,1778.4,34.8,8 +verl-966,generation,1783.4,66.4,8 +verl-966,generation,1788.4,89.8,8 +verl-966,generation,1793.4,59.4,8 +verl-966,generation,1798.4,67.4,8 +verl-966,generation,1803.4,39.8,8 +verl-966,generation,1808.4,99.8,8 +verl-966,generation,1813.4,99.9,8 +verl-966,generation,1818.4,99.9,8 +verl-966,generation,1823.4,87.6,8 +verl-966,generation,1828.4,98.8,8 +verl-966,generation,1833.4,99.9,8 +verl-966,generation,1838.4,76.6,8 +verl-966,generation,1843.4,89.1,8 +verl-966,generation,1848.4,91.6,8 +verl-966,generation,1853.4,83.9,8 +verl-966,generation,1858.4,82.4,8 +verl-966,generation,1863.4,99.9,8 +verl-966,generation,1868.4,0.0,8 +verl-966,generation,1873.4,0.0,8 +verl-966,generation,1878.4,0.0,8 +verl-966,generation,1883.4,0.0,8 +verl-966,generation,1888.4,0.0,8 +verl-966,generation,1893.4,0.0,8 +verl-966,generation,1898.4,0.0,8 +verl-966,generation,1903.4,2.2,8 +verl-966,generation,1908.4,86.6,8 +verl-966,generation,1913.4,93.4,8 +verl-966,generation,1918.4,100.0,8 +verl-966,generation,1923.4,100.0,8 +verl-966,generation,1928.4,99.5,8 +verl-966,generation,1933.4,99.8,8 +verl-966,generation,1938.4,99.9,8 +verl-966,generation,1943.4,99.6,8 +verl-966,generation,1948.4,0.8,8 +verl-966,generation,1953.4,0.0,8 +verl-966,generation,1958.4,0.0,8 +verl-966,generation,1963.4,0.0,8 +verl-966,generation,1968.4,0.0,8 +verl-966,generation,1973.4,0.0,8 +verl-966,generation,1978.4,0.0,8 +verl-966,generation,1983.4,0.0,8 +verl-966,generation,1988.4,0.0,8 +verl-966,generation,1993.4,0.0,8 +verl-966,generation,1998.4,0.0,8 +verl-966,generation,2003.4,0.0,8 +verl-966,generation,2008.4,0.0,8 +verl-966,generation,2013.4,0.0,8 +verl-966,generation,2018.4,80.1,8 +verl-966,generation,2023.4,72.8,8 +verl-966,generation,2028.4,87.4,8 +verl-966,generation,2033.4,87.9,8 +verl-966,generation,2038.4,99.0,8 +verl-966,generation,2043.4,99.8,8 +verl-966,generation,2048.4,99.5,8 +verl-966,generation,2053.4,0.6,8 +verl-966,generation,2058.4,0.0,8 +verl-966,generation,2063.4,0.0,8 +verl-966,generation,2068.4,0.0,8 +verl-966,generation,2073.4,0.0,8 +verl-966,generation,2078.4,0.0,8 +verl-966,generation,2083.4,0.0,8 +verl-966,generation,2088.4,0.0,8 +verl-966,generation,2093.4,0.0,8 +verl-966,generation,2098.4,0.0,8 +verl-966,generation,2103.4,0.0,8 +verl-966,generation,2108.4,0.0,8 +verl-966,generation,2113.4,0.0,8 +verl-966,generation,2118.4,0.0,8 +verl-966,generation,2123.4,53.8,8 +verl-966,generation,2128.4,99.9,8 +verl-966,generation,2133.4,99.8,8 +verl-966,generation,2138.4,100.0,8 +verl-966,generation,2143.4,99.9,8 +verl-966,generation,2148.4,99.8,8 +verl-966,generation,2153.4,99.8,8 +verl-966,generation,2158.4,99.4,8 +verl-966,generation,2163.4,86.4,8 +verl-966,generation,2168.4,87.5,8 +verl-966,generation,2173.4,91.5,8 +verl-966,generation,2178.4,0.2,8 +verl-966,generation,2183.4,55.8,8 +verl-966,generation,2188.4,0.0,8 +verl-966,generation,2193.4,0.0,8 +verl-966,generation,2198.4,0.0,8 +verl-966,generation,2203.4,0.0,8 +verl-966,generation,2208.4,0.0,8 +verl-966,generation,2213.4,0.0,8 +verl-966,generation,2218.4,0.0,8 +verl-966,generation,2223.4,0.0,8 +verl-966,generation,2228.4,0.0,8 +verl-966,generation,2233.4,24.0,8 +verl-966,generation,2238.4,100.0,8 +verl-966,generation,2243.4,100.0,8 +verl-966,generation,2248.4,100.0,8 +verl-966,generation,2253.4,100.0,8 +verl-966,generation,2258.4,100.0,8 +verl-966,generation,2263.4,97.0,8 +verl-966,generation,2268.4,87.2,8 +verl-966,generation,2273.4,0.0,8 +verl-966,generation,2278.4,0.0,8 +verl-966,generation,2283.4,0.0,8 +verl-966,generation,2288.4,0.0,8 +verl-966,generation,2293.4,0.0,8 +verl-966,generation,2298.4,0.0,8 +verl-966,generation,2303.4,0.0,8 +verl-966,generation,2308.4,0.0,8 +verl-966,generation,2313.4,0.0,8 +verl-966,generation,2318.4,0.0,8 +verl-966,generation,2323.4,0.0,8 +verl-966,generation,2328.4,0.0,8 +verl-966,generation,2333.4,87.5,8 +verl-966,generation,2338.4,77.4,8 +verl-966,generation,2343.4,100.0,8 +verl-966,generation,2348.4,100.0,8 +verl-966,generation,2353.4,100.0,8 +verl-966,generation,2358.4,99.8,8 +verl-966,generation,2363.4,81.6,8 +verl-966,generation,2368.4,89.2,8 +verl-966,generation,2373.4,0.0,8 +verl-966,generation,2378.4,0.0,8 +verl-966,generation,2383.4,0.0,8 +verl-966,generation,2388.4,0.0,8 +verl-966,generation,2393.4,0.0,8 +verl-966,generation,2398.4,0.0,8 +verl-966,generation,2403.4,0.0,8 +verl-966,generation,2408.5,0.0,8 +verl-966,generation,2413.5,0.0,8 +verl-966,generation,2418.5,0.0,8 +verl-966,generation,2423.5,0.0,8 +verl-966,generation,2428.5,74.2,8 +verl-966,generation,2433.5,99.9,8 +verl-966,generation,2438.5,99.8,8 +verl-966,generation,2443.5,100.0,8 +verl-966,generation,2448.5,99.8,8 +verl-966,generation,2453.5,92.1,8 +verl-966,generation,2458.5,0.0,8 +verl-966,generation,2463.5,0.0,8 +verl-966,generation,2468.5,0.0,8 +verl-966,generation,2473.5,0.0,8 +verl-966,generation,2478.5,0.0,8 +verl-966,generation,2483.5,0.0,8 +verl-966,generation,2488.5,0.0,8 +verl-966,generation,2493.5,0.0,8 +verl-966,generation,2498.5,0.0,8 +verl-966,generation,2503.5,0.0,8 +verl-966,generation,2508.5,0.0,8 +verl-966,generation,2513.5,0.0,8 +verl-966,generation,2518.5,0.0,8 +verl-966,generation,2523.5,72.8,8 +verl-966,generation,2528.5,99.8,8 +verl-966,generation,2533.5,99.8,8 +verl-966,generation,2538.5,100.0,8 +verl-966,generation,2543.5,99.0,8 +verl-966,generation,2548.5,82.0,8 +verl-966,generation,2553.5,93.4,8 +verl-966,generation,2558.5,52.4,8 +verl-966,generation,2563.5,0.0,8 +verl-966,generation,2568.5,0.0,8 +verl-966,generation,2573.5,0.0,8 +verl-966,generation,2578.5,0.0,8 +verl-966,generation,2583.5,0.0,8 +verl-966,generation,2588.5,0.0,8 +verl-966,generation,2593.5,0.0,8 +verl-966,generation,2598.5,0.0,8 +verl-966,generation,2603.5,0.0,8 +verl-966,generation,2608.5,0.0,8 +verl-966,generation,2613.5,0.0,8 +verl-966,generation,2618.5,0.0,8 +verl-966,generation,2623.5,0.0,8 +verl-966,generation,2628.5,53.8,8 +verl-966,generation,2633.5,99.6,8 +verl-966,generation,2638.5,99.9,8 +verl-966,generation,2643.5,100.0,8 +verl-966,generation,2648.5,99.8,8 +verl-966,generation,2653.5,94.0,8 +verl-966,generation,2658.5,0.0,8 +verl-966,generation,2663.5,95.9,8 +verl-966,generation,2668.5,0.0,8 +verl-966,generation,2673.5,0.0,8 +verl-966,generation,2678.5,0.0,8 +verl-966,generation,2683.5,0.0,8 +verl-966,generation,2688.5,0.0,8 +verl-966,generation,2693.5,0.0,8 +verl-966,generation,2698.5,0.0,8 +verl-966,generation,2703.5,0.0,8 +verl-966,generation,2708.5,0.0,8 +verl-966,generation,2713.5,0.0,8 +verl-966,generation,2718.5,0.0,8 +verl-966,generation,2723.5,0.0,8 +verl-966,generation,2728.5,0.0,8 +verl-966,generation,2733.5,0.0,8 +verl-966,generation,2738.5,0.0,8 +verl-966,generation,2743.5,0.0,8 +verl-966,generation,2748.5,0.0,8 +verl-966,generation,2753.5,0.0,8 +verl-966,generation,2758.5,0.0,8 +verl-966,generation,2763.5,0.0,8 +verl-966,generation,2768.5,0.0,8 +verl-966,generation,2773.5,0.0,8 +verl-966,generation,2778.5,0.0,8 +verl-966,generation,2783.5,0.0,8 +verl-966,generation,2788.5,0.0,8 +verl-966,generation,2793.5,0.0,8 +verl-966,generation,2798.5,0.0,8 +verl-966,generation,2803.5,0.0,8 +verl-966,generation,2808.5,0.0,8 +verl-966,generation,2813.5,0.0,8 +verl-966,generation,2818.5,0.0,8 +verl-966,generation,2823.5,0.0,8 +verl-966,generation,2828.5,0.0,8 +verl-966,generation,2833.5,0.0,8 +verl-966,generation,2838.5,0.0,8 +verl-966,generation,2843.5,0.0,8 +verl-966,generation,2848.5,0.0,8 +verl-966,generation,2853.5,0.0,8 +verl-966,generation,2858.5,0.0,8 +verl-966,generation,2863.5,0.0,8 +verl-966,generation,2868.5,0.0,8 +verl-966,generation,2873.5,0.0,8 +verl-966,generation,2878.5,0.0,8 +verl-966,generation,2883.5,0.0,8 +verl-966,generation,2888.5,0.0,8 +verl-966,generation,2893.5,0.0,8 +verl-966,generation,2898.5,0.0,8 +verl-966,generation,2903.5,0.0,8 +verl-966,generation,2908.5,23.5,8 +verl-966,generation,2913.5,90.5,8 +verl-966,generation,2918.5,99.8,8 +verl-966,generation,2923.5,97.5,8 +verl-966,generation,2928.5,100.0,8 +verl-966,generation,2933.5,100.0,8 +verl-966,generation,2938.5,100.0,8 +verl-966,generation,2943.5,95.2,8 +verl-966,generation,2948.5,92.5,8 +verl-966,generation,2953.5,99.8,8 +verl-966,generation,2958.5,99.9,8 +verl-966,generation,2963.5,0.2,8 +verl-966,generation,2968.5,0.0,8 +verl-966,generation,2973.5,0.0,8 +verl-966,generation,2978.5,0.0,8 +verl-966,generation,2983.5,0.0,8 +verl-966,generation,2988.5,0.0,8 +verl-966,generation,2993.5,0.0,8 +verl-966,generation,2998.5,0.0,8 +verl-966,generation,3003.5,0.0,8 +verl-966,generation,3008.5,0.0,8 +verl-966,generation,3013.5,0.0,8 +verl-966,generation,3018.5,0.0,8 +verl-966,generation,3023.5,0.0,8 +verl-966,generation,3028.5,0.0,8 +verl-966,generation,3033.5,10.4,8 +verl-966,generation,3038.5,88.0,8 +verl-966,generation,3043.5,76.5,8 +verl-966,generation,3048.5,80.0,8 +verl-966,generation,3053.5,93.2,8 +verl-966,generation,3058.5,0.0,8 +verl-966,generation,3063.5,72.2,8 +verl-966,generation,3068.5,0.0,8 +verl-966,generation,3073.5,0.0,8 +verl-966,generation,3078.5,0.0,8 +verl-966,generation,3083.5,0.0,8 +verl-966,generation,3088.5,0.0,8 +verl-966,generation,3093.5,0.0,8 +verl-966,generation,3098.5,0.0,8 +verl-966,generation,3103.5,0.0,8 +verl-966,generation,3108.5,0.0,8 +verl-966,generation,3113.5,0.0,8 +verl-966,generation,3118.5,0.0,8 +verl-966,generation,3123.5,0.0,8 +verl-966,generation,3128.5,0.0,8 +verl-966,generation,3133.5,0.0,8 +verl-966,generation,3138.5,78.6,8 +verl-966,generation,3143.5,99.5,8 +verl-966,generation,3148.5,99.8,8 +verl-966,generation,3153.5,99.8,8 +verl-966,generation,3158.5,83.4,8 +verl-966,generation,3163.5,97.8,8 +verl-966,generation,3168.5,99.8,8 +verl-966,generation,3173.5,81.9,8 +verl-966,generation,3178.5,0.0,8 +verl-966,generation,3183.6,0.2,8 +verl-966,generation,3188.6,0.0,8 +verl-966,generation,3193.6,0.0,8 +verl-966,generation,3198.6,0.0,8 +verl-966,generation,3203.6,0.0,8 +verl-966,generation,3208.6,0.0,8 +verl-966,generation,3213.6,0.0,8 +verl-966,generation,3218.6,0.0,8 +verl-966,generation,3223.6,0.0,8 +verl-966,generation,3228.6,0.0,8 +verl-966,generation,3233.6,0.0,8 +verl-966,generation,3238.6,0.0,8 +verl-966,generation,3243.6,87.0,8 +verl-966,generation,3248.6,77.1,8 +verl-966,generation,3253.6,99.4,8 +verl-966,generation,3258.6,99.0,8 +verl-966,generation,3263.6,86.5,8 +verl-966,generation,3268.6,100.0,8 +verl-966,generation,3273.6,89.2,8 +verl-966,generation,3278.6,98.6,8 +verl-966,generation,3283.6,99.6,8 +verl-966,generation,3288.6,87.5,8 +verl-966,generation,3293.6,0.5,8 +verl-966,generation,3298.6,0.0,8 +verl-966,generation,3303.6,0.0,8 +verl-966,generation,3308.6,0.0,8 +verl-966,generation,3313.6,0.0,8 +verl-966,generation,3318.6,0.0,8 +verl-966,generation,3323.6,0.0,8 +verl-966,generation,3328.6,0.0,8 +verl-966,generation,3333.6,0.0,8 +verl-966,generation,3338.6,0.0,8 +verl-966,generation,3343.6,0.0,8 +verl-966,generation,3348.6,0.0,8 +verl-966,generation,3353.6,42.4,8 +verl-966,generation,3358.6,99.2,8 +verl-966,generation,3363.6,99.5,8 +verl-966,generation,3368.6,98.1,8 +verl-966,generation,3373.6,80.4,8 +verl-966,generation,3378.6,99.9,8 +verl-966,generation,3383.6,99.6,8 +verl-966,generation,3388.6,99.9,8 +verl-966,generation,3393.6,99.9,8 +verl-966,generation,3398.6,99.8,8 +verl-966,generation,3403.6,0.0,8 +verl-966,generation,3408.6,0.0,8 +verl-966,generation,3413.6,0.0,8 +verl-966,generation,3418.6,0.0,8 +verl-966,generation,3423.6,0.0,8 +verl-966,generation,3428.6,0.0,8 +verl-966,generation,3433.6,0.0,8 +verl-966,generation,3438.6,0.0,8 +verl-966,generation,3443.6,0.0,8 +verl-966,generation,3448.6,0.0,8 +verl-966,generation,3453.6,0.0,8 +verl-966,generation,3458.6,0.0,8 +verl-966,generation,3463.6,0.0,8 +verl-966,generation,3468.6,87.5,8 +verl-966,generation,3473.6,100.0,8 +verl-966,generation,3478.6,99.6,8 +verl-966,generation,3483.6,100.0,8 +verl-966,generation,3488.6,98.9,8 +verl-966,generation,3493.6,87.0,8 +verl-966,generation,3498.6,98.4,8 +verl-966,generation,3503.6,99.9,8 +verl-966,generation,3508.6,99.6,8 +verl-966,generation,3513.6,0.2,8 +verl-966,generation,3518.6,0.0,8 +verl-966,generation,3523.6,0.0,8 +verl-966,generation,3528.6,0.0,8 +verl-966,generation,3533.6,0.0,8 +verl-966,generation,3538.6,0.0,8 +verl-966,generation,3543.6,0.0,8 +verl-966,generation,3548.6,0.0,8 +verl-966,generation,3553.6,0.0,8 +verl-966,generation,3558.6,0.0,8 +verl-966,generation,3563.6,0.0,8 +verl-966,generation,3568.6,0.0,8 +verl-966,generation,3573.6,0.0,8 +verl-966,generation,3578.6,0.0,8 +verl-966,generation,3583.6,74.1,8 +verl-966,generation,3588.6,82.0,8 +verl-966,generation,3593.6,86.4,8 +verl-966,generation,3598.6,86.5,8 +verl-966,generation,3603.6,100.0,8 +verl-966,generation,3608.6,96.5,8 +verl-966,generation,3613.6,99.9,8 +verl-966,generation,3618.6,99.8,8 +verl-966,generation,3623.6,71.5,8 +verl-966,generation,3628.6,85.4,8 +verl-966,generation,3633.6,0.0,8 +verl-966,generation,3638.6,0.0,8 +verl-966,generation,3643.6,0.0,8 +verl-966,generation,3648.6,0.0,8 +verl-966,generation,3653.6,0.0,8 +verl-966,generation,3658.6,0.0,8 +verl-966,generation,3663.6,0.0,8 +verl-966,generation,3668.6,0.0,8 +verl-966,generation,3673.6,0.0,8 +verl-966,generation,3678.6,0.0,8 +verl-966,generation,3683.6,0.0,8 +verl-966,generation,3688.6,27.5,8 +verl-966,generation,3693.6,100.0,8 +verl-966,generation,3698.6,100.0,8 +verl-966,generation,3703.6,90.6,8 +verl-966,generation,3708.6,96.9,8 +verl-966,generation,3713.6,96.5,8 +verl-966,generation,3718.6,99.8,8 +verl-966,generation,3723.6,99.9,8 +verl-966,generation,3728.6,99.8,8 +verl-966,generation,3733.6,0.0,8 +verl-966,generation,3738.6,0.0,8 +verl-966,generation,3743.6,0.0,8 +verl-966,generation,3748.6,0.0,8 +verl-966,generation,3753.6,0.0,8 +verl-966,generation,3758.6,0.0,8 +verl-966,generation,3763.6,0.0,8 +verl-966,generation,3768.6,0.0,8 +verl-966,generation,3773.6,0.0,8 +verl-966,generation,3778.6,0.0,8 +verl-966,generation,3783.6,0.0,8 +verl-966,generation,3788.6,0.0,8 +verl-966,generation,3793.6,0.0,8 +verl-966,generation,3798.6,87.5,8 +verl-966,generation,3803.6,84.5,8 +verl-966,generation,3808.6,99.9,8 +verl-966,generation,3813.6,96.0,8 +verl-966,generation,3818.6,86.6,8 +verl-966,generation,3823.6,94.1,8 +verl-966,generation,3828.6,98.4,8 +verl-966,generation,3833.6,99.9,8 +verl-966,generation,3838.6,99.9,8 +verl-966,generation,3843.6,0.6,8 +verl-966,generation,3848.6,0.0,8 +verl-966,generation,3853.6,0.0,8 +verl-966,generation,3858.6,0.0,8 +verl-966,generation,3863.6,0.0,8 +verl-966,generation,3868.6,0.0,8 +verl-966,generation,3873.6,0.0,8 +verl-966,generation,3878.6,0.0,8 +verl-966,generation,3883.6,0.0,8 +verl-966,generation,3888.6,0.0,8 +verl-966,generation,3893.6,0.0,8 +verl-966,generation,3898.6,0.0,8 +verl-966,generation,3903.6,0.0,8 +verl-966,generation,3908.6,0.0,8 +verl-966,generation,3913.6,0.0,8 +verl-966,generation,3918.6,0.0,8 +verl-966,generation,3923.6,0.0,8 +verl-966,generation,3928.6,0.0,8 +verl-966,generation,3933.6,0.0,8 +verl-966,generation,3938.6,0.8,8 +verl-966,generation,3943.6,72.8,8 +verl-966,generation,3948.6,87.5,8 +verl-966,generation,3953.7,99.8,8 +verl-966,generation,3958.7,87.5,8 +verl-966,generation,3963.7,97.2,8 +verl-966,generation,3968.7,86.5,8 +verl-966,generation,3973.7,99.9,8 +verl-966,generation,3978.7,99.8,8 +verl-966,generation,3983.7,99.8,8 +verl-966,generation,3988.7,98.5,8 +verl-966,generation,3993.7,99.9,8 +verl-966,generation,3998.7,0.0,8 +verl-966,generation,4003.7,0.0,8 +verl-966,generation,4008.7,0.0,8 +verl-966,generation,4013.7,0.0,8 +verl-966,generation,4018.7,0.0,8 +verl-966,generation,4023.7,0.0,8 +verl-966,generation,4028.7,0.0,8 +verl-966,generation,4033.7,0.0,8 +verl-966,generation,4038.7,0.0,8 +verl-966,generation,4043.7,0.0,8 +verl-966,generation,4048.7,0.0,8 +verl-966,generation,4053.7,0.0,8 +verl-966,generation,4058.7,0.0,8 +verl-966,generation,4063.7,0.0,8 +verl-966,generation,4068.7,0.0,8 +verl-966,generation,4073.7,0.0,8 +verl-966,generation,4078.7,0.0,8 +verl-966,generation,4083.7,0.0,8 +verl-966,generation,4088.7,0.0,8 +verl-966,generation,4093.7,0.0,8 +verl-966,generation,4098.7,0.0,8 +verl-966,generation,4103.7,0.0,8 +verl-966,generation,4108.7,0.0,8 +verl-966,generation,4113.7,0.0,8 +verl-966,generation,4118.7,0.0,8 +verl-966,generation,4123.7,0.0,8 +verl-966,generation,4128.7,0.0,8 +verl-966,generation,4133.7,0.0,8 +verl-966,generation,4138.7,0.0,8 +verl-966,generation,4143.7,0.0,8 +verl-966,generation,4148.7,0.0,8 +verl-966,generation,4153.7,0.0,8 +verl-966,generation,4158.7,0.0,8 +verl-966,generation,4163.7,0.0,8 +verl-966,generation,4168.7,0.0,8 +verl-966,generation,4173.7,0.0,8 +verl-966,generation,4178.7,0.0,8 +verl-966,generation,4183.7,0.0,8 +verl-966,generation,4188.7,0.0,8 +verl-966,generation,4193.7,0.0,8 +verl-966,generation,4198.7,0.0,8 +verl-966,generation,4203.7,0.0,8 +verl-966,generation,4208.7,14.2,8 +verl-966,generation,4213.7,0.0,8 +verl-966,generation,4218.7,0.0,8 +verl-966,generation,4223.7,0.0,8 +verl-966,generation,4228.7,0.5,8 +verl-966,generation,4233.7,2.0,8 +verl-966,generation,4238.7,1.6,8 +verl-966,generation,4243.7,0.1,8 +verl-966,generation,4248.7,0.9,8 +verl-966,generation,4253.7,1.0,8 +verl-966,generation,4258.7,0.6,8 +verl-966,generation,4263.7,0.0,8 +verl-966,generation,4268.7,0.0,8 +verl-966,learner,8.1,0.0,7 +verl-966,learner,13.1,0.0,7 +verl-966,learner,18.1,0.0,7 +verl-966,learner,23.1,0.0,7 +verl-966,learner,28.1,0.0,7 +verl-966,learner,33.1,0.0,7 +verl-966,learner,38.1,0.0,7 +verl-966,learner,43.1,0.0,7 +verl-966,learner,48.1,0.0,7 +verl-966,learner,53.1,0.0,7 +verl-966,learner,58.1,0.0,7 +verl-966,learner,63.1,0.0,7 +verl-966,learner,68.1,0.0,7 +verl-966,learner,73.1,0.0,7 +verl-966,learner,78.1,0.0,7 +verl-966,learner,83.1,0.0,7 +verl-966,learner,88.1,0.0,7 +verl-966,learner,93.1,0.0,7 +verl-966,learner,98.1,0.0,7 +verl-966,learner,103.1,0.0,7 +verl-966,learner,108.1,0.0,7 +verl-966,learner,113.1,0.0,7 +verl-966,learner,118.1,0.0,7 +verl-966,learner,123.1,0.0,7 +verl-966,learner,128.1,0.0,7 +verl-966,learner,133.1,0.0,7 +verl-966,learner,138.1,0.0,7 +verl-966,learner,143.1,0.0,7 +verl-966,learner,148.1,0.0,7 +verl-966,learner,153.1,0.0,7 +verl-966,learner,158.2,0.0,7 +verl-966,learner,163.2,0.0,7 +verl-966,learner,168.2,0.0,7 +verl-966,learner,173.2,0.0,7 +verl-966,learner,178.2,0.0,7 +verl-966,learner,183.2,0.0,7 +verl-966,learner,188.2,0.0,7 +verl-966,learner,193.2,0.0,7 +verl-966,learner,198.2,0.0,7 +verl-966,learner,203.2,0.0,7 +verl-966,learner,208.2,0.0,7 +verl-966,learner,213.2,0.0,7 +verl-966,learner,218.2,0.0,7 +verl-966,learner,223.2,0.0,7 +verl-966,learner,228.2,0.0,7 +verl-966,learner,233.2,0.0,7 +verl-966,learner,238.2,0.0,7 +verl-966,learner,243.2,0.0,7 +verl-966,learner,248.2,0.0,7 +verl-966,learner,253.2,0.0,7 +verl-966,learner,258.2,0.0,7 +verl-966,learner,263.2,0.0,7 +verl-966,learner,268.2,0.0,7 +verl-966,learner,273.2,0.0,7 +verl-966,learner,278.2,0.0,7 +verl-966,learner,283.2,0.0,7 +verl-966,learner,288.2,0.0,7 +verl-966,learner,293.2,0.0,7 +verl-966,learner,298.2,0.0,7 +verl-966,learner,303.2,0.0,7 +verl-966,learner,308.2,0.0,7 +verl-966,learner,313.2,0.0,7 +verl-966,learner,318.2,0.0,7 +verl-966,learner,323.2,0.0,7 +verl-966,learner,328.2,0.0,7 +verl-966,learner,333.2,0.0,7 +verl-966,learner,338.2,0.0,7 +verl-966,learner,343.2,0.0,7 +verl-966,learner,348.2,0.0,7 +verl-966,learner,353.2,0.0,7 +verl-966,learner,358.2,0.0,7 +verl-966,learner,363.2,0.0,7 +verl-966,learner,368.2,0.0,7 +verl-966,learner,373.2,0.0,7 +verl-966,learner,378.2,0.0,7 +verl-966,learner,383.2,0.0,7 +verl-966,learner,388.2,0.0,7 +verl-966,learner,393.2,0.0,7 +verl-966,learner,398.2,0.0,7 +verl-966,learner,403.2,0.0,7 +verl-966,learner,408.2,0.0,7 +verl-966,learner,413.2,0.0,7 +verl-966,learner,418.2,0.0,7 +verl-966,learner,423.2,0.0,7 +verl-966,learner,428.2,0.0,7 +verl-966,learner,433.2,0.0,7 +verl-966,learner,438.2,0.0,7 +verl-966,learner,443.2,0.0,7 +verl-966,learner,448.2,0.0,7 +verl-966,learner,453.2,0.6,7 +verl-966,learner,458.2,1.1,7 +verl-966,learner,463.2,1.0,7 +verl-966,learner,468.2,1.0,7 +verl-966,learner,473.2,1.3,7 +verl-966,learner,478.2,0.0,7 +verl-966,learner,483.2,0.0,7 +verl-966,learner,488.2,0.0,7 +verl-966,learner,493.2,0.0,7 +verl-966,learner,498.2,0.0,7 +verl-966,learner,503.2,0.0,7 +verl-966,learner,508.2,0.0,7 +verl-966,learner,513.2,0.0,7 +verl-966,learner,518.2,0.0,7 +verl-966,learner,523.2,0.0,7 +verl-966,learner,528.2,0.0,7 +verl-966,learner,533.2,0.0,7 +verl-966,learner,538.2,0.0,7 +verl-966,learner,543.2,0.0,7 +verl-966,learner,548.2,0.0,7 +verl-966,learner,553.2,0.0,7 +verl-966,learner,558.2,0.0,7 +verl-966,learner,563.2,0.0,7 +verl-966,learner,568.2,0.0,7 +verl-966,learner,573.2,0.0,7 +verl-966,learner,578.2,0.0,7 +verl-966,learner,583.2,0.0,7 +verl-966,learner,588.2,0.0,7 +verl-966,learner,593.2,0.0,7 +verl-966,learner,598.2,0.0,7 +verl-966,learner,603.2,0.0,7 +verl-966,learner,608.2,0.0,7 +verl-966,learner,613.2,0.0,7 +verl-966,learner,618.2,0.0,7 +verl-966,learner,623.2,0.0,7 +verl-966,learner,628.2,0.0,7 +verl-966,learner,633.2,0.0,7 +verl-966,learner,638.2,0.0,7 +verl-966,learner,643.2,0.0,7 +verl-966,learner,648.2,0.0,7 +verl-966,learner,653.2,0.0,7 +verl-966,learner,658.2,0.0,7 +verl-966,learner,663.2,0.0,7 +verl-966,learner,668.2,0.0,7 +verl-966,learner,673.2,0.0,7 +verl-966,learner,678.2,0.0,7 +verl-966,learner,683.2,0.0,7 +verl-966,learner,688.2,0.0,7 +verl-966,learner,693.2,0.0,7 +verl-966,learner,698.2,0.0,7 +verl-966,learner,703.2,0.0,7 +verl-966,learner,708.2,0.0,7 +verl-966,learner,713.2,0.0,7 +verl-966,learner,718.2,0.0,7 +verl-966,learner,723.2,0.0,7 +verl-966,learner,728.2,0.0,7 +verl-966,learner,733.2,0.0,7 +verl-966,learner,738.2,0.0,7 +verl-966,learner,743.2,0.0,7 +verl-966,learner,748.2,0.0,7 +verl-966,learner,753.2,0.0,7 +verl-966,learner,758.2,0.0,7 +verl-966,learner,763.2,0.0,7 +verl-966,learner,768.2,0.0,7 +verl-966,learner,773.2,0.0,7 +verl-966,learner,778.2,0.0,7 +verl-966,learner,783.2,0.0,7 +verl-966,learner,788.2,0.0,7 +verl-966,learner,793.2,0.0,7 +verl-966,learner,798.2,0.0,7 +verl-966,learner,803.2,0.0,7 +verl-966,learner,808.2,0.0,7 +verl-966,learner,813.2,0.0,7 +verl-966,learner,818.2,0.0,7 +verl-966,learner,823.2,0.0,7 +verl-966,learner,828.2,0.0,7 +verl-966,learner,833.2,0.0,7 +verl-966,learner,838.2,0.0,7 +verl-966,learner,843.2,0.0,7 +verl-966,learner,848.2,0.0,7 +verl-966,learner,853.2,0.0,7 +verl-966,learner,858.2,0.0,7 +verl-966,learner,863.2,0.0,7 +verl-966,learner,868.2,0.0,7 +verl-966,learner,873.2,0.0,7 +verl-966,learner,878.2,0.0,7 +verl-966,learner,883.2,0.0,7 +verl-966,learner,888.2,0.0,7 +verl-966,learner,893.2,0.0,7 +verl-966,learner,898.2,0.0,7 +verl-966,learner,903.2,0.0,7 +verl-966,learner,908.2,0.0,7 +verl-966,learner,913.2,0.0,7 +verl-966,learner,918.2,0.0,7 +verl-966,learner,923.2,0.0,7 +verl-966,learner,928.2,0.0,7 +verl-966,learner,933.2,0.0,7 +verl-966,learner,938.2,0.0,7 +verl-966,learner,943.2,0.0,7 +verl-966,learner,948.2,0.0,7 +verl-966,learner,953.2,0.0,7 +verl-966,learner,958.2,0.0,7 +verl-966,learner,963.2,0.0,7 +verl-966,learner,968.2,0.0,7 +verl-966,learner,973.2,0.0,7 +verl-966,learner,978.2,0.0,7 +verl-966,learner,983.2,0.0,7 +verl-966,learner,988.2,0.0,7 +verl-966,learner,993.2,0.0,7 +verl-966,learner,998.2,0.0,7 +verl-966,learner,1003.2,0.0,7 +verl-966,learner,1008.2,0.0,7 +verl-966,learner,1013.2,0.0,7 +verl-966,learner,1018.2,0.0,7 +verl-966,learner,1023.2,0.0,7 +verl-966,learner,1028.2,0.0,7 +verl-966,learner,1033.2,0.6,7 +verl-966,learner,1038.2,9.6,7 +verl-966,learner,1043.2,5.6,7 +verl-966,learner,1048.2,3.6,7 +verl-966,learner,1053.2,4.7,7 +verl-966,learner,1058.2,0.0,7 +verl-966,learner,1063.2,0.0,7 +verl-966,learner,1068.2,0.0,7 +verl-966,learner,1073.2,0.0,7 +verl-966,learner,1078.2,0.0,7 +verl-966,learner,1083.2,0.0,7 +verl-966,learner,1088.2,6.9,7 +verl-966,learner,1093.2,0.0,7 +verl-966,learner,1098.2,0.0,7 +verl-966,learner,1103.2,0.0,7 +verl-966,learner,1108.2,0.0,7 +verl-966,learner,1113.2,0.0,7 +verl-966,learner,1118.2,0.0,7 +verl-966,learner,1123.2,0.0,7 +verl-966,learner,1128.2,0.0,7 +verl-966,learner,1133.2,0.0,7 +verl-966,learner,1138.2,0.0,7 +verl-966,learner,1143.2,0.0,7 +verl-966,learner,1148.2,0.0,7 +verl-966,learner,1153.2,0.0,7 +verl-966,learner,1158.2,0.0,7 +verl-966,learner,1163.2,0.0,7 +verl-966,learner,1168.2,0.0,7 +verl-966,learner,1173.2,0.0,7 +verl-966,learner,1178.2,0.0,7 +verl-966,learner,1183.2,17.0,7 +verl-966,learner,1188.2,0.7,7 +verl-966,learner,1193.2,0.0,7 +verl-966,learner,1198.2,0.0,7 +verl-966,learner,1203.2,0.0,7 +verl-966,learner,1208.2,0.0,7 +verl-966,learner,1213.2,0.0,7 +verl-966,learner,1218.2,0.0,7 +verl-966,learner,1223.2,0.0,7 +verl-966,learner,1228.2,8.1,7 +verl-966,learner,1233.2,0.0,7 +verl-966,learner,1238.2,0.0,7 +verl-966,learner,1243.2,4.7,7 +verl-966,learner,1248.2,9.0,7 +verl-966,learner,1253.2,5.9,7 +verl-966,learner,1258.2,21.0,7 +verl-966,learner,1263.2,34.0,7 +verl-966,learner,1268.2,32.9,7 +verl-966,learner,1273.2,27.0,7 +verl-966,learner,1278.2,5.0,7 +verl-966,learner,1283.2,17.4,7 +verl-966,learner,1288.2,11.7,7 +verl-966,learner,1293.2,16.3,7 +verl-966,learner,1298.2,42.9,7 +verl-966,learner,1303.2,0.0,7 +verl-966,learner,1308.2,0.0,7 +verl-966,learner,1313.2,0.0,7 +verl-966,learner,1318.2,0.0,7 +verl-966,learner,1323.2,0.0,7 +verl-966,learner,1328.2,12.7,7 +verl-966,learner,1333.2,0.0,7 +verl-966,learner,1338.2,36.3,7 +verl-966,learner,1343.2,0.0,7 +verl-966,learner,1348.2,0.0,7 +verl-966,learner,1353.2,85.7,7 +verl-966,learner,1358.2,85.7,7 +verl-966,learner,1363.2,85.7,7 +verl-966,learner,1368.2,85.7,7 +verl-966,learner,1373.2,85.7,7 +verl-966,learner,1378.2,85.7,7 +verl-966,learner,1383.2,85.7,7 +verl-966,learner,1388.2,85.7,7 +verl-966,learner,1393.2,85.7,7 +verl-966,learner,1398.2,85.7,7 +verl-966,learner,1403.2,85.7,7 +verl-966,learner,1408.2,85.7,7 +verl-966,learner,1413.2,85.7,7 +verl-966,learner,1418.2,85.7,7 +verl-966,learner,1423.2,85.7,7 +verl-966,learner,1428.2,85.7,7 +verl-966,learner,1433.2,85.7,7 +verl-966,learner,1438.2,85.7,7 +verl-966,learner,1443.2,85.7,7 +verl-966,learner,1448.2,85.7,7 +verl-966,learner,1453.2,85.7,7 +verl-966,learner,1458.2,85.7,7 +verl-966,learner,1463.2,85.7,7 +verl-966,learner,1468.2,85.7,7 +verl-966,learner,1473.2,85.7,7 +verl-966,learner,1478.2,85.7,7 +verl-966,learner,1483.2,85.7,7 +verl-966,learner,1488.2,85.7,7 +verl-966,learner,1493.2,85.7,7 +verl-966,learner,1498.2,85.7,7 +verl-966,learner,1503.2,85.7,7 +verl-966,learner,1508.2,85.7,7 +verl-966,learner,1513.2,85.7,7 +verl-966,learner,1518.2,85.7,7 +verl-966,learner,1523.2,85.7,7 +verl-966,learner,1528.2,85.7,7 +verl-966,learner,1533.2,85.7,7 +verl-966,learner,1538.2,85.7,7 +verl-966,learner,1543.2,85.7,7 +verl-966,learner,1548.2,85.7,7 +verl-966,learner,1553.2,28.6,7 +verl-966,learner,1558.2,0.0,7 +verl-966,learner,1563.2,85.7,7 +verl-966,learner,1568.2,85.7,7 +verl-966,learner,1573.2,85.7,7 +verl-966,learner,1578.2,85.7,7 +verl-966,learner,1583.2,85.7,7 +verl-966,learner,1588.2,85.7,7 +verl-966,learner,1593.2,93.1,7 +verl-966,learner,1598.2,93.7,7 +verl-966,learner,1603.2,85.7,7 +verl-966,learner,1608.2,85.7,7 +verl-966,learner,1613.2,92.6,7 +verl-966,learner,1618.2,85.7,7 +verl-966,learner,1623.2,85.7,7 +verl-966,learner,1628.2,85.7,7 +verl-966,learner,1633.2,93.1,7 +verl-966,learner,1638.2,85.7,7 +verl-966,learner,1643.2,100.0,7 +verl-966,learner,1648.2,100.0,7 +verl-966,learner,1653.2,85.7,7 +verl-966,learner,1658.2,85.7,7 +verl-966,learner,1663.2,85.7,7 +verl-966,learner,1668.2,85.7,7 +verl-966,learner,1673.2,85.7,7 +verl-966,learner,1678.2,90.6,7 +verl-966,learner,1683.2,99.0,7 +verl-966,learner,1688.2,85.7,7 +verl-966,learner,1693.2,85.7,7 +verl-966,learner,1698.2,85.7,7 +verl-966,learner,1703.2,85.7,7 +verl-966,learner,1708.2,100.0,7 +verl-966,learner,1713.2,97.1,7 +verl-966,learner,1718.2,85.7,7 +verl-966,learner,1723.2,0.0,7 +verl-966,learner,1728.2,0.0,7 +verl-966,learner,1733.2,85.7,7 +verl-966,learner,1738.2,85.7,7 +verl-966,learner,1743.2,85.7,7 +verl-966,learner,1748.2,85.7,7 +verl-966,learner,1753.2,100.0,7 +verl-966,learner,1758.2,100.0,7 +verl-966,learner,1763.2,85.7,7 +verl-966,learner,1768.2,85.7,7 +verl-966,learner,1773.2,85.7,7 +verl-966,learner,1778.2,85.7,7 +verl-966,learner,1783.2,85.7,7 +verl-966,learner,1788.2,100.0,7 +verl-966,learner,1793.2,90.4,7 +verl-966,learner,1798.2,85.7,7 +verl-966,learner,1803.2,85.7,7 +verl-966,learner,1808.2,85.7,7 +verl-966,learner,1813.2,85.7,7 +verl-966,learner,1818.2,85.7,7 +verl-966,learner,1823.2,85.7,7 +verl-966,learner,1828.2,86.1,7 +verl-966,learner,1833.2,100.0,7 +verl-966,learner,1838.2,85.7,7 +verl-966,learner,1843.2,85.7,7 +verl-966,learner,1848.2,85.7,7 +verl-966,learner,1853.2,85.7,7 +verl-966,learner,1858.2,100.0,7 +verl-966,learner,1863.2,61.9,7 +verl-966,learner,1868.2,4.4,7 +verl-966,learner,1873.2,85.7,7 +verl-966,learner,1878.2,85.7,7 +verl-966,learner,1883.2,87.7,7 +verl-966,learner,1888.2,85.7,7 +verl-966,learner,1893.2,85.7,7 +verl-966,learner,1898.2,85.7,7 +verl-966,learner,1903.2,85.7,7 +verl-966,learner,1908.2,99.3,7 +verl-966,learner,1913.2,90.1,7 +verl-966,learner,1918.2,100.0,7 +verl-966,learner,1923.2,85.7,7 +verl-966,learner,1928.2,85.7,7 +verl-966,learner,1933.2,85.7,7 +verl-966,learner,1938.2,87.7,7 +verl-966,learner,1943.2,57.0,7 +verl-966,learner,1948.2,0.0,7 +verl-966,learner,1953.2,85.7,7 +verl-966,learner,1958.2,85.7,7 +verl-966,learner,1963.2,85.7,7 +verl-966,learner,1968.2,85.7,7 +verl-966,learner,1973.2,85.7,7 +verl-966,learner,1978.2,85.7,7 +verl-966,learner,1983.2,85.7,7 +verl-966,learner,1988.2,92.4,7 +verl-966,learner,1993.2,85.7,7 +verl-966,learner,1998.2,85.7,7 +verl-966,learner,2003.2,85.7,7 +verl-966,learner,2008.2,85.7,7 +verl-966,learner,2013.2,99.4,7 +verl-966,learner,2018.2,85.7,7 +verl-966,learner,2023.2,85.7,7 +verl-966,learner,2028.2,100.0,7 +verl-966,learner,2033.2,100.0,7 +verl-966,learner,2038.2,85.7,7 +verl-966,learner,2043.2,100.0,7 +verl-966,learner,2048.2,15.7,7 +verl-966,learner,2053.2,0.0,7 +verl-966,learner,2058.2,95.4,7 +verl-966,learner,2063.2,85.7,7 +verl-966,learner,2068.2,85.7,7 +verl-966,learner,2073.2,100.0,7 +verl-966,learner,2078.2,85.7,7 +verl-966,learner,2083.2,91.0,7 +verl-966,learner,2088.2,85.7,7 +verl-966,learner,2093.2,85.7,7 +verl-966,learner,2098.2,85.7,7 +verl-966,learner,2103.2,85.7,7 +verl-966,learner,2108.2,99.9,7 +verl-966,learner,2113.2,100.0,7 +verl-966,learner,2118.2,97.9,7 +verl-966,learner,2123.2,85.7,7 +verl-966,learner,2128.2,89.0,7 +verl-966,learner,2133.2,85.7,7 +verl-966,learner,2138.2,85.7,7 +verl-966,learner,2143.2,85.7,7 +verl-966,learner,2148.2,96.4,7 +verl-966,learner,2153.3,100.0,7 +verl-966,learner,2158.3,85.7,7 +verl-966,learner,2163.3,85.7,7 +verl-966,learner,2168.3,100.0,7 +verl-966,learner,2173.3,85.7,7 +verl-966,learner,2178.3,1.1,7 +verl-966,learner,2183.3,69.3,7 +verl-966,learner,2188.3,64.6,7 +verl-966,learner,2193.3,90.0,7 +verl-966,learner,2198.3,85.7,7 +verl-966,learner,2203.3,94.7,7 +verl-966,learner,2208.3,85.7,7 +verl-966,learner,2213.3,85.7,7 +verl-966,learner,2218.3,85.7,7 +verl-966,learner,2223.3,85.7,7 +verl-966,learner,2228.3,85.7,7 +verl-966,learner,2233.3,85.7,7 +verl-966,learner,2238.3,85.7,7 +verl-966,learner,2243.3,99.9,7 +verl-966,learner,2248.3,85.7,7 +verl-966,learner,2253.3,100.0,7 +verl-966,learner,2258.3,90.6,7 +verl-966,learner,2263.3,85.7,7 +verl-966,learner,2268.3,85.7,7 +verl-966,learner,2273.3,14.1,7 +verl-966,learner,2278.3,33.9,7 +verl-966,learner,2283.3,85.7,7 +verl-966,learner,2288.3,85.7,7 +verl-966,learner,2293.3,85.7,7 +verl-966,learner,2298.3,85.7,7 +verl-966,learner,2303.3,90.0,7 +verl-966,learner,2308.3,100.0,7 +verl-966,learner,2313.3,100.0,7 +verl-966,learner,2318.3,85.7,7 +verl-966,learner,2323.3,85.7,7 +verl-966,learner,2328.3,85.7,7 +verl-966,learner,2333.3,85.7,7 +verl-966,learner,2338.3,85.7,7 +verl-966,learner,2343.3,100.0,7 +verl-966,learner,2348.3,85.7,7 +verl-966,learner,2353.3,85.7,7 +verl-966,learner,2358.3,85.7,7 +verl-966,learner,2363.3,85.7,7 +verl-966,learner,2368.3,38.1,7 +verl-966,learner,2373.3,58.9,7 +verl-966,learner,2378.3,96.9,7 +verl-966,learner,2383.3,85.7,7 +verl-966,learner,2388.3,85.7,7 +verl-966,learner,2393.3,85.7,7 +verl-966,learner,2398.3,85.7,7 +verl-966,learner,2403.3,85.7,7 +verl-966,learner,2408.3,85.7,7 +verl-966,learner,2413.3,96.4,7 +verl-966,learner,2418.3,85.7,7 +verl-966,learner,2423.3,85.7,7 +verl-966,learner,2428.3,85.7,7 +verl-966,learner,2433.3,85.7,7 +verl-966,learner,2438.3,85.7,7 +verl-966,learner,2443.3,85.7,7 +verl-966,learner,2448.3,85.7,7 +verl-966,learner,2453.3,85.7,7 +verl-966,learner,2458.3,19.3,7 +verl-966,learner,2463.3,29.6,7 +verl-966,learner,2468.3,77.9,7 +verl-966,learner,2473.3,85.7,7 +verl-966,learner,2478.3,100.0,7 +verl-966,learner,2483.3,85.7,7 +verl-966,learner,2488.3,85.7,7 +verl-966,learner,2493.3,85.7,7 +verl-966,learner,2498.3,85.7,7 +verl-966,learner,2503.3,85.7,7 +verl-966,learner,2508.3,85.7,7 +verl-966,learner,2513.3,85.7,7 +verl-966,learner,2518.3,85.7,7 +verl-966,learner,2523.3,85.7,7 +verl-966,learner,2528.3,98.9,7 +verl-966,learner,2533.3,85.7,7 +verl-966,learner,2538.3,100.0,7 +verl-966,learner,2543.3,100.0,7 +verl-966,learner,2548.3,95.4,7 +verl-966,learner,2553.3,0.7,7 +verl-966,learner,2558.3,64.1,7 +verl-966,learner,2563.3,18.1,7 +verl-966,learner,2568.3,85.7,7 +verl-966,learner,2573.3,85.7,7 +verl-966,learner,2578.3,85.7,7 +verl-966,learner,2583.3,85.7,7 +verl-966,learner,2588.3,85.7,7 +verl-966,learner,2593.3,100.0,7 +verl-966,learner,2598.3,85.7,7 +verl-966,learner,2603.3,85.7,7 +verl-966,learner,2608.3,85.7,7 +verl-966,learner,2613.3,85.7,7 +verl-966,learner,2618.3,85.7,7 +verl-966,learner,2623.3,100.0,7 +verl-966,learner,2628.3,85.7,7 +verl-966,learner,2633.3,100.0,7 +verl-966,learner,2638.3,100.0,7 +verl-966,learner,2643.3,85.7,7 +verl-966,learner,2648.3,85.7,7 +verl-966,learner,2653.3,85.7,7 +verl-966,learner,2658.3,37.3,7 +verl-966,learner,2663.3,52.0,7 +verl-966,learner,2668.3,9.6,7 +verl-966,learner,2673.3,85.7,7 +verl-966,learner,2678.3,85.7,7 +verl-966,learner,2683.3,85.7,7 +verl-966,learner,2688.3,85.7,7 +verl-966,learner,2693.3,94.7,7 +verl-966,learner,2698.3,85.7,7 +verl-966,learner,2703.3,100.0,7 +verl-966,learner,2708.3,89.3,7 +verl-966,learner,2713.3,85.7,7 +verl-966,learner,2718.3,85.7,7 +verl-966,learner,2723.3,100.0,7 +verl-966,learner,2728.3,100.0,7 +verl-966,learner,2733.3,85.7,7 +verl-966,learner,2738.3,85.7,7 +verl-966,learner,2743.3,100.0,7 +verl-966,learner,2748.3,100.0,7 +verl-966,learner,2753.3,85.7,7 +verl-966,learner,2758.3,85.7,7 +verl-966,learner,2763.3,85.7,7 +verl-966,learner,2768.3,85.7,7 +verl-966,learner,2773.3,85.7,7 +verl-966,learner,2778.3,85.7,7 +verl-966,learner,2783.3,85.7,7 +verl-966,learner,2788.3,100.0,7 +verl-966,learner,2793.3,85.7,7 +verl-966,learner,2798.3,85.7,7 +verl-966,learner,2803.3,85.7,7 +verl-966,learner,2808.3,85.7,7 +verl-966,learner,2813.3,100.0,7 +verl-966,learner,2818.3,85.7,7 +verl-966,learner,2823.3,85.7,7 +verl-966,learner,2828.3,85.7,7 +verl-966,learner,2833.3,85.7,7 +verl-966,learner,2838.3,85.7,7 +verl-966,learner,2843.3,85.7,7 +verl-966,learner,2848.3,85.7,7 +verl-966,learner,2853.3,85.7,7 +verl-966,learner,2858.3,85.7,7 +verl-966,learner,2863.3,85.7,7 +verl-966,learner,2868.3,85.7,7 +verl-966,learner,2873.3,85.7,7 +verl-966,learner,2878.3,85.7,7 +verl-966,learner,2883.3,85.7,7 +verl-966,learner,2888.3,85.7,7 +verl-966,learner,2893.3,85.7,7 +verl-966,learner,2898.3,28.6,7 +verl-966,learner,2903.3,85.7,7 +verl-966,learner,2908.3,85.7,7 +verl-966,learner,2913.3,85.7,7 +verl-966,learner,2918.3,85.7,7 +verl-966,learner,2923.3,85.7,7 +verl-966,learner,2928.3,85.7,7 +verl-966,learner,2933.3,85.7,7 +verl-966,learner,2938.3,85.7,7 +verl-966,learner,2943.3,92.6,7 +verl-966,learner,2948.3,100.0,7 +verl-966,learner,2953.3,85.7,7 +verl-966,learner,2958.3,21.1,7 +verl-966,learner,2963.3,0.0,7 +verl-966,learner,2968.3,85.7,7 +verl-966,learner,2973.3,100.0,7 +verl-966,learner,2978.3,85.7,7 +verl-966,learner,2983.3,85.7,7 +verl-966,learner,2988.3,96.7,7 +verl-966,learner,2993.3,85.7,7 +verl-966,learner,2998.3,85.7,7 +verl-966,learner,3003.3,85.7,7 +verl-966,learner,3008.3,85.7,7 +verl-966,learner,3013.3,85.7,7 +verl-966,learner,3018.3,85.7,7 +verl-966,learner,3023.3,85.7,7 +verl-966,learner,3028.3,86.3,7 +verl-966,learner,3033.3,99.7,7 +verl-966,learner,3038.3,85.7,7 +verl-966,learner,3043.3,85.7,7 +verl-966,learner,3048.3,97.1,7 +verl-966,learner,3053.3,98.7,7 +verl-966,learner,3058.3,0.0,7 +verl-966,learner,3063.3,63.3,7 +verl-966,learner,3068.3,19.7,7 +verl-966,learner,3073.3,85.7,7 +verl-966,learner,3078.3,85.7,7 +verl-966,learner,3083.3,91.7,7 +verl-966,learner,3088.3,85.7,7 +verl-966,learner,3093.3,85.7,7 +verl-966,learner,3098.3,86.4,7 +verl-966,learner,3103.3,100.0,7 +verl-966,learner,3108.3,91.1,7 +verl-966,learner,3113.3,85.7,7 +verl-966,learner,3118.3,85.7,7 +verl-966,learner,3123.3,85.7,7 +verl-966,learner,3128.3,95.1,7 +verl-966,learner,3133.3,88.9,7 +verl-966,learner,3138.3,85.7,7 +verl-966,learner,3143.3,85.7,7 +verl-966,learner,3148.3,85.7,7 +verl-966,learner,3153.3,86.9,7 +verl-966,learner,3158.3,85.7,7 +verl-966,learner,3163.3,85.7,7 +verl-966,learner,3168.3,85.7,7 +verl-966,learner,3173.3,0.0,7 +verl-966,learner,3178.3,0.0,7 +verl-966,learner,3183.3,0.0,7 +verl-966,learner,3188.3,85.7,7 +verl-966,learner,3193.3,85.7,7 +verl-966,learner,3198.3,100.0,7 +verl-966,learner,3203.3,85.7,7 +verl-966,learner,3208.3,85.7,7 +verl-966,learner,3213.3,88.1,7 +verl-966,learner,3218.3,85.7,7 +verl-966,learner,3223.3,85.7,7 +verl-966,learner,3228.3,85.7,7 +verl-966,learner,3233.3,100.0,7 +verl-966,learner,3238.3,85.7,7 +verl-966,learner,3243.3,85.7,7 +verl-966,learner,3248.3,85.7,7 +verl-966,learner,3253.3,98.6,7 +verl-966,learner,3258.3,100.0,7 +verl-966,learner,3263.3,85.7,7 +verl-966,learner,3268.3,97.1,7 +verl-966,learner,3273.3,85.7,7 +verl-966,learner,3278.3,85.7,7 +verl-966,learner,3283.3,100.0,7 +verl-966,learner,3288.3,13.1,7 +verl-966,learner,3293.3,0.0,7 +verl-966,learner,3298.3,85.7,7 +verl-966,learner,3303.3,85.9,7 +verl-966,learner,3308.3,100.0,7 +verl-966,learner,3313.3,88.1,7 +verl-966,learner,3318.3,85.7,7 +verl-966,learner,3323.3,100.0,7 +verl-966,learner,3328.3,85.7,7 +verl-966,learner,3333.3,100.0,7 +verl-966,learner,3338.3,100.0,7 +verl-966,learner,3343.3,96.1,7 +verl-966,learner,3348.3,85.7,7 +verl-966,learner,3353.3,99.9,7 +verl-966,learner,3358.3,85.7,7 +verl-966,learner,3363.3,85.7,7 +verl-966,learner,3368.3,85.7,7 +verl-966,learner,3373.3,85.7,7 +verl-966,learner,3378.3,85.7,7 +verl-966,learner,3383.3,85.7,7 +verl-966,learner,3388.3,99.9,7 +verl-966,learner,3393.3,100.0,7 +verl-966,learner,3398.3,65.6,7 +verl-966,learner,3403.3,8.4,7 +verl-966,learner,3408.3,85.7,7 +verl-966,learner,3413.3,99.0,7 +verl-966,learner,3418.3,85.7,7 +verl-966,learner,3423.3,85.7,7 +verl-966,learner,3428.3,94.1,7 +verl-966,learner,3433.3,85.7,7 +verl-966,learner,3438.3,85.7,7 +verl-966,learner,3443.3,100.0,7 +verl-966,learner,3448.3,85.7,7 +verl-966,learner,3453.3,100.0,7 +verl-966,learner,3458.3,85.7,7 +verl-966,learner,3463.3,85.7,7 +verl-966,learner,3468.3,85.7,7 +verl-966,learner,3473.3,98.9,7 +verl-966,learner,3478.3,85.7,7 +verl-966,learner,3483.3,85.7,7 +verl-966,learner,3488.3,88.3,7 +verl-966,learner,3493.3,85.7,7 +verl-966,learner,3498.3,85.7,7 +verl-966,learner,3503.3,85.7,7 +verl-966,learner,3508.3,44.3,7 +verl-966,learner,3513.3,0.0,7 +verl-966,learner,3518.3,15.7,7 +verl-966,learner,3523.3,85.7,7 +verl-966,learner,3528.3,85.7,7 +verl-966,learner,3533.3,85.7,7 +verl-966,learner,3538.3,92.1,7 +verl-966,learner,3543.3,85.7,7 +verl-966,learner,3548.3,85.7,7 +verl-966,learner,3553.3,100.0,7 +verl-966,learner,3558.3,85.7,7 +verl-966,learner,3563.3,85.7,7 +verl-966,learner,3568.3,85.7,7 +verl-966,learner,3573.3,85.7,7 +verl-966,learner,3578.3,85.7,7 +verl-966,learner,3583.3,85.7,7 +verl-966,learner,3588.3,85.7,7 +verl-966,learner,3593.3,85.7,7 +verl-966,learner,3598.3,85.7,7 +verl-966,learner,3603.3,85.7,7 +verl-966,learner,3608.3,85.7,7 +verl-966,learner,3613.3,100.0,7 +verl-966,learner,3618.3,100.0,7 +verl-966,learner,3623.3,2.3,7 +verl-966,learner,3628.3,56.0,7 +verl-966,learner,3633.3,7.6,7 +verl-966,learner,3638.3,85.7,7 +verl-966,learner,3643.3,85.7,7 +verl-966,learner,3648.3,85.7,7 +verl-966,learner,3653.3,85.7,7 +verl-966,learner,3658.3,100.0,7 +verl-966,learner,3663.3,100.0,7 +verl-966,learner,3668.3,100.0,7 +verl-966,learner,3673.3,94.6,7 +verl-966,learner,3678.3,92.3,7 +verl-966,learner,3683.3,85.7,7 +verl-966,learner,3688.3,85.7,7 +verl-966,learner,3693.3,88.0,7 +verl-966,learner,3698.3,95.0,7 +verl-966,learner,3703.3,85.7,7 +verl-966,learner,3708.3,85.7,7 +verl-966,learner,3713.3,99.3,7 +verl-966,learner,3718.3,85.7,7 +verl-966,learner,3723.3,85.7,7 +verl-966,learner,3728.3,57.9,7 +verl-966,learner,3733.3,9.9,7 +verl-966,learner,3738.3,85.7,7 +verl-966,learner,3743.3,85.7,7 +verl-966,learner,3748.3,85.7,7 +verl-966,learner,3753.3,85.7,7 +verl-966,learner,3758.3,100.0,7 +verl-966,learner,3763.3,94.0,7 +verl-966,learner,3768.3,85.7,7 +verl-966,learner,3773.3,85.7,7 +verl-966,learner,3778.3,99.0,7 +verl-966,learner,3783.3,91.6,7 +verl-966,learner,3788.3,100.0,7 +verl-966,learner,3793.3,85.7,7 +verl-966,learner,3798.3,85.7,7 +verl-966,learner,3803.3,85.7,7 +verl-966,learner,3808.3,100.0,7 +verl-966,learner,3813.3,85.7,7 +verl-966,learner,3818.3,98.1,7 +verl-966,learner,3823.3,85.7,7 +verl-966,learner,3828.3,85.7,7 +verl-966,learner,3833.3,85.7,7 +verl-966,learner,3838.3,67.6,7 +verl-966,learner,3843.3,0.0,7 +verl-966,learner,3848.3,85.7,7 +verl-966,learner,3853.3,85.7,7 +verl-966,learner,3858.3,85.7,7 +verl-966,learner,3863.3,85.7,7 +verl-966,learner,3868.3,100.0,7 +verl-966,learner,3873.3,100.0,7 +verl-966,learner,3878.3,100.0,7 +verl-966,learner,3883.3,85.7,7 +verl-966,learner,3888.3,85.7,7 +verl-966,learner,3893.3,100.0,7 +verl-966,learner,3898.3,85.7,7 +verl-966,learner,3903.3,85.7,7 +verl-966,learner,3908.3,100.0,7 +verl-966,learner,3913.3,85.7,7 +verl-966,learner,3918.3,57.1,7 +verl-966,learner,3923.3,57.1,7 +verl-966,learner,3928.3,28.6,7 +verl-966,learner,3933.3,28.6,7 +verl-966,learner,3938.3,0.0,7 +verl-966,learner,3943.3,0.0,7 +verl-966,learner,3948.3,0.0,7 +verl-966,learner,3953.3,0.0,7 +verl-966,learner,3958.3,0.0,7 +verl-966,learner,3963.3,0.0,7 +verl-966,learner,3968.3,0.0,7 +verl-966,learner,3973.3,0.0,7 +verl-966,learner,3978.3,0.0,7 +verl-966,learner,3983.3,0.0,7 +verl-966,learner,3988.3,0.0,7 +verl-966,learner,3993.3,64.7,7 +verl-966,learner,3998.3,5.1,7 +verl-966,learner,4003.3,85.7,7 +verl-966,learner,4008.3,85.7,7 +verl-966,learner,4013.3,85.7,7 +verl-966,learner,4018.3,85.7,7 +verl-966,learner,4023.3,85.7,7 +verl-966,learner,4028.3,85.7,7 +verl-966,learner,4033.3,85.7,7 +verl-966,learner,4038.3,85.7,7 +verl-966,learner,4043.3,85.7,7 +verl-966,learner,4048.3,85.7,7 +verl-966,learner,4053.3,85.7,7 +verl-966,learner,4058.3,85.7,7 +verl-966,learner,4063.3,85.7,7 +verl-966,learner,4068.3,85.7,7 +verl-966,learner,4073.3,85.7,7 +verl-966,learner,4078.3,85.7,7 +verl-966,learner,4083.3,85.7,7 +verl-966,learner,4088.3,85.7,7 +verl-966,learner,4093.3,85.7,7 +verl-966,learner,4098.3,85.7,7 +verl-966,learner,4103.3,85.7,7 +verl-966,learner,4108.3,85.7,7 +verl-966,learner,4113.3,85.7,7 +verl-966,learner,4118.3,85.7,7 +verl-966,learner,4123.3,85.7,7 +verl-966,learner,4128.3,85.7,7 +verl-966,learner,4133.3,85.7,7 +verl-966,learner,4138.3,85.7,7 +verl-966,learner,4143.3,85.7,7 +verl-966,learner,4148.3,85.7,7 +verl-966,learner,4153.3,85.7,7 +verl-966,learner,4158.3,85.7,7 +verl-966,learner,4163.3,85.7,7 +verl-966,learner,4168.3,85.7,7 +verl-966,learner,4173.3,85.7,7 +verl-966,learner,4178.3,85.7,7 +verl-966,learner,4183.3,85.7,7 +verl-966,learner,4188.3,85.7,7 +verl-966,learner,4193.3,28.6,7 +verl-966,learner,4198.3,28.6,7 +verl-966,learner,4203.3,0.0,7 +verl-966,learner,4208.3,0.0,7 +verl-966,learner,4213.3,0.0,7 +verl-966,learner,4218.3,0.0,7 +verl-966,learner,4223.3,0.0,7 +verl-966,learner,4228.3,0.0,7 +verl-966,learner,4233.3,0.0,7 +verl-966,learner,4238.3,0.0,7 +verl-966,learner,4243.3,0.0,7 +verl-966,learner,4248.3,0.0,7 +verl-966,learner,4253.3,0.0,7 +verl-966,learner,4258.3,0.0,7 +verl-966,learner,4263.3,0.0,7 +prime-rl-1017,generation,14.1,0.0,7 +prime-rl-1017,generation,19.1,0.0,7 +prime-rl-1017,generation,24.1,0.0,7 +prime-rl-1017,generation,29.1,0.0,7 +prime-rl-1017,generation,34.1,0.0,7 +prime-rl-1017,generation,39.1,0.0,7 +prime-rl-1017,generation,44.1,0.0,7 +prime-rl-1017,generation,49.1,0.0,7 +prime-rl-1017,generation,54.1,0.0,7 +prime-rl-1017,generation,59.1,0.9,7 +prime-rl-1017,generation,64.1,1.0,7 +prime-rl-1017,generation,69.1,0.6,7 +prime-rl-1017,generation,74.1,1.1,7 +prime-rl-1017,generation,79.1,1.7,7 +prime-rl-1017,generation,84.1,3.9,7 +prime-rl-1017,generation,89.1,3.4,7 +prime-rl-1017,generation,94.1,13.3,7 +prime-rl-1017,generation,99.1,36.7,7 +prime-rl-1017,generation,104.1,12.7,7 +prime-rl-1017,generation,109.1,13.0,7 +prime-rl-1017,generation,114.1,21.1,7 +prime-rl-1017,generation,119.1,4.4,7 +prime-rl-1017,generation,124.2,0.0,7 +prime-rl-1017,generation,129.2,0.0,7 +prime-rl-1017,generation,134.2,0.0,7 +prime-rl-1017,generation,139.2,0.0,7 +prime-rl-1017,generation,144.2,0.0,7 +prime-rl-1017,generation,149.2,0.0,7 +prime-rl-1017,generation,154.2,0.0,7 +prime-rl-1017,generation,159.2,0.0,7 +prime-rl-1017,generation,164.2,0.0,7 +prime-rl-1017,generation,169.2,0.0,7 +prime-rl-1017,generation,174.2,0.0,7 +prime-rl-1017,generation,179.2,0.0,7 +prime-rl-1017,generation,184.2,0.0,7 +prime-rl-1017,generation,189.2,34.9,7 +prime-rl-1017,generation,194.2,0.0,7 +prime-rl-1017,generation,199.2,85.3,7 +prime-rl-1017,generation,204.2,85.7,7 +prime-rl-1017,generation,209.2,85.7,7 +prime-rl-1017,generation,214.2,85.7,7 +prime-rl-1017,generation,219.2,89.3,7 +prime-rl-1017,generation,224.2,85.7,7 +prime-rl-1017,generation,229.2,94.4,7 +prime-rl-1017,generation,234.2,85.7,7 +prime-rl-1017,generation,239.2,85.7,7 +prime-rl-1017,generation,244.2,85.7,7 +prime-rl-1017,generation,249.2,85.7,7 +prime-rl-1017,generation,254.2,85.7,7 +prime-rl-1017,generation,259.2,85.7,7 +prime-rl-1017,generation,264.2,93.3,7 +prime-rl-1017,generation,269.2,85.7,7 +prime-rl-1017,generation,274.2,92.3,7 +prime-rl-1017,generation,279.2,85.7,7 +prime-rl-1017,generation,284.2,85.6,7 +prime-rl-1017,generation,289.2,85.7,7 +prime-rl-1017,generation,294.2,96.4,7 +prime-rl-1017,generation,299.2,96.9,7 +prime-rl-1017,generation,304.2,94.0,7 +prime-rl-1017,generation,309.2,85.7,7 +prime-rl-1017,generation,314.2,87.4,7 +prime-rl-1017,generation,319.2,92.3,7 +prime-rl-1017,generation,324.2,85.7,7 +prime-rl-1017,generation,329.2,48.4,7 +prime-rl-1017,generation,334.2,85.6,7 +prime-rl-1017,generation,339.2,98.3,7 +prime-rl-1017,generation,344.2,96.3,7 +prime-rl-1017,generation,349.2,89.4,7 +prime-rl-1017,generation,354.2,88.1,7 +prime-rl-1017,generation,359.2,85.7,7 +prime-rl-1017,generation,364.2,52.3,7 +prime-rl-1017,generation,369.2,85.7,7 +prime-rl-1017,generation,374.2,85.7,7 +prime-rl-1017,generation,379.2,85.7,7 +prime-rl-1017,generation,384.2,85.7,7 +prime-rl-1017,generation,389.2,85.7,7 +prime-rl-1017,generation,394.2,85.7,7 +prime-rl-1017,generation,399.2,85.7,7 +prime-rl-1017,generation,404.2,85.7,7 +prime-rl-1017,generation,409.2,85.7,7 +prime-rl-1017,generation,414.2,85.7,7 +prime-rl-1017,generation,419.2,85.7,7 +prime-rl-1017,generation,424.2,85.7,7 +prime-rl-1017,generation,429.2,93.3,7 +prime-rl-1017,generation,434.2,85.7,7 +prime-rl-1017,generation,439.2,85.7,7 +prime-rl-1017,generation,444.2,85.7,7 +prime-rl-1017,generation,449.2,85.7,7 +prime-rl-1017,generation,454.2,85.7,7 +prime-rl-1017,generation,459.2,90.0,7 +prime-rl-1017,generation,464.2,85.7,7 +prime-rl-1017,generation,469.2,85.7,7 +prime-rl-1017,generation,474.2,85.7,7 +prime-rl-1017,generation,479.2,98.7,7 +prime-rl-1017,generation,484.2,92.9,7 +prime-rl-1017,generation,489.2,93.0,7 +prime-rl-1017,generation,494.2,91.6,7 +prime-rl-1017,generation,499.2,97.1,7 +prime-rl-1017,generation,504.2,85.7,7 +prime-rl-1017,generation,509.2,85.7,7 +prime-rl-1017,generation,514.2,28.6,7 +prime-rl-1017,generation,519.2,39.4,7 +prime-rl-1017,generation,524.2,85.7,7 +prime-rl-1017,generation,529.2,83.4,7 +prime-rl-1017,generation,534.2,87.4,7 +prime-rl-1017,generation,539.2,88.0,7 +prime-rl-1017,generation,544.2,95.7,7 +prime-rl-1017,generation,549.2,99.7,7 +prime-rl-1017,generation,554.2,36.7,7 +prime-rl-1017,generation,559.2,85.7,7 +prime-rl-1017,generation,564.2,85.7,7 +prime-rl-1017,generation,569.2,85.7,7 +prime-rl-1017,generation,574.2,85.7,7 +prime-rl-1017,generation,579.2,85.7,7 +prime-rl-1017,generation,584.2,85.7,7 +prime-rl-1017,generation,589.2,85.7,7 +prime-rl-1017,generation,594.2,85.7,7 +prime-rl-1017,generation,599.2,85.7,7 +prime-rl-1017,generation,604.2,94.9,7 +prime-rl-1017,generation,609.2,85.7,7 +prime-rl-1017,generation,614.2,94.9,7 +prime-rl-1017,generation,619.2,85.7,7 +prime-rl-1017,generation,624.2,85.7,7 +prime-rl-1017,generation,629.2,85.7,7 +prime-rl-1017,generation,634.2,97.1,7 +prime-rl-1017,generation,639.2,94.4,7 +prime-rl-1017,generation,644.2,85.7,7 +prime-rl-1017,generation,649.2,85.7,7 +prime-rl-1017,generation,654.2,85.7,7 +prime-rl-1017,generation,659.2,94.4,7 +prime-rl-1017,generation,664.2,85.7,7 +prime-rl-1017,generation,669.2,85.7,7 +prime-rl-1017,generation,674.2,85.7,7 +prime-rl-1017,generation,679.2,85.7,7 +prime-rl-1017,generation,684.2,94.7,7 +prime-rl-1017,generation,689.2,89.1,7 +prime-rl-1017,generation,694.2,85.7,7 +prime-rl-1017,generation,699.2,100.0,7 +prime-rl-1017,generation,704.2,84.1,7 +prime-rl-1017,generation,709.2,12.3,7 +prime-rl-1017,generation,714.2,85.7,7 +prime-rl-1017,generation,719.2,85.7,7 +prime-rl-1017,generation,724.2,85.7,7 +prime-rl-1017,generation,729.2,85.7,7 +prime-rl-1017,generation,734.2,85.7,7 +prime-rl-1017,generation,739.2,92.0,7 +prime-rl-1017,generation,744.2,99.9,7 +prime-rl-1017,generation,749.2,100.0,7 +prime-rl-1017,generation,754.2,85.7,7 +prime-rl-1017,generation,759.2,85.7,7 +prime-rl-1017,generation,764.2,100.0,7 +prime-rl-1017,generation,769.2,85.7,7 +prime-rl-1017,generation,774.2,35.0,7 +prime-rl-1017,generation,779.2,85.7,7 +prime-rl-1017,generation,784.2,85.7,7 +prime-rl-1017,generation,789.2,85.7,7 +prime-rl-1017,generation,794.2,90.3,7 +prime-rl-1017,generation,799.2,85.7,7 +prime-rl-1017,generation,804.2,85.7,7 +prime-rl-1017,generation,809.2,85.7,7 +prime-rl-1017,generation,814.2,85.7,7 +prime-rl-1017,generation,819.2,85.7,7 +prime-rl-1017,generation,824.2,85.7,7 +prime-rl-1017,generation,829.2,89.4,7 +prime-rl-1017,generation,834.2,94.9,7 +prime-rl-1017,generation,839.2,85.7,7 +prime-rl-1017,generation,844.2,98.7,7 +prime-rl-1017,generation,849.2,85.7,7 +prime-rl-1017,generation,854.2,85.7,7 +prime-rl-1017,generation,859.2,93.3,7 +prime-rl-1017,generation,864.2,95.3,7 +prime-rl-1017,generation,869.2,85.7,7 +prime-rl-1017,generation,874.2,88.7,7 +prime-rl-1017,generation,879.2,99.4,7 +prime-rl-1017,generation,884.2,85.7,7 +prime-rl-1017,generation,889.2,85.7,7 +prime-rl-1017,generation,894.2,97.7,7 +prime-rl-1017,generation,899.2,1.4,7 +prime-rl-1017,generation,904.2,45.4,7 +prime-rl-1017,generation,909.2,90.6,7 +prime-rl-1017,generation,914.2,94.9,7 +prime-rl-1017,generation,919.2,90.4,7 +prime-rl-1017,generation,924.2,99.1,7 +prime-rl-1017,generation,929.2,98.1,7 +prime-rl-1017,generation,934.2,98.6,7 +prime-rl-1017,generation,939.2,87.7,7 +prime-rl-1017,generation,944.2,0.0,7 +prime-rl-1017,generation,949.2,38.3,7 +prime-rl-1017,generation,954.2,85.7,7 +prime-rl-1017,generation,959.2,85.7,7 +prime-rl-1017,generation,964.2,86.9,7 +prime-rl-1017,generation,969.2,88.0,7 +prime-rl-1017,generation,974.2,85.7,7 +prime-rl-1017,generation,979.2,84.7,7 +prime-rl-1017,generation,984.2,85.7,7 +prime-rl-1017,generation,989.2,85.7,7 +prime-rl-1017,generation,994.3,85.7,7 +prime-rl-1017,generation,999.3,92.6,7 +prime-rl-1017,generation,1004.3,85.7,7 +prime-rl-1017,generation,1009.3,85.7,7 +prime-rl-1017,generation,1014.3,85.7,7 +prime-rl-1017,generation,1019.3,97.3,7 +prime-rl-1017,generation,1024.3,94.6,7 +prime-rl-1017,generation,1029.3,85.7,7 +prime-rl-1017,generation,1034.3,85.7,7 +prime-rl-1017,generation,1039.3,85.7,7 +prime-rl-1017,generation,1044.3,87.3,7 +prime-rl-1017,generation,1049.3,85.7,7 +prime-rl-1017,generation,1054.3,97.6,7 +prime-rl-1017,generation,1059.3,94.1,7 +prime-rl-1017,generation,1064.3,60.1,7 +prime-rl-1017,generation,1069.3,2.9,7 +prime-rl-1017,generation,1074.3,85.7,7 +prime-rl-1017,generation,1079.3,85.7,7 +prime-rl-1017,generation,1084.3,85.7,7 +prime-rl-1017,generation,1089.3,85.7,7 +prime-rl-1017,generation,1094.3,95.6,7 +prime-rl-1017,generation,1099.3,97.9,7 +prime-rl-1017,generation,1104.3,85.7,7 +prime-rl-1017,generation,1109.3,97.1,7 +prime-rl-1017,generation,1114.3,87.9,7 +prime-rl-1017,generation,1119.3,28.6,7 +prime-rl-1017,generation,1124.3,32.6,7 +prime-rl-1017,generation,1129.3,85.7,7 +prime-rl-1017,generation,1134.3,85.9,7 +prime-rl-1017,generation,1139.3,85.7,7 +prime-rl-1017,generation,1144.3,85.7,7 +prime-rl-1017,generation,1149.3,86.1,7 +prime-rl-1017,generation,1154.3,85.7,7 +prime-rl-1017,generation,1159.3,89.0,7 +prime-rl-1017,generation,1164.3,85.7,7 +prime-rl-1017,generation,1169.3,91.4,7 +prime-rl-1017,generation,1174.3,85.7,7 +prime-rl-1017,generation,1179.3,85.7,7 +prime-rl-1017,generation,1184.3,85.7,7 +prime-rl-1017,generation,1189.3,85.7,7 +prime-rl-1017,generation,1194.3,85.7,7 +prime-rl-1017,generation,1199.3,91.4,7 +prime-rl-1017,generation,1204.3,85.7,7 +prime-rl-1017,generation,1209.3,85.7,7 +prime-rl-1017,generation,1214.3,91.6,7 +prime-rl-1017,generation,1219.3,85.7,7 +prime-rl-1017,generation,1224.3,85.7,7 +prime-rl-1017,generation,1229.3,85.7,7 +prime-rl-1017,generation,1234.3,85.7,7 +prime-rl-1017,generation,1239.3,99.9,7 +prime-rl-1017,generation,1244.3,97.6,7 +prime-rl-1017,generation,1249.3,39.7,7 +prime-rl-1017,generation,1254.3,91.9,7 +prime-rl-1017,generation,1259.3,85.7,7 +prime-rl-1017,generation,1264.3,85.7,7 +prime-rl-1017,generation,1269.3,85.7,7 +prime-rl-1017,generation,1274.3,85.7,7 +prime-rl-1017,generation,1279.3,85.7,7 +prime-rl-1017,generation,1284.3,96.4,7 +prime-rl-1017,generation,1289.3,98.7,7 +prime-rl-1017,generation,1294.3,85.7,7 +prime-rl-1017,generation,1299.3,87.9,7 +prime-rl-1017,generation,1304.3,85.7,7 +prime-rl-1017,generation,1309.3,95.4,7 +prime-rl-1017,generation,1314.3,100.0,7 +prime-rl-1017,generation,1319.3,86.0,7 +prime-rl-1017,generation,1324.3,28.6,7 +prime-rl-1017,generation,1329.3,33.3,7 +prime-rl-1017,generation,1334.3,85.7,7 +prime-rl-1017,generation,1339.3,85.7,7 +prime-rl-1017,generation,1344.3,85.7,7 +prime-rl-1017,generation,1349.3,85.7,7 +prime-rl-1017,generation,1354.3,85.7,7 +prime-rl-1017,generation,1359.3,85.7,7 +prime-rl-1017,generation,1364.3,85.7,7 +prime-rl-1017,generation,1369.3,85.7,7 +prime-rl-1017,generation,1374.3,85.7,7 +prime-rl-1017,generation,1379.3,85.7,7 +prime-rl-1017,generation,1384.3,85.7,7 +prime-rl-1017,generation,1389.3,85.7,7 +prime-rl-1017,generation,1394.3,85.7,7 +prime-rl-1017,generation,1399.3,93.0,7 +prime-rl-1017,generation,1404.3,85.7,7 +prime-rl-1017,generation,1409.3,85.7,7 +prime-rl-1017,generation,1414.3,85.7,7 +prime-rl-1017,generation,1419.3,85.7,7 +prime-rl-1017,generation,1424.3,89.1,7 +prime-rl-1017,generation,1429.3,85.7,7 +prime-rl-1017,generation,1434.3,85.7,7 +prime-rl-1017,generation,1439.3,85.7,7 +prime-rl-1017,generation,1444.3,85.7,7 +prime-rl-1017,generation,1449.3,67.9,7 +prime-rl-1017,generation,1454.3,41.1,7 +prime-rl-1017,generation,1459.3,92.1,7 +prime-rl-1017,generation,1464.3,85.7,7 +prime-rl-1017,generation,1469.3,88.3,7 +prime-rl-1017,generation,1474.3,85.7,7 +prime-rl-1017,generation,1479.3,96.1,7 +prime-rl-1017,generation,1484.3,85.7,7 +prime-rl-1017,generation,1489.3,85.7,7 +prime-rl-1017,generation,1494.3,85.7,7 +prime-rl-1017,generation,1499.3,85.7,7 +prime-rl-1017,generation,1504.3,85.7,7 +prime-rl-1017,generation,1509.3,85.7,7 +prime-rl-1017,generation,1514.3,85.7,7 +prime-rl-1017,generation,1519.3,85.7,7 +prime-rl-1017,generation,1524.3,98.4,7 +prime-rl-1017,generation,1529.3,85.7,7 +prime-rl-1017,generation,1534.3,33.9,7 +prime-rl-1017,generation,1539.3,85.7,7 +prime-rl-1017,generation,1544.3,85.7,7 +prime-rl-1017,generation,1549.3,85.7,7 +prime-rl-1017,generation,1554.3,85.7,7 +prime-rl-1017,generation,1559.3,90.0,7 +prime-rl-1017,generation,1564.3,85.7,7 +prime-rl-1017,generation,1569.3,85.7,7 +prime-rl-1017,generation,1574.3,85.7,7 +prime-rl-1017,generation,1579.3,85.7,7 +prime-rl-1017,generation,1584.3,85.7,7 +prime-rl-1017,generation,1589.3,85.7,7 +prime-rl-1017,generation,1594.3,85.7,7 +prime-rl-1017,generation,1599.3,85.7,7 +prime-rl-1017,generation,1604.3,90.0,7 +prime-rl-1017,generation,1609.3,95.3,7 +prime-rl-1017,generation,1614.3,85.7,7 +prime-rl-1017,generation,1619.3,97.0,7 +prime-rl-1017,generation,1624.3,85.7,7 +prime-rl-1017,generation,1629.3,90.6,7 +prime-rl-1017,generation,1634.3,85.7,7 +prime-rl-1017,generation,1639.3,93.6,7 +prime-rl-1017,generation,1644.3,86.3,7 +prime-rl-1017,generation,1649.3,85.7,7 +prime-rl-1017,generation,1654.3,89.9,7 +prime-rl-1017,generation,1659.3,33.0,7 +prime-rl-1017,generation,1664.3,33.1,7 +prime-rl-1017,generation,1669.3,98.3,7 +prime-rl-1017,generation,1674.3,97.4,7 +prime-rl-1017,generation,1679.3,85.7,7 +prime-rl-1017,generation,1684.3,85.7,7 +prime-rl-1017,generation,1689.3,85.7,7 +prime-rl-1017,generation,1694.3,84.6,7 +prime-rl-1017,generation,1699.3,85.7,7 +prime-rl-1017,generation,1704.3,100.0,7 +prime-rl-1017,generation,1709.3,88.4,7 +prime-rl-1017,generation,1714.3,85.7,7 +prime-rl-1017,generation,1719.3,93.9,7 +prime-rl-1017,generation,1724.3,93.3,7 +prime-rl-1017,generation,1729.3,100.0,7 +prime-rl-1017,generation,1734.3,68.3,7 +prime-rl-1017,generation,1739.3,34.4,7 +prime-rl-1017,generation,1744.3,85.7,7 +prime-rl-1017,generation,1749.3,85.7,7 +prime-rl-1017,generation,1754.3,85.7,7 +prime-rl-1017,generation,1759.3,85.7,7 +prime-rl-1017,generation,1764.3,88.0,7 +prime-rl-1017,generation,1769.3,85.7,7 +prime-rl-1017,generation,1774.3,85.7,7 +prime-rl-1017,generation,1779.3,89.3,7 +prime-rl-1017,generation,1784.3,85.7,7 +prime-rl-1017,generation,1789.3,85.7,7 +prime-rl-1017,generation,1794.3,85.7,7 +prime-rl-1017,generation,1799.3,85.7,7 +prime-rl-1017,generation,1804.3,85.7,7 +prime-rl-1017,generation,1809.3,85.7,7 +prime-rl-1017,generation,1814.3,92.1,7 +prime-rl-1017,generation,1819.3,85.7,7 +prime-rl-1017,generation,1824.3,99.1,7 +prime-rl-1017,generation,1829.3,99.4,7 +prime-rl-1017,generation,1834.4,93.6,7 +prime-rl-1017,generation,1839.4,85.7,7 +prime-rl-1017,generation,1844.4,85.7,7 +prime-rl-1017,generation,1849.4,85.7,7 +prime-rl-1017,generation,1854.4,85.7,7 +prime-rl-1017,generation,1859.4,41.9,7 +prime-rl-1017,generation,1864.4,99.7,7 +prime-rl-1017,generation,1869.4,85.7,7 +prime-rl-1017,generation,1874.4,85.7,7 +prime-rl-1017,generation,1879.4,99.4,7 +prime-rl-1017,generation,1884.4,85.7,7 +prime-rl-1017,generation,1889.4,99.9,7 +prime-rl-1017,generation,1894.4,92.7,7 +prime-rl-1017,generation,1899.4,91.3,7 +prime-rl-1017,generation,1904.4,85.7,7 +prime-rl-1017,generation,1909.4,99.1,7 +prime-rl-1017,generation,1914.4,98.9,7 +prime-rl-1017,generation,1919.4,85.7,7 +prime-rl-1017,generation,1924.4,94.3,7 +prime-rl-1017,generation,1929.4,88.9,7 +prime-rl-1017,generation,1934.4,91.0,7 +prime-rl-1017,generation,1939.4,85.7,7 +prime-rl-1017,generation,1944.4,85.7,7 +prime-rl-1017,generation,1949.4,20.6,7 +prime-rl-1017,generation,1954.4,40.9,7 +prime-rl-1017,generation,1959.4,85.7,7 +prime-rl-1017,generation,1964.4,85.7,7 +prime-rl-1017,generation,1969.4,85.7,7 +prime-rl-1017,generation,1974.4,91.4,7 +prime-rl-1017,generation,1979.4,85.7,7 +prime-rl-1017,generation,1984.4,97.9,7 +prime-rl-1017,generation,1989.4,85.7,7 +prime-rl-1017,generation,1994.4,85.7,7 +prime-rl-1017,generation,1999.4,85.7,7 +prime-rl-1017,generation,2004.4,85.7,7 +prime-rl-1017,generation,2009.4,89.6,7 +prime-rl-1017,generation,2014.4,85.7,7 +prime-rl-1017,generation,2019.4,98.7,7 +prime-rl-1017,generation,2024.4,95.1,7 +prime-rl-1017,generation,2029.4,85.7,7 +prime-rl-1017,generation,2034.4,85.7,7 +prime-rl-1017,generation,2039.4,85.7,7 +prime-rl-1017,generation,2044.4,30.4,7 +prime-rl-1017,generation,2049.4,85.7,7 +prime-rl-1017,generation,2054.4,90.3,7 +prime-rl-1017,generation,2059.4,97.1,7 +prime-rl-1017,generation,2064.4,92.7,7 +prime-rl-1017,generation,2069.4,85.7,7 +prime-rl-1017,generation,2074.4,85.7,7 +prime-rl-1017,generation,2079.4,85.7,7 +prime-rl-1017,generation,2084.4,94.7,7 +prime-rl-1017,generation,2089.4,95.3,7 +prime-rl-1017,generation,2094.4,96.7,7 +prime-rl-1017,generation,2099.4,85.7,7 +prime-rl-1017,generation,2104.4,85.7,7 +prime-rl-1017,generation,2109.4,85.7,7 +prime-rl-1017,generation,2114.4,85.7,7 +prime-rl-1017,generation,2119.4,85.7,7 +prime-rl-1017,generation,2124.4,85.7,7 +prime-rl-1017,generation,2129.4,85.7,7 +prime-rl-1017,generation,2134.4,1.9,7 +prime-rl-1017,generation,2139.4,34.4,7 +prime-rl-1017,generation,2144.4,90.9,7 +prime-rl-1017,generation,2149.4,94.4,7 +prime-rl-1017,generation,2154.4,93.0,7 +prime-rl-1017,generation,2159.4,85.7,7 +prime-rl-1017,generation,2164.4,85.7,7 +prime-rl-1017,generation,2169.4,85.7,7 +prime-rl-1017,generation,2174.4,85.7,7 +prime-rl-1017,generation,2179.4,85.7,7 +prime-rl-1017,generation,2184.4,85.7,7 +prime-rl-1017,generation,2189.4,85.7,7 +prime-rl-1017,generation,2194.4,85.7,7 +prime-rl-1017,generation,2199.4,85.7,7 +prime-rl-1017,generation,2204.4,85.7,7 +prime-rl-1017,generation,2209.4,85.7,7 +prime-rl-1017,generation,2214.4,85.7,7 +prime-rl-1017,generation,2219.4,85.7,7 +prime-rl-1017,generation,2224.4,85.7,7 +prime-rl-1017,generation,2229.4,85.7,7 +prime-rl-1017,generation,2234.4,85.7,7 +prime-rl-1017,generation,2239.4,85.7,7 +prime-rl-1017,generation,2244.4,85.7,7 +prime-rl-1017,generation,2249.4,85.7,7 +prime-rl-1017,generation,2254.4,85.7,7 +prime-rl-1017,generation,2259.4,85.7,7 +prime-rl-1017,generation,2264.4,85.7,7 +prime-rl-1017,generation,2269.4,85.7,7 +prime-rl-1017,generation,2274.4,85.7,7 +prime-rl-1017,generation,2279.4,85.7,7 +prime-rl-1017,generation,2284.4,85.7,7 +prime-rl-1017,generation,2289.4,85.7,7 +prime-rl-1017,generation,2294.4,85.7,7 +prime-rl-1017,generation,2299.4,85.7,7 +prime-rl-1017,generation,2304.4,85.7,7 +prime-rl-1017,generation,2309.4,85.7,7 +prime-rl-1017,generation,2314.4,85.7,7 +prime-rl-1017,generation,2319.4,85.7,7 +prime-rl-1017,generation,2324.4,85.7,7 +prime-rl-1017,generation,2329.4,85.7,7 +prime-rl-1017,generation,2334.4,57.1,7 +prime-rl-1017,generation,2339.4,57.1,7 +prime-rl-1017,generation,2344.4,57.1,7 +prime-rl-1017,generation,2349.4,0.0,7 +prime-rl-1017,generation,2354.4,0.0,7 +prime-rl-1017,generation,2359.4,0.0,7 +prime-rl-1017,generation,2364.4,0.0,7 +prime-rl-1017,generation,2369.4,0.0,7 +prime-rl-1017,learner,14.1,0.0,8 +prime-rl-1017,learner,19.1,0.0,8 +prime-rl-1017,learner,24.1,0.0,8 +prime-rl-1017,learner,29.1,0.0,8 +prime-rl-1017,learner,34.2,0.0,8 +prime-rl-1017,learner,39.2,0.0,8 +prime-rl-1017,learner,44.2,0.1,8 +prime-rl-1017,learner,49.2,19.9,8 +prime-rl-1017,learner,54.2,2.6,8 +prime-rl-1017,learner,59.2,16.5,8 +prime-rl-1017,learner,64.2,87.5,8 +prime-rl-1017,learner,69.2,87.5,8 +prime-rl-1017,learner,74.2,87.5,8 +prime-rl-1017,learner,79.2,87.5,8 +prime-rl-1017,learner,84.2,87.5,8 +prime-rl-1017,learner,89.2,87.5,8 +prime-rl-1017,learner,94.2,87.5,8 +prime-rl-1017,learner,99.2,87.5,8 +prime-rl-1017,learner,104.2,87.5,8 +prime-rl-1017,learner,109.2,87.5,8 +prime-rl-1017,learner,114.2,87.5,8 +prime-rl-1017,learner,119.2,87.5,8 +prime-rl-1017,learner,124.2,87.5,8 +prime-rl-1017,learner,129.2,87.5,8 +prime-rl-1017,learner,134.2,87.5,8 +prime-rl-1017,learner,139.2,87.5,8 +prime-rl-1017,learner,144.2,87.5,8 +prime-rl-1017,learner,149.2,87.5,8 +prime-rl-1017,learner,154.2,87.5,8 +prime-rl-1017,learner,159.2,87.5,8 +prime-rl-1017,learner,164.2,87.5,8 +prime-rl-1017,learner,169.2,87.5,8 +prime-rl-1017,learner,174.2,87.5,8 +prime-rl-1017,learner,179.2,87.5,8 +prime-rl-1017,learner,184.2,87.5,8 +prime-rl-1017,learner,189.2,99.8,8 +prime-rl-1017,learner,194.2,0.0,8 +prime-rl-1017,learner,199.2,0.0,8 +prime-rl-1017,learner,204.2,0.0,8 +prime-rl-1017,learner,209.2,0.0,8 +prime-rl-1017,learner,214.2,0.0,8 +prime-rl-1017,learner,219.2,0.0,8 +prime-rl-1017,learner,224.2,0.0,8 +prime-rl-1017,learner,229.2,0.0,8 +prime-rl-1017,learner,234.2,0.0,8 +prime-rl-1017,learner,239.2,0.0,8 +prime-rl-1017,learner,244.2,0.0,8 +prime-rl-1017,learner,249.2,0.0,8 +prime-rl-1017,learner,254.2,0.0,8 +prime-rl-1017,learner,259.2,0.0,8 +prime-rl-1017,learner,264.2,0.0,8 +prime-rl-1017,learner,269.2,0.0,8 +prime-rl-1017,learner,274.2,0.0,8 +prime-rl-1017,learner,279.2,0.0,8 +prime-rl-1017,learner,284.2,0.0,8 +prime-rl-1017,learner,289.2,0.0,8 +prime-rl-1017,learner,294.2,84.5,8 +prime-rl-1017,learner,299.2,100.0,8 +prime-rl-1017,learner,304.2,1.2,8 +prime-rl-1017,learner,309.2,4.1,8 +prime-rl-1017,learner,314.2,3.8,8 +prime-rl-1017,learner,319.2,3.9,8 +prime-rl-1017,learner,324.2,87.5,8 +prime-rl-1017,learner,329.2,99.8,8 +prime-rl-1017,learner,334.2,90.8,8 +prime-rl-1017,learner,339.2,99.5,8 +prime-rl-1017,learner,344.2,100.0,8 +prime-rl-1017,learner,349.2,7.1,8 +prime-rl-1017,learner,354.2,7.4,8 +prime-rl-1017,learner,359.2,1.0,8 +prime-rl-1017,learner,364.2,99.4,8 +prime-rl-1017,learner,369.2,0.0,8 +prime-rl-1017,learner,374.2,0.0,8 +prime-rl-1017,learner,379.2,0.0,8 +prime-rl-1017,learner,384.2,0.0,8 +prime-rl-1017,learner,389.2,0.0,8 +prime-rl-1017,learner,394.2,0.0,8 +prime-rl-1017,learner,399.2,0.0,8 +prime-rl-1017,learner,404.2,0.0,8 +prime-rl-1017,learner,409.2,0.0,8 +prime-rl-1017,learner,414.2,0.0,8 +prime-rl-1017,learner,419.2,0.0,8 +prime-rl-1017,learner,424.2,0.0,8 +prime-rl-1017,learner,429.2,0.0,8 +prime-rl-1017,learner,434.2,0.0,8 +prime-rl-1017,learner,439.2,0.0,8 +prime-rl-1017,learner,444.2,0.0,8 +prime-rl-1017,learner,449.2,0.0,8 +prime-rl-1017,learner,454.2,0.0,8 +prime-rl-1017,learner,459.2,0.0,8 +prime-rl-1017,learner,464.2,0.0,8 +prime-rl-1017,learner,469.2,0.0,8 +prime-rl-1017,learner,474.2,0.0,8 +prime-rl-1017,learner,479.2,100.0,8 +prime-rl-1017,learner,484.2,87.5,8 +prime-rl-1017,learner,489.2,89.8,8 +prime-rl-1017,learner,494.2,87.5,8 +prime-rl-1017,learner,499.2,7.5,8 +prime-rl-1017,learner,504.2,7.5,8 +prime-rl-1017,learner,509.2,3.6,8 +prime-rl-1017,learner,514.2,87.5,8 +prime-rl-1017,learner,519.2,99.8,8 +prime-rl-1017,learner,524.2,62.9,8 +prime-rl-1017,learner,529.2,100.0,8 +prime-rl-1017,learner,534.2,100.0,8 +prime-rl-1017,learner,539.2,7.8,8 +prime-rl-1017,learner,544.2,7.4,8 +prime-rl-1017,learner,549.2,87.5,8 +prime-rl-1017,learner,554.2,99.8,8 +prime-rl-1017,learner,559.2,0.0,8 +prime-rl-1017,learner,564.2,0.0,8 +prime-rl-1017,learner,569.2,0.0,8 +prime-rl-1017,learner,574.2,0.0,8 +prime-rl-1017,learner,579.2,0.0,8 +prime-rl-1017,learner,584.2,0.0,8 +prime-rl-1017,learner,589.2,0.0,8 +prime-rl-1017,learner,594.2,0.0,8 +prime-rl-1017,learner,599.2,0.0,8 +prime-rl-1017,learner,604.2,0.0,8 +prime-rl-1017,learner,609.2,0.0,8 +prime-rl-1017,learner,614.2,0.0,8 +prime-rl-1017,learner,619.2,0.0,8 +prime-rl-1017,learner,624.2,0.0,8 +prime-rl-1017,learner,629.2,0.0,8 +prime-rl-1017,learner,634.2,0.0,8 +prime-rl-1017,learner,639.2,0.0,8 +prime-rl-1017,learner,644.2,0.0,8 +prime-rl-1017,learner,649.2,0.0,8 +prime-rl-1017,learner,654.2,0.0,8 +prime-rl-1017,learner,659.2,0.0,8 +prime-rl-1017,learner,664.2,0.0,8 +prime-rl-1017,learner,669.2,0.0,8 +prime-rl-1017,learner,674.2,0.0,8 +prime-rl-1017,learner,679.2,89.4,8 +prime-rl-1017,learner,684.2,95.4,8 +prime-rl-1017,learner,689.2,7.5,8 +prime-rl-1017,learner,694.2,7.4,8 +prime-rl-1017,learner,699.2,6.2,8 +prime-rl-1017,learner,704.2,51.1,8 +prime-rl-1017,learner,709.2,4.0,8 +prime-rl-1017,learner,714.2,0.0,8 +prime-rl-1017,learner,719.2,0.0,8 +prime-rl-1017,learner,724.2,0.0,8 +prime-rl-1017,learner,729.2,0.0,8 +prime-rl-1017,learner,734.2,0.0,8 +prime-rl-1017,learner,739.2,100.0,8 +prime-rl-1017,learner,744.2,100.0,8 +prime-rl-1017,learner,749.2,92.2,8 +prime-rl-1017,learner,754.2,7.0,8 +prime-rl-1017,learner,759.2,7.0,8 +prime-rl-1017,learner,764.2,2.9,8 +prime-rl-1017,learner,769.2,87.5,8 +prime-rl-1017,learner,774.2,99.8,8 +prime-rl-1017,learner,779.2,0.0,8 +prime-rl-1017,learner,784.2,0.0,8 +prime-rl-1017,learner,789.2,0.0,8 +prime-rl-1017,learner,794.2,0.0,8 +prime-rl-1017,learner,799.2,0.0,8 +prime-rl-1017,learner,804.2,0.0,8 +prime-rl-1017,learner,809.2,0.0,8 +prime-rl-1017,learner,814.2,0.0,8 +prime-rl-1017,learner,819.2,0.0,8 +prime-rl-1017,learner,824.2,0.0,8 +prime-rl-1017,learner,829.2,0.0,8 +prime-rl-1017,learner,834.2,0.0,8 +prime-rl-1017,learner,839.2,0.0,8 +prime-rl-1017,learner,844.2,0.0,8 +prime-rl-1017,learner,849.2,0.0,8 +prime-rl-1017,learner,854.2,0.0,8 +prime-rl-1017,learner,859.2,0.0,8 +prime-rl-1017,learner,864.2,0.0,8 +prime-rl-1017,learner,869.2,84.1,8 +prime-rl-1017,learner,874.2,100.0,8 +prime-rl-1017,learner,879.2,100.0,8 +prime-rl-1017,learner,884.2,7.5,8 +prime-rl-1017,learner,889.2,7.4,8 +prime-rl-1017,learner,894.2,87.5,8 +prime-rl-1017,learner,899.2,87.0,8 +prime-rl-1017,learner,904.2,99.8,8 +prime-rl-1017,learner,909.2,0.0,8 +prime-rl-1017,learner,914.2,0.0,8 +prime-rl-1017,learner,919.2,100.0,8 +prime-rl-1017,learner,924.2,100.0,8 +prime-rl-1017,learner,929.2,7.6,8 +prime-rl-1017,learner,934.2,7.9,8 +prime-rl-1017,learner,939.2,13.0,8 +prime-rl-1017,learner,944.2,87.5,8 +prime-rl-1017,learner,949.2,100.0,8 +prime-rl-1017,learner,954.2,0.0,8 +prime-rl-1017,learner,959.2,0.0,8 +prime-rl-1017,learner,964.2,0.0,8 +prime-rl-1017,learner,969.2,0.0,8 +prime-rl-1017,learner,974.2,0.0,8 +prime-rl-1017,learner,979.2,0.0,8 +prime-rl-1017,learner,984.2,0.0,8 +prime-rl-1017,learner,989.2,0.0,8 +prime-rl-1017,learner,994.2,0.0,8 +prime-rl-1017,learner,999.2,0.0,8 +prime-rl-1017,learner,1004.2,0.0,8 +prime-rl-1017,learner,1009.2,0.0,8 +prime-rl-1017,learner,1014.2,0.0,8 +prime-rl-1017,learner,1019.2,0.0,8 +prime-rl-1017,learner,1024.2,0.0,8 +prime-rl-1017,learner,1029.2,0.0,8 +prime-rl-1017,learner,1034.2,0.0,8 +prime-rl-1017,learner,1039.2,96.5,8 +prime-rl-1017,learner,1044.2,100.0,8 +prime-rl-1017,learner,1049.2,7.5,8 +prime-rl-1017,learner,1054.2,7.5,8 +prime-rl-1017,learner,1059.2,26.1,8 +prime-rl-1017,learner,1064.2,94.8,8 +prime-rl-1017,learner,1069.2,0.0,8 +prime-rl-1017,learner,1074.2,0.0,8 +prime-rl-1017,learner,1079.2,0.0,8 +prime-rl-1017,learner,1084.2,0.0,8 +prime-rl-1017,learner,1089.2,0.0,8 +prime-rl-1017,learner,1094.2,100.0,8 +prime-rl-1017,learner,1099.2,100.0,8 +prime-rl-1017,learner,1104.2,49.1,8 +prime-rl-1017,learner,1109.2,7.2,8 +prime-rl-1017,learner,1114.2,7.4,8 +prime-rl-1017,learner,1119.2,87.5,8 +prime-rl-1017,learner,1124.2,100.0,8 +prime-rl-1017,learner,1129.2,0.0,8 +prime-rl-1017,learner,1134.2,0.0,8 +prime-rl-1017,learner,1139.3,0.0,8 +prime-rl-1017,learner,1144.3,0.0,8 +prime-rl-1017,learner,1149.3,0.0,8 +prime-rl-1017,learner,1154.3,0.0,8 +prime-rl-1017,learner,1159.3,0.0,8 +prime-rl-1017,learner,1164.3,0.0,8 +prime-rl-1017,learner,1169.3,0.0,8 +prime-rl-1017,learner,1174.3,0.0,8 +prime-rl-1017,learner,1179.3,0.0,8 +prime-rl-1017,learner,1184.3,0.0,8 +prime-rl-1017,learner,1189.3,0.0,8 +prime-rl-1017,learner,1194.3,0.0,8 +prime-rl-1017,learner,1199.3,0.0,8 +prime-rl-1017,learner,1204.3,0.0,8 +prime-rl-1017,learner,1209.3,0.0,8 +prime-rl-1017,learner,1214.3,0.0,8 +prime-rl-1017,learner,1219.3,0.0,8 +prime-rl-1017,learner,1224.3,98.5,8 +prime-rl-1017,learner,1229.3,100.0,8 +prime-rl-1017,learner,1234.3,7.1,8 +prime-rl-1017,learner,1239.3,7.0,8 +prime-rl-1017,learner,1244.3,13.2,8 +prime-rl-1017,learner,1249.3,99.4,8 +prime-rl-1017,learner,1254.3,0.0,8 +prime-rl-1017,learner,1259.3,0.0,8 +prime-rl-1017,learner,1264.3,0.0,8 +prime-rl-1017,learner,1269.3,0.0,8 +prime-rl-1017,learner,1274.3,0.0,8 +prime-rl-1017,learner,1279.3,0.0,8 +prime-rl-1017,learner,1284.3,0.0,8 +prime-rl-1017,learner,1289.3,0.0,8 +prime-rl-1017,learner,1294.3,0.0,8 +prime-rl-1017,learner,1299.3,100.0,8 +prime-rl-1017,learner,1304.3,91.9,8 +prime-rl-1017,learner,1309.3,8.1,8 +prime-rl-1017,learner,1314.3,7.1,8 +prime-rl-1017,learner,1319.3,7.4,8 +prime-rl-1017,learner,1324.3,87.5,8 +prime-rl-1017,learner,1329.3,99.8,8 +prime-rl-1017,learner,1334.3,0.0,8 +prime-rl-1017,learner,1339.3,0.0,8 +prime-rl-1017,learner,1344.3,0.0,8 +prime-rl-1017,learner,1349.3,0.0,8 +prime-rl-1017,learner,1354.3,0.0,8 +prime-rl-1017,learner,1359.3,0.0,8 +prime-rl-1017,learner,1364.3,0.0,8 +prime-rl-1017,learner,1369.3,0.0,8 +prime-rl-1017,learner,1374.3,0.0,8 +prime-rl-1017,learner,1379.3,0.0,8 +prime-rl-1017,learner,1384.3,0.0,8 +prime-rl-1017,learner,1389.3,0.0,8 +prime-rl-1017,learner,1394.3,0.0,8 +prime-rl-1017,learner,1399.3,0.0,8 +prime-rl-1017,learner,1404.3,0.0,8 +prime-rl-1017,learner,1409.3,0.0,8 +prime-rl-1017,learner,1414.3,0.0,8 +prime-rl-1017,learner,1419.3,0.0,8 +prime-rl-1017,learner,1424.3,100.0,8 +prime-rl-1017,learner,1429.3,100.0,8 +prime-rl-1017,learner,1434.3,7.4,8 +prime-rl-1017,learner,1439.3,7.5,8 +prime-rl-1017,learner,1444.3,4.1,8 +prime-rl-1017,learner,1449.3,87.5,8 +prime-rl-1017,learner,1454.3,99.8,8 +prime-rl-1017,learner,1459.3,0.0,8 +prime-rl-1017,learner,1464.3,0.0,8 +prime-rl-1017,learner,1469.3,0.0,8 +prime-rl-1017,learner,1474.3,0.0,8 +prime-rl-1017,learner,1479.3,0.0,8 +prime-rl-1017,learner,1484.3,0.0,8 +prime-rl-1017,learner,1489.3,0.0,8 +prime-rl-1017,learner,1494.3,0.0,8 +prime-rl-1017,learner,1499.3,0.0,8 +prime-rl-1017,learner,1504.3,100.0,8 +prime-rl-1017,learner,1509.3,100.0,8 +prime-rl-1017,learner,1514.3,100.0,8 +prime-rl-1017,learner,1519.3,7.9,8 +prime-rl-1017,learner,1524.3,7.2,8 +prime-rl-1017,learner,1529.3,87.5,8 +prime-rl-1017,learner,1534.3,99.8,8 +prime-rl-1017,learner,1539.3,0.0,8 +prime-rl-1017,learner,1544.3,0.0,8 +prime-rl-1017,learner,1549.3,0.0,8 +prime-rl-1017,learner,1554.3,0.0,8 +prime-rl-1017,learner,1559.3,0.0,8 +prime-rl-1017,learner,1564.3,0.0,8 +prime-rl-1017,learner,1569.3,0.0,8 +prime-rl-1017,learner,1574.3,0.0,8 +prime-rl-1017,learner,1579.3,0.0,8 +prime-rl-1017,learner,1584.3,0.0,8 +prime-rl-1017,learner,1589.3,0.0,8 +prime-rl-1017,learner,1594.3,0.0,8 +prime-rl-1017,learner,1599.3,0.0,8 +prime-rl-1017,learner,1604.3,0.0,8 +prime-rl-1017,learner,1609.3,0.0,8 +prime-rl-1017,learner,1614.3,0.0,8 +prime-rl-1017,learner,1619.3,0.0,8 +prime-rl-1017,learner,1624.3,0.0,8 +prime-rl-1017,learner,1629.3,0.0,8 +prime-rl-1017,learner,1634.3,90.9,8 +prime-rl-1017,learner,1639.3,99.1,8 +prime-rl-1017,learner,1644.3,7.4,8 +prime-rl-1017,learner,1649.3,6.9,8 +prime-rl-1017,learner,1654.3,13.5,8 +prime-rl-1017,learner,1659.3,85.9,8 +prime-rl-1017,learner,1664.3,99.8,8 +prime-rl-1017,learner,1669.3,0.0,8 +prime-rl-1017,learner,1674.3,0.0,8 +prime-rl-1017,learner,1679.3,0.0,8 +prime-rl-1017,learner,1684.3,0.0,8 +prime-rl-1017,learner,1689.3,0.0,8 +prime-rl-1017,learner,1694.3,0.0,8 +prime-rl-1017,learner,1699.3,0.0,8 +prime-rl-1017,learner,1704.3,0.0,8 +prime-rl-1017,learner,1709.3,100.0,8 +prime-rl-1017,learner,1714.3,100.0,8 +prime-rl-1017,learner,1719.3,42.4,8 +prime-rl-1017,learner,1724.3,7.5,8 +prime-rl-1017,learner,1729.3,7.1,8 +prime-rl-1017,learner,1734.3,87.5,8 +prime-rl-1017,learner,1739.3,99.8,8 +prime-rl-1017,learner,1744.3,0.0,8 +prime-rl-1017,learner,1749.3,0.0,8 +prime-rl-1017,learner,1754.3,0.0,8 +prime-rl-1017,learner,1759.3,0.0,8 +prime-rl-1017,learner,1764.3,0.0,8 +prime-rl-1017,learner,1769.3,0.0,8 +prime-rl-1017,learner,1774.3,0.0,8 +prime-rl-1017,learner,1779.3,0.0,8 +prime-rl-1017,learner,1784.3,0.0,8 +prime-rl-1017,learner,1789.3,0.0,8 +prime-rl-1017,learner,1794.3,0.0,8 +prime-rl-1017,learner,1799.3,0.0,8 +prime-rl-1017,learner,1804.3,0.0,8 +prime-rl-1017,learner,1809.3,0.0,8 +prime-rl-1017,learner,1814.3,0.0,8 +prime-rl-1017,learner,1819.3,0.0,8 +prime-rl-1017,learner,1824.3,0.0,8 +prime-rl-1017,learner,1829.3,100.0,8 +prime-rl-1017,learner,1834.3,100.0,8 +prime-rl-1017,learner,1839.3,7.5,8 +prime-rl-1017,learner,1844.3,7.9,8 +prime-rl-1017,learner,1849.3,0.6,8 +prime-rl-1017,learner,1854.3,87.5,8 +prime-rl-1017,learner,1859.3,99.8,8 +prime-rl-1017,learner,1864.3,0.0,8 +prime-rl-1017,learner,1869.3,0.0,8 +prime-rl-1017,learner,1874.3,0.0,8 +prime-rl-1017,learner,1879.3,0.0,8 +prime-rl-1017,learner,1884.3,0.0,8 +prime-rl-1017,learner,1889.3,100.0,8 +prime-rl-1017,learner,1894.3,88.2,8 +prime-rl-1017,learner,1899.3,87.5,8 +prime-rl-1017,learner,1904.3,91.9,8 +prime-rl-1017,learner,1909.3,100.0,8 +prime-rl-1017,learner,1914.3,87.5,8 +prime-rl-1017,learner,1919.3,89.1,8 +prime-rl-1017,learner,1924.3,99.0,8 +prime-rl-1017,learner,1929.3,100.0,8 +prime-rl-1017,learner,1934.3,6.8,8 +prime-rl-1017,learner,1939.3,7.6,8 +prime-rl-1017,learner,1944.3,2.6,8 +prime-rl-1017,learner,1949.3,84.2,8 +prime-rl-1017,learner,1954.3,99.5,8 +prime-rl-1017,learner,1959.3,0.0,8 +prime-rl-1017,learner,1964.3,0.0,8 +prime-rl-1017,learner,1969.3,0.0,8 +prime-rl-1017,learner,1974.3,0.0,8 +prime-rl-1017,learner,1979.3,0.0,8 +prime-rl-1017,learner,1984.3,0.0,8 +prime-rl-1017,learner,1989.3,0.0,8 +prime-rl-1017,learner,1994.3,0.0,8 +prime-rl-1017,learner,1999.3,0.0,8 +prime-rl-1017,learner,2004.3,0.0,8 +prime-rl-1017,learner,2009.3,0.0,8 +prime-rl-1017,learner,2014.3,99.6,8 +prime-rl-1017,learner,2019.3,100.0,8 +prime-rl-1017,learner,2024.3,100.0,8 +prime-rl-1017,learner,2029.3,7.5,8 +prime-rl-1017,learner,2034.3,7.1,8 +prime-rl-1017,learner,2039.3,87.5,8 +prime-rl-1017,learner,2044.3,99.8,8 +prime-rl-1017,learner,2049.3,0.0,8 +prime-rl-1017,learner,2054.3,0.0,8 +prime-rl-1017,learner,2059.3,0.0,8 +prime-rl-1017,learner,2064.3,0.0,8 +prime-rl-1017,learner,2069.3,0.0,8 +prime-rl-1017,learner,2074.3,0.0,8 +prime-rl-1017,learner,2079.3,0.0,8 +prime-rl-1017,learner,2084.3,0.0,8 +prime-rl-1017,learner,2089.3,0.0,8 +prime-rl-1017,learner,2094.3,0.0,8 +prime-rl-1017,learner,2099.3,0.0,8 +prime-rl-1017,learner,2104.3,100.0,8 +prime-rl-1017,learner,2109.3,100.0,8 +prime-rl-1017,learner,2114.3,100.0,8 +prime-rl-1017,learner,2119.3,7.8,8 +prime-rl-1017,learner,2124.4,7.2,8 +prime-rl-1017,learner,2129.4,14.4,8 +prime-rl-1017,learner,2134.4,85.9,8 +prime-rl-1017,learner,2139.4,99.8,8 +prime-rl-1017,learner,2144.4,7.1,8 +prime-rl-1017,learner,2149.4,7.6,8 +prime-rl-1017,learner,2154.4,0.0,8 +prime-rl-1017,learner,2159.4,1.9,8 +prime-rl-1017,learner,2164.4,2.2,8 +prime-rl-1017,learner,2169.4,2.0,8 +prime-rl-1017,learner,2174.4,0.0,8 +prime-rl-1017,learner,2179.4,0.0,8 +prime-rl-1017,learner,2184.4,1.9,8 +prime-rl-1017,learner,2189.4,1.9,8 +prime-rl-1017,learner,2194.4,2.6,8 +prime-rl-1017,learner,2199.4,2.6,8 +prime-rl-1017,learner,2204.4,87.1,8 +prime-rl-1017,learner,2209.4,0.6,8 +prime-rl-1017,learner,2233.8,0.0,8 +prime-rl-1017,learner,2238.8,0.0,8 +prime-rl-1017,learner,2243.8,0.0,8 +prime-rl-1017,learner,2248.8,0.0,8 +prime-rl-1017,learner,2253.8,0.0,8 +prime-rl-1017,learner,2258.8,0.0,8 +prime-rl-1017,learner,2263.9,0.0,8 +prime-rl-1017,learner,2268.9,0.0,8 +prime-rl-1017,learner,2273.9,0.0,8 +prime-rl-1017,learner,2278.9,0.0,8 +prime-rl-1017,learner,2283.9,0.0,8 +prime-rl-1017,learner,2288.9,0.0,8 +prime-rl-1017,learner,2293.9,0.0,8 +prime-rl-1017,learner,2298.9,0.0,8 +prime-rl-1017,learner,2303.9,0.0,8 +prime-rl-1017,learner,2308.9,0.0,8 +prime-rl-1017,learner,2313.9,0.0,8 +prime-rl-1017,learner,2318.9,0.0,8 +prime-rl-1017,learner,2323.9,0.0,8 +prime-rl-1017,learner,2328.9,0.0,8 +prime-rl-1017,learner,2333.9,0.0,8 +prime-rl-1017,learner,2338.9,0.0,8 +prime-rl-1017,learner,2343.9,0.0,8 +prime-rl-1017,learner,2348.9,0.0,8 +prime-rl-1017,learner,2353.9,0.0,8 +slime-979,generation,32.8,0.0,7 +slime-979,generation,37.8,0.0,7 +slime-979,generation,42.8,0.0,7 +slime-979,generation,47.8,0.0,7 +slime-979,generation,52.8,0.0,7 +slime-979,generation,57.8,0.0,7 +slime-979,generation,62.8,0.0,7 +slime-979,generation,67.8,0.1,7 +slime-979,generation,72.8,0.6,7 +slime-979,generation,77.8,0.1,7 +slime-979,generation,82.8,0.4,7 +slime-979,generation,87.8,0.1,7 +slime-979,generation,92.8,0.0,7 +slime-979,generation,97.8,0.4,7 +slime-979,generation,102.8,0.1,7 +slime-979,generation,107.8,0.6,7 +slime-979,generation,112.8,0.1,7 +slime-979,generation,117.9,0.1,7 +slime-979,generation,122.9,0.9,7 +slime-979,generation,127.9,0.3,7 +slime-979,generation,132.9,0.0,7 +slime-979,generation,137.9,0.0,7 +slime-979,generation,142.9,0.0,7 +slime-979,generation,147.9,0.0,7 +slime-979,generation,152.9,0.0,7 +slime-979,generation,157.9,0.0,7 +slime-979,generation,162.9,0.0,7 +slime-979,generation,167.9,0.0,7 +slime-979,generation,172.9,0.0,7 +slime-979,generation,177.9,0.0,7 +slime-979,generation,182.9,0.0,7 +slime-979,generation,187.9,0.0,7 +slime-979,generation,192.9,0.0,7 +slime-979,generation,197.9,0.0,7 +slime-979,generation,202.9,0.0,7 +slime-979,generation,207.9,0.0,7 +slime-979,generation,212.9,0.0,7 +slime-979,generation,217.9,0.0,7 +slime-979,generation,222.9,0.0,7 +slime-979,generation,227.9,0.0,7 +slime-979,generation,232.9,0.0,7 +slime-979,generation,237.9,0.0,7 +slime-979,generation,242.9,0.4,7 +slime-979,generation,247.9,0.4,7 +slime-979,generation,252.9,0.1,7 +slime-979,generation,257.9,0.1,7 +slime-979,generation,262.9,0.3,7 +slime-979,generation,267.9,0.7,7 +slime-979,generation,272.9,0.0,7 +slime-979,generation,277.9,0.0,7 +slime-979,generation,282.9,0.6,7 +slime-979,generation,287.9,0.4,7 +slime-979,generation,292.9,0.0,7 +slime-979,generation,297.9,0.1,7 +slime-979,generation,302.9,0.9,7 +slime-979,generation,307.9,0.0,7 +slime-979,generation,312.9,0.6,7 +slime-979,generation,317.9,0.4,7 +slime-979,generation,322.9,0.4,7 +slime-979,generation,327.9,0.0,7 +slime-979,generation,332.9,0.0,7 +slime-979,generation,337.9,0.4,7 +slime-979,generation,342.9,0.9,7 +slime-979,generation,347.9,0.3,7 +slime-979,generation,352.9,0.7,7 +slime-979,generation,357.9,0.4,7 +slime-979,generation,362.9,0.4,7 +slime-979,generation,367.9,0.3,7 +slime-979,generation,372.9,0.9,7 +slime-979,generation,377.9,0.0,7 +slime-979,generation,382.9,0.4,7 +slime-979,generation,387.9,0.0,7 +slime-979,generation,392.9,7.6,7 +slime-979,generation,397.9,0.0,7 +slime-979,generation,402.9,78.4,7 +slime-979,generation,407.9,6.6,7 +slime-979,generation,412.9,14.9,7 +slime-979,generation,417.9,0.0,7 +slime-979,generation,422.9,0.0,7 +slime-979,generation,427.9,0.0,7 +slime-979,generation,432.9,21.4,7 +slime-979,generation,437.9,0.4,7 +slime-979,generation,442.9,3.4,7 +slime-979,generation,447.9,0.0,7 +slime-979,generation,452.9,61.4,7 +slime-979,generation,457.9,85.7,7 +slime-979,generation,462.9,85.7,7 +slime-979,generation,467.9,85.7,7 +slime-979,generation,472.9,85.7,7 +slime-979,generation,477.9,85.7,7 +slime-979,generation,482.9,85.7,7 +slime-979,generation,487.9,85.7,7 +slime-979,generation,492.9,85.7,7 +slime-979,generation,497.9,85.7,7 +slime-979,generation,502.9,85.7,7 +slime-979,generation,507.9,85.6,7 +slime-979,generation,512.9,85.7,7 +slime-979,generation,517.9,85.7,7 +slime-979,generation,522.9,85.7,7 +slime-979,generation,527.9,85.7,7 +slime-979,generation,532.9,85.7,7 +slime-979,generation,537.9,85.7,7 +slime-979,generation,542.9,85.7,7 +slime-979,generation,547.9,85.7,7 +slime-979,generation,552.9,85.7,7 +slime-979,generation,557.9,85.7,7 +slime-979,generation,562.9,85.7,7 +slime-979,generation,567.9,85.7,7 +slime-979,generation,572.9,85.7,7 +slime-979,generation,577.9,85.7,7 +slime-979,generation,582.9,85.7,7 +slime-979,generation,587.9,85.7,7 +slime-979,generation,592.9,85.7,7 +slime-979,generation,597.9,85.7,7 +slime-979,generation,602.9,85.7,7 +slime-979,generation,607.9,85.7,7 +slime-979,generation,612.9,85.7,7 +slime-979,generation,617.9,85.7,7 +slime-979,generation,622.9,85.7,7 +slime-979,generation,627.9,85.7,7 +slime-979,generation,632.9,85.7,7 +slime-979,generation,637.9,85.7,7 +slime-979,generation,642.9,85.7,7 +slime-979,generation,647.9,85.7,7 +slime-979,generation,652.9,85.7,7 +slime-979,generation,657.9,85.7,7 +slime-979,generation,662.9,85.7,7 +slime-979,generation,667.9,85.7,7 +slime-979,generation,672.9,85.7,7 +slime-979,generation,677.9,85.7,7 +slime-979,generation,682.9,85.7,7 +slime-979,generation,687.9,85.7,7 +slime-979,generation,692.9,28.6,7 +slime-979,generation,697.9,28.6,7 +slime-979,generation,702.9,28.6,7 +slime-979,generation,707.9,28.6,7 +slime-979,generation,712.9,0.0,7 +slime-979,generation,717.9,85.7,7 +slime-979,generation,722.9,85.7,7 +slime-979,generation,727.9,85.7,7 +slime-979,generation,732.9,85.7,7 +slime-979,generation,737.9,85.7,7 +slime-979,generation,742.9,85.7,7 +slime-979,generation,747.9,85.7,7 +slime-979,generation,752.9,85.7,7 +slime-979,generation,757.9,85.7,7 +slime-979,generation,762.9,85.7,7 +slime-979,generation,767.9,85.7,7 +slime-979,generation,772.9,85.7,7 +slime-979,generation,777.9,90.3,7 +slime-979,generation,782.9,85.7,7 +slime-979,generation,787.9,85.7,7 +slime-979,generation,792.9,85.7,7 +slime-979,generation,797.9,85.7,7 +slime-979,generation,802.9,85.7,7 +slime-979,generation,807.9,85.7,7 +slime-979,generation,812.9,93.6,7 +slime-979,generation,817.9,85.7,7 +slime-979,generation,822.9,96.9,7 +slime-979,generation,827.9,85.7,7 +slime-979,generation,832.9,85.7,7 +slime-979,generation,837.9,94.1,7 +slime-979,generation,842.9,85.7,7 +slime-979,generation,847.9,85.7,7 +slime-979,generation,852.9,85.7,7 +slime-979,generation,857.9,99.7,7 +slime-979,generation,862.9,85.7,7 +slime-979,generation,867.9,85.7,7 +slime-979,generation,872.9,85.7,7 +slime-979,generation,877.9,57.1,7 +slime-979,generation,882.9,28.6,7 +slime-979,generation,887.9,28.6,7 +slime-979,generation,892.9,84.6,7 +slime-979,generation,897.9,85.7,7 +slime-979,generation,902.9,85.7,7 +slime-979,generation,907.9,85.7,7 +slime-979,generation,912.9,85.7,7 +slime-979,generation,917.9,85.7,7 +slime-979,generation,922.9,93.7,7 +slime-979,generation,927.9,85.7,7 +slime-979,generation,932.9,92.6,7 +slime-979,generation,937.9,96.0,7 +slime-979,generation,942.9,85.7,7 +slime-979,generation,947.9,85.7,7 +slime-979,generation,952.9,85.7,7 +slime-979,generation,957.9,85.7,7 +slime-979,generation,962.9,85.7,7 +slime-979,generation,967.9,85.7,7 +slime-979,generation,972.9,86.7,7 +slime-979,generation,977.9,85.7,7 +slime-979,generation,982.9,85.7,7 +slime-979,generation,987.9,85.7,7 +slime-979,generation,992.9,98.6,7 +slime-979,generation,997.9,85.7,7 +slime-979,generation,1002.9,100.0,7 +slime-979,generation,1007.9,85.7,7 +slime-979,generation,1012.9,85.7,7 +slime-979,generation,1017.9,90.6,7 +slime-979,generation,1022.9,85.7,7 +slime-979,generation,1027.9,85.7,7 +slime-979,generation,1032.9,85.7,7 +slime-979,generation,1037.9,85.7,7 +slime-979,generation,1042.9,85.7,7 +slime-979,generation,1047.9,100.0,7 +slime-979,generation,1052.9,100.0,7 +slime-979,generation,1057.9,85.7,7 +slime-979,generation,1062.9,85.7,7 +slime-979,generation,1067.9,85.7,7 +slime-979,generation,1072.9,28.6,7 +slime-979,generation,1077.9,0.0,7 +slime-979,generation,1082.9,0.0,7 +slime-979,generation,1087.9,0.0,7 +slime-979,generation,1092.9,0.0,7 +slime-979,generation,1097.9,0.0,7 +slime-979,generation,1102.9,0.0,7 +slime-979,generation,1107.9,0.0,7 +slime-979,generation,1112.9,0.0,7 +slime-979,generation,1118.0,0.0,7 +slime-979,generation,1123.0,0.0,7 +slime-979,generation,1128.0,0.0,7 +slime-979,generation,1133.0,0.0,7 +slime-979,generation,1138.0,0.0,7 +slime-979,generation,1143.0,0.0,7 +slime-979,generation,1148.0,0.0,7 +slime-979,generation,1153.0,0.0,7 +slime-979,generation,1158.0,0.0,7 +slime-979,generation,1163.0,0.0,7 +slime-979,generation,1168.0,84.9,7 +slime-979,generation,1173.0,85.7,7 +slime-979,generation,1178.0,85.7,7 +slime-979,generation,1183.0,85.7,7 +slime-979,generation,1188.0,85.7,7 +slime-979,generation,1193.0,85.7,7 +slime-979,generation,1198.0,85.7,7 +slime-979,generation,1203.0,85.7,7 +slime-979,generation,1208.0,85.7,7 +slime-979,generation,1213.0,85.7,7 +slime-979,generation,1218.0,85.7,7 +slime-979,generation,1223.0,85.7,7 +slime-979,generation,1228.0,85.7,7 +slime-979,generation,1233.0,85.7,7 +slime-979,generation,1238.0,85.7,7 +slime-979,generation,1243.0,85.7,7 +slime-979,generation,1248.0,86.4,7 +slime-979,generation,1253.0,100.0,7 +slime-979,generation,1258.0,97.0,7 +slime-979,generation,1263.0,85.7,7 +slime-979,generation,1268.0,85.7,7 +slime-979,generation,1273.0,90.1,7 +slime-979,generation,1278.0,85.7,7 +slime-979,generation,1283.0,85.7,7 +slime-979,generation,1288.0,92.3,7 +slime-979,generation,1293.0,85.7,7 +slime-979,generation,1298.0,99.4,7 +slime-979,generation,1303.0,85.7,7 +slime-979,generation,1308.0,85.7,7 +slime-979,generation,1313.0,85.7,7 +slime-979,generation,1318.0,85.7,7 +slime-979,generation,1323.0,85.7,7 +slime-979,generation,1328.0,85.7,7 +slime-979,generation,1333.0,85.7,7 +slime-979,generation,1338.0,85.7,7 +slime-979,generation,1343.0,42.9,7 +slime-979,generation,1348.0,28.6,7 +slime-979,generation,1353.0,28.6,7 +slime-979,generation,1358.0,28.6,7 +slime-979,generation,1363.0,28.6,7 +slime-979,generation,1368.0,0.0,7 +slime-979,generation,1373.0,85.7,7 +slime-979,generation,1378.0,85.7,7 +slime-979,generation,1383.0,85.7,7 +slime-979,generation,1388.0,85.7,7 +slime-979,generation,1393.0,85.7,7 +slime-979,generation,1398.0,85.7,7 +slime-979,generation,1403.0,87.9,7 +slime-979,generation,1408.0,85.7,7 +slime-979,generation,1413.0,85.7,7 +slime-979,generation,1418.0,85.7,7 +slime-979,generation,1423.0,87.1,7 +slime-979,generation,1428.0,85.7,7 +slime-979,generation,1433.0,85.7,7 +slime-979,generation,1438.0,85.7,7 +slime-979,generation,1443.0,92.4,7 +slime-979,generation,1448.0,85.7,7 +slime-979,generation,1453.0,100.0,7 +slime-979,generation,1458.0,87.9,7 +slime-979,generation,1463.0,85.7,7 +slime-979,generation,1468.0,85.7,7 +slime-979,generation,1473.0,85.7,7 +slime-979,generation,1478.0,100.0,7 +slime-979,generation,1483.0,85.7,7 +slime-979,generation,1488.0,85.7,7 +slime-979,generation,1493.0,85.7,7 +slime-979,generation,1498.0,85.7,7 +slime-979,generation,1503.0,85.7,7 +slime-979,generation,1508.0,85.7,7 +slime-979,generation,1513.0,85.7,7 +slime-979,generation,1518.0,85.7,7 +slime-979,generation,1523.0,85.7,7 +slime-979,generation,1528.0,85.7,7 +slime-979,generation,1533.0,85.7,7 +slime-979,generation,1538.0,85.7,7 +slime-979,generation,1543.0,57.1,7 +slime-979,generation,1548.0,57.1,7 +slime-979,generation,1553.0,28.6,7 +slime-979,generation,1558.0,28.6,7 +slime-979,generation,1563.0,0.0,7 +slime-979,generation,1568.0,85.7,7 +slime-979,generation,1573.0,85.7,7 +slime-979,generation,1578.0,85.7,7 +slime-979,generation,1583.0,85.7,7 +slime-979,generation,1588.0,85.7,7 +slime-979,generation,1593.0,85.7,7 +slime-979,generation,1598.0,85.7,7 +slime-979,generation,1603.0,85.7,7 +slime-979,generation,1608.0,85.7,7 +slime-979,generation,1613.0,85.7,7 +slime-979,generation,1618.0,85.7,7 +slime-979,generation,1623.0,95.4,7 +slime-979,generation,1628.0,85.7,7 +slime-979,generation,1633.0,85.7,7 +slime-979,generation,1638.0,88.9,7 +slime-979,generation,1643.0,85.7,7 +slime-979,generation,1648.0,85.7,7 +slime-979,generation,1653.0,85.7,7 +slime-979,generation,1658.0,94.4,7 +slime-979,generation,1663.0,85.7,7 +slime-979,generation,1668.0,95.1,7 +slime-979,generation,1673.0,85.7,7 +slime-979,generation,1678.0,85.7,7 +slime-979,generation,1683.0,85.7,7 +slime-979,generation,1688.0,100.0,7 +slime-979,generation,1693.0,96.6,7 +slime-979,generation,1698.0,85.7,7 +slime-979,generation,1703.0,85.7,7 +slime-979,generation,1708.0,100.0,7 +slime-979,generation,1713.0,57.1,7 +slime-979,generation,1718.0,57.1,7 +slime-979,generation,1723.0,57.1,7 +slime-979,generation,1728.0,57.1,7 +slime-979,generation,1733.0,57.1,7 +slime-979,generation,1738.0,57.1,7 +slime-979,generation,1743.0,57.1,7 +slime-979,generation,1748.0,28.6,7 +slime-979,generation,1753.0,28.6,7 +slime-979,generation,1758.0,14.1,7 +slime-979,generation,1763.0,62.1,7 +slime-979,generation,1768.0,85.7,7 +slime-979,generation,1773.0,85.7,7 +slime-979,generation,1778.0,85.7,7 +slime-979,generation,1783.0,85.7,7 +slime-979,generation,1788.0,85.7,7 +slime-979,generation,1793.0,85.7,7 +slime-979,generation,1798.0,85.7,7 +slime-979,generation,1803.0,85.7,7 +slime-979,generation,1808.0,90.0,7 +slime-979,generation,1813.0,85.7,7 +slime-979,generation,1818.0,85.7,7 +slime-979,generation,1823.0,85.7,7 +slime-979,generation,1828.0,85.7,7 +slime-979,generation,1833.0,85.7,7 +slime-979,generation,1838.0,85.7,7 +slime-979,generation,1843.0,85.7,7 +slime-979,generation,1848.0,85.7,7 +slime-979,generation,1853.0,85.7,7 +slime-979,generation,1858.0,85.7,7 +slime-979,generation,1863.0,85.7,7 +slime-979,generation,1868.0,85.7,7 +slime-979,generation,1873.0,85.7,7 +slime-979,generation,1878.0,85.7,7 +slime-979,generation,1883.0,85.7,7 +slime-979,generation,1888.0,89.3,7 +slime-979,generation,1893.0,85.7,7 +slime-979,generation,1898.0,85.7,7 +slime-979,generation,1903.0,85.7,7 +slime-979,generation,1908.0,85.7,7 +slime-979,generation,1913.0,85.7,7 +slime-979,generation,1918.0,85.7,7 +slime-979,generation,1923.0,57.1,7 +slime-979,generation,1928.0,57.1,7 +slime-979,generation,1933.0,57.1,7 +slime-979,generation,1938.0,57.1,7 +slime-979,generation,1943.0,28.6,7 +slime-979,generation,1948.0,28.6,7 +slime-979,generation,1953.0,60.6,7 +slime-979,generation,1958.0,85.7,7 +slime-979,generation,1963.0,85.7,7 +slime-979,generation,1968.0,85.7,7 +slime-979,generation,1973.0,85.7,7 +slime-979,generation,1978.0,85.7,7 +slime-979,generation,1983.0,85.7,7 +slime-979,generation,1988.0,85.7,7 +slime-979,generation,1993.0,85.7,7 +slime-979,generation,1998.0,96.7,7 +slime-979,generation,2003.0,85.7,7 +slime-979,generation,2008.0,85.7,7 +slime-979,generation,2013.0,85.7,7 +slime-979,generation,2018.0,97.4,7 +slime-979,generation,2023.0,99.9,7 +slime-979,generation,2028.0,85.7,7 +slime-979,generation,2033.0,85.7,7 +slime-979,generation,2038.0,96.6,7 +slime-979,generation,2043.0,85.7,7 +slime-979,generation,2048.0,85.7,7 +slime-979,generation,2053.0,100.0,7 +slime-979,generation,2058.0,85.7,7 +slime-979,generation,2063.0,85.7,7 +slime-979,generation,2068.0,90.4,7 +slime-979,generation,2073.0,85.7,7 +slime-979,generation,2078.0,85.7,7 +slime-979,generation,2083.0,85.7,7 +slime-979,generation,2088.0,85.7,7 +slime-979,generation,2093.1,85.7,7 +slime-979,generation,2098.1,85.7,7 +slime-979,generation,2103.1,65.4,7 +slime-979,generation,2108.1,57.1,7 +slime-979,generation,2113.1,57.1,7 +slime-979,generation,2118.1,57.1,7 +slime-979,generation,2123.1,57.1,7 +slime-979,generation,2128.1,57.1,7 +slime-979,generation,2133.1,57.1,7 +slime-979,generation,2138.1,42.1,7 +slime-979,generation,2143.1,0.0,7 +slime-979,generation,2148.1,85.7,7 +slime-979,generation,2153.1,85.7,7 +slime-979,generation,2158.1,85.7,7 +slime-979,generation,2163.1,85.7,7 +slime-979,generation,2168.1,85.7,7 +slime-979,generation,2173.1,85.7,7 +slime-979,generation,2178.1,85.7,7 +slime-979,generation,2183.1,88.9,7 +slime-979,generation,2188.1,85.7,7 +slime-979,generation,2193.1,85.7,7 +slime-979,generation,2198.1,94.9,7 +slime-979,generation,2203.1,85.7,7 +slime-979,generation,2208.1,85.7,7 +slime-979,generation,2213.1,85.7,7 +slime-979,generation,2218.1,85.7,7 +slime-979,generation,2223.1,85.7,7 +slime-979,generation,2228.1,85.7,7 +slime-979,generation,2233.1,85.7,7 +slime-979,generation,2238.1,91.6,7 +slime-979,generation,2243.1,86.1,7 +slime-979,generation,2248.1,100.0,7 +slime-979,generation,2253.1,92.9,7 +slime-979,generation,2258.1,85.7,7 +slime-979,generation,2263.1,85.7,7 +slime-979,generation,2268.1,85.7,7 +slime-979,generation,2273.1,97.0,7 +slime-979,generation,2278.1,85.7,7 +slime-979,generation,2283.1,85.7,7 +slime-979,generation,2288.1,87.9,7 +slime-979,generation,2293.1,85.7,7 +slime-979,generation,2298.1,100.0,7 +slime-979,generation,2303.1,85.7,7 +slime-979,generation,2308.1,85.7,7 +slime-979,generation,2313.1,57.1,7 +slime-979,generation,2318.1,57.1,7 +slime-979,generation,2323.1,49.6,7 +slime-979,generation,2328.1,28.6,7 +slime-979,generation,2333.1,28.6,7 +slime-979,generation,2338.1,0.0,7 +slime-979,generation,2343.1,30.6,7 +slime-979,generation,2348.1,85.7,7 +slime-979,generation,2353.1,89.6,7 +slime-979,generation,2358.1,85.7,7 +slime-979,generation,2363.1,85.7,7 +slime-979,generation,2368.1,89.1,7 +slime-979,generation,2373.1,85.7,7 +slime-979,generation,2378.1,85.7,7 +slime-979,generation,2383.1,85.7,7 +slime-979,generation,2388.1,93.0,7 +slime-979,generation,2393.1,85.7,7 +slime-979,generation,2398.1,85.7,7 +slime-979,generation,2403.1,96.1,7 +slime-979,generation,2408.1,85.7,7 +slime-979,generation,2413.1,85.7,7 +slime-979,generation,2418.1,85.7,7 +slime-979,generation,2423.1,85.7,7 +slime-979,generation,2428.1,85.7,7 +slime-979,generation,2433.1,85.7,7 +slime-979,generation,2438.1,85.7,7 +slime-979,generation,2443.1,85.7,7 +slime-979,generation,2448.1,85.7,7 +slime-979,generation,2453.1,97.6,7 +slime-979,generation,2458.1,85.7,7 +slime-979,generation,2463.1,85.7,7 +slime-979,generation,2468.1,85.7,7 +slime-979,generation,2473.1,99.7,7 +slime-979,generation,2478.1,85.7,7 +slime-979,generation,2483.1,85.7,7 +slime-979,generation,2488.1,85.7,7 +slime-979,generation,2493.1,85.7,7 +slime-979,generation,2498.1,85.7,7 +slime-979,generation,2503.1,85.7,7 +slime-979,generation,2508.1,85.7,7 +slime-979,generation,2513.1,85.7,7 +slime-979,generation,2518.1,42.9,7 +slime-979,generation,2523.1,28.6,7 +slime-979,generation,2528.1,28.6,7 +slime-979,generation,2533.1,0.0,7 +slime-979,generation,2538.1,85.7,7 +slime-979,generation,2543.1,86.9,7 +slime-979,generation,2548.1,85.7,7 +slime-979,generation,2553.1,85.7,7 +slime-979,generation,2558.1,85.7,7 +slime-979,generation,2563.1,85.7,7 +slime-979,generation,2568.1,85.7,7 +slime-979,generation,2573.1,85.7,7 +slime-979,generation,2578.1,85.7,7 +slime-979,generation,2583.1,85.7,7 +slime-979,generation,2588.1,97.4,7 +slime-979,generation,2593.1,85.7,7 +slime-979,generation,2598.1,90.4,7 +slime-979,generation,2603.1,85.7,7 +slime-979,generation,2608.1,85.7,7 +slime-979,generation,2613.1,85.7,7 +slime-979,generation,2618.1,85.7,7 +slime-979,generation,2623.1,85.7,7 +slime-979,generation,2628.1,85.7,7 +slime-979,generation,2633.1,85.7,7 +slime-979,generation,2638.1,86.7,7 +slime-979,generation,2643.1,85.7,7 +slime-979,generation,2648.1,85.7,7 +slime-979,generation,2653.1,85.7,7 +slime-979,generation,2658.1,85.7,7 +slime-979,generation,2663.1,85.7,7 +slime-979,generation,2668.1,85.7,7 +slime-979,generation,2673.1,85.7,7 +slime-979,generation,2678.1,85.7,7 +slime-979,generation,2683.1,85.7,7 +slime-979,generation,2688.1,85.7,7 +slime-979,generation,2693.1,85.7,7 +slime-979,generation,2698.1,85.7,7 +slime-979,generation,2703.1,85.7,7 +slime-979,generation,2708.1,85.7,7 +slime-979,generation,2713.1,42.9,7 +slime-979,generation,2718.1,14.3,7 +slime-979,generation,2723.1,62.7,7 +slime-979,generation,2728.1,85.7,7 +slime-979,generation,2733.1,85.7,7 +slime-979,generation,2738.1,85.7,7 +slime-979,generation,2743.1,85.7,7 +slime-979,generation,2748.1,85.7,7 +slime-979,generation,2753.1,85.7,7 +slime-979,generation,2758.1,85.7,7 +slime-979,generation,2763.1,86.1,7 +slime-979,generation,2768.1,85.7,7 +slime-979,generation,2773.1,98.0,7 +slime-979,generation,2778.1,85.7,7 +slime-979,generation,2783.1,85.7,7 +slime-979,generation,2788.1,94.1,7 +slime-979,generation,2793.1,85.7,7 +slime-979,generation,2798.1,86.7,7 +slime-979,generation,2803.1,85.7,7 +slime-979,generation,2808.1,85.7,7 +slime-979,generation,2813.1,85.7,7 +slime-979,generation,2818.1,97.9,7 +slime-979,generation,2823.1,85.7,7 +slime-979,generation,2828.1,85.7,7 +slime-979,generation,2833.1,98.7,7 +slime-979,generation,2838.1,85.7,7 +slime-979,generation,2843.1,97.7,7 +slime-979,generation,2848.1,85.7,7 +slime-979,generation,2853.1,85.7,7 +slime-979,generation,2858.1,85.7,7 +slime-979,generation,2863.1,100.0,7 +slime-979,generation,2868.1,85.7,7 +slime-979,generation,2873.1,85.7,7 +slime-979,generation,2878.1,85.7,7 +slime-979,generation,2883.1,85.7,7 +slime-979,generation,2888.1,85.7,7 +slime-979,generation,2893.1,85.7,7 +slime-979,generation,2898.1,57.1,7 +slime-979,generation,2903.1,57.1,7 +slime-979,generation,2908.1,28.6,7 +slime-979,generation,2913.1,0.0,7 +slime-979,generation,2918.1,84.3,7 +slime-979,generation,2923.1,85.7,7 +slime-979,generation,2928.1,85.7,7 +slime-979,generation,2933.1,85.7,7 +slime-979,generation,2938.1,85.7,7 +slime-979,generation,2943.1,85.7,7 +slime-979,generation,2948.1,85.7,7 +slime-979,generation,2953.1,85.7,7 +slime-979,generation,2958.1,85.7,7 +slime-979,generation,2963.1,85.7,7 +slime-979,generation,2968.1,85.7,7 +slime-979,generation,2973.1,85.7,7 +slime-979,generation,2978.1,85.7,7 +slime-979,generation,2983.1,85.7,7 +slime-979,generation,2988.1,85.7,7 +slime-979,generation,2993.1,85.7,7 +slime-979,generation,2998.1,85.7,7 +slime-979,generation,3003.1,85.7,7 +slime-979,generation,3008.1,85.7,7 +slime-979,generation,3013.1,85.7,7 +slime-979,generation,3018.1,85.7,7 +slime-979,generation,3023.1,85.7,7 +slime-979,generation,3028.1,85.7,7 +slime-979,generation,3033.1,85.7,7 +slime-979,generation,3038.1,85.7,7 +slime-979,generation,3043.1,85.7,7 +slime-979,generation,3048.1,85.7,7 +slime-979,generation,3053.1,85.7,7 +slime-979,generation,3058.1,85.7,7 +slime-979,generation,3063.1,85.7,7 +slime-979,generation,3068.1,85.7,7 +slime-979,generation,3073.1,85.7,7 +slime-979,generation,3078.1,85.7,7 +slime-979,generation,3083.2,85.7,7 +slime-979,generation,3088.2,85.7,7 +slime-979,generation,3093.2,85.7,7 +slime-979,generation,3098.2,85.7,7 +slime-979,generation,3103.2,85.7,7 +slime-979,generation,3108.2,85.7,7 +slime-979,generation,3113.2,85.7,7 +slime-979,generation,3118.2,85.7,7 +slime-979,generation,3123.2,85.7,7 +slime-979,generation,3128.2,85.7,7 +slime-979,generation,3133.2,57.1,7 +slime-979,generation,3138.2,57.1,7 +slime-979,generation,3143.2,57.1,7 +slime-979,generation,3148.2,57.1,7 +slime-979,generation,3153.2,57.1,7 +slime-979,generation,3158.2,5.4,7 +slime-979,generation,3163.2,85.7,7 +slime-979,generation,3168.2,85.7,7 +slime-979,generation,3173.2,85.7,7 +slime-979,generation,3178.2,85.7,7 +slime-979,generation,3183.2,85.9,7 +slime-979,generation,3188.2,85.7,7 +slime-979,generation,3193.2,85.7,7 +slime-979,generation,3198.2,92.0,7 +slime-979,generation,3203.2,85.7,7 +slime-979,generation,3208.2,85.7,7 +slime-979,generation,3213.2,85.7,7 +slime-979,generation,3218.2,85.7,7 +slime-979,generation,3223.2,85.7,7 +slime-979,generation,3228.2,85.7,7 +slime-979,generation,3233.2,97.7,7 +slime-979,generation,3238.2,97.9,7 +slime-979,generation,3243.2,85.7,7 +slime-979,generation,3248.2,85.7,7 +slime-979,generation,3253.2,100.0,7 +slime-979,generation,3258.2,85.7,7 +slime-979,generation,3263.2,85.7,7 +slime-979,generation,3268.2,98.6,7 +slime-979,generation,3273.2,85.7,7 +slime-979,generation,3278.2,85.7,7 +slime-979,generation,3283.2,85.7,7 +slime-979,generation,3288.2,85.7,7 +slime-979,generation,3293.2,85.7,7 +slime-979,generation,3298.2,85.7,7 +slime-979,generation,3303.2,85.7,7 +slime-979,generation,3308.2,85.7,7 +slime-979,generation,3313.2,100.0,7 +slime-979,generation,3318.2,85.7,7 +slime-979,generation,3323.2,85.7,7 +slime-979,generation,3328.2,85.7,7 +slime-979,generation,3333.2,57.1,7 +slime-979,generation,3338.2,28.6,7 +slime-979,generation,3343.2,28.6,7 +slime-979,generation,3348.2,28.6,7 +slime-979,generation,3353.2,0.0,7 +slime-979,generation,3358.2,85.4,7 +slime-979,generation,3363.2,85.7,7 +slime-979,generation,3368.2,85.7,7 +slime-979,generation,3373.2,85.7,7 +slime-979,generation,3378.2,85.7,7 +slime-979,generation,3383.2,85.7,7 +slime-979,generation,3388.2,85.7,7 +slime-979,generation,3393.2,85.7,7 +slime-979,generation,3398.2,85.7,7 +slime-979,generation,3403.2,85.7,7 +slime-979,generation,3408.2,85.7,7 +slime-979,generation,3413.2,85.7,7 +slime-979,generation,3418.2,85.7,7 +slime-979,generation,3423.2,85.7,7 +slime-979,generation,3428.2,85.7,7 +slime-979,generation,3433.2,85.7,7 +slime-979,generation,3438.2,85.7,7 +slime-979,generation,3443.2,85.7,7 +slime-979,generation,3448.2,85.7,7 +slime-979,generation,3453.2,85.7,7 +slime-979,generation,3458.2,85.7,7 +slime-979,generation,3463.2,85.7,7 +slime-979,generation,3468.2,85.7,7 +slime-979,generation,3473.2,85.7,7 +slime-979,generation,3478.2,85.7,7 +slime-979,generation,3483.2,100.0,7 +slime-979,generation,3488.2,85.7,7 +slime-979,generation,3493.2,85.7,7 +slime-979,generation,3498.2,85.7,7 +slime-979,generation,3503.2,85.7,7 +slime-979,generation,3508.2,85.7,7 +slime-979,generation,3513.2,94.6,7 +slime-979,generation,3518.2,85.7,7 +slime-979,generation,3523.2,85.7,7 +slime-979,generation,3528.2,85.7,7 +slime-979,generation,3533.2,85.7,7 +slime-979,generation,3538.2,85.7,7 +slime-979,generation,3543.2,70.6,7 +slime-979,generation,3548.2,28.6,7 +slime-979,generation,3553.2,13.1,7 +slime-979,generation,3558.2,59.1,7 +slime-979,generation,3563.2,85.7,7 +slime-979,generation,3568.2,85.7,7 +slime-979,generation,3573.2,85.7,7 +slime-979,generation,3578.2,85.7,7 +slime-979,generation,3583.2,85.7,7 +slime-979,generation,3588.2,85.7,7 +slime-979,generation,3593.2,85.7,7 +slime-979,generation,3598.2,85.7,7 +slime-979,generation,3603.2,85.7,7 +slime-979,generation,3608.2,85.7,7 +slime-979,generation,3613.2,92.3,7 +slime-979,generation,3618.2,85.7,7 +slime-979,generation,3623.2,85.7,7 +slime-979,generation,3628.2,85.7,7 +slime-979,generation,3633.2,85.7,7 +slime-979,generation,3638.2,93.7,7 +slime-979,generation,3643.2,85.7,7 +slime-979,generation,3648.2,85.7,7 +slime-979,generation,3653.2,100.0,7 +slime-979,generation,3658.2,85.7,7 +slime-979,generation,3663.2,85.7,7 +slime-979,generation,3668.2,85.7,7 +slime-979,generation,3673.2,85.7,7 +slime-979,generation,3678.2,98.9,7 +slime-979,generation,3683.2,85.7,7 +slime-979,generation,3688.2,95.4,7 +slime-979,generation,3693.2,100.0,7 +slime-979,generation,3698.2,85.7,7 +slime-979,generation,3703.2,85.7,7 +slime-979,generation,3708.2,89.6,7 +slime-979,generation,3713.2,85.7,7 +slime-979,generation,3718.2,85.7,7 +slime-979,generation,3723.2,85.7,7 +slime-979,generation,3728.2,85.7,7 +slime-979,generation,3733.2,57.1,7 +slime-979,generation,3738.2,57.1,7 +slime-979,generation,3743.2,57.1,7 +slime-979,generation,3748.2,28.6,7 +slime-979,generation,3753.2,0.0,7 +slime-979,generation,3758.2,1.9,7 +slime-979,generation,3763.2,85.7,7 +slime-979,generation,3768.2,85.7,7 +slime-979,generation,3773.2,85.7,7 +slime-979,generation,3778.2,85.7,7 +slime-979,generation,3783.2,85.7,7 +slime-979,generation,3788.2,85.7,7 +slime-979,generation,3793.2,85.7,7 +slime-979,generation,3798.2,85.7,7 +slime-979,generation,3803.2,85.7,7 +slime-979,generation,3808.2,97.0,7 +slime-979,generation,3813.2,85.7,7 +slime-979,generation,3818.2,85.7,7 +slime-979,generation,3823.2,97.4,7 +slime-979,generation,3828.2,85.7,7 +slime-979,generation,3833.2,85.7,7 +slime-979,generation,3838.2,85.7,7 +slime-979,generation,3843.2,85.7,7 +slime-979,generation,3848.2,85.7,7 +slime-979,generation,3853.2,85.7,7 +slime-979,generation,3858.2,85.7,7 +slime-979,generation,3863.2,85.7,7 +slime-979,generation,3868.2,100.0,7 +slime-979,generation,3873.2,85.7,7 +slime-979,generation,3878.2,88.4,7 +slime-979,generation,3883.2,99.7,7 +slime-979,generation,3888.2,85.7,7 +slime-979,generation,3893.2,85.7,7 +slime-979,generation,3898.2,100.0,7 +slime-979,generation,3903.2,89.6,7 +slime-979,generation,3908.2,85.7,7 +slime-979,generation,3913.2,85.7,7 +slime-979,generation,3918.2,85.7,7 +slime-979,generation,3923.2,85.7,7 +slime-979,generation,3928.2,85.7,7 +slime-979,generation,3933.2,85.7,7 +slime-979,generation,3938.2,85.7,7 +slime-979,generation,3943.2,85.7,7 +slime-979,generation,3948.2,34.0,7 +slime-979,generation,3953.2,0.0,7 +slime-979,generation,3958.2,85.7,7 +slime-979,generation,3963.2,85.7,7 +slime-979,generation,3968.2,85.7,7 +slime-979,generation,3973.2,85.7,7 +slime-979,generation,3978.2,85.7,7 +slime-979,generation,3983.2,85.7,7 +slime-979,generation,3988.2,85.7,7 +slime-979,generation,3993.2,85.7,7 +slime-979,generation,3998.3,85.7,7 +slime-979,generation,4003.3,85.7,7 +slime-979,generation,4008.3,85.7,7 +slime-979,generation,4013.3,91.7,7 +slime-979,generation,4018.3,85.7,7 +slime-979,generation,4023.3,85.7,7 +slime-979,generation,4028.3,85.7,7 +slime-979,generation,4033.3,85.7,7 +slime-979,generation,4038.3,85.7,7 +slime-979,generation,4043.3,85.7,7 +slime-979,generation,4048.3,85.7,7 +slime-979,generation,4053.3,85.7,7 +slime-979,generation,4058.3,85.7,7 +slime-979,generation,4063.3,90.4,7 +slime-979,generation,4068.3,85.7,7 +slime-979,generation,4073.3,85.7,7 +slime-979,generation,4078.3,85.7,7 +slime-979,generation,4083.3,85.7,7 +slime-979,generation,4088.3,85.7,7 +slime-979,generation,4093.3,85.7,7 +slime-979,generation,4098.3,85.7,7 +slime-979,generation,4103.3,85.7,7 +slime-979,generation,4108.3,85.7,7 +slime-979,generation,4113.3,85.7,7 +slime-979,generation,4118.3,100.0,7 +slime-979,generation,4123.3,85.7,7 +slime-979,generation,4128.3,85.7,7 +slime-979,generation,4133.3,57.1,7 +slime-979,generation,4138.3,37.7,7 +slime-979,generation,4143.3,28.6,7 +slime-979,generation,4148.3,28.6,7 +slime-979,generation,4153.3,28.6,7 +slime-979,generation,4158.3,0.0,7 +slime-979,generation,4163.3,85.7,7 +slime-979,generation,4168.3,85.7,7 +slime-979,generation,4173.3,85.7,7 +slime-979,generation,4178.3,85.7,7 +slime-979,generation,4183.3,85.7,7 +slime-979,generation,4188.3,85.7,7 +slime-979,generation,4193.3,85.7,7 +slime-979,generation,4198.3,85.7,7 +slime-979,generation,4203.3,85.7,7 +slime-979,generation,4208.3,85.7,7 +slime-979,generation,4213.3,85.7,7 +slime-979,generation,4218.3,85.7,7 +slime-979,generation,4223.3,85.7,7 +slime-979,generation,4228.3,98.6,7 +slime-979,generation,4233.3,85.7,7 +slime-979,generation,4238.3,85.7,7 +slime-979,generation,4243.3,97.0,7 +slime-979,generation,4248.3,85.7,7 +slime-979,generation,4253.3,85.7,7 +slime-979,generation,4258.3,85.7,7 +slime-979,generation,4263.3,85.7,7 +slime-979,generation,4268.3,85.7,7 +slime-979,generation,4273.3,85.7,7 +slime-979,generation,4278.3,85.7,7 +slime-979,generation,4283.3,85.7,7 +slime-979,generation,4288.3,85.7,7 +slime-979,generation,4293.3,85.7,7 +slime-979,generation,4298.3,85.7,7 +slime-979,generation,4303.3,85.7,7 +slime-979,generation,4308.3,100.0,7 +slime-979,generation,4313.3,86.1,7 +slime-979,generation,4318.3,85.7,7 +slime-979,generation,4323.3,85.7,7 +slime-979,generation,4328.3,85.7,7 +slime-979,generation,4333.3,85.7,7 +slime-979,generation,4338.3,99.9,7 +slime-979,generation,4343.3,28.6,7 +slime-979,generation,4348.3,16.9,7 +slime-979,generation,4353.3,62.7,7 +slime-979,generation,4358.3,85.7,7 +slime-979,generation,4363.3,85.7,7 +slime-979,generation,4368.3,85.7,7 +slime-979,generation,4373.3,85.7,7 +slime-979,generation,4378.3,87.3,7 +slime-979,generation,4383.3,85.7,7 +slime-979,generation,4388.3,85.7,7 +slime-979,generation,4393.3,85.7,7 +slime-979,generation,4398.3,85.7,7 +slime-979,generation,4403.3,85.7,7 +slime-979,generation,4408.3,85.7,7 +slime-979,generation,4413.3,99.6,7 +slime-979,generation,4418.3,93.3,7 +slime-979,generation,4423.3,85.7,7 +slime-979,generation,4428.3,85.7,7 +slime-979,generation,4433.3,85.7,7 +slime-979,generation,4438.3,85.7,7 +slime-979,generation,4443.3,85.7,7 +slime-979,generation,4448.3,87.6,7 +slime-979,generation,4453.3,85.7,7 +slime-979,generation,4458.3,85.7,7 +slime-979,generation,4463.3,85.7,7 +slime-979,generation,4468.3,97.1,7 +slime-979,generation,4473.3,99.0,7 +slime-979,generation,4478.3,85.7,7 +slime-979,generation,4483.3,85.7,7 +slime-979,generation,4488.3,85.7,7 +slime-979,generation,4493.3,85.7,7 +slime-979,generation,4498.3,85.7,7 +slime-979,generation,4503.3,85.7,7 +slime-979,generation,4508.3,100.0,7 +slime-979,generation,4513.3,85.7,7 +slime-979,generation,4518.3,85.7,7 +slime-979,generation,4523.3,85.7,7 +slime-979,generation,4528.3,57.1,7 +slime-979,generation,4533.3,64.0,7 +slime-979,generation,4538.3,57.1,7 +slime-979,generation,4543.3,57.1,7 +slime-979,generation,4548.3,58.6,7 +slime-979,generation,4553.3,28.6,7 +slime-979,generation,4558.3,14.3,7 +slime-979,generation,4563.3,62.4,7 +slime-979,generation,4568.3,85.7,7 +slime-979,generation,4573.3,85.7,7 +slime-979,generation,4578.3,85.7,7 +slime-979,generation,4583.3,85.7,7 +slime-979,generation,4588.3,85.7,7 +slime-979,generation,4593.3,85.7,7 +slime-979,generation,4598.3,85.7,7 +slime-979,generation,4603.3,85.7,7 +slime-979,generation,4608.3,85.7,7 +slime-979,generation,4613.3,85.7,7 +slime-979,generation,4618.3,85.7,7 +slime-979,generation,4623.3,85.7,7 +slime-979,generation,4628.3,85.7,7 +slime-979,generation,4633.3,99.7,7 +slime-979,generation,4638.3,85.7,7 +slime-979,generation,4643.3,85.7,7 +slime-979,generation,4648.3,85.7,7 +slime-979,generation,4653.3,85.7,7 +slime-979,generation,4658.3,85.7,7 +slime-979,generation,4663.3,85.7,7 +slime-979,generation,4668.3,85.7,7 +slime-979,generation,4673.3,85.7,7 +slime-979,generation,4678.3,85.7,7 +slime-979,generation,4683.3,85.7,7 +slime-979,generation,4688.3,85.7,7 +slime-979,generation,4693.3,85.7,7 +slime-979,generation,4698.3,85.7,7 +slime-979,generation,4703.3,85.7,7 +slime-979,generation,4708.3,85.7,7 +slime-979,generation,4713.3,85.7,7 +slime-979,generation,4718.3,100.0,7 +slime-979,generation,4723.3,85.7,7 +slime-979,generation,4728.3,100.0,7 +slime-979,generation,4733.3,92.0,7 +slime-979,generation,4738.3,85.7,7 +slime-979,generation,4743.3,60.3,7 +slime-979,generation,4748.3,28.6,7 +slime-979,generation,4753.3,14.3,7 +slime-979,generation,4758.3,60.7,7 +slime-979,generation,4763.3,85.7,7 +slime-979,generation,4768.3,85.7,7 +slime-979,generation,4773.3,85.7,7 +slime-979,generation,4778.3,85.7,7 +slime-979,generation,4783.3,85.7,7 +slime-979,generation,4788.3,85.7,7 +slime-979,generation,4793.3,85.7,7 +slime-979,generation,4798.3,85.7,7 +slime-979,generation,4803.3,85.7,7 +slime-979,generation,4808.3,85.7,7 +slime-979,generation,4813.3,85.7,7 +slime-979,generation,4818.3,85.7,7 +slime-979,generation,4823.3,85.7,7 +slime-979,generation,4828.3,86.1,7 +slime-979,generation,4833.3,85.7,7 +slime-979,generation,4838.3,85.7,7 +slime-979,generation,4843.3,85.7,7 +slime-979,generation,4848.3,97.4,7 +slime-979,generation,4853.3,85.7,7 +slime-979,generation,4858.3,85.7,7 +slime-979,generation,4863.3,85.7,7 +slime-979,generation,4868.3,85.7,7 +slime-979,generation,4873.3,99.0,7 +slime-979,generation,4878.3,85.7,7 +slime-979,generation,4883.3,85.7,7 +slime-979,generation,4888.3,85.7,7 +slime-979,generation,4893.3,85.7,7 +slime-979,generation,4898.3,98.3,7 +slime-979,generation,4903.3,85.7,7 +slime-979,generation,4908.3,85.7,7 +slime-979,generation,4913.3,85.7,7 +slime-979,generation,4918.3,85.7,7 +slime-979,generation,4923.3,57.1,7 +slime-979,generation,4928.3,57.1,7 +slime-979,generation,4933.3,57.1,7 +slime-979,generation,4938.3,57.1,7 +slime-979,generation,4943.3,42.9,7 +slime-979,generation,4948.3,0.0,7 +slime-979,generation,4953.3,0.0,7 +slime-979,generation,4958.3,0.0,7 +slime-979,generation,4963.3,0.0,7 +slime-979,generation,4968.3,0.0,7 +slime-979,generation,4973.3,0.0,7 +slime-979,generation,4978.3,0.0,7 +slime-979,generation,4983.3,0.0,7 +slime-979,generation,4988.3,0.0,7 +slime-979,generation,4993.3,0.0,7 +slime-979,generation,4998.3,0.0,7 +slime-979,generation,5003.3,0.0,7 +slime-979,generation,5008.3,0.0,7 +slime-979,generation,5013.3,0.0,7 +slime-979,generation,5018.4,0.0,7 +slime-979,generation,5023.4,0.0,7 +slime-979,generation,5028.4,0.0,7 +slime-979,generation,5033.4,0.0,7 +slime-979,generation,5038.4,0.0,7 +slime-979,generation,5043.4,0.0,7 +slime-979,generation,5048.4,0.0,7 +slime-979,generation,5053.4,0.0,7 +slime-979,generation,5058.4,0.0,7 +slime-979,generation,5063.4,26.6,7 +slime-979,generation,5068.4,85.7,7 +slime-979,generation,5073.4,85.7,7 +slime-979,generation,5078.4,85.7,7 +slime-979,generation,5083.4,85.7,7 +slime-979,generation,5088.4,85.7,7 +slime-979,generation,5093.4,85.7,7 +slime-979,generation,5098.4,85.7,7 +slime-979,generation,5103.4,85.7,7 +slime-979,generation,5108.4,85.7,7 +slime-979,generation,5113.4,85.7,7 +slime-979,generation,5118.4,85.7,7 +slime-979,generation,5123.4,85.7,7 +slime-979,generation,5128.4,85.7,7 +slime-979,generation,5133.4,85.7,7 +slime-979,generation,5138.4,85.7,7 +slime-979,generation,5143.4,85.7,7 +slime-979,generation,5148.4,85.7,7 +slime-979,generation,5153.4,85.7,7 +slime-979,generation,5158.4,85.7,7 +slime-979,generation,5163.4,85.7,7 +slime-979,generation,5168.4,85.7,7 +slime-979,generation,5173.4,85.7,7 +slime-979,generation,5178.4,85.7,7 +slime-979,generation,5183.4,85.7,7 +slime-979,generation,5188.4,85.7,7 +slime-979,generation,5193.4,85.7,7 +slime-979,generation,5198.4,85.7,7 +slime-979,generation,5203.4,85.7,7 +slime-979,generation,5208.4,85.7,7 +slime-979,generation,5213.4,85.7,7 +slime-979,generation,5218.4,85.7,7 +slime-979,generation,5223.4,85.7,7 +slime-979,generation,5228.4,85.7,7 +slime-979,generation,5233.4,85.7,7 +slime-979,generation,5238.4,85.7,7 +slime-979,generation,5243.4,85.7,7 +slime-979,generation,5248.4,85.7,7 +slime-979,generation,5253.4,85.7,7 +slime-979,generation,5258.4,85.7,7 +slime-979,generation,5263.4,85.7,7 +slime-979,generation,5268.4,85.7,7 +slime-979,generation,5273.4,85.7,7 +slime-979,generation,5278.4,85.7,7 +slime-979,generation,5283.4,57.1,7 +slime-979,generation,5288.4,57.1,7 +slime-979,generation,5293.4,57.1,7 +slime-979,generation,5298.4,57.1,7 +slime-979,generation,5303.4,57.1,7 +slime-979,generation,5308.4,28.6,7 +slime-979,generation,5313.4,28.6,7 +slime-979,generation,5318.4,0.0,7 +slime-979,generation,5323.4,0.0,7 +slime-979,generation,5328.4,0.0,7 +slime-979,generation,5333.4,0.0,7 +slime-979,generation,5338.4,0.0,7 +slime-979,generation,5343.4,0.0,7 +slime-979,learner,33.8,0.0,8 +slime-979,learner,38.8,0.0,8 +slime-979,learner,43.8,0.0,8 +slime-979,learner,48.8,0.0,8 +slime-979,learner,53.8,0.0,8 +slime-979,learner,58.8,0.0,8 +slime-979,learner,63.8,0.0,8 +slime-979,learner,68.8,0.0,8 +slime-979,learner,73.8,0.0,8 +slime-979,learner,78.8,0.0,8 +slime-979,learner,83.8,0.0,8 +slime-979,learner,88.8,0.0,8 +slime-979,learner,93.8,0.0,8 +slime-979,learner,98.8,0.0,8 +slime-979,learner,103.8,0.0,8 +slime-979,learner,108.8,0.0,8 +slime-979,learner,113.8,0.0,8 +slime-979,learner,118.8,0.0,8 +slime-979,learner,123.8,0.0,8 +slime-979,learner,128.8,0.0,8 +slime-979,learner,133.8,0.0,8 +slime-979,learner,138.8,0.0,8 +slime-979,learner,143.8,0.0,8 +slime-979,learner,148.8,0.0,8 +slime-979,learner,153.8,0.0,8 +slime-979,learner,158.8,0.0,8 +slime-979,learner,163.8,0.0,8 +slime-979,learner,168.8,0.0,8 +slime-979,learner,173.8,0.0,8 +slime-979,learner,178.8,0.0,8 +slime-979,learner,183.8,0.0,8 +slime-979,learner,188.8,0.0,8 +slime-979,learner,193.8,0.0,8 +slime-979,learner,198.8,0.0,8 +slime-979,learner,203.8,0.0,8 +slime-979,learner,208.8,0.0,8 +slime-979,learner,213.8,1.0,8 +slime-979,learner,218.8,0.0,8 +slime-979,learner,223.8,37.5,8 +slime-979,learner,228.8,0.0,8 +slime-979,learner,233.8,1.0,8 +slime-979,learner,238.8,1.1,8 +slime-979,learner,243.8,26.0,8 +slime-979,learner,248.8,37.6,8 +slime-979,learner,253.8,7.1,8 +slime-979,learner,258.8,0.0,8 +slime-979,learner,263.8,0.0,8 +slime-979,learner,268.8,0.0,8 +slime-979,learner,273.8,0.0,8 +slime-979,learner,278.8,0.0,8 +slime-979,learner,283.8,0.0,8 +slime-979,learner,288.8,0.0,8 +slime-979,learner,293.8,0.0,8 +slime-979,learner,298.8,0.0,8 +slime-979,learner,303.8,0.0,8 +slime-979,learner,308.8,0.0,8 +slime-979,learner,313.8,0.0,8 +slime-979,learner,318.8,0.0,8 +slime-979,learner,323.8,0.0,8 +slime-979,learner,328.8,0.0,8 +slime-979,learner,333.8,0.0,8 +slime-979,learner,338.8,0.0,8 +slime-979,learner,343.8,0.0,8 +slime-979,learner,348.8,0.0,8 +slime-979,learner,353.8,0.0,8 +slime-979,learner,358.8,0.0,8 +slime-979,learner,363.8,0.0,8 +slime-979,learner,368.8,0.0,8 +slime-979,learner,373.8,0.0,8 +slime-979,learner,378.8,0.0,8 +slime-979,learner,383.8,0.0,8 +slime-979,learner,388.8,0.0,8 +slime-979,learner,393.8,0.0,8 +slime-979,learner,398.8,0.0,8 +slime-979,learner,403.8,0.0,8 +slime-979,learner,408.8,0.0,8 +slime-979,learner,413.8,0.0,8 +slime-979,learner,418.8,0.0,8 +slime-979,learner,423.8,0.0,8 +slime-979,learner,428.8,0.0,8 +slime-979,learner,433.8,0.0,8 +slime-979,learner,438.8,0.0,8 +slime-979,learner,443.8,0.0,8 +slime-979,learner,448.8,0.0,8 +slime-979,learner,453.8,95.4,8 +slime-979,learner,458.8,0.0,8 +slime-979,learner,463.8,0.0,8 +slime-979,learner,468.8,0.0,8 +slime-979,learner,473.8,0.0,8 +slime-979,learner,478.8,0.0,8 +slime-979,learner,483.8,0.0,8 +slime-979,learner,488.8,0.0,8 +slime-979,learner,493.8,0.0,8 +slime-979,learner,498.8,0.0,8 +slime-979,learner,503.8,0.0,8 +slime-979,learner,508.8,0.0,8 +slime-979,learner,513.8,0.0,8 +slime-979,learner,518.8,0.0,8 +slime-979,learner,523.8,0.0,8 +slime-979,learner,528.8,0.0,8 +slime-979,learner,533.8,0.0,8 +slime-979,learner,538.8,0.0,8 +slime-979,learner,543.8,0.0,8 +slime-979,learner,548.8,0.0,8 +slime-979,learner,553.8,0.0,8 +slime-979,learner,558.8,0.0,8 +slime-979,learner,563.8,0.0,8 +slime-979,learner,568.8,0.0,8 +slime-979,learner,573.8,0.0,8 +slime-979,learner,578.8,0.0,8 +slime-979,learner,583.8,0.0,8 +slime-979,learner,588.8,0.0,8 +slime-979,learner,593.8,0.0,8 +slime-979,learner,598.8,0.0,8 +slime-979,learner,603.8,0.0,8 +slime-979,learner,608.8,0.0,8 +slime-979,learner,613.8,0.0,8 +slime-979,learner,618.8,0.0,8 +slime-979,learner,623.8,0.0,8 +slime-979,learner,628.8,0.0,8 +slime-979,learner,633.8,0.0,8 +slime-979,learner,638.8,0.0,8 +slime-979,learner,643.8,0.0,8 +slime-979,learner,648.8,0.0,8 +slime-979,learner,653.8,0.0,8 +slime-979,learner,658.8,0.0,8 +slime-979,learner,663.8,0.0,8 +slime-979,learner,668.8,0.0,8 +slime-979,learner,673.8,0.0,8 +slime-979,learner,678.8,0.0,8 +slime-979,learner,683.8,0.0,8 +slime-979,learner,688.8,0.0,8 +slime-979,learner,693.8,0.0,8 +slime-979,learner,698.8,0.0,8 +slime-979,learner,703.8,0.0,8 +slime-979,learner,708.8,0.0,8 +slime-979,learner,713.8,0.0,8 +slime-979,learner,718.8,0.0,8 +slime-979,learner,723.9,0.0,8 +slime-979,learner,728.9,0.0,8 +slime-979,learner,733.9,0.0,8 +slime-979,learner,738.9,0.0,8 +slime-979,learner,743.9,0.0,8 +slime-979,learner,748.9,0.0,8 +slime-979,learner,753.9,0.0,8 +slime-979,learner,758.9,0.0,8 +slime-979,learner,763.9,0.0,8 +slime-979,learner,768.9,0.0,8 +slime-979,learner,773.9,0.0,8 +slime-979,learner,778.9,0.0,8 +slime-979,learner,783.9,0.0,8 +slime-979,learner,788.9,0.0,8 +slime-979,learner,793.9,0.0,8 +slime-979,learner,798.9,0.0,8 +slime-979,learner,803.9,0.0,8 +slime-979,learner,808.9,0.0,8 +slime-979,learner,813.9,0.0,8 +slime-979,learner,818.9,0.0,8 +slime-979,learner,823.9,0.0,8 +slime-979,learner,828.9,0.0,8 +slime-979,learner,833.9,0.0,8 +slime-979,learner,838.9,0.0,8 +slime-979,learner,843.9,0.0,8 +slime-979,learner,848.9,0.0,8 +slime-979,learner,853.9,0.0,8 +slime-979,learner,858.9,0.0,8 +slime-979,learner,863.9,0.0,8 +slime-979,learner,868.9,0.0,8 +slime-979,learner,873.9,0.0,8 +slime-979,learner,878.9,0.0,8 +slime-979,learner,883.9,0.0,8 +slime-979,learner,888.9,0.0,8 +slime-979,learner,893.9,24.6,8 +slime-979,learner,898.9,50.0,8 +slime-979,learner,903.9,0.0,8 +slime-979,learner,908.9,0.0,8 +slime-979,learner,913.9,0.0,8 +slime-979,learner,918.9,24.0,8 +slime-979,learner,923.9,0.0,8 +slime-979,learner,928.9,0.0,8 +slime-979,learner,933.9,0.0,8 +slime-979,learner,938.9,0.1,8 +slime-979,learner,943.9,0.0,8 +slime-979,learner,948.9,23.1,8 +slime-979,learner,953.9,14.8,8 +slime-979,learner,958.9,0.0,8 +slime-979,learner,963.9,0.0,8 +slime-979,learner,968.9,98.8,8 +slime-979,learner,973.9,87.5,8 +slime-979,learner,978.9,87.5,8 +slime-979,learner,983.9,36.4,8 +slime-979,learner,988.9,84.9,8 +slime-979,learner,993.9,97.2,8 +slime-979,learner,998.9,84.6,8 +slime-979,learner,1003.9,88.1,8 +slime-979,learner,1008.9,74.4,8 +slime-979,learner,1013.9,87.1,8 +slime-979,learner,1018.9,1.4,8 +slime-979,learner,1023.9,14.9,8 +slime-979,learner,1028.9,25.1,8 +slime-979,learner,1033.9,3.9,8 +slime-979,learner,1038.9,11.2,8 +slime-979,learner,1043.9,7.5,8 +slime-979,learner,1048.9,0.0,8 +slime-979,learner,1053.9,0.0,8 +slime-979,learner,1058.9,0.0,8 +slime-979,learner,1063.9,0.0,8 +slime-979,learner,1068.9,0.0,8 +slime-979,learner,1073.9,0.0,8 +slime-979,learner,1078.9,0.0,8 +slime-979,learner,1083.9,12.9,8 +slime-979,learner,1088.9,11.0,8 +slime-979,learner,1093.9,87.2,8 +slime-979,learner,1098.9,12.8,8 +slime-979,learner,1103.9,91.2,8 +slime-979,learner,1108.9,87.9,8 +slime-979,learner,1113.9,75.0,8 +slime-979,learner,1118.9,97.0,8 +slime-979,learner,1123.9,75.0,8 +slime-979,learner,1128.9,98.5,8 +slime-979,learner,1133.9,83.1,8 +slime-979,learner,1138.9,81.4,8 +slime-979,learner,1143.9,86.8,8 +slime-979,learner,1148.9,63.1,8 +slime-979,learner,1153.9,63.5,8 +slime-979,learner,1158.9,54.6,8 +slime-979,learner,1163.9,96.8,8 +slime-979,learner,1168.9,95.9,8 +slime-979,learner,1173.9,12.5,8 +slime-979,learner,1178.9,83.2,8 +slime-979,learner,1183.9,82.9,8 +slime-979,learner,1188.9,12.5,8 +slime-979,learner,1193.9,14.0,8 +slime-979,learner,1198.9,89.4,8 +slime-979,learner,1203.9,99.8,8 +slime-979,learner,1208.9,99.6,8 +slime-979,learner,1213.9,99.6,8 +slime-979,learner,1218.9,100.0,8 +slime-979,learner,1223.9,83.1,8 +slime-979,learner,1228.9,94.0,8 +slime-979,learner,1233.9,47.2,8 +slime-979,learner,1238.9,0.0,8 +slime-979,learner,1243.9,0.0,8 +slime-979,learner,1248.9,0.0,8 +slime-979,learner,1253.9,0.0,8 +slime-979,learner,1258.9,0.0,8 +slime-979,learner,1263.9,0.0,8 +slime-979,learner,1268.9,0.0,8 +slime-979,learner,1273.9,0.0,8 +slime-979,learner,1278.9,0.0,8 +slime-979,learner,1283.9,0.0,8 +slime-979,learner,1288.9,0.0,8 +slime-979,learner,1293.9,0.0,8 +slime-979,learner,1298.9,0.0,8 +slime-979,learner,1303.9,0.0,8 +slime-979,learner,1308.9,0.0,8 +slime-979,learner,1313.9,0.0,8 +slime-979,learner,1318.9,0.0,8 +slime-979,learner,1323.9,0.0,8 +slime-979,learner,1328.9,0.0,8 +slime-979,learner,1333.9,0.0,8 +slime-979,learner,1338.9,0.0,8 +slime-979,learner,1348.9,0.0,8 +slime-979,learner,1353.9,0.0,8 +slime-979,learner,1358.9,0.0,8 +slime-979,learner,1363.9,0.0,8 +slime-979,learner,1368.9,95.9,8 +slime-979,learner,1373.9,87.5,8 +slime-979,learner,1378.9,89.0,8 +slime-979,learner,1383.9,80.6,8 +slime-979,learner,1388.9,91.8,8 +slime-979,learner,1393.9,65.5,8 +slime-979,learner,1398.9,100.0,8 +slime-979,learner,1403.9,87.5,8 +slime-979,learner,1408.9,95.6,8 +slime-979,learner,1413.9,100.0,8 +slime-979,learner,1418.9,99.8,8 +slime-979,learner,1423.9,74.8,8 +slime-979,learner,1428.9,75.0,8 +slime-979,learner,1433.9,99.9,8 +slime-979,learner,1438.9,68.4,8 +slime-979,learner,1443.9,62.5,8 +slime-979,learner,1448.9,99.6,8 +slime-979,learner,1453.9,0.0,8 +slime-979,learner,1458.9,0.0,8 +slime-979,learner,1463.9,0.0,8 +slime-979,learner,1468.9,0.0,8 +slime-979,learner,1473.9,0.0,8 +slime-979,learner,1478.9,0.0,8 +slime-979,learner,1483.9,0.0,8 +slime-979,learner,1488.9,0.0,8 +slime-979,learner,1493.9,0.0,8 +slime-979,learner,1498.9,0.0,8 +slime-979,learner,1503.9,0.0,8 +slime-979,learner,1508.9,0.0,8 +slime-979,learner,1513.9,0.0,8 +slime-979,learner,1518.9,0.0,8 +slime-979,learner,1523.9,0.0,8 +slime-979,learner,1528.9,0.0,8 +slime-979,learner,1533.9,0.0,8 +slime-979,learner,1538.9,0.0,8 +slime-979,learner,1543.9,0.0,8 +slime-979,learner,1548.9,0.0,8 +slime-979,learner,1553.9,0.0,8 +slime-979,learner,1558.9,0.0,8 +slime-979,learner,1564.0,65.9,8 +slime-979,learner,1568.9,87.4,8 +slime-979,learner,1573.9,97.1,8 +slime-979,learner,1578.9,99.6,8 +slime-979,learner,1583.9,100.0,8 +slime-979,learner,1589.0,99.1,8 +slime-979,learner,1599.0,100.0,8 +slime-979,learner,1604.0,15.2,8 +slime-979,learner,1609.0,0.0,8 +slime-979,learner,1614.0,0.0,8 +slime-979,learner,1619.0,0.0,8 +slime-979,learner,1624.0,0.0,8 +slime-979,learner,1629.0,0.0,8 +slime-979,learner,1634.0,0.0,8 +slime-979,learner,1639.0,0.0,8 +slime-979,learner,1644.0,0.0,8 +slime-979,learner,1649.0,0.0,8 +slime-979,learner,1659.0,0.0,8 +slime-979,learner,1664.0,0.0,8 +slime-979,learner,1669.0,0.0,8 +slime-979,learner,1674.0,0.0,8 +slime-979,learner,1679.0,0.0,8 +slime-979,learner,1684.0,0.0,8 +slime-979,learner,1689.0,0.0,8 +slime-979,learner,1694.0,0.0,8 +slime-979,learner,1699.0,0.0,8 +slime-979,learner,1704.0,0.0,8 +slime-979,learner,1709.0,0.0,8 +slime-979,learner,1714.0,0.0,8 +slime-979,learner,1719.0,0.0,8 +slime-979,learner,1724.0,0.0,8 +slime-979,learner,1729.0,0.0,8 +slime-979,learner,1734.0,0.0,8 +slime-979,learner,1739.0,0.0,8 +slime-979,learner,1744.0,0.0,8 +slime-979,learner,1749.0,0.0,8 +slime-979,learner,1754.0,0.0,8 +slime-979,learner,1759.0,0.0,8 +slime-979,learner,1764.0,100.0,8 +slime-979,learner,1769.0,75.0,8 +slime-979,learner,1774.0,100.0,8 +slime-979,learner,1779.0,100.0,8 +slime-979,learner,1784.0,100.0,8 +slime-979,learner,1789.0,78.0,8 +slime-979,learner,1794.0,74.8,8 +slime-979,learner,1799.0,99.4,8 +slime-979,learner,1804.0,0.0,8 +slime-979,learner,1809.0,0.0,8 +slime-979,learner,1814.0,0.0,8 +slime-979,learner,1819.0,0.0,8 +slime-979,learner,1824.0,0.0,8 +slime-979,learner,1829.0,0.0,8 +slime-979,learner,1834.0,0.0,8 +slime-979,learner,1839.0,0.0,8 +slime-979,learner,1844.0,0.0,8 +slime-979,learner,1849.0,0.0,8 +slime-979,learner,1854.0,0.0,8 +slime-979,learner,1859.0,0.0,8 +slime-979,learner,1864.0,0.0,8 +slime-979,learner,1869.0,0.0,8 +slime-979,learner,1874.0,0.0,8 +slime-979,learner,1879.0,0.0,8 +slime-979,learner,1884.0,0.0,8 +slime-979,learner,1889.0,0.0,8 +slime-979,learner,1899.0,0.0,8 +slime-979,learner,1904.0,0.0,8 +slime-979,learner,1909.0,0.0,8 +slime-979,learner,1914.0,0.0,8 +slime-979,learner,1919.0,0.0,8 +slime-979,learner,1924.0,0.0,8 +slime-979,learner,1929.0,0.0,8 +slime-979,learner,1934.0,0.0,8 +slime-979,learner,1939.0,0.0,8 +slime-979,learner,1944.0,0.0,8 +slime-979,learner,1949.0,0.0,8 +slime-979,learner,1954.0,91.6,8 +slime-979,learner,1959.0,100.0,8 +slime-979,learner,1964.0,100.0,8 +slime-979,learner,1969.0,99.6,8 +slime-979,learner,1974.0,100.0,8 +slime-979,learner,1979.0,61.2,8 +slime-979,learner,1984.0,0.0,8 +slime-979,learner,1989.0,0.0,8 +slime-979,learner,1994.0,0.0,8 +slime-979,learner,1999.0,0.0,8 +slime-979,learner,2004.0,0.0,8 +slime-979,learner,2009.0,0.0,8 +slime-979,learner,2014.0,0.0,8 +slime-979,learner,2019.0,0.0,8 +slime-979,learner,2024.0,0.0,8 +slime-979,learner,2029.0,0.0,8 +slime-979,learner,2034.0,0.0,8 +slime-979,learner,2039.0,0.0,8 +slime-979,learner,2044.0,0.0,8 +slime-979,learner,2049.0,0.0,8 +slime-979,learner,2054.0,0.0,8 +slime-979,learner,2059.0,0.0,8 +slime-979,learner,2064.0,0.0,8 +slime-979,learner,2069.0,0.0,8 +slime-979,learner,2074.0,0.0,8 +slime-979,learner,2079.0,0.0,8 +slime-979,learner,2084.0,0.0,8 +slime-979,learner,2089.0,0.0,8 +slime-979,learner,2094.0,0.0,8 +slime-979,learner,2099.0,0.0,8 +slime-979,learner,2104.0,0.0,8 +slime-979,learner,2109.0,0.0,8 +slime-979,learner,2114.0,0.0,8 +slime-979,learner,2119.0,0.0,8 +slime-979,learner,2124.0,0.0,8 +slime-979,learner,2129.0,0.0,8 +slime-979,learner,2134.0,0.0,8 +slime-979,learner,2139.0,0.0,8 +slime-979,learner,2144.0,96.8,8 +slime-979,learner,2149.0,100.0,8 +slime-979,learner,2154.0,95.9,8 +slime-979,learner,2159.0,99.6,8 +slime-979,learner,2164.0,100.0,8 +slime-979,learner,2169.0,99.9,8 +slime-979,learner,2174.0,99.6,8 +slime-979,learner,2179.0,77.8,8 +slime-979,learner,2184.0,82.6,8 +slime-979,learner,2189.0,62.5,8 +slime-979,learner,2199.0,0.0,8 +slime-979,learner,2204.0,0.0,8 +slime-979,learner,2209.0,0.0,8 +slime-979,learner,2214.0,0.0,8 +slime-979,learner,2219.0,0.0,8 +slime-979,learner,2224.0,0.0,8 +slime-979,learner,2229.0,0.0,8 +slime-979,learner,2234.0,0.0,8 +slime-979,learner,2239.0,0.0,8 +slime-979,learner,2244.0,0.0,8 +slime-979,learner,2249.0,0.0,8 +slime-979,learner,2254.0,0.0,8 +slime-979,learner,2259.0,0.0,8 +slime-979,learner,2264.0,0.0,8 +slime-979,learner,2269.0,0.0,8 +slime-979,learner,2274.1,0.0,8 +slime-979,learner,2279.1,0.0,8 +slime-979,learner,2284.1,0.0,8 +slime-979,learner,2289.1,0.0,8 +slime-979,learner,2294.1,0.0,8 +slime-979,learner,2299.1,0.0,8 +slime-979,learner,2304.1,0.0,8 +slime-979,learner,2309.1,0.0,8 +slime-979,learner,2314.1,0.0,8 +slime-979,learner,2319.1,0.0,8 +slime-979,learner,2324.1,0.0,8 +slime-979,learner,2329.1,0.0,8 +slime-979,learner,2334.1,0.0,8 +slime-979,learner,2339.1,16.8,8 +slime-979,learner,2344.1,99.8,8 +slime-979,learner,2349.1,84.9,8 +slime-979,learner,2354.1,100.0,8 +slime-979,learner,2359.1,99.8,8 +slime-979,learner,2364.1,99.5,8 +slime-979,learner,2369.1,100.0,8 +slime-979,learner,2374.1,75.0,8 +slime-979,learner,2379.1,75.0,8 +slime-979,learner,2384.1,0.0,8 +slime-979,learner,2389.1,0.0,8 +slime-979,learner,2394.1,0.0,8 +slime-979,learner,2399.1,0.0,8 +slime-979,learner,2404.1,0.0,8 +slime-979,learner,2409.1,0.0,8 +slime-979,learner,2414.1,0.0,8 +slime-979,learner,2419.1,0.0,8 +slime-979,learner,2424.1,0.0,8 +slime-979,learner,2429.1,0.0,8 +slime-979,learner,2434.1,0.0,8 +slime-979,learner,2439.1,0.0,8 +slime-979,learner,2444.1,0.0,8 +slime-979,learner,2449.1,0.0,8 +slime-979,learner,2454.1,0.0,8 +slime-979,learner,2459.1,0.0,8 +slime-979,learner,2464.1,0.0,8 +slime-979,learner,2469.1,0.0,8 +slime-979,learner,2474.1,0.0,8 +slime-979,learner,2479.1,0.0,8 +slime-979,learner,2484.1,0.0,8 +slime-979,learner,2489.1,0.0,8 +slime-979,learner,2499.1,0.0,8 +slime-979,learner,2504.1,0.0,8 +slime-979,learner,2509.1,0.0,8 +slime-979,learner,2514.1,0.0,8 +slime-979,learner,2519.1,0.0,8 +slime-979,learner,2524.1,0.0,8 +slime-979,learner,2529.1,0.0,8 +slime-979,learner,2534.1,94.8,8 +slime-979,learner,2539.1,99.9,8 +slime-979,learner,2544.1,96.9,8 +slime-979,learner,2549.1,97.5,8 +slime-979,learner,2554.1,100.0,8 +slime-979,learner,2559.1,100.0,8 +slime-979,learner,2564.1,99.1,8 +slime-979,learner,2569.1,98.1,8 +slime-979,learner,2574.1,87.5,8 +slime-979,learner,2579.1,37.4,8 +slime-979,learner,2584.1,0.0,8 +slime-979,learner,2589.1,0.0,8 +slime-979,learner,2594.1,0.0,8 +slime-979,learner,2599.1,0.0,8 +slime-979,learner,2604.1,0.0,8 +slime-979,learner,2609.1,0.0,8 +slime-979,learner,2614.1,0.0,8 +slime-979,learner,2619.1,0.0,8 +slime-979,learner,2624.1,0.0,8 +slime-979,learner,2629.1,0.0,8 +slime-979,learner,2634.1,0.0,8 +slime-979,learner,2639.1,0.0,8 +slime-979,learner,2644.1,0.0,8 +slime-979,learner,2649.1,0.0,8 +slime-979,learner,2654.1,0.0,8 +slime-979,learner,2659.1,0.0,8 +slime-979,learner,2664.1,0.0,8 +slime-979,learner,2669.1,0.0,8 +slime-979,learner,2674.1,0.0,8 +slime-979,learner,2679.1,0.0,8 +slime-979,learner,2684.1,0.0,8 +slime-979,learner,2689.1,0.0,8 +slime-979,learner,2694.1,0.0,8 +slime-979,learner,2699.1,0.0,8 +slime-979,learner,2704.1,0.0,8 +slime-979,learner,2709.1,0.0,8 +slime-979,learner,2714.1,0.0,8 +slime-979,learner,2719.1,0.0,8 +slime-979,learner,2724.1,93.5,8 +slime-979,learner,2729.1,100.0,8 +slime-979,learner,2734.1,100.0,8 +slime-979,learner,2739.1,100.0,8 +slime-979,learner,2744.1,99.9,8 +slime-979,learner,2749.1,0.0,8 +slime-979,learner,2754.1,0.0,8 +slime-979,learner,2759.1,0.0,8 +slime-979,learner,2764.1,0.0,8 +slime-979,learner,2769.1,0.0,8 +slime-979,learner,2774.1,0.0,8 +slime-979,learner,2779.1,0.0,8 +slime-979,learner,2784.1,0.0,8 +slime-979,learner,2789.1,0.0,8 +slime-979,learner,2799.1,0.0,8 +slime-979,learner,2804.1,0.0,8 +slime-979,learner,2809.1,0.0,8 +slime-979,learner,2814.1,0.0,8 +slime-979,learner,2819.1,0.0,8 +slime-979,learner,2824.1,0.0,8 +slime-979,learner,2829.1,0.0,8 +slime-979,learner,2834.1,0.0,8 +slime-979,learner,2839.1,0.0,8 +slime-979,learner,2844.1,0.0,8 +slime-979,learner,2849.1,0.0,8 +slime-979,learner,2854.1,0.0,8 +slime-979,learner,2859.1,0.0,8 +slime-979,learner,2864.1,0.0,8 +slime-979,learner,2869.1,0.0,8 +slime-979,learner,2874.1,0.0,8 +slime-979,learner,2879.1,0.0,8 +slime-979,learner,2884.1,0.0,8 +slime-979,learner,2889.1,0.0,8 +slime-979,learner,2894.1,0.0,8 +slime-979,learner,2899.1,0.0,8 +slime-979,learner,2904.1,0.0,8 +slime-979,learner,2909.1,0.0,8 +slime-979,learner,2914.1,41.8,8 +slime-979,learner,2919.1,0.0,8 +slime-979,learner,2924.1,0.0,8 +slime-979,learner,2929.1,0.0,8 +slime-979,learner,2934.1,0.0,8 +slime-979,learner,2939.1,0.0,8 +slime-979,learner,2944.1,0.0,8 +slime-979,learner,2949.1,0.0,8 +slime-979,learner,2954.1,0.0,8 +slime-979,learner,2959.1,0.0,8 +slime-979,learner,2964.1,0.0,8 +slime-979,learner,2969.1,0.0,8 +slime-979,learner,2974.1,0.0,8 +slime-979,learner,2979.1,0.0,8 +slime-979,learner,2984.1,0.0,8 +slime-979,learner,2989.1,0.0,8 +slime-979,learner,2994.1,0.0,8 +slime-979,learner,2999.1,0.0,8 +slime-979,learner,3004.2,0.0,8 +slime-979,learner,3009.2,0.0,8 +slime-979,learner,3014.2,0.0,8 +slime-979,learner,3019.2,0.0,8 +slime-979,learner,3024.2,0.0,8 +slime-979,learner,3029.2,0.0,8 +slime-979,learner,3034.2,0.0,8 +slime-979,learner,3039.2,0.0,8 +slime-979,learner,3044.2,0.0,8 +slime-979,learner,3049.2,0.0,8 +slime-979,learner,3054.2,0.0,8 +slime-979,learner,3059.2,0.0,8 +slime-979,learner,3064.2,0.0,8 +slime-979,learner,3069.2,0.0,8 +slime-979,learner,3074.2,0.0,8 +slime-979,learner,3079.2,0.0,8 +slime-979,learner,3084.2,0.0,8 +slime-979,learner,3089.2,0.0,8 +slime-979,learner,3099.2,0.0,8 +slime-979,learner,3104.2,0.0,8 +slime-979,learner,3109.2,0.0,8 +slime-979,learner,3114.2,0.0,8 +slime-979,learner,3119.2,0.0,8 +slime-979,learner,3124.2,0.0,8 +slime-979,learner,3129.2,0.0,8 +slime-979,learner,3134.2,0.0,8 +slime-979,learner,3139.2,0.0,8 +slime-979,learner,3144.2,0.0,8 +slime-979,learner,3149.2,0.0,8 +slime-979,learner,3154.2,0.0,8 +slime-979,learner,3159.2,0.0,8 +slime-979,learner,3164.2,99.9,8 +slime-979,learner,3169.2,99.8,8 +slime-979,learner,3174.2,100.0,8 +slime-979,learner,3179.2,100.0,8 +slime-979,learner,3184.2,31.4,8 +slime-979,learner,3189.2,0.0,8 +slime-979,learner,3194.2,0.0,8 +slime-979,learner,3199.2,0.0,8 +slime-979,learner,3204.2,0.0,8 +slime-979,learner,3209.2,0.0,8 +slime-979,learner,3214.2,0.0,8 +slime-979,learner,3219.2,0.0,8 +slime-979,learner,3224.2,0.0,8 +slime-979,learner,3229.2,0.0,8 +slime-979,learner,3234.2,0.0,8 +slime-979,learner,3239.2,0.0,8 +slime-979,learner,3244.2,0.0,8 +slime-979,learner,3249.2,0.0,8 +slime-979,learner,3254.2,0.0,8 +slime-979,learner,3259.2,0.0,8 +slime-979,learner,3264.2,0.0,8 +slime-979,learner,3269.2,0.0,8 +slime-979,learner,3274.2,0.0,8 +slime-979,learner,3279.2,0.0,8 +slime-979,learner,3284.2,0.0,8 +slime-979,learner,3289.2,0.0,8 +slime-979,learner,3294.2,0.0,8 +slime-979,learner,3299.2,0.0,8 +slime-979,learner,3304.2,0.0,8 +slime-979,learner,3309.2,0.0,8 +slime-979,learner,3314.2,0.0,8 +slime-979,learner,3319.2,0.0,8 +slime-979,learner,3324.2,0.0,8 +slime-979,learner,3329.2,0.0,8 +slime-979,learner,3334.2,0.0,8 +slime-979,learner,3339.2,0.0,8 +slime-979,learner,3344.2,0.0,8 +slime-979,learner,3349.2,0.0,8 +slime-979,learner,3354.2,30.5,8 +slime-979,learner,3359.2,99.8,8 +slime-979,learner,3364.2,100.0,8 +slime-979,learner,3369.2,100.0,8 +slime-979,learner,3374.2,99.9,8 +slime-979,learner,3379.2,99.4,8 +slime-979,learner,3384.2,0.0,8 +slime-979,learner,3389.2,0.0,8 +slime-979,learner,3399.2,0.0,8 +slime-979,learner,3404.2,0.0,8 +slime-979,learner,3414.2,0.0,8 +slime-979,learner,3419.2,0.0,8 +slime-979,learner,3424.2,0.0,8 +slime-979,learner,3429.2,0.0,8 +slime-979,learner,3434.2,0.0,8 +slime-979,learner,3439.2,0.0,8 +slime-979,learner,3444.2,0.0,8 +slime-979,learner,3449.2,0.0,8 +slime-979,learner,3454.2,0.0,8 +slime-979,learner,3459.2,0.0,8 +slime-979,learner,3464.2,0.0,8 +slime-979,learner,3469.2,0.0,8 +slime-979,learner,3474.2,0.0,8 +slime-979,learner,3479.2,0.0,8 +slime-979,learner,3484.2,0.0,8 +slime-979,learner,3489.2,0.0,8 +slime-979,learner,3494.2,0.0,8 +slime-979,learner,3499.2,0.0,8 +slime-979,learner,3504.2,0.0,8 +slime-979,learner,3509.2,0.0,8 +slime-979,learner,3514.2,0.0,8 +slime-979,learner,3519.2,0.0,8 +slime-979,learner,3524.2,0.0,8 +slime-979,learner,3529.2,0.0,8 +slime-979,learner,3534.2,0.0,8 +slime-979,learner,3539.2,0.0,8 +slime-979,learner,3544.2,0.0,8 +slime-979,learner,3549.2,0.0,8 +slime-979,learner,3554.2,0.0,8 +slime-979,learner,3559.2,90.8,8 +slime-979,learner,3569.2,81.6,8 +slime-979,learner,3574.2,100.0,8 +slime-979,learner,3579.2,99.6,8 +slime-979,learner,3584.2,99.5,8 +slime-979,learner,3589.2,74.6,8 +slime-979,learner,3594.2,83.4,8 +slime-979,learner,3599.2,100.0,8 +slime-979,learner,3604.2,0.0,8 +slime-979,learner,3609.2,0.0,8 +slime-979,learner,3614.2,0.0,8 +slime-979,learner,3619.2,0.0,8 +slime-979,learner,3624.2,0.0,8 +slime-979,learner,3629.2,0.0,8 +slime-979,learner,3634.2,0.0,8 +slime-979,learner,3639.2,0.0,8 +slime-979,learner,3644.2,0.0,8 +slime-979,learner,3649.2,0.0,8 +slime-979,learner,3654.2,0.0,8 +slime-979,learner,3659.2,0.0,8 +slime-979,learner,3664.2,0.0,8 +slime-979,learner,3669.2,0.0,8 +slime-979,learner,3674.2,0.0,8 +slime-979,learner,3679.2,0.0,8 +slime-979,learner,3684.2,0.0,8 +slime-979,learner,3689.2,0.0,8 +slime-979,learner,3699.2,0.0,8 +slime-979,learner,3704.2,0.0,8 +slime-979,learner,3709.2,0.0,8 +slime-979,learner,3714.2,0.0,8 +slime-979,learner,3719.2,0.0,8 +slime-979,learner,3724.2,0.0,8 +slime-979,learner,3729.2,0.0,8 +slime-979,learner,3734.2,0.0,8 +slime-979,learner,3739.2,0.0,8 +slime-979,learner,3744.3,0.0,8 +slime-979,learner,3749.3,0.0,8 +slime-979,learner,3754.3,48.9,8 +slime-979,learner,3759.3,99.8,8 +slime-979,learner,3764.3,100.0,8 +slime-979,learner,3769.3,100.0,8 +slime-979,learner,3774.3,99.9,8 +slime-979,learner,3779.3,99.9,8 +slime-979,learner,3784.3,0.0,8 +slime-979,learner,3789.3,0.0,8 +slime-979,learner,3794.3,0.0,8 +slime-979,learner,3799.3,0.0,8 +slime-979,learner,3804.3,0.0,8 +slime-979,learner,3809.3,0.0,8 +slime-979,learner,3814.3,0.0,8 +slime-979,learner,3819.3,0.0,8 +slime-979,learner,3824.3,0.0,8 +slime-979,learner,3829.3,0.0,8 +slime-979,learner,3834.3,0.0,8 +slime-979,learner,3839.3,0.0,8 +slime-979,learner,3844.3,0.0,8 +slime-979,learner,3849.3,0.0,8 +slime-979,learner,3854.3,0.0,8 +slime-979,learner,3859.3,0.0,8 +slime-979,learner,3864.3,0.0,8 +slime-979,learner,3869.3,0.0,8 +slime-979,learner,3874.3,0.0,8 +slime-979,learner,3879.3,0.0,8 +slime-979,learner,3884.3,0.0,8 +slime-979,learner,3889.3,0.0,8 +slime-979,learner,3894.3,0.0,8 +slime-979,learner,3899.3,0.0,8 +slime-979,learner,3904.3,0.0,8 +slime-979,learner,3909.3,0.0,8 +slime-979,learner,3914.3,0.0,8 +slime-979,learner,3919.3,0.0,8 +slime-979,learner,3924.3,0.0,8 +slime-979,learner,3929.3,0.0,8 +slime-979,learner,3934.3,0.0,8 +slime-979,learner,3939.3,0.0,8 +slime-979,learner,3944.3,0.0,8 +slime-979,learner,3949.3,0.0,8 +slime-979,learner,3954.3,96.0,8 +slime-979,learner,3959.3,99.9,8 +slime-979,learner,3964.3,99.8,8 +slime-979,learner,3969.3,100.0,8 +slime-979,learner,3974.3,99.8,8 +slime-979,learner,3979.3,100.0,8 +slime-979,learner,3984.3,99.9,8 +slime-979,learner,3989.3,0.0,8 +slime-979,learner,3999.3,0.0,8 +slime-979,learner,4004.3,0.0,8 +slime-979,learner,4009.3,0.0,8 +slime-979,learner,4014.3,0.0,8 +slime-979,learner,4019.3,0.0,8 +slime-979,learner,4024.3,0.0,8 +slime-979,learner,4029.3,0.0,8 +slime-979,learner,4034.3,0.0,8 +slime-979,learner,4039.3,0.0,8 +slime-979,learner,4044.3,0.0,8 +slime-979,learner,4049.3,0.0,8 +slime-979,learner,4054.3,0.0,8 +slime-979,learner,4059.3,0.0,8 +slime-979,learner,4064.3,0.0,8 +slime-979,learner,4069.3,0.0,8 +slime-979,learner,4074.3,0.0,8 +slime-979,learner,4079.3,0.0,8 +slime-979,learner,4084.3,0.0,8 +slime-979,learner,4089.3,0.0,8 +slime-979,learner,4094.3,0.0,8 +slime-979,learner,4099.3,0.0,8 +slime-979,learner,4104.3,0.0,8 +slime-979,learner,4109.3,0.0,8 +slime-979,learner,4114.3,0.0,8 +slime-979,learner,4119.3,0.0,8 +slime-979,learner,4124.3,0.0,8 +slime-979,learner,4129.3,0.0,8 +slime-979,learner,4134.3,0.0,8 +slime-979,learner,4139.3,0.0,8 +slime-979,learner,4144.3,0.0,8 +slime-979,learner,4149.3,0.0,8 +slime-979,learner,4154.3,0.0,8 +slime-979,learner,4159.3,95.5,8 +slime-979,learner,4164.3,99.9,8 +slime-979,learner,4169.3,87.5,8 +slime-979,learner,4174.3,100.0,8 +slime-979,learner,4179.3,99.6,8 +slime-979,learner,4184.3,99.5,8 +slime-979,learner,4189.3,26.4,8 +slime-979,learner,4194.3,0.0,8 +slime-979,learner,4199.3,0.0,8 +slime-979,learner,4204.3,0.0,8 +slime-979,learner,4209.3,0.0,8 +slime-979,learner,4214.3,0.0,8 +slime-979,learner,4219.3,0.0,8 +slime-979,learner,4224.3,0.0,8 +slime-979,learner,4229.3,0.0,8 +slime-979,learner,4234.3,0.0,8 +slime-979,learner,4239.3,0.0,8 +slime-979,learner,4244.3,0.0,8 +slime-979,learner,4249.3,0.0,8 +slime-979,learner,4254.3,0.0,8 +slime-979,learner,4259.3,0.0,8 +slime-979,learner,4264.3,0.0,8 +slime-979,learner,4269.3,0.0,8 +slime-979,learner,4274.3,0.0,8 +slime-979,learner,4279.3,0.0,8 +slime-979,learner,4284.3,0.0,8 +slime-979,learner,4289.3,0.0,8 +slime-979,learner,4299.3,0.0,8 +slime-979,learner,4304.3,0.0,8 +slime-979,learner,4309.3,0.0,8 +slime-979,learner,4314.3,0.0,8 +slime-979,learner,4319.3,0.0,8 +slime-979,learner,4324.3,0.0,8 +slime-979,learner,4329.3,0.0,8 +slime-979,learner,4334.3,0.0,8 +slime-979,learner,4339.3,0.0,8 +slime-979,learner,4344.3,0.0,8 +slime-979,learner,4349.3,0.0,8 +slime-979,learner,4354.3,92.1,8 +slime-979,learner,4359.3,98.4,8 +slime-979,learner,4364.3,100.0,8 +slime-979,learner,4369.3,100.0,8 +slime-979,learner,4374.3,100.0,8 +slime-979,learner,4379.3,0.0,8 +slime-979,learner,4384.3,0.0,8 +slime-979,learner,4389.3,0.0,8 +slime-979,learner,4394.3,0.0,8 +slime-979,learner,4399.3,0.0,8 +slime-979,learner,4404.3,0.0,8 +slime-979,learner,4409.3,0.0,8 +slime-979,learner,4414.3,0.0,8 +slime-979,learner,4419.3,0.0,8 +slime-979,learner,4424.3,0.0,8 +slime-979,learner,4429.3,0.0,8 +slime-979,learner,4434.3,0.0,8 +slime-979,learner,4439.3,0.0,8 +slime-979,learner,4444.3,0.0,8 +slime-979,learner,4449.3,0.0,8 +slime-979,learner,4454.3,0.0,8 +slime-979,learner,4459.3,0.0,8 +slime-979,learner,4464.3,0.0,8 +slime-979,learner,4469.3,0.0,8 +slime-979,learner,4474.3,0.0,8 +slime-979,learner,4479.3,0.0,8 +slime-979,learner,4484.3,0.0,8 +slime-979,learner,4489.3,0.0,8 +slime-979,learner,4494.3,0.0,8 +slime-979,learner,4499.3,0.0,8 +slime-979,learner,4504.4,0.0,8 +slime-979,learner,4509.3,0.0,8 +slime-979,learner,4514.4,0.0,8 +slime-979,learner,4519.4,0.0,8 +slime-979,learner,4524.4,0.0,8 +slime-979,learner,4529.4,0.0,8 +slime-979,learner,4534.4,0.0,8 +slime-979,learner,4539.4,0.0,8 +slime-979,learner,4544.4,0.0,8 +slime-979,learner,4549.4,0.0,8 +slime-979,learner,4554.4,0.0,8 +slime-979,learner,4559.4,0.0,8 +slime-979,learner,4564.4,90.8,8 +slime-979,learner,4569.4,99.9,8 +slime-979,learner,4574.4,100.0,8 +slime-979,learner,4579.4,100.0,8 +slime-979,learner,4584.4,99.6,8 +slime-979,learner,4589.4,98.8,8 +slime-979,learner,4594.4,0.0,8 +slime-979,learner,4599.4,0.0,8 +slime-979,learner,4604.4,0.0,8 +slime-979,learner,4609.4,0.0,8 +slime-979,learner,4614.4,0.0,8 +slime-979,learner,4619.4,0.0,8 +slime-979,learner,4624.4,0.0,8 +slime-979,learner,4629.4,0.0,8 +slime-979,learner,4634.4,0.0,8 +slime-979,learner,4639.4,0.0,8 +slime-979,learner,4644.4,0.0,8 +slime-979,learner,4649.4,0.0,8 +slime-979,learner,4654.4,0.0,8 +slime-979,learner,4659.4,0.0,8 +slime-979,learner,4664.4,0.0,8 +slime-979,learner,4669.4,0.0,8 +slime-979,learner,4674.4,0.0,8 +slime-979,learner,4679.4,0.0,8 +slime-979,learner,4684.4,0.0,8 +slime-979,learner,4689.4,0.0,8 +slime-979,learner,4694.4,0.0,8 +slime-979,learner,4699.4,0.0,8 +slime-979,learner,4704.4,0.0,8 +slime-979,learner,4709.4,0.0,8 +slime-979,learner,4714.4,0.0,8 +slime-979,learner,4719.4,0.0,8 +slime-979,learner,4724.4,0.0,8 +slime-979,learner,4729.4,0.0,8 +slime-979,learner,4734.4,0.0,8 +slime-979,learner,4739.4,0.0,8 +slime-979,learner,4744.4,0.0,8 +slime-979,learner,4749.4,0.0,8 +slime-979,learner,4754.4,0.0,8 +slime-979,learner,4759.4,96.9,8 +slime-979,learner,4764.4,100.0,8 +slime-979,learner,4769.4,99.5,8 +slime-979,learner,4774.4,99.8,8 +slime-979,learner,4779.4,100.0,8 +slime-979,learner,4784.4,0.0,8 +slime-979,learner,4789.4,0.0,8 +slime-979,learner,4794.4,0.0,8 +slime-979,learner,4799.4,0.0,8 +slime-979,learner,4804.4,0.0,8 +slime-979,learner,4809.4,0.0,8 +slime-979,learner,4814.4,0.0,8 +slime-979,learner,4819.4,0.0,8 +slime-979,learner,4824.4,0.0,8 +slime-979,learner,4829.4,0.0,8 +slime-979,learner,4834.4,0.0,8 +slime-979,learner,4839.4,0.0,8 +slime-979,learner,4844.4,0.0,8 +slime-979,learner,4849.4,0.0,8 +slime-979,learner,4854.4,0.0,8 +slime-979,learner,4859.4,0.0,8 +slime-979,learner,4864.4,0.0,8 +slime-979,learner,4869.4,0.0,8 +slime-979,learner,4874.4,0.0,8 +slime-979,learner,4879.4,0.0,8 +slime-979,learner,4884.4,0.0,8 +slime-979,learner,4889.4,0.0,8 +slime-979,learner,4899.4,0.0,8 +slime-979,learner,4904.4,0.0,8 +slime-979,learner,4909.4,0.0,8 +slime-979,learner,4914.4,0.0,8 +slime-979,learner,4919.4,0.0,8 +slime-979,learner,4924.4,0.0,8 +slime-979,learner,4929.4,0.0,8 +slime-979,learner,4934.4,0.0,8 +slime-979,learner,4939.4,0.0,8 +slime-979,learner,4944.4,0.0,8 +slime-979,learner,4949.4,96.2,8 +slime-979,learner,4954.4,99.8,8 +slime-979,learner,4959.4,99.2,8 +slime-979,learner,4964.4,99.6,8 +slime-979,learner,4969.4,100.0,8 +slime-979,learner,4974.4,67.6,8 +slime-979,learner,4979.4,0.0,8 +slime-979,learner,4984.4,0.0,8 +slime-979,learner,4989.4,16.8,8 +slime-979,learner,4994.4,74.5,8 +slime-979,learner,5004.4,87.5,8 +slime-979,learner,5009.4,75.0,8 +slime-979,learner,5014.4,89.1,8 +slime-979,learner,5019.4,88.9,8 +slime-979,learner,5024.4,87.4,8 +slime-979,learner,5029.4,88.4,8 +slime-979,learner,5034.4,87.4,8 +slime-979,learner,5039.4,88.5,8 +slime-979,learner,5044.4,88.9,8 +slime-979,learner,5049.4,87.2,8 +slime-979,learner,5054.4,88.0,8 +slime-979,learner,5059.4,87.5,8 +slime-979,learner,5064.4,96.1,8 +slime-979,learner,5069.4,0.0,8 +slime-979,learner,5074.4,0.0,8 +slime-979,learner,5084.4,0.0,8 +slime-979,learner,5094.4,0.0,8 +slime-979,learner,5099.4,0.0,8 +slime-979,learner,5104.4,0.0,8 +slime-979,learner,5109.4,0.0,8 +slime-979,learner,5114.4,0.0,8 +slime-979,learner,5119.4,0.0,8 +slime-979,learner,5124.4,0.0,8 +slime-979,learner,5129.4,0.0,8 +slime-979,learner,5134.4,0.0,8 +slime-979,learner,5139.4,0.0,8 +slime-979,learner,5144.4,0.0,8 +slime-979,learner,5149.4,0.0,8 +slime-979,learner,5154.4,0.0,8 +slime-979,learner,5159.4,0.0,8 +slime-979,learner,5164.4,0.0,8 +slime-979,learner,5169.4,0.0,8 +slime-979,learner,5174.4,0.0,8 +slime-979,learner,5184.4,0.0,8 +slime-979,learner,5189.4,0.0,8 +slime-979,learner,5199.4,0.0,8 +slime-979,learner,5204.4,0.0,8 +slime-979,learner,5209.4,0.0,8 +slime-979,learner,5214.4,0.0,8 +slime-979,learner,5219.4,0.0,8 +slime-979,learner,5224.4,0.0,8 +slime-979,learner,5229.4,0.0,8 +slime-979,learner,5234.4,0.0,8 +slime-979,learner,5239.4,0.0,8 +slime-979,learner,5244.4,0.0,8 +slime-979,learner,5254.5,0.0,8 +slime-979,learner,5259.5,0.0,8 +slime-979,learner,5264.5,0.0,8 +slime-979,learner,5269.5,0.0,8 +slime-979,learner,5274.5,0.0,8 +slime-979,learner,5279.5,0.0,8 +slime-979,learner,5284.5,0.0,8 +slime-979,learner,5289.5,0.0,8 +slime-979,learner,5294.5,0.0,8 +slime-979,learner,5299.5,0.0,8 +slime-979,learner,5304.5,0.0,8 +slime-979,learner,5309.5,0.0,8 +slime-979,learner,5314.5,0.0,8 diff --git a/docs/caesar-opd-2026-09-27/data/miles-1021/summary.json b/docs/caesar-opd-2026-09-27/data/miles-1021/summary.json new file mode 100644 index 0000000..e7991f0 --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/miles-1021/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/miles-caesar-02/checkpoints/miles-1021/hf/iter_19", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 380.1805899143219, + "output_tokens_per_second": 44872.39341662493, + "domains": { + "caesar_cipher": { + "avg@8": 0.7757936507936508, + "pass@8": 0.9920634920634921, + "finished_thinking": 0.9464285714285714, + "truncated": 0.057539682539682536, + "response_tokens_mean": 13055.827380952382, + "response_tokens_median": 11226.5 + }, + "simple_geometry": { + "avg@8": 0.6591796875, + "pass@8": 0.9921875, + "finished_thinking": 0.99609375, + "truncated": 0.00390625, + "response_tokens_mean": 3807.9482421875, + "response_tokens_median": 3623.5 + } + } +} diff --git a/docs/caesar-opd-2026-09-27/data/miles-965/summary.json b/docs/caesar-opd-2026-09-27/data/miles-965/summary.json new file mode 100644 index 0000000..d39b55f --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/miles-965/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/miles-caesar-01/checkpoints/miles-965/hf/iter_19", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 376.73781752586365, + "output_tokens_per_second": 44616.56148667914, + "domains": { + "caesar_cipher": { + "avg@8": 0.7738095238095238, + "pass@8": 1.0, + "finished_thinking": 0.9523809523809523, + "truncated": 0.047619047619047616, + "response_tokens_mean": 12752.33134920635, + "response_tokens_median": 11012.0 + }, + "simple_geometry": { + "avg@8": 0.6806640625, + "pass@8": 1.0, + "finished_thinking": 0.99609375, + "truncated": 0.00390625, + "response_tokens_mean": 3861.71484375, + "response_tokens_median": 3729.5 + } + } +} diff --git a/docs/caesar-opd-2026-09-27/data/prime-rl-1017/summary.json b/docs/caesar-opd-2026-09-27/data/prime-rl-1017/summary.json new file mode 100644 index 0000000..ae63e32 --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/prime-rl-1017/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/prime-rl-caesar-01/checkpoints/prime-rl-1017/checkpoints/step_20/weights", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 388.1367473602295, + "output_tokens_per_second": 44536.09228594062, + "domains": { + "caesar_cipher": { + "avg@8": 0.7787698412698413, + "pass@8": 0.9920634920634921, + "finished_thinking": 0.9484126984126984, + "truncated": 0.05654761904761905, + "response_tokens_mean": 13256.776785714286, + "response_tokens_median": 11490.0 + }, + "simple_geometry": { + "avg@8": 0.7041015625, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 3831.3115234375, + "response_tokens_median": 3815.5 + } + } +} diff --git a/docs/caesar-opd-2026-09-27/data/prime-rl-1023-step10/summary.json b/docs/caesar-opd-2026-09-27/data/prime-rl-1023-step10/summary.json new file mode 100644 index 0000000..ac9e8af --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/prime-rl-1023-step10/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/prime-rl-caesar-03/checkpoints/prime-rl-1023/checkpoints/step_10/weights", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 380.51149344444275, + "output_tokens_per_second": 44913.90744940873, + "domains": { + "caesar_cipher": { + "avg@8": 0.7757936507936508, + "pass@8": 0.9841269841269841, + "finished_thinking": 0.9503968253968254, + "truncated": 0.054563492063492064, + "response_tokens_mean": 12868.04365079365, + "response_tokens_median": 10746.0 + }, + "simple_geometry": { + "avg@8": 0.703125, + "pass@8": 1.0, + "finished_thinking": 0.9951171875, + "truncated": 0.0048828125, + "response_tokens_mean": 4022.724609375, + "response_tokens_median": 3807.5 + } + } +} diff --git a/docs/caesar-opd-2026-09-27/data/prime-rl-1023/summary.json b/docs/caesar-opd-2026-09-27/data/prime-rl-1023/summary.json new file mode 100644 index 0000000..023a638 --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/prime-rl-1023/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/prime-rl-caesar-03/checkpoints/prime-rl-1023/checkpoints/step_20/weights", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 380.4039845466614, + "output_tokens_per_second": 45266.50271689729, + "domains": { + "caesar_cipher": { + "avg@8": 0.7688492063492064, + "pass@8": 1.0, + "finished_thinking": 0.9474206349206349, + "truncated": 0.05555555555555555, + "response_tokens_mean": 13187.689484126984, + "response_tokens_median": 11100.5 + }, + "simple_geometry": { + "avg@8": 0.712890625, + "pass@8": 0.9921875, + "finished_thinking": 0.9970703125, + "truncated": 0.0029296875, + "response_tokens_mean": 3834.3427734375, + "response_tokens_median": 3776.5 + } + } +} diff --git a/docs/caesar-opd-2026-09-27/data/prime-rl-968/summary.json b/docs/caesar-opd-2026-09-27/data/prime-rl-968/summary.json new file mode 100644 index 0000000..2401a5b --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/prime-rl-968/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/prime-rl-caesar-01/checkpoints/prime-rl-968/checkpoints/step_20/weights", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 376.5782730579376, + "output_tokens_per_second": 45369.84266580717, + "domains": { + "caesar_cipher": { + "avg@8": 0.5803571428571429, + "pass@8": 0.9603174603174603, + "finished_thinking": 0.9503968253968254, + "truncated": 0.05357142857142857, + "response_tokens_mean": 13076.5, + "response_tokens_median": 11290.5 + }, + "simple_geometry": { + "avg@8": 0.58203125, + "pass@8": 1.0, + "finished_thinking": 0.99609375, + "truncated": 0.00390625, + "response_tokens_mean": 3812.6806640625, + "response_tokens_median": 3440.5 + } + } +} diff --git a/docs/caesar-opd-2026-09-27/data/slime-979/summary.json b/docs/caesar-opd-2026-09-27/data/slime-979/summary.json new file mode 100644 index 0000000..872bd78 --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/slime-979/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/slime-caesar-01/checkpoints/slime-979/hf/iter_19", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 372.3330216407776, + "output_tokens_per_second": 44909.12443466367, + "domains": { + "caesar_cipher": { + "avg@8": 0.7767857142857143, + "pass@8": 0.9920634920634921, + "finished_thinking": 0.9583333333333334, + "truncated": 0.04563492063492063, + "response_tokens_mean": 12722.262896825398, + "response_tokens_median": 10932.5 + }, + "simple_geometry": { + "avg@8": 0.7177734375, + "pass@8": 1.0, + "finished_thinking": 0.9970703125, + "truncated": 0.0029296875, + "response_tokens_mean": 3805.7705078125, + "response_tokens_median": 3681.0 + } + } +} diff --git a/docs/caesar-opd-2026-09-27/data/teacher-caesar/summary.json b/docs/caesar-opd-2026-09-27/data/teacher-caesar/summary.json new file mode 100644 index 0000000..999c3da --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/teacher-caesar/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-assets/teachers/caesar_cipher", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 366.0560100078583, + "output_tokens_per_second": 45223.71589977342, + "domains": { + "caesar_cipher": { + "avg@8": 0.7212301587301587, + "pass@8": 0.9841269841269841, + "finished_thinking": 0.9563492063492064, + "truncated": 0.044642857142857144, + "response_tokens_mean": 12579.578373015873, + "response_tokens_median": 11050.5 + }, + "simple_geometry": { + "avg@8": 0.587890625, + "pass@8": 0.9921875, + "finished_thinking": 0.998046875, + "truncated": 0.001953125, + "response_tokens_mean": 3783.396484375, + "response_tokens_median": 3549.0 + } + } +} diff --git a/docs/caesar-opd-2026-09-27/data/telemetry-miles-965.json b/docs/caesar-opd-2026-09-27/data/telemetry-miles-965.json new file mode 100644 index 0000000..3a9893e --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/telemetry-miles-965.json @@ -0,0 +1,423 @@ +[ + { + "run": "miles-965", + "job_id": "965", + "nodes": [ + "gpu-nodes-2", + "gpu-nodes-0" + ], + "exit_code": 0, + "wall_seconds": 3814.208867788315, + "updates": { + "updates": 20, + "per_update": [ + { + "step_s": 548.016720533371, + "train_s": 151.45704221725464, + "wait_for_rollouts_s": 396.55967831611633, + "weight_sync_s": 4.334013223648071, + "mfu": 0.004595942941394495, + "response_tokens_mean": 5603.546875, + "reverse_kl": 0.0031590794678777456, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 96.81705236434937, + "train_s": 93.7499406337738, + "wait_for_rollouts_s": 3.0671117305755615, + "weight_sync_s": 2.675510883331299, + "mfu": 0.01248329518106836, + "response_tokens_mean": 8470.640625, + "reverse_kl": 0.002333808457478881, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 46.43102192878723, + "train_s": 43.10495138168335, + "wait_for_rollouts_s": 3.326070547103882, + "weight_sync_s": 2.918860912322998, + "mfu": 0.04054062190511205, + "response_tokens_mean": 11112.6640625, + "reverse_kl": 0.0024879397824406624, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 103.40338206291199, + "train_s": 37.8379967212677, + "wait_for_rollouts_s": 65.56538534164429, + "weight_sync_s": 2.9061319828033447, + "mfu": 0.02553929922762038, + "response_tokens_mean": 7234.7109375, + "reverse_kl": 0.0018574441783130169, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 151.65496563911438, + "train_s": 55.97388935089111, + "wait_for_rollouts_s": 95.68107628822327, + "weight_sync_s": 2.9090235233306885, + "mfu": 0.026246178314353476, + "response_tokens_mean": 10024.2109375, + "reverse_kl": 0.001354292850010097, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 96.64260721206665, + "train_s": 24.301031827926636, + "wait_for_rollouts_s": 72.34157538414001, + "weight_sync_s": 3.1500658988952637, + "mfu": 0.05509765406931508, + "response_tokens_mean": 9218.484375, + "reverse_kl": 0.0014172287192195654, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 134.86347150802612, + "train_s": 36.54276704788208, + "wait_for_rollouts_s": 98.32070446014404, + "weight_sync_s": 3.050025701522827, + "mfu": 0.03296625264406311, + "response_tokens_mean": 8578.859375, + "reverse_kl": 0.00147082912735641, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 124.09599900245667, + "train_s": 30.81938648223877, + "wait_for_rollouts_s": 93.2766125202179, + "weight_sync_s": 2.718616247177124, + "mfu": 0.037336032257095295, + "response_tokens_mean": 8317.6171875, + "reverse_kl": 0.0012234770692884922, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 108.73579835891724, + "train_s": 21.125571250915527, + "wait_for_rollouts_s": 87.61022710800171, + "weight_sync_s": 2.9796061515808105, + "mfu": 0.053155509673796564, + "response_tokens_mean": 8182.4375, + "reverse_kl": 0.0011541444109752774, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 132.81878876686096, + "train_s": 24.954744815826416, + "wait_for_rollouts_s": 107.86404395103455, + "weight_sync_s": 2.993527412414551, + "mfu": 0.047209447682400235, + "response_tokens_mean": 8371.59375, + "reverse_kl": 0.0010920073837041855, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 466.37772274017334, + "train_s": 28.66751742362976, + "wait_for_rollouts_s": 437.7102053165436, + "weight_sync_s": 2.9297940731048584, + "mfu": 0.041710639283943175, + "response_tokens_mean": 8646.2421875, + "reverse_kl": 0.0011140494607388973, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 69.68702149391174, + "train_s": 66.63796305656433, + "wait_for_rollouts_s": 3.049058437347412, + "weight_sync_s": 2.6310746669769287, + "mfu": 0.0403742670792218, + "response_tokens_mean": 15110.484375, + "reverse_kl": 0.0010307823540642858, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 38.776514530181885, + "train_s": 24.20430636405945, + "wait_for_rollouts_s": 14.572208166122437, + "weight_sync_s": 2.8823344707489014, + "mfu": 0.03697350063326541, + "response_tokens_mean": 6841.578125, + "reverse_kl": 0.001273116678930819, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 161.63240838050842, + "train_s": 37.37427234649658, + "wait_for_rollouts_s": 124.25813603401184, + "weight_sync_s": 3.1547036170959473, + "mfu": 0.03180074313368438, + "response_tokens_mean": 8509.15625, + "reverse_kl": 0.001133687561377883, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 107.68330574035645, + "train_s": 26.521886825561523, + "wait_for_rollouts_s": 81.16141891479492, + "weight_sync_s": 2.870715618133545, + "mfu": 0.05428789793217284, + "response_tokens_mean": 9810.203125, + "reverse_kl": 0.0011100568808615208, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 125.97569012641907, + "train_s": 25.59818983078003, + "wait_for_rollouts_s": 100.37750029563904, + "weight_sync_s": 3.022256851196289, + "mfu": 0.04726139562614121, + "response_tokens_mean": 8673.7890625, + "reverse_kl": 0.0011934987269341946, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 132.13553190231323, + "train_s": 33.37939977645874, + "wait_for_rollouts_s": 98.75613212585449, + "weight_sync_s": 3.199458360671997, + "mfu": 0.037011174084739334, + "response_tokens_mean": 8708.9375, + "reverse_kl": 0.0014676849823445082, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 130.2577257156372, + "train_s": 32.50129961967468, + "wait_for_rollouts_s": 97.75642609596252, + "weight_sync_s": 3.5366978645324707, + "mfu": 0.03853176649863027, + "response_tokens_mean": 8861.546875, + "reverse_kl": 0.001207289402373135, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 134.38482308387756, + "train_s": 39.28387188911438, + "wait_for_rollouts_s": 95.10095119476318, + "weight_sync_s": 2.7791523933410645, + "mfu": 0.03710260950177865, + "response_tokens_mean": 9864.2421875, + "reverse_kl": 0.0011596750700846314, + "train_score": null, + "dropped_stale": null + }, + { + "step_s": 126.17139434814453, + "train_s": 31.33073329925537, + "wait_for_rollouts_s": 94.84066104888916, + "weight_sync_s": 2.857374668121338, + "mfu": 0.046047079785706393, + "response_tokens_mean": 9865.109375, + "reverse_kl": 0.0012012431398034096, + "train_score": null, + "dropped_stale": null + } + ], + "step_s_median": 126.0735422372818, + "step_s_total": 3036.561945438385, + "train_s_median": 32.94034969806671, + "train_s_total": 865.3667621612549, + "wait_for_rollouts_s_median": 94.97080612182617, + "wait_for_rollouts_s_total": 2171.19518327713, + "weight_sync_s_median": 2.9243274927139282, + "weight_sync_s_total": 60.49894452095032, + "mfu_mean": 0.03731356537277512, + "response_tokens_mean_mean": 9000.302734375, + "reverse_kl_first": 0.0031590794678777456, + "reverse_kl_last": 0.0012012431398034096 + }, + "roles": { + "generation": { + "samples": 754, + "seconds": 3765.4289886951447, + "gpus": { + "0": { + "utilization_mean_pct": 17.624668435013263, + "busy_fraction": 0.2493368700265252, + "memory_peak_gib": 160.9033203125, + "power_mean_w": 395.02693633952254, + "energy_kwh": 0.4133829777973321 + }, + "1": { + "utilization_mean_pct": 0.0, + "busy_fraction": 0.0, + "memory_peak_gib": 0.0, + "power_mean_w": 194.24220159151193, + "energy_kwh": 0.203168910554364 + }, + "2": { + "utilization_mean_pct": 84.53713527851458, + "busy_fraction": 0.8580901856763926, + "memory_peak_gib": 129.966796875, + "power_mean_w": 712.4853183023873, + "energy_kwh": 0.74585982976619 + }, + "3": { + "utilization_mean_pct": 84.51458885941645, + "busy_fraction": 0.8594164456233422, + "memory_peak_gib": 129.978515625, + "power_mean_w": 717.2951591511937, + "energy_kwh": 0.7508938594942544 + }, + "4": { + "utilization_mean_pct": 84.57029177718833, + "busy_fraction": 0.8607427055702918, + "memory_peak_gib": 129.93359375, + "power_mean_w": 703.941326259947, + "energy_kwh": 0.7369205499702017 + }, + "5": { + "utilization_mean_pct": 84.60079575596816, + "busy_fraction": 0.8607427055702918, + "memory_peak_gib": 129.93359375, + "power_mean_w": 711.5424933687003, + "energy_kwh": 0.7448698301989065 + }, + "6": { + "utilization_mean_pct": 90.18037135278514, + "busy_fraction": 0.9177718832891246, + "memory_peak_gib": 132.91796875, + "power_mean_w": 786.6584350132625, + "energy_kwh": 0.8235505307746106 + }, + "7": { + "utilization_mean_pct": 90.22413793103448, + "busy_fraction": 0.9177718832891246, + "memory_peak_gib": 132.9140625, + "power_mean_w": 816.8420291777188, + "energy_kwh": 0.8551471039219075 + } + }, + "gpu_energy_kwh": 5.273793592477767, + "cpu_utilization_pct": 2.9685669553457528, + "network": { + "rx_gb": 0.431261505, + "tx_gb": 4.780067339, + "rx_gbit_s": 0.0009162547083899675, + "tx_gbit_s": 0.010155692439509187 + }, + "engines": { + "127.0.0.1:30000": { + "seconds": 3685.4200615882874, + "generation_tokens": 8.0, + "generation_tokens_per_s": 0.0021707159201147562, + "prompt_tokens": 56053331.0, + "prompt_tokens_per_s": 15209.482247145248 + } + } + }, + "learner": { + "samples": 748, + "seconds": 3737.6995339393616, + "gpus": { + "0": { + "utilization_mean_pct": 17.447121820615795, + "busy_fraction": 0.2248995983935743, + "memory_peak_gib": 178.1962890625, + "power_mean_w": 293.34788487282464, + "energy_kwh": 0.3046233002873785 + }, + "1": { + "utilization_mean_pct": 19.049531459170012, + "busy_fraction": 0.22623828647925034, + "memory_peak_gib": 178.3095703125, + "power_mean_w": 289.22432396251673, + "energy_kwh": 0.30026228754880163 + }, + "2": { + "utilization_mean_pct": 19.041499330655956, + "busy_fraction": 0.21285140562248997, + "memory_peak_gib": 177.2900390625, + "power_mean_w": 288.4752744310576, + "energy_kwh": 0.2995680282294823 + }, + "3": { + "utilization_mean_pct": 19.203480589022757, + "busy_fraction": 0.20615796519410978, + "memory_peak_gib": 178.1064453125, + "power_mean_w": 296.2988621151272, + "energy_kwh": 0.30760697538884413 + }, + "4": { + "utilization_mean_pct": 18.635876840696117, + "busy_fraction": 0.20214190093708165, + "memory_peak_gib": 177.8720703125, + "power_mean_w": 283.9928246318608, + "energy_kwh": 0.29491303687337045 + }, + "5": { + "utilization_mean_pct": 17.266398929049533, + "busy_fraction": 0.19678714859437751, + "memory_peak_gib": 178.0693359375, + "power_mean_w": 287.0240026773762, + "energy_kwh": 0.2980638212287876 + }, + "6": { + "utilization_mean_pct": 18.72556894243641, + "busy_fraction": 0.20883534136546184, + "memory_peak_gib": 177.5888671875, + "power_mean_w": 288.1708969210174, + "energy_kwh": 0.2992243043790108 + }, + "7": { + "utilization_mean_pct": 16.092369477911646, + "busy_fraction": 0.17938420348058903, + "memory_peak_gib": 177.4755859375, + "power_mean_w": 285.0921686746988, + "energy_kwh": 0.2960594197585649 + } + }, + "gpu_energy_kwh": 2.4003211736942403, + "cpu_utilization_pct": 1.530006764711589, + "network": { + "rx_gb": 4.769379744, + "tx_gb": 0.431149291, + "rx_gbit_s": 0.01020816082339994, + "tx_gbit_s": 0.0009228120924863935 + }, + "engines": { + "10.244.218.33:20000": { + "seconds": 3502.6734211444855, + "generation_tokens": 18359489.0, + "generation_tokens_per_s": 5241.56459725015, + "prompt_tokens": 197251.0, + "prompt_tokens_per_s": 56.314413673070604 + }, + "10.244.218.33:20035": { + "seconds": 3502.6734211444855, + "generation_tokens": 27044189.0, + "generation_tokens_per_s": 7721.013565450648, + "prompt_tokens": 289231.0, + "prompt_tokens_per_s": 82.57435542063607 + }, + "10.244.218.33:20070": { + "seconds": 3502.6734211444855, + "generation_tokens": 18640742.0, + "generation_tokens_per_s": 5321.8612638768955, + "prompt_tokens": 202947.0, + "prompt_tokens_per_s": 57.940600107014205 + } + } + } + } + } +] diff --git a/docs/caesar-opd-2026-09-27/data/telemetry-prime-rl-968.json b/docs/caesar-opd-2026-09-27/data/telemetry-prime-rl-968.json new file mode 100644 index 0000000..57dc127 --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/telemetry-prime-rl-968.json @@ -0,0 +1,428 @@ +[ + { + "run": "prime-rl-968", + "job_id": "968", + "nodes": [ + "gpu-nodes-2", + "gpu-nodes-0" + ], + "exit_code": 0, + "wall_seconds": 3079.591822385788, + "updates": { + "updates": 20, + "per_update": [ + { + "step_s": 529.1642922339961, + "train_s": 192.49921798391733, + "wait_for_rollouts_s": 325.34216199291404, + "weight_sync_s": 6.2794303559930995, + "mfu": 0.009282242402542901, + "response_tokens_mean": 8538.734375, + "reverse_kl": -0.00047136563807725906, + "train_score": 0.2734375, + "dropped_stale": 0.0 + }, + { + "step_s": 104.96455219900236, + "train_s": 98.21152711706236, + "wait_for_rollouts_s": 3.154994919896126e-05, + "weight_sync_s": 6.536054615979083, + "mfu": 0.021592478250890043, + "response_tokens_mean": 8462.5, + "reverse_kl": 9.559861791785806e-05, + "train_score": 0.234375, + "dropped_stale": 0.0 + }, + { + "step_s": 126.76679878891446, + "train_s": 100.04035399504937, + "wait_for_rollouts_s": 17.231033449992537, + "weight_sync_s": 9.307145943981595, + "mfu": 0.02080759026493673, + "response_tokens_mean": 10042.306709265176, + "reverse_kl": 0.00019746931502595544, + "train_score": 0.25559105431309903, + "dropped_stale": 185.0 + }, + { + "step_s": 105.52144598704763, + "train_s": 42.87825250194874, + "wait_for_rollouts_s": 53.45416821807157, + "weight_sync_s": 8.985557502950542, + "mfu": 0.048606123045532836, + "response_tokens_mean": 9980.773234200744, + "reverse_kl": 0.00032566016307100654, + "train_score": 0.3345724907063197, + "dropped_stale": 141.0 + }, + { + "step_s": 86.69881049799733, + "train_s": 17.801031296956353, + "wait_for_rollouts_s": 60.78861289692577, + "weight_sync_s": 7.908716163947247, + "mfu": 0.11617866285575676, + "response_tokens_mean": 9136.551136363636, + "reverse_kl": 0.0006093847332522273, + "train_score": 0.39204545454545453, + "dropped_stale": 48.0 + }, + { + "step_s": 104.43410309602041, + "train_s": 17.899157947977073, + "wait_for_rollouts_s": 78.16571401804686, + "weight_sync_s": 8.174887432018295, + "mfu": 0.0593538817967208, + "response_tokens_mean": 8041.748299319728, + "reverse_kl": 0.0008295756997540593, + "train_score": 0.4557823129251701, + "dropped_stale": 19.0 + }, + { + "step_s": 93.22696455498226, + "train_s": 17.673263796023093, + "wait_for_rollouts_s": 65.58713516301941, + "weight_sync_s": 9.742642987985164, + "mfu": 0.11787242841394775, + "response_tokens_mean": 8709.232876712329, + "reverse_kl": 0.0003179615014232695, + "train_score": 0.3561643835616438, + "dropped_stale": 18.0 + }, + { + "step_s": 93.72490900801495, + "train_s": 17.831136393011548, + "wait_for_rollouts_s": 67.11461894691456, + "weight_sync_s": 8.577460916014388, + "mfu": 0.11635745675156325, + "response_tokens_mean": 8530.509933774834, + "reverse_kl": 0.0009700859664008021, + "train_score": 0.5364238410596026, + "dropped_stale": 23.0 + }, + { + "step_s": 132.16780913900584, + "train_s": 55.53849826392252, + "wait_for_rollouts_s": 68.77997702895664, + "weight_sync_s": 7.64045638195239, + "mfu": 0.02029819010086872, + "response_tokens_mean": 8483.881944444445, + "reverse_kl": 0.0006685081752948463, + "train_score": 0.4861111111111111, + "dropped_stale": 16.0 + }, + { + "step_s": 111.10036526399199, + "train_s": 57.68315007293131, + "wait_for_rollouts_s": 43.45418237801641, + "weight_sync_s": 9.748705380945466, + "mfu": 0.045205051331039824, + "response_tokens_mean": 9809.781609195403, + "reverse_kl": 0.0006194195011630654, + "train_score": 0.46551724137931033, + "dropped_stale": 46.0 + }, + { + "step_s": 139.00324622693006, + "train_s": 35.42562606802676, + "wait_for_rollouts_s": 95.30031715007499, + "weight_sync_s": 8.057248550001532, + "mfu": 0.06661285470488915, + "response_tokens_mean": 10808.148936170213, + "reverse_kl": 0.0007953346939757466, + "train_score": 0.5, + "dropped_stale": 60.0 + }, + { + "step_s": 147.270811320981, + "train_s": 21.205964643973857, + "wait_for_rollouts_s": 117.81079692195635, + "weight_sync_s": 8.052056271000765, + "mfu": 0.12286263977912089, + "response_tokens_mean": 9625.025806451613, + "reverse_kl": 0.0007957383058965206, + "train_score": 0.5096774193548387, + "dropped_stale": 27.0 + }, + { + "step_s": 135.10662515601143, + "train_s": 24.091835576924495, + "wait_for_rollouts_s": 103.46373246901203, + "weight_sync_s": 7.332935592974536, + "mfu": 0.10681731117759233, + "response_tokens_mean": 10142.598540145986, + "reverse_kl": 0.001023666700348258, + "train_score": 0.5620437956204379, + "dropped_stale": 9.0 + }, + { + "step_s": 92.48810204700567, + "train_s": 20.672371904016472, + "wait_for_rollouts_s": 63.055761488038115, + "weight_sync_s": 8.552990477066487, + "mfu": 0.06723479335045987, + "response_tokens_mean": 9256.427536231884, + "reverse_kl": 0.0007763210451230407, + "train_score": 0.5942028985507246, + "dropped_stale": 10.0 + }, + { + "step_s": 119.50264672306366, + "train_s": 33.13773320696782, + "wait_for_rollouts_s": 78.05186863790732, + "weight_sync_s": 8.119539730949327, + "mfu": 0.06278390381696958, + "response_tokens_mean": 8481.077922077922, + "reverse_kl": 0.0008169288048520684, + "train_score": 0.5194805194805194, + "dropped_stale": 26.0 + }, + { + "step_s": 80.46022387593985, + "train_s": 20.07488451502286, + "wait_for_rollouts_s": 51.31910520209931, + "weight_sync_s": 8.86391131603159, + "mfu": 0.057809576024360265, + "response_tokens_mean": 9227.723684210527, + "reverse_kl": 0.0007599100936204195, + "train_score": 0.5986842105263158, + "dropped_stale": 24.0 + }, + { + "step_s": 100.31909861892927, + "train_s": 18.82646247802768, + "wait_for_rollouts_s": 72.91362332506105, + "weight_sync_s": 8.39562535600271, + "mfu": 0.11069960416034083, + "response_tokens_mean": 7861.0822784810125, + "reverse_kl": 0.0008074099896475673, + "train_score": 0.6012658227848101, + "dropped_stale": 30.0 + }, + { + "step_s": 89.54183928796556, + "train_s": 17.5268189170165, + "wait_for_rollouts_s": 63.10842035198584, + "weight_sync_s": 8.693991623003967, + "mfu": 0.11891543468242234, + "response_tokens_mean": 8364.45390070922, + "reverse_kl": 0.0009619193151593208, + "train_score": 0.6028368794326241, + "dropped_stale": 13.0 + }, + { + "step_s": 118.7571992619196, + "train_s": 35.006838553003035, + "wait_for_rollouts_s": 76.30982511793263, + "weight_sync_s": 7.246274672914296, + "mfu": 0.029606722235762228, + "response_tokens_mean": 8754.493670886075, + "reverse_kl": 0.0007734878454357386, + "train_score": 0.5759493670886076, + "dropped_stale": 30.0 + }, + { + "step_s": 82.84999850799795, + "train_s": 21.063905091956258, + "wait_for_rollouts_s": 53.89681686996482, + "weight_sync_s": 7.694233054993674, + "mfu": 0.123634573671898, + "response_tokens_mean": 9179.376623376624, + "reverse_kl": 0.0007314156973734498, + "train_score": 0.6363636363636364, + "dropped_stale": 26.0 + } + ], + "step_s_median": 105.24299909302499, + "step_s_total": 2593.0698417957174, + "train_s_median": 22.648900110449176, + "train_s_total": 865.0880303237354, + "wait_for_rollouts_s_median": 66.35087705496699, + "wait_for_rollouts_s_total": 1555.1479031768395, + "weight_sync_s_median": 8.147213581483811, + "weight_sync_s_total": 163.90986432670616, + "mfu_mean": 0.07212657594088076, + "response_tokens_mean_mean": 9071.821450850868, + "reverse_kl_first": -0.00047136563807725906, + "reverse_kl_last": 0.0007314156973734498, + "train_score_first": 0.2734375, + "train_score_last": 0.6363636363636364, + "dropped_stale_total": 751.0 + }, + "roles": { + "generation": { + "samples": 613, + "seconds": 3060.326251268387, + "gpus": { + "0": { + "utilization_mean_pct": 18.773246329526916, + "busy_fraction": 0.3132137030995106, + "memory_peak_gib": 169.51953125, + "power_mean_w": 374.1085318107667, + "energy_kwh": 0.31820277639260225 + }, + "1": { + "utilization_mean_pct": 0.0, + "busy_fraction": 0.0, + "memory_peak_gib": 0.0, + "power_mean_w": 193.92066884176182, + "energy_kwh": 0.16485081086628173 + }, + "2": { + "utilization_mean_pct": 88.43556280587276, + "busy_fraction": 0.9200652528548124, + "memory_peak_gib": 149.220703125, + "power_mean_w": 722.3237683523654, + "energy_kwh": 0.6146854632263105 + }, + "3": { + "utilization_mean_pct": 88.78303425774878, + "busy_fraction": 0.9168026101141925, + "memory_peak_gib": 149.23828125, + "power_mean_w": 742.0151712887439, + "energy_kwh": 0.6314420833581799 + }, + "4": { + "utilization_mean_pct": 87.9021207177814, + "busy_fraction": 0.9119086460032626, + "memory_peak_gib": 149.185546875, + "power_mean_w": 706.3587765089723, + "energy_kwh": 0.6011050537656341 + }, + "5": { + "utilization_mean_pct": 87.97063621533442, + "busy_fraction": 0.9151712887438825, + "memory_peak_gib": 149.19921875, + "power_mean_w": 727.4530831973899, + "energy_kwh": 0.6190519554378404 + }, + "6": { + "utilization_mean_pct": 81.0326264274062, + "busy_fraction": 0.8482871125611745, + "memory_peak_gib": 149.19140625, + "power_mean_w": 664.4599021207177, + "energy_kwh": 0.5654227689089703 + }, + "7": { + "utilization_mean_pct": 81.23164763458401, + "busy_fraction": 0.8466557911908646, + "memory_peak_gib": 149.181640625, + "power_mean_w": 698.411370309951, + "energy_kwh": 0.5943149089877996 + } + }, + "gpu_energy_kwh": 4.109075820943619, + "cpu_utilization_pct": 3.914734550188914, + "network": { + "rx_gb": 0.150082118, + "tx_gb": 7.713288294, + "rx_gbit_s": 0.00039232972089246174, + "tx_gbit_s": 0.020163309819149225 + }, + "engines": { + "127.0.0.1:29000": { + "seconds": 3055.325673341751 + }, + "127.0.0.1:8100": { + "seconds": 2845.3053905963898, + "generation_tokens": 20756047.0, + "generation_tokens_per_s": 7294.839797723587, + "prompt_tokens": 265780.0, + "prompt_tokens_per_s": 93.41000824670395 + }, + "127.0.0.1:8110": { + "seconds": 2845.3053905963898, + "generation_tokens": 20216445.0, + "generation_tokens_per_s": 7105.193371092772, + "prompt_tokens": 257824.0, + "prompt_tokens_per_s": 90.61382333583491 + }, + "127.0.0.1:8120": { + "seconds": 2845.3053905963898, + "generation_tokens": 18455607.0, + "generation_tokens_per_s": 6486.336075204784, + "prompt_tokens": 246911.0, + "prompt_tokens_per_s": 86.77838267063707 + }, + "127.0.0.1:8200": { + "seconds": 2915.3127222061157, + "generation_tokens": 3359.0, + "generation_tokens_per_s": 1.1521920013637958, + "prompt_tokens": 31335390.0, + "prompt_tokens_per_s": 10748.551865916961 + } + } + }, + "learner": { + "samples": 612, + "seconds": 3055.2958657741547, + "gpus": { + "0": { + "utilization_mean_pct": 24.225859247135844, + "busy_fraction": 0.2978723404255319, + "memory_peak_gib": 69.79296875, + "power_mean_w": 311.3138134206219, + "energy_kwh": 0.2642923050939838 + }, + "1": { + "utilization_mean_pct": 32.55646481178396, + "busy_fraction": 0.3698854337152209, + "memory_peak_gib": 69.724609375, + "power_mean_w": 308.11471358428804, + "energy_kwh": 0.2615809138969223 + }, + "2": { + "utilization_mean_pct": 30.170212765957448, + "busy_fraction": 0.3518821603927987, + "memory_peak_gib": 69.3359375, + "power_mean_w": 308.2317512274959, + "energy_kwh": 0.2616805104802019 + }, + "3": { + "utilization_mean_pct": 29.518821603927986, + "busy_fraction": 0.3404255319148936, + "memory_peak_gib": 69.52734375, + "power_mean_w": 318.38690671031094, + "energy_kwh": 0.27030031177256375 + }, + "4": { + "utilization_mean_pct": 31.309328968903436, + "busy_fraction": 0.3649754500818331, + "memory_peak_gib": 69.703125, + "power_mean_w": 304.3416039279869, + "energy_kwh": 0.25837393723843366 + }, + "5": { + "utilization_mean_pct": 29.42389525368249, + "busy_fraction": 0.3469721767594108, + "memory_peak_gib": 69.373046875, + "power_mean_w": 312.2945171849427, + "energy_kwh": 0.26512922030914754 + }, + "6": { + "utilization_mean_pct": 30.384615384615383, + "busy_fraction": 0.3453355155482815, + "memory_peak_gib": 69.529296875, + "power_mean_w": 310.0766939443535, + "energy_kwh": 0.26324463226111333 + }, + "7": { + "utilization_mean_pct": 30.60229132569558, + "busy_fraction": 0.3436988543371522, + "memory_peak_gib": 70.24609375, + "power_mean_w": 312.12931260229135, + "energy_kwh": 0.26498985351393156 + } + }, + "gpu_energy_kwh": 2.109591684566298, + "cpu_utilization_pct": 2.031318689317596, + "network": { + "rx_gb": 7.795754054, + "tx_gb": 0.15032705, + "rx_gbit_s": 0.020412436363572147, + "tx_gbit_s": 0.00039361700235707924 + }, + "engines": {} + } + } + } +] diff --git a/docs/caesar-opd-2026-09-27/data/telemetry.json b/docs/caesar-opd-2026-09-27/data/telemetry.json new file mode 100644 index 0000000..7e6ed62 --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/telemetry.json @@ -0,0 +1,1657 @@ +[ + { + "run": "miles-1021", + "job_id": "1021", + "nodes": [ + "gpu-nodes-2", + "gpu-nodes-0" + ], + "exit_code": 0, + "wall_seconds": 4048.0985386371613, + "updates": { + "updates": 20, + "per_update": [ + { + "step_s": 597.6700809001923, + "train_s": 204.63222193717957, + "wait_for_rollouts_s": 393.0378589630127, + "weight_sync_s": 4.391594409942627, + "mfu": 0.0030243524968915306, + "response_tokens_mean": 5095.9609375, + "reverse_kl": 0.003356889123097062, + "train_score": 0.21875, + "dropped_stale": null + }, + { + "step_s": 85.3843035697937, + "train_s": 82.20186066627502, + "wait_for_rollouts_s": 3.1824429035186768, + "weight_sync_s": 2.7689473628997803, + "mfu": 0.01529696303178155, + "response_tokens_mean": 8960.015625, + "reverse_kl": 0.0025822308380156755, + "train_score": 0.1796875, + "dropped_stale": null + }, + { + "step_s": 49.67130136489868, + "train_s": 46.45985817909241, + "wait_for_rollouts_s": 3.2114431858062744, + "weight_sync_s": 2.8194637298583984, + "mfu": 0.03316376684281018, + "response_tokens_mean": 10167.5546875, + "reverse_kl": 0.0023739386815577745, + "train_score": 0.2421875, + "dropped_stale": null + }, + { + "step_s": 52.70484638214111, + "train_s": 19.810572624206543, + "wait_for_rollouts_s": 32.89427375793457, + "weight_sync_s": 2.857557535171509, + "mfu": 0.05063639663957783, + "response_tokens_mean": 7352.8828125, + "reverse_kl": 0.00221398938447237, + "train_score": 0.3046875, + "dropped_stale": null + }, + { + "step_s": 173.03067803382874, + "train_s": 75.47511911392212, + "wait_for_rollouts_s": 97.55555891990662, + "weight_sync_s": 3.0035948753356934, + "mfu": 0.016169032739310852, + "response_tokens_mean": 8722.484375, + "reverse_kl": 0.001588120823726058, + "train_score": 0.40625, + "dropped_stale": null + }, + { + "step_s": 92.14266276359558, + "train_s": 32.0719735622406, + "wait_for_rollouts_s": 60.07068920135498, + "weight_sync_s": 2.7061779499053955, + "mfu": 0.04048767712516813, + "response_tokens_mean": 9136.859375, + "reverse_kl": 0.0013828104129061103, + "train_score": 0.5390625, + "dropped_stale": null + }, + { + "step_s": 125.42454051971436, + "train_s": 31.142021417617798, + "wait_for_rollouts_s": 94.28251910209656, + "weight_sync_s": 3.3669371604919434, + "mfu": 0.040927128628428414, + "response_tokens_mean": 8904.140625, + "reverse_kl": 0.0011549877235665917, + "train_score": 0.6875, + "dropped_stale": null + }, + { + "step_s": 136.48607563972473, + "train_s": 46.07018208503723, + "wait_for_rollouts_s": 90.4158935546875, + "weight_sync_s": 2.8655331134796143, + "mfu": 0.02446357999777555, + "response_tokens_mean": 8183.2109375, + "reverse_kl": 0.0011093395296484232, + "train_score": 0.7890625, + "dropped_stale": null + }, + { + "step_s": 103.02271604537964, + "train_s": 25.668120622634888, + "wait_for_rollouts_s": 77.35459542274475, + "weight_sync_s": 3.088804006576538, + "mfu": 0.04676611893380043, + "response_tokens_mean": 8570.671875, + "reverse_kl": 0.0012070070952177048, + "train_score": 0.7421875, + "dropped_stale": null + }, + { + "step_s": 123.96711587905884, + "train_s": 28.636270999908447, + "wait_for_rollouts_s": 95.33084487915039, + "weight_sync_s": 3.118975877761841, + "mfu": 0.04124240737702938, + "response_tokens_mean": 8383.578125, + "reverse_kl": 0.0010992811294272542, + "train_score": 0.7890625, + "dropped_stale": null + }, + { + "step_s": 489.05251717567444, + "train_s": 39.53125739097595, + "wait_for_rollouts_s": 449.5212597846985, + "weight_sync_s": 2.8751144409179688, + "mfu": 0.03226402350750038, + "response_tokens_mean": 8958.8671875, + "reverse_kl": 0.0010076625039801002, + "train_score": 0.84375, + "dropped_stale": null + }, + { + "step_s": 62.75617790222168, + "train_s": 59.522186040878296, + "wait_for_rollouts_s": 3.233991861343384, + "weight_sync_s": 2.8978049755096436, + "mfu": 0.0462182180338812, + "response_tokens_mean": 15503.3828125, + "reverse_kl": 0.001099989516660571, + "train_score": 0.7421875, + "dropped_stale": null + }, + { + "step_s": 39.768423318862915, + "train_s": 20.961050510406494, + "wait_for_rollouts_s": 18.80737280845642, + "weight_sync_s": 3.0133984088897705, + "mfu": 0.04256496277746578, + "response_tokens_mean": 6876.3046875, + "reverse_kl": 0.0011806434486061335, + "train_score": 0.8515625, + "dropped_stale": null + }, + { + "step_s": 150.1284806728363, + "train_s": 24.47415065765381, + "wait_for_rollouts_s": 125.6543300151825, + "weight_sync_s": 3.081162214279175, + "mfu": 0.04836823928516223, + "response_tokens_mean": 8475.109375, + "reverse_kl": 0.001345189055427909, + "train_score": 0.875, + "dropped_stale": null + }, + { + "step_s": 114.9930911064148, + "train_s": 34.51295280456543, + "wait_for_rollouts_s": 80.48013830184937, + "weight_sync_s": 2.8879215717315674, + "mfu": 0.03823127627445985, + "response_tokens_mean": 9262.421875, + "reverse_kl": 0.0013528410345315933, + "train_score": 0.8515625, + "dropped_stale": null + }, + { + "step_s": 159.2911238670349, + "train_s": 57.04923367500305, + "wait_for_rollouts_s": 102.24189019203186, + "weight_sync_s": 2.877606153488159, + "mfu": 0.021527706706656023, + "response_tokens_mean": 8760.3203125, + "reverse_kl": 0.0011987559264525771, + "train_score": 0.9140625, + "dropped_stale": null + }, + { + "step_s": 113.69561553001404, + "train_s": 51.25014662742615, + "wait_for_rollouts_s": 62.44546890258789, + "weight_sync_s": 2.8931775093078613, + "mfu": 0.025519899096840878, + "response_tokens_mean": 9162.0078125, + "reverse_kl": 0.0011458165245130658, + "train_score": 0.828125, + "dropped_stale": null + }, + { + "step_s": 101.72297954559326, + "train_s": 26.842811346054077, + "wait_for_rollouts_s": 74.88016819953918, + "weight_sync_s": 3.022257089614868, + "mfu": 0.0413513915153491, + "response_tokens_mean": 8136.84375, + "reverse_kl": 0.0010855741566047072, + "train_score": 0.8515625, + "dropped_stale": null + }, + { + "step_s": 127.55266189575195, + "train_s": 27.07021951675415, + "wait_for_rollouts_s": 100.4824423789978, + "weight_sync_s": 2.7347960472106934, + "mfu": 0.04767658626396127, + "response_tokens_mean": 9042.09375, + "reverse_kl": 0.001144160982221365, + "train_score": 0.8515625, + "dropped_stale": null + }, + { + "step_s": 144.55442261695862, + "train_s": 39.72490930557251, + "wait_for_rollouts_s": 104.82951331138611, + "weight_sync_s": 2.762937545776367, + "mfu": 0.03517531648738338, + "response_tokens_mean": 9667.96875, + "reverse_kl": 0.0012208757689222693, + "train_score": 0.8515625, + "dropped_stale": null + } + ], + "step_s_median": 119.48010349273682, + "step_s_total": 3043.0198147296906, + "train_s_median": 37.02210509777069, + "train_s_total": 973.1071190834045, + "wait_for_rollouts_s_median": 85.44801592826843, + "wait_for_rollouts_s_total": 2069.912695646286, + "weight_sync_s_median": 2.8905495405197144, + "weight_sync_s_total": 60.033761978149414, + "mfu_mean": 0.0345537521880617, + "response_tokens_mean_mean": 8866.133984375, + "reverse_kl_first": 0.003356889123097062, + "reverse_kl_last": 0.0012208757689222693, + "train_score_first": 0.21875, + "train_score_last": 0.8515625 + }, + "roles": { + "generation": { + "samples": 801, + "seconds": 4000.437460422516, + "gpus": { + "0": { + "utilization_mean_pct": 19.057428214731587, + "busy_fraction": 0.2958801498127341, + "memory_peak_gib": 154.7607421875, + "power_mean_w": 390.94309612983767, + "energy_kwh": 0.434628261650361 + }, + "1": { + "utilization_mean_pct": 0.0, + "busy_fraction": 0.0, + "memory_peak_gib": 0.0009765625, + "power_mean_w": 194.32460674157304, + "energy_kwh": 0.21594068232663935 + }, + "2": { + "utilization_mean_pct": 79.28214731585518, + "busy_fraction": 0.8177278401997503, + "memory_peak_gib": 129.96875, + "power_mean_w": 679.6140574282147, + "energy_kwh": 0.7558027726408084 + }, + "3": { + "utilization_mean_pct": 79.29588014981273, + "busy_fraction": 0.8214731585518102, + "memory_peak_gib": 129.97265625, + "power_mean_w": 683.6854681647941, + "energy_kwh": 0.760329091615414 + }, + "4": { + "utilization_mean_pct": 79.71660424469414, + "busy_fraction": 0.8214731585518102, + "memory_peak_gib": 129.93359375, + "power_mean_w": 673.9209488139826, + "energy_kwh": 0.7494771332631137 + }, + "5": { + "utilization_mean_pct": 79.6903870162297, + "busy_fraction": 0.8227215980024969, + "memory_peak_gib": 129.93359375, + "power_mean_w": 680.8332958801498, + "energy_kwh": 0.7571555772976544 + }, + "6": { + "utilization_mean_pct": 85.35705368289638, + "busy_fraction": 0.8751560549313359, + "memory_peak_gib": 132.859375, + "power_mean_w": 758.4422347066167, + "energy_kwh": 0.8435082399570293 + }, + "7": { + "utilization_mean_pct": 85.52684144818976, + "busy_fraction": 0.8826466916354557, + "memory_peak_gib": 132.85546875, + "power_mean_w": 786.9060424469413, + "energy_kwh": 0.8751614755566365 + } + }, + "gpu_energy_kwh": 5.392003234307657, + "cpu_utilization_pct": 3.2885624462766883, + "network": { + "rx_gb": 0.429601338, + "tx_gb": 4.739228189, + "rx_gbit_s": 0.0008591087194841469, + "tx_gbit_s": 0.009477419878973846 + }, + "engines": { + "127.0.0.1:30000": { + "seconds": 3875.421417951584, + "generation_tokens": 8.0, + "generation_tokens_per_s": 0.002064291631083705, + "prompt_tokens": 55652626.0, + "prompt_tokens_per_s": 14360.406262453926 + } + } + }, + "learner": { + "samples": 796, + "seconds": 3975.4714756011963, + "gpus": { + "0": { + "utilization_mean_pct": 16.95088161209068, + "busy_fraction": 0.21536523929471033, + "memory_peak_gib": 177.5888671875, + "power_mean_w": 282.9374307304786, + "energy_kwh": 0.3124238267899142 + }, + "1": { + "utilization_mean_pct": 17.025188916876573, + "busy_fraction": 0.2052896725440806, + "memory_peak_gib": 177.5126953125, + "power_mean_w": 284.02084382871533, + "energy_kwh": 0.31359329558685356 + }, + "2": { + "utilization_mean_pct": 18.200251889168765, + "busy_fraction": 0.21032745591939547, + "memory_peak_gib": 178.3427734375, + "power_mean_w": 278.69400503778337, + "energy_kwh": 0.30780221652624606 + }, + "3": { + "utilization_mean_pct": 17.78715365239295, + "busy_fraction": 0.19899244332493704, + "memory_peak_gib": 178.3427734375, + "power_mean_w": 288.6645843828715, + "energy_kwh": 0.31881108195999264 + }, + "4": { + "utilization_mean_pct": 16.326196473551637, + "busy_fraction": 0.181360201511335, + "memory_peak_gib": 178.2841796875, + "power_mean_w": 277.8652141057935, + "energy_kwh": 0.30688493149320284 + }, + "5": { + "utilization_mean_pct": 17.384130982367758, + "busy_fraction": 0.19269521410579346, + "memory_peak_gib": 178.1865234375, + "power_mean_w": 281.06507556675064, + "energy_kwh": 0.31042213752542536 + }, + "6": { + "utilization_mean_pct": 17.12846347607053, + "busy_fraction": 0.190176322418136, + "memory_peak_gib": 178.3134765625, + "power_mean_w": 280.70523929471034, + "energy_kwh": 0.3099466911036326 + }, + "7": { + "utilization_mean_pct": 16.612090680100756, + "busy_fraction": 0.190176322418136, + "memory_peak_gib": 178.0146484375, + "power_mean_w": 281.2691183879093, + "energy_kwh": 0.3106525896345013 + } + }, + "gpu_energy_kwh": 2.4905367706197685, + "cpu_utilization_pct": 2.0622629259894776, + "network": { + "rx_gb": 4.734458411, + "tx_gb": 0.429478109, + "rx_gbit_s": 0.009527339718183288, + "tx_gbit_s": 0.0008642559487816253 + }, + "engines": { + "10.244.218.33:20000": { + "seconds": 3530.431127309799, + "generation_tokens": 18208571.0, + "generation_tokens_per_s": 5157.60550011777, + "prompt_tokens": 200294.0, + "prompt_tokens_per_s": 56.733580907617004 + }, + "10.244.218.33:20032": { + "seconds": 3530.431127309799, + "generation_tokens": 18306751.0, + "generation_tokens_per_s": 5185.415134822304, + "prompt_tokens": 202230.0, + "prompt_tokens_per_s": 57.28195585962329 + }, + "10.244.218.33:20033": { + "seconds": 3530.431127309799, + "generation_tokens": 27100773.0, + "generation_tokens_per_s": 7676.335275417448, + "prompt_tokens": 289146.0, + "prompt_tokens_per_s": 81.90104538884752 + } + } + } + } + }, + { + "run": "verl-966", + "job_id": "966", + "nodes": [ + "gpu-nodes-3", + "gpu-nodes-1" + ], + "exit_code": 0, + "wall_seconds": 4275.904811620712, + "updates": { + "updates": 19, + "per_update": [ + { + "step_s": 172.75680740200914, + "train_s": 81.34275495109614, + "wait_for_rollouts_s": 80.99835253704805, + "weight_sync_s": 10.398343034088612, + "mfu": 0.011889213156342155, + "response_tokens_mean": 5790.7578125, + "reverse_kl": 0.0025430515815969557, + "train_score": 0.234375, + "dropped_stale": 0.0 + }, + { + "step_s": 136.59216520702466, + "train_s": 94.73462918994483, + "wait_for_rollouts_s": 32.20524190401193, + "weight_sync_s": 9.643874311004765, + "mfu": 0.02952618674129811, + "response_tokens_mean": 15179.5625, + "reverse_kl": 0.0014494075439870358, + "train_score": 0.328125, + "dropped_stale": 0.0 + }, + { + "step_s": 81.73101178300567, + "train_s": 38.952785343979485, + "wait_for_rollouts_s": 32.53947107202839, + "weight_sync_s": 10.230698341038078, + "mfu": 0.06453590741609518, + "response_tokens_mean": 13393.6171875, + "reverse_kl": 0.0013652302382070047, + "train_score": 0.28125, + "dropped_stale": 0.0 + }, + { + "step_s": 105.64325875905342, + "train_s": 31.148920328938402, + "wait_for_rollouts_s": 64.30604800803121, + "weight_sync_s": 10.17914696491789, + "mfu": 0.06022929812880029, + "response_tokens_mean": 10347.109375, + "reverse_kl": 0.0014484586426988244, + "train_score": 0.3828125, + "dropped_stale": 0.0 + }, + { + "step_s": 132.14637610001955, + "train_s": 56.625891607953236, + "wait_for_rollouts_s": 64.37457262503449, + "weight_sync_s": 11.136496708961204, + "mfu": 0.04316318917389978, + "response_tokens_mean": 13335.1953125, + "reverse_kl": 0.0012391659206514305, + "train_score": 0.3984375, + "dropped_stale": 0.0 + }, + { + "step_s": 94.60119743400719, + "train_s": 43.05516555800568, + "wait_for_rollouts_s": 41.55802140396554, + "weight_sync_s": 9.97943758906331, + "mfu": 0.06503260522656014, + "response_tokens_mean": 14871.390625, + "reverse_kl": 0.0011688995277836511, + "train_score": 0.546875, + "dropped_stale": 0.0 + }, + { + "step_s": 90.62578713393304, + "train_s": 32.734760722960345, + "wait_for_rollouts_s": 48.36108275002334, + "weight_sync_s": 9.521579338004813, + "mfu": 0.0666384353442171, + "response_tokens_mean": 11808.2578125, + "reverse_kl": 0.0011762890717363916, + "train_score": 0.6796875, + "dropped_stale": 0.0 + }, + { + "step_s": 94.74705709191039, + "train_s": 33.67108616500627, + "wait_for_rollouts_s": 50.96918722696137, + "weight_sync_s": 10.098212170996703, + "mfu": 0.060401314461821586, + "response_tokens_mean": 11319.6640625, + "reverse_kl": 0.0011188759576725715, + "train_score": 0.7265625, + "dropped_stale": 0.0 + }, + { + "step_s": 95.14703081804328, + "train_s": 30.722841551993042, + "wait_for_rollouts_s": 52.7104216030566, + "weight_sync_s": 11.704338183975779, + "mfu": 0.06581042522093368, + "response_tokens_mean": 11139.609375, + "reverse_kl": 0.0011161956790601835, + "train_score": 0.765625, + "dropped_stale": 0.0 + }, + { + "step_s": 105.18525407405104, + "train_s": 34.75646492501255, + "wait_for_rollouts_s": 60.12909598695114, + "weight_sync_s": 10.29059942997992, + "mfu": 0.06517252193855584, + "response_tokens_mean": 12332.8671875, + "reverse_kl": 0.0010885439041885547, + "train_score": 0.765625, + "dropped_stale": 0.0 + }, + { + "step_s": 63.93575842201244, + "train_s": 52.95747886807658, + "wait_for_rollouts_s": 0.8133310710545629, + "weight_sync_s": 10.15547273599077, + "mfu": 0.0672027109787465, + "response_tokens_mean": 18583.7578125, + "reverse_kl": 0.0009980842023651348, + "train_score": 0.65625, + "dropped_stale": 0.0 + }, + { + "step_s": 102.12371621606871, + "train_s": 26.902333624078892, + "wait_for_rollouts_s": 65.00271527608857, + "weight_sync_s": 10.20806024200283, + "mfu": 0.054292389062245885, + "response_tokens_mean": 8397.875, + "reverse_kl": 0.001154337865045818, + "train_score": 0.8671875, + "dropped_stale": 0.0 + }, + { + "step_s": 118.19352968095336, + "train_s": 35.898216894012876, + "wait_for_rollouts_s": 66.49870058090892, + "weight_sync_s": 15.786726179998368, + "mfu": 0.06645883978820495, + "response_tokens_mean": 13056.46875, + "reverse_kl": 0.0010989276031523332, + "train_score": 0.828125, + "dropped_stale": 0.0 + }, + { + "step_s": 108.6782383149257, + "train_s": 43.15239067701623, + "wait_for_rollouts_s": 56.164290760993026, + "weight_sync_s": 9.350678071961738, + "mfu": 0.06103391322413672, + "response_tokens_mean": 13988.1953125, + "reverse_kl": 0.0011210037073396961, + "train_score": 0.78125, + "dropped_stale": 0.0 + }, + { + "step_s": 108.5387348550139, + "train_s": 43.42840760096442, + "wait_for_rollouts_s": 55.90573708200827, + "weight_sync_s": 9.193898663972504, + "mfu": 0.0647361849519656, + "response_tokens_mean": 14930.3515625, + "reverse_kl": 0.0010788498270812852, + "train_score": 0.6953125, + "dropped_stale": 0.0 + }, + { + "step_s": 114.77016145701054, + "train_s": 40.98146297293715, + "wait_for_rollouts_s": 60.788905465975404, + "weight_sync_s": 12.99071964493487, + "mfu": 0.0613098745445717, + "response_tokens_mean": 13386.625, + "reverse_kl": 0.0010952472630378907, + "train_score": 0.78125, + "dropped_stale": 0.0 + }, + { + "step_s": 114.75836611504201, + "train_s": 43.65625631902367, + "wait_for_rollouts_s": 59.794136806041934, + "weight_sync_s": 11.300301382085308, + "mfu": 0.058232281320206136, + "response_tokens_mean": 13662.296875, + "reverse_kl": 0.0010441575916502188, + "train_score": 0.78125, + "dropped_stale": 0.0 + }, + { + "step_s": 100.74360607098788, + "train_s": 38.74039398995228, + "wait_for_rollouts_s": 52.35113900597207, + "weight_sync_s": 9.64348242606502, + "mfu": 0.06680400259620764, + "response_tokens_mean": 13830.140625, + "reverse_kl": 0.001022967876679104, + "train_score": 0.7265625, + "dropped_stale": 0.0 + }, + { + "step_s": 111.06763072696049, + "train_s": 39.85531761904713, + "wait_for_rollouts_s": 61.64706286799628, + "weight_sync_s": 9.55637810495682, + "mfu": 0.06325120809042098, + "response_tokens_mean": 13472.2421875, + "reverse_kl": 0.0010555996393577516, + "train_score": 0.7890625, + "dropped_stale": 0.0 + } + ], + "step_s_median": 105.64325875905342, + "step_s_total": 2051.9856876620324, + "train_s_median": 39.85531761904713, + "train_s_total": 843.3175589099992, + "wait_for_rollouts_s_median": 56.164290760993026, + "wait_for_rollouts_s_total": 1007.1175140341511, + "weight_sync_s_median": 10.17914696491789, + "weight_sync_s_total": 201.3684435239993, + "mfu_mean": 0.05766950007185421, + "response_tokens_mean_mean": 12780.314967105263, + "reverse_kl_first": 0.0025430515815969557, + "reverse_kl_last": 0.0010555996393577516, + "train_score_first": 0.234375, + "train_score_last": 0.7890625, + "dropped_stale_total": 0.0 + }, + "roles": { + "generation": { + "samples": 853, + "seconds": 4260.567019462585, + "gpus": { + "0": { + "utilization_mean_pct": 20.303634232121922, + "busy_fraction": 0.2321219226260258, + "memory_peak_gib": 122.0029296875, + "power_mean_w": 323.9786283704572, + "energy_kwh": 0.38361796352130173 + }, + "1": { + "utilization_mean_pct": 20.172332942555684, + "busy_fraction": 0.2321219226260258, + "memory_peak_gib": 123.4111328125, + "power_mean_w": 329.9781008206331, + "energy_kwh": 0.39071048662178653 + }, + "2": { + "utilization_mean_pct": 19.902696365767877, + "busy_fraction": 0.22626025791324736, + "memory_peak_gib": 123.2177734375, + "power_mean_w": 319.4224267291911, + "energy_kwh": 0.37821249557000197 + }, + "3": { + "utilization_mean_pct": 19.533411488862836, + "busy_fraction": 0.2250879249706917, + "memory_peak_gib": 123.4111328125, + "power_mean_w": 319.29635404454865, + "energy_kwh": 0.3780656793170849 + }, + "4": { + "utilization_mean_pct": 19.844079718640092, + "busy_fraction": 0.2250879249706917, + "memory_peak_gib": 123.1181640625, + "power_mean_w": 323.18451348182884, + "energy_kwh": 0.3826684324148522 + }, + "5": { + "utilization_mean_pct": 19.67409144196952, + "busy_fraction": 0.22039859320046892, + "memory_peak_gib": 123.1259765625, + "power_mean_w": 320.46508792497065, + "energy_kwh": 0.3794448559432672 + }, + "6": { + "utilization_mean_pct": 18.80773739742087, + "busy_fraction": 0.21453692848769051, + "memory_peak_gib": 122.9326171875, + "power_mean_w": 314.17090269636583, + "energy_kwh": 0.37199628868521717 + }, + "7": { + "utilization_mean_pct": 18.587338804220398, + "busy_fraction": 0.21688159437280188, + "memory_peak_gib": 123.0986328125, + "power_mean_w": 324.09770222743265, + "energy_kwh": 0.3837484729040947 + } + }, + "gpu_energy_kwh": 3.0484646749776063, + "cpu_utilization_pct": 1.6495872616619778, + "network": { + "rx_gb": 16.46186278, + "tx_gb": 11.716861822, + "rx_gbit_s": 0.0309101820575543, + "tx_gbit_s": 0.022000568034210487 + }, + "engines": {} + }, + "learner": { + "samples": 852, + "seconds": 4255.208590745926, + "gpus": { + "0": { + "utilization_mean_pct": 12.051643192488262, + "busy_fraction": 0.1643192488262911, + "memory_peak_gib": 146.8818359375, + "power_mean_w": 339.5544014084507, + "energy_kwh": 0.40146741572147737 + }, + "1": { + "utilization_mean_pct": 61.786384976525824, + "busy_fraction": 0.6431924882629108, + "memory_peak_gib": 147.259765625, + "power_mean_w": 552.1531103286385, + "energy_kwh": 0.6530386909225033 + }, + "2": { + "utilization_mean_pct": 61.571596244131456, + "busy_fraction": 0.6431924882629108, + "memory_peak_gib": 147.259765625, + "power_mean_w": 539.9368192488263, + "energy_kwh": 0.6385901339863624 + }, + "3": { + "utilization_mean_pct": 60.94131455399061, + "busy_fraction": 0.6349765258215962, + "memory_peak_gib": 147.220703125, + "power_mean_w": 555.2690610328639, + "energy_kwh": 0.6567225809774265 + }, + "4": { + "utilization_mean_pct": 60.955399061032864, + "busy_fraction": 0.6431924882629108, + "memory_peak_gib": 147.220703125, + "power_mean_w": 532.6197652582159, + "energy_kwh": 0.6299283946275506 + }, + "5": { + "utilization_mean_pct": 61.536384976525824, + "busy_fraction": 0.6431924882629108, + "memory_peak_gib": 147.26171875, + "power_mean_w": 549.093485915493, + "energy_kwh": 0.6494210580419872 + }, + "6": { + "utilization_mean_pct": 61.71478873239437, + "busy_fraction": 0.6467136150234741, + "memory_peak_gib": 147.26171875, + "power_mean_w": 541.8520187793428, + "energy_kwh": 0.6408472181258161 + }, + "7": { + "utilization_mean_pct": 0.0, + "busy_fraction": 0.0, + "memory_peak_gib": 0.00390625, + "power_mean_w": 195.7943544600939, + "energy_kwh": 0.23143097907787427 + } + }, + "gpu_energy_kwh": 4.501446471480998, + "cpu_utilization_pct": 3.117016575409126, + "network": { + "rx_gb": 16.931485415, + "tx_gb": 16.486443839, + "rx_gbit_s": 0.03183201961346287, + "tx_gbit_s": 0.03099531971213655 + }, + "engines": {} + } + } + }, + { + "run": "prime-rl-1017", + "job_id": "1017", + "nodes": [ + "gpu-nodes-2", + "gpu-nodes-0" + ], + "exit_code": 0, + "wall_seconds": 2376.532244205475, + "updates": { + "updates": 20, + "per_update": [ + { + "step_s": 144.4788833790226, + "train_s": 32.086351781967096, + "wait_for_rollouts_s": 96.74467610602733, + "weight_sync_s": 10.755033813999034, + "mfu": 0.03870436191907464, + "response_tokens_mean": 3394.25, + "reverse_kl": -0.001533877570182085, + "train_score": 0.1796875, + "dropped_stale": 0.0 + }, + { + "step_s": 35.96255283092614, + "train_s": 24.556351256906055, + "wait_for_rollouts_s": 3.9659091271460056e-05, + "weight_sync_s": 11.224737305077724, + "mfu": 0.07731820607279659, + "response_tokens_mean": 5236.0390625, + "reverse_kl": 0.00015920778969302773, + "train_score": 0.265625, + "dropped_stale": 0.0 + }, + { + "step_s": 153.62348753295373, + "train_s": 33.05279436905403, + "wait_for_rollouts_s": 107.73067473201081, + "weight_sync_s": 12.681057763984427, + "mfu": 0.03922179376981037, + "response_tokens_mean": 4740.947826086956, + "reverse_kl": 0.00014395834296010435, + "train_score": 0.34782608695652173, + "dropped_stale": 102.0 + }, + { + "step_s": 36.31380651902873, + "train_s": 24.277177523938008, + "wait_for_rollouts_s": 3.5470002330839634e-05, + "weight_sync_s": 11.857437881990336, + "mfu": 0.07970278897712857, + "response_tokens_mean": 5880.207142857143, + "reverse_kl": 0.0010230080224573612, + "train_score": 0.36428571428571427, + "dropped_stale": 12.0 + }, + { + "step_s": 150.43857803498395, + "train_s": 24.991932337055914, + "wait_for_rollouts_s": 116.00409923610277, + "weight_sync_s": 9.276290082023479, + "mfu": 0.07775381815339669, + "response_tokens_mean": 4997.0327868852455, + "reverse_kl": 0.00011915319919353351, + "train_score": 0.5737704918032787, + "dropped_stale": 55.0 + }, + { + "step_s": 68.43897247395944, + "train_s": 27.408474265947007, + "wait_for_rollouts_s": 27.956947628059424, + "weight_sync_s": 12.887561341980472, + "mfu": 0.04173735843524286, + "response_tokens_mean": 6313.371428571429, + "reverse_kl": 0.0008889507735148072, + "train_score": 0.6, + "dropped_stale": 12.0 + }, + { + "step_s": 129.25354096095543, + "train_s": 23.72248961997684, + "wait_for_rollouts_s": 90.21377467701677, + "weight_sync_s": 15.151672738022171, + "mfu": 0.036036611260475054, + "response_tokens_mean": 5625.515789473684, + "reverse_kl": 0.0005825496045872569, + "train_score": 0.7631578947368421, + "dropped_stale": 62.0 + }, + { + "step_s": 43.08054239896592, + "train_s": 23.41111319209449, + "wait_for_rollouts_s": 7.928687057923526, + "weight_sync_s": 11.566743440926075, + "mfu": 0.08178071753942047, + "response_tokens_mean": 5488.267515923567, + "reverse_kl": 0.0005513469222933054, + "train_score": 0.8407643312101911, + "dropped_stale": 29.0 + }, + { + "step_s": 119.02783853793517, + "train_s": 20.237185698002577, + "wait_for_rollouts_s": 87.77124396606814, + "weight_sync_s": 10.861088745994493, + "mfu": 0.06413808924692681, + "response_tokens_mean": 4699.148936170212, + "reverse_kl": 0.0005713208811357617, + "train_score": 0.9095744680851063, + "dropped_stale": 60.0 + }, + { + "step_s": 57.25693124590907, + "train_s": 23.92832764191553, + "wait_for_rollouts_s": 22.758872546022758, + "weight_sync_s": 10.396850169985555, + "mfu": 0.07888313305361348, + "response_tokens_mean": 5613.855263157895, + "reverse_kl": 0.0006913860561326146, + "train_score": 0.8881578947368421, + "dropped_stale": 24.0 + }, + { + "step_s": 127.65362238604575, + "train_s": 23.68334661296103, + "wait_for_rollouts_s": 93.61718370509334, + "weight_sync_s": 10.17816310597118, + "mfu": 0.03735566098933637, + "response_tokens_mean": 5412.478723404255, + "reverse_kl": 0.0004626796580851078, + "train_score": 0.9042553191489362, + "dropped_stale": 60.0 + }, + { + "step_s": 76.7000254529994, + "train_s": 23.787804755964316, + "wait_for_rollouts_s": 42.435447611962445, + "weight_sync_s": 10.300998560036533, + "mfu": 0.08168968016059644, + "response_tokens_mean": 6000.940789473684, + "reverse_kl": 0.0008522993302904069, + "train_score": 0.875, + "dropped_stale": 24.0 + }, + { + "step_s": 125.83909300994128, + "train_s": 23.576739586074837, + "wait_for_rollouts_s": 90.33605520799756, + "weight_sync_s": 11.754040983971208, + "mfu": 0.08251423831581124, + "response_tokens_mean": 5806.933333333333, + "reverse_kl": 0.00042326434049755335, + "train_score": 0.8888888888888888, + "dropped_stale": 52.0 + }, + { + "step_s": 81.85396343201865, + "train_s": 23.599713323055767, + "wait_for_rollouts_s": 46.86137806100305, + "weight_sync_s": 11.219248073990457, + "mfu": 0.07987831038126825, + "response_tokens_mean": 6162.547169811321, + "reverse_kl": 0.00042032721103169024, + "train_score": 0.8930817610062893, + "dropped_stale": 31.0 + }, + { + "step_s": 126.21146486699581, + "train_s": 23.49435645400081, + "wait_for_rollouts_s": 91.71109386801254, + "weight_sync_s": 10.832084667985328, + "mfu": 0.0815238374944362, + "response_tokens_mean": 5827.4802259887, + "reverse_kl": 0.0006552739650942385, + "train_score": 0.9096045197740112, + "dropped_stale": 49.0 + }, + { + "step_s": 77.31017562898342, + "train_s": 24.781482699909247, + "wait_for_rollouts_s": 42.72640538599808, + "weight_sync_s": 9.628259644960053, + "mfu": 0.0756686788395719, + "response_tokens_mean": 6099.672727272728, + "reverse_kl": 0.0005843224353156984, + "train_score": 0.9333333333333333, + "dropped_stale": 37.0 + }, + { + "step_s": 120.87991812103428, + "train_s": 23.461442642030306, + "wait_for_rollouts_s": 82.7778837919468, + "weight_sync_s": 14.47798943601083, + "mfu": 0.035789213100589495, + "response_tokens_mean": 5533.695906432748, + "reverse_kl": 0.0009353603818453848, + "train_score": 0.9298245614035088, + "dropped_stale": 43.0 + }, + { + "step_s": 91.78918998595327, + "train_s": 58.399746520095505, + "wait_for_rollouts_s": 23.419462776044384, + "weight_sync_s": 9.788741883006878, + "mfu": 0.03258254620773529, + "response_tokens_mean": 5757.636363636364, + "reverse_kl": 0.0006311070173978806, + "train_score": 0.8863636363636364, + "dropped_stale": 48.0 + }, + { + "step_s": 93.61296518403105, + "train_s": 23.698215640964918, + "wait_for_rollouts_s": 58.57251729595009, + "weight_sync_s": 11.174247522023506, + "mfu": 0.08195336849039331, + "response_tokens_mean": 6132.431451612903, + "reverse_kl": 0.0005433842306956649, + "train_score": 0.9032258064516129, + "dropped_stale": 120.0 + }, + { + "step_s": 92.54560050205328, + "train_s": 23.580536207999103, + "wait_for_rollouts_s": 55.15934814698994, + "weight_sync_s": 13.63917431992013, + "mfu": 0.08247575415399876, + "response_tokens_mean": 5405.511764705882, + "reverse_kl": 0.0004075725737493485, + "train_score": 0.8941176470588236, + "dropped_stale": 42.0 + } + ], + "step_s_median": 93.07928284304217, + "step_s_total": 1952.2711524846964, + "train_s_median": 23.75514718797058, + "train_s_total": 529.7355821299134, + "wait_for_rollouts_s_median": 56.86593272147002, + "wait_for_rollouts_s_total": 1184.7258269293234, + "weight_sync_s_median": 11.196747798006982, + "weight_sync_s_total": 229.65142148185987, + "mfu_mean": 0.06433540832808113, + "response_tokens_mean_mean": 5506.398210364902, + "reverse_kl_first": -0.001533877570182085, + "reverse_kl_last": 0.0004075725737493485, + "train_score_first": 0.1796875, + "train_score_last": 0.8941176470588236, + "dropped_stale_total": 862.0 + }, + "roles": { + "generation": { + "samples": 472, + "seconds": 2355.2709283828735, + "gpus": { + "0": { + "utilization_mean_pct": 17.616525423728813, + "busy_fraction": 0.326271186440678, + "memory_peak_gib": 162.53125, + "power_mean_w": 362.32213983050843, + "energy_kwh": 0.23720655410410826 + }, + "1": { + "utilization_mean_pct": 0.0, + "busy_fraction": 0.0, + "memory_peak_gib": 0.0009765625, + "power_mean_w": 194.06019067796612, + "energy_kwh": 0.12696275846684113 + }, + "2": { + "utilization_mean_pct": 87.04661016949153, + "busy_fraction": 0.913135593220339, + "memory_peak_gib": 167.73046875, + "power_mean_w": 820.9327330508474, + "energy_kwh": 0.537872627958807 + }, + "3": { + "utilization_mean_pct": 87.42372881355932, + "busy_fraction": 0.9152542372881356, + "memory_peak_gib": 167.734375, + "power_mean_w": 844.7290466101695, + "energy_kwh": 0.5534665584633748 + }, + "4": { + "utilization_mean_pct": 86.30296610169492, + "busy_fraction": 0.9004237288135594, + "memory_peak_gib": 167.6953125, + "power_mean_w": 804.9197669491525, + "energy_kwh": 0.527385811288033 + }, + "5": { + "utilization_mean_pct": 86.3177966101695, + "busy_fraction": 0.9046610169491526, + "memory_peak_gib": 167.6953125, + "power_mean_w": 829.92875, + "energy_kwh": 0.5437686698511408 + }, + "6": { + "utilization_mean_pct": 86.99788135593221, + "busy_fraction": 0.9110169491525424, + "memory_peak_gib": 167.6953125, + "power_mean_w": 805.635063559322, + "energy_kwh": 0.5278458847598215 + }, + "7": { + "utilization_mean_pct": 87.45127118644068, + "busy_fraction": 0.9067796610169492, + "memory_peak_gib": 167.69140625, + "power_mean_w": 847.1306355932203, + "energy_kwh": 0.5550335684943524 + } + }, + "gpu_energy_kwh": 3.609542433386479, + "cpu_utilization_pct": 4.929428167940342, + "network": { + "rx_gb": 0.117251284, + "tx_gb": 4.843333213, + "rx_gbit_s": 0.00039826003059615603, + "tx_gbit_s": 0.016451044012420014 + }, + "engines": { + "127.0.0.1:29000": { + "seconds": 2350.2705631256104 + }, + "127.0.0.1:8100": { + "seconds": 2230.2589445114136, + "generation_tokens": 30518217.0, + "generation_tokens_per_s": 13683.710169666274, + "prompt_tokens": 641430.0, + "prompt_tokens_per_s": 287.6033751948562 + }, + "127.0.0.1:8110": { + "seconds": 2230.2589445114136, + "generation_tokens": 29985663.0, + "generation_tokens_per_s": 13444.924444219192, + "prompt_tokens": 625217.0, + "prompt_tokens_per_s": 280.3338157385878 + }, + "127.0.0.1:8120": { + "seconds": 2230.2589445114136, + "generation_tokens": 29472280.0, + "generation_tokens_per_s": 13214.73458161897, + "prompt_tokens": 612173.0, + "prompt_tokens_per_s": 274.4851675212583 + }, + "127.0.0.1:8200": { + "seconds": 2245.260773897171, + "generation_tokens": 3509.0, + "generation_tokens_per_s": 1.5628474165650328, + "prompt_tokens": 19885354.0, + "prompt_tokens_per_s": 8856.589947672026 + } + } + }, + "learner": { + "samples": 466, + "seconds": 2339.7137932777405, + "gpus": { + "0": { + "utilization_mean_pct": 17.39569892473118, + "busy_fraction": 0.3139784946236559, + "memory_peak_gib": 74.25, + "power_mean_w": 301.2775053763441, + "energy_kwh": 0.19569068143930302 + }, + "1": { + "utilization_mean_pct": 26.412903225806453, + "busy_fraction": 0.4021505376344086, + "memory_peak_gib": 71.626953125, + "power_mean_w": 298.1489247311828, + "energy_kwh": 0.19367158370235893 + }, + "2": { + "utilization_mean_pct": 27.653763440860214, + "busy_fraction": 0.3978494623655914, + "memory_peak_gib": 71.625, + "power_mean_w": 296.2956559139785, + "energy_kwh": 0.19246670420889125 + }, + "3": { + "utilization_mean_pct": 26.135483870967743, + "busy_fraction": 0.4043010752688172, + "memory_peak_gib": 71.62890625, + "power_mean_w": 303.6988387096774, + "energy_kwh": 0.19728359207862087 + }, + "4": { + "utilization_mean_pct": 26.76989247311828, + "busy_fraction": 0.3978494623655914, + "memory_peak_gib": 71.626953125, + "power_mean_w": 292.375376344086, + "energy_kwh": 0.18992824606548878 + }, + "5": { + "utilization_mean_pct": 26.941935483870967, + "busy_fraction": 0.3913978494623656, + "memory_peak_gib": 71.626953125, + "power_mean_w": 299.8402150537635, + "energy_kwh": 0.19477787205054098 + }, + "6": { + "utilization_mean_pct": 26.333333333333332, + "busy_fraction": 0.3827956989247312, + "memory_peak_gib": 71.626953125, + "power_mean_w": 301.1916129032258, + "energy_kwh": 0.19565228934521542 + }, + "7": { + "utilization_mean_pct": 24.455913978494625, + "busy_fraction": 0.3763440860215054, + "memory_peak_gib": 72.35546875, + "power_mean_w": 299.5030967741936, + "energy_kwh": 0.1945668244530241 + } + }, + "gpu_energy_kwh": 1.5540377933434433, + "cpu_utilization_pct": 3.901847593437291, + "network": { + "rx_gb": 5.064161989, + "tx_gb": 0.117989125, + "rx_gbit_s": 0.01731549218900159, + "tx_gbit_s": 0.0004034309678012617 + }, + "engines": {} + } + } + }, + { + "run": "slime-979", + "job_id": "979", + "nodes": [ + "gpu-nodes-2", + "gpu-nodes-0" + ], + "exit_code": 0, + "wall_seconds": 5362.61212515831, + "updates": { + "updates": 20, + "per_update": [ + { + "step_s": 907.9744830131531, + "train_s": 270.8119933605194, + "wait_for_rollouts_s": 637.1624896526337, + "weight_sync_s": 5.656439781188965, + "mfu": 0.015341763004965486, + "response_tokens_mean": 10988.671875, + "reverse_kl": 0.0008959413462434895, + "train_score": 0.203125, + "dropped_stale": null + }, + { + "step_s": 71.38871788978577, + "train_s": 68.27548480033875, + "wait_for_rollouts_s": 3.1132330894470215, + "weight_sync_s": 2.8400778770446777, + "mfu": 0.06334983694655988, + "response_tokens_mean": 12940.0, + "reverse_kl": 0.0006864221322757658, + "train_score": 0.2421875, + "dropped_stale": null + }, + { + "step_s": 216.13840222358704, + "train_s": 79.45052528381348, + "wait_for_rollouts_s": 136.68787693977356, + "weight_sync_s": 2.8856163024902344, + "mfu": 0.05935560229279665, + "response_tokens_mean": 13557.46875, + "reverse_kl": 0.00043477039798744954, + "train_score": 0.171875, + "dropped_stale": null + }, + { + "step_s": 153.32333326339722, + "train_s": 36.86121344566345, + "wait_for_rollouts_s": 116.46211981773376, + "weight_sync_s": 2.882200241088867, + "mfu": 0.10671118377002763, + "response_tokens_mean": 12815.0703125, + "reverse_kl": 0.00015857577318456606, + "train_score": 0.34375, + "dropped_stale": null + }, + { + "step_s": 198.3151457309723, + "train_s": 38.58327317237854, + "wait_for_rollouts_s": 159.73187255859375, + "weight_sync_s": 2.776104211807251, + "mfu": 0.10202261438171342, + "response_tokens_mean": 12848.328125, + "reverse_kl": 0.00012835218672080373, + "train_score": 0.46875, + "dropped_stale": null + }, + { + "step_s": 177.99560856819153, + "train_s": 25.422216653823853, + "wait_for_rollouts_s": 152.57339191436768, + "weight_sync_s": 2.877518653869629, + "mfu": 0.1464668564371025, + "response_tokens_mean": 13254.53125, + "reverse_kl": -7.13282964852624e-05, + "train_score": 0.578125, + "dropped_stale": null + }, + { + "step_s": 210.0331301689148, + "train_s": 43.756181955337524, + "wait_for_rollouts_s": 166.27694821357727, + "weight_sync_s": 2.8555424213409424, + "mfu": 0.0907029354909207, + "response_tokens_mean": 12170.9765625, + "reverse_kl": 0.00020243220706106513, + "train_score": 0.6796875, + "dropped_stale": null + }, + { + "step_s": 192.8955590724945, + "train_s": 39.85651993751526, + "wait_for_rollouts_s": 153.03903913497925, + "weight_sync_s": 2.919271230697632, + "mfu": 0.1028257998343201, + "response_tokens_mean": 13263.921875, + "reverse_kl": 0.0001263862468476873, + "train_score": 0.734375, + "dropped_stale": null + }, + { + "step_s": 196.71664237976074, + "train_s": 43.3758864402771, + "wait_for_rollouts_s": 153.34075593948364, + "weight_sync_s": 2.8044750690460205, + "mfu": 0.09227341419175943, + "response_tokens_mean": 12802.8046875, + "reverse_kl": 0.00015785531468281988, + "train_score": 0.7421875, + "dropped_stale": null + }, + { + "step_s": 168.2114543914795, + "train_s": 23.25459122657776, + "wait_for_rollouts_s": 144.95686316490173, + "weight_sync_s": 2.879070281982422, + "mfu": 0.15071029482430587, + "response_tokens_mean": 12317.9140625, + "reverse_kl": -2.1807356915815035e-06, + "train_score": 0.7421875, + "dropped_stale": null + }, + { + "step_s": 436.8235514163971, + "train_s": 23.493101835250854, + "wait_for_rollouts_s": 413.33044958114624, + "weight_sync_s": 2.849902629852295, + "mfu": 0.15884048144808863, + "response_tokens_mean": 12917.953125, + "reverse_kl": 8.776813285749085e-05, + "train_score": 0.828125, + "dropped_stale": null + }, + { + "step_s": 197.64992928504944, + "train_s": 24.56040859222412, + "wait_for_rollouts_s": 173.08952069282532, + "weight_sync_s": 2.8877220153808594, + "mfu": 0.14624873487526235, + "response_tokens_mean": 12915.734375, + "reverse_kl": 0.00025330824246339034, + "train_score": 0.84375, + "dropped_stale": null + }, + { + "step_s": 218.90509963035583, + "train_s": 41.19570779800415, + "wait_for_rollouts_s": 177.70939183235168, + "weight_sync_s": 3.194890260696411, + "mfu": 0.11059231669572066, + "response_tokens_mean": 14241.0703125, + "reverse_kl": 5.21245674462989e-05, + "train_score": 0.75, + "dropped_stale": null + }, + { + "step_s": 180.08714818954468, + "train_s": 23.814002513885498, + "wait_for_rollouts_s": 156.27314567565918, + "weight_sync_s": 2.801879644393921, + "mfu": 0.16105268978700543, + "response_tokens_mean": 13206.1796875, + "reverse_kl": 0.00011198982156201964, + "train_score": 0.8125, + "dropped_stale": null + }, + { + "step_s": 204.65650701522827, + "train_s": 30.005844831466675, + "wait_for_rollouts_s": 174.6506621837616, + "weight_sync_s": 2.8053951263427734, + "mfu": 0.1438595143770505, + "response_tokens_mean": 14824.53125, + "reverse_kl": 4.038979545839538e-05, + "train_score": 0.8125, + "dropped_stale": null + }, + { + "step_s": 203.25741934776306, + "train_s": 27.348483324050903, + "wait_for_rollouts_s": 175.90893602371216, + "weight_sync_s": 2.8152928352355957, + "mfu": 0.1464480192808345, + "response_tokens_mean": 13768.53125, + "reverse_kl": 0.00010395546223662677, + "train_score": 0.7734375, + "dropped_stale": null + }, + { + "step_s": 189.09022545814514, + "train_s": 23.549643754959106, + "wait_for_rollouts_s": 165.54058170318604, + "weight_sync_s": 2.8402538299560547, + "mfu": 0.1607316125099345, + "response_tokens_mean": 13172.15625, + "reverse_kl": 0.0001436661249840654, + "train_score": 0.796875, + "dropped_stale": null + }, + { + "step_s": 213.54103302955627, + "train_s": 26.71874761581421, + "wait_for_rollouts_s": 186.82228541374207, + "weight_sync_s": 2.862229824066162, + "mfu": 0.16284128845380993, + "response_tokens_mean": 14360.265625, + "reverse_kl": 5.980591504339827e-05, + "train_score": 0.6953125, + "dropped_stale": null + }, + { + "step_s": 191.62022972106934, + "train_s": 23.989046573638916, + "wait_for_rollouts_s": 167.63118314743042, + "weight_sync_s": 2.89442777633667, + "mfu": 0.16298006323970865, + "response_tokens_mean": 13447.375, + "reverse_kl": 9.776106048775546e-05, + "train_score": 0.8125, + "dropped_stale": null + }, + { + "step_s": 191.1463441848755, + "train_s": 23.25367522239685, + "wait_for_rollouts_s": 167.89266896247864, + "weight_sync_s": 2.998922824859619, + "mfu": 0.1524318826081285, + "response_tokens_mean": 12529.9921875, + "reverse_kl": 0.00016122279998853628, + "train_score": 0.75, + "dropped_stale": null + } + ], + "step_s_median": 197.1832858324051, + "step_s_total": 4719.769963979721, + "train_s_median": 28.67716407775879, + "train_s_total": 937.5765483379364, + "wait_for_rollouts_s_median": 165.90876495838165, + "wait_for_rollouts_s_total": 3782.1934156417847, + "weight_sync_s_median": 2.8698742389678955, + "weight_sync_s_total": 60.327232837677, + "mfu_mean": 0.12178934522250076, + "response_tokens_mean_mean": 13117.173828125, + "reverse_kl_first": 0.0008959413462434895, + "reverse_kl_last": 0.00016122279998853628, + "train_score_first": 0.203125, + "train_score_last": 0.75 + }, + "roles": { + "generation": { + "samples": 1063, + "seconds": 5310.5423991680145, + "gpus": { + "0": { + "utilization_mean_pct": 8.094073377234242, + "busy_fraction": 0.13358419567262464, + "memory_peak_gib": 152.8828125, + "power_mean_w": 309.90571966133587, + "energy_kwh": 0.4572448552071101 + }, + "1": { + "utilization_mean_pct": 0.0, + "busy_fraction": 0.0, + "memory_peak_gib": 0.0, + "power_mean_w": 193.79902163687674, + "energy_kwh": 0.2858827536464009 + }, + "2": { + "utilization_mean_pct": 84.38664158043274, + "busy_fraction": 0.8579492003762935, + "memory_peak_gib": 124.57421875, + "power_mean_w": 629.2946190028223, + "energy_kwh": 0.9289052429202596 + }, + "3": { + "utilization_mean_pct": 84.42615239887112, + "busy_fraction": 0.8645343367826905, + "memory_peak_gib": 124.578125, + "power_mean_w": 625.6601787394168, + "energy_kwh": 0.9235348962452842 + }, + "4": { + "utilization_mean_pct": 78.60865475070555, + "busy_fraction": 0.8024459078080903, + "memory_peak_gib": 124.529296875, + "power_mean_w": 576.2607902163687, + "energy_kwh": 0.8506080176884293 + }, + "5": { + "utilization_mean_pct": 78.66698024459077, + "busy_fraction": 0.8052681091251176, + "memory_peak_gib": 124.529296875, + "power_mean_w": 577.8371495766698, + "energy_kwh": 0.8529273672752334 + }, + "6": { + "utilization_mean_pct": 81.46190028222013, + "busy_fraction": 0.8287864534336783, + "memory_peak_gib": 124.529296875, + "power_mean_w": 595.8065757290686, + "energy_kwh": 0.8794612066605191 + }, + "7": { + "utilization_mean_pct": 81.25587958607714, + "busy_fraction": 0.8297271872060207, + "memory_peak_gib": 124.453125, + "power_mean_w": 604.6716368767638, + "energy_kwh": 0.8925416353657133 + } + }, + "gpu_energy_kwh": 6.07110597500895, + "cpu_utilization_pct": 2.902079957027759, + "network": { + "rx_gb": 0.394265338, + "tx_gb": 3.2156128, + "rx_gbit_s": 0.0005939360741181816, + "tx_gbit_s": 0.0048441195769438235 + }, + "engines": { + "127.0.0.1:30000": { + "seconds": 5205.528298139572, + "generation_tokens": 8.0, + "generation_tokens_per_s": 0.001536827684302313, + "prompt_tokens": 33959924.0, + "prompt_tokens_per_s": 6523.818920000319 + } + } + }, + "learner": { + "samples": 1059, + "seconds": 5290.7078239917755, + "gpus": { + "0": { + "utilization_mean_pct": 15.166988416988417, + "busy_fraction": 0.16988416988416988, + "memory_peak_gib": 168.6142578125, + "power_mean_w": 299.02312741312744, + "energy_kwh": 0.43830363499085306 + }, + "1": { + "utilization_mean_pct": 16.86969111969112, + "busy_fraction": 0.1776061776061776, + "memory_peak_gib": 167.5615234375, + "power_mean_w": 299.70228764478765, + "energy_kwh": 0.43975352939688367 + }, + "2": { + "utilization_mean_pct": 16.857142857142858, + "busy_fraction": 0.1776061776061776, + "memory_peak_gib": 166.9892578125, + "power_mean_w": 296.76278957528956, + "energy_kwh": 0.4348436475433648 + }, + "3": { + "utilization_mean_pct": 15.595559845559846, + "busy_fraction": 0.166988416988417, + "memory_peak_gib": 167.9716796875, + "power_mean_w": 307.5958494208494, + "energy_kwh": 0.45087851541061663 + }, + "4": { + "utilization_mean_pct": 16.233590733590734, + "busy_fraction": 0.17664092664092665, + "memory_peak_gib": 167.9853515625, + "power_mean_w": 295.7286003861004, + "energy_kwh": 0.4335966228102571 + }, + "5": { + "utilization_mean_pct": 16.927606177606176, + "busy_fraction": 0.1805019305019305, + "memory_peak_gib": 166.9111328125, + "power_mean_w": 299.64416023166024, + "energy_kwh": 0.439378844377525 + }, + "6": { + "utilization_mean_pct": 16.2027027027027, + "busy_fraction": 0.17374517374517376, + "memory_peak_gib": 168.0341796875, + "power_mean_w": 301.352750965251, + "energy_kwh": 0.4420954993822376 + }, + "7": { + "utilization_mean_pct": 16.381274131274132, + "busy_fraction": 0.17664092664092665, + "memory_peak_gib": 166.1552734375, + "power_mean_w": 301.6312548262548, + "energy_kwh": 0.4425657687004785 + } + }, + "gpu_energy_kwh": 3.521416062612216, + "cpu_utilization_pct": 1.4590698666959612, + "network": { + "rx_gb": 3.215783097, + "tx_gb": 0.39412358, + "rx_gbit_s": 0.004862537420671596, + "tx_gbit_s": 0.000595948357930888 + }, + "engines": { + "10.244.218.33:15000": { + "seconds": 4865.648669958115, + "generation_tokens": 14636070.0, + "generation_tokens_per_s": 3008.0408580190383, + "prompt_tokens": 155637.0, + "prompt_tokens_per_s": 31.986896415465974 + }, + "10.244.218.33:15002": { + "seconds": 4865.648669958115, + "generation_tokens": 12785729.0, + "generation_tokens_per_s": 2627.7542558595924, + "prompt_tokens": 135450.0, + "prompt_tokens_per_s": 27.83801486455577 + }, + "10.244.218.33:15004": { + "seconds": 4865.648669958115, + "generation_tokens": 13569640.0, + "generation_tokens_per_s": 2788.865559443858, + "prompt_tokens": 140272.0, + "prompt_tokens_per_s": 28.8290440832851 + } + } + } + } + } +] diff --git a/docs/caesar-opd-2026-09-27/data/verl-966/summary.json b/docs/caesar-opd-2026-09-27/data/verl-966/summary.json new file mode 100644 index 0000000..924a66f --- /dev/null +++ b/docs/caesar-opd-2026-09-27/data/verl-966/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/verl-caesar-01/checkpoints/verl-966/global_step_20/actor/huggingface-merged", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 389.5900001525879, + "output_tokens_per_second": 44689.9458229956, + "domains": { + "caesar_cipher": { + "avg@8": 0.7837301587301587, + "pass@8": 1.0, + "finished_thinking": 0.9444444444444444, + "truncated": 0.05654761904761905, + "response_tokens_mean": 13253.779761904761, + "response_tokens_median": 11453.0 + }, + "simple_geometry": { + "avg@8": 0.6865234375, + "pass@8": 1.0, + "finished_thinking": 0.9970703125, + "truncated": 0.0029296875, + "response_tokens_mean": 3956.001953125, + "response_tokens_median": 3704.0 + } + } +} diff --git a/docs/caesar-opd-2026-09-27/figures/gpu_utilization.png b/docs/caesar-opd-2026-09-27/figures/gpu_utilization.png new file mode 100644 index 0000000..699285b Binary files /dev/null and b/docs/caesar-opd-2026-09-27/figures/gpu_utilization.png differ diff --git a/docs/caesar-opd-2026-09-27/figures/miles_vs_prime_rl.png b/docs/caesar-opd-2026-09-27/figures/miles_vs_prime_rl.png new file mode 100644 index 0000000..8d7f9b3 Binary files /dev/null and b/docs/caesar-opd-2026-09-27/figures/miles_vs_prime_rl.png differ diff --git a/docs/caesar-opd-2026-09-27/figures/per_update.png b/docs/caesar-opd-2026-09-27/figures/per_update.png new file mode 100644 index 0000000..610ce7b Binary files /dev/null and b/docs/caesar-opd-2026-09-27/figures/per_update.png differ diff --git a/docs/caesar-opd-2026-09-27/figures/quality.png b/docs/caesar-opd-2026-09-27/figures/quality.png new file mode 100644 index 0000000..8cd51ca Binary files /dev/null and b/docs/caesar-opd-2026-09-27/figures/quality.png differ diff --git a/docs/caesar-opd-2026-09-27/nemo-rl-log.md b/docs/caesar-opd-2026-09-27/nemo-rl-log.md new file mode 100644 index 0000000..71c670a --- /dev/null +++ b/docs/caesar-opd-2026-09-27/nemo-rl-log.md @@ -0,0 +1,139 @@ +# NeMo-RL attempt log + +What we tried with NeMo-RL for the [caesar_cipher OPD benchmark](../CAESAR-OPD-2026-09-27.md), and why no +training update completed. + +- Upstream: `NVIDIA-NeMo/RL`. We first read `main` at `4aaa48fabd178a4bf481e52c49e9125995e0f2a8` + (2026-09-24), then ran the NGC container `nvcr.io/nvidia/nemo-rl:v0.7.0`, which is built from `19244a09`. +- Mode: MOPD on async GRPO (`adv_estimator: opd`) with NeMo Gym rollouts, per + `docs/about/algorithms/mopd.md` and `examples/configs/recipes/llm/mopd-qwen3-1.7b-3n8g-megatron-pack.yaml`. + +## Findings before any GPU time + +- **Two distillation paths.** `run_distillation.py` does teacher-logit KD and "currently supports the + DTensor and vLLM generation backend. Megatron generation/training paths are not supported yet". MOPD + is async GRPO with a sampled-token teacher-minus-student advantage, the same objective as the other + frameworks. (The on-policy-distillation guide's Megatron note conflicts with a checked-in + `distillation_math_megatron.yaml`; not investigated, since we used MOPD.) +- **Layout.** MOPD teachers are non-colocated and reserve whole nodes (`opd_teacher_nodes += num_nodes` + in `nemo_rl/algorithms/grpo.py`). With more than one policy node, generation must also take whole nodes: + `policy.generation.colocated.resources.gpus_per_node` "must be explicitly set and equal to + cluster.gpus_per_node". Final layout: trainer node (8 GPUs), vLLM node (8 GPUs, 4 × TP2), teacher node + (1 GPU used, 7 idle). That is 24 GPUs held, against 16 for the other frameworks, which share one node + between the teacher (1 GPU) and the policy (6 GPUs, 3 × TP2). +- **Recipes.** The reference MOPD recipe is a self-distillation smoke test (student == teacher, "OPD loss + stays near zero"). There is no MOPD recipe for Qwen3.5/3.6, so model settings came from the + Qwen3.5-35B-A3B GRPO recipes (Megatron EP16, Automodel EP16). +- **Runtime.** The README recommends the NGC container. `main` has no matching container, so we used the + latest release, v0.7.0 (tag `81aa43dd`, 2026-07-29), which already contains MOPD, the Qwen3.5-35B-A3B + recipes and a NeMo Gym pin with the `reasoning_gym` server. The image pulled anonymously from NGC. +- **Container provenance.** The image's `/opt/nemo-rl` and `NEMO_RL_COMMIT` are `19244a09` on `r0.7.0`, + one commit before the tag, which only bumps docs and `package_info.py`. The package pins `19244a09` and + the controller refuses to start if the container's `NEMO_RL_COMMIT` differs. Image details: 43 GB, + `/opt/nemo_rl_venv`, prebuilt worker venvs in `/opt/ray_venvs`, Gym venvs in `/opt/gym_venvs`, Python + 3.13.13. +- **NeMo Gym.** Pinned as a submodule (`267305e2` on `main`, `d67ad661` in v0.7.0). Its `reasoning_gym` + resources server takes rows of `responses_create_params.input` (chat messages) + `question`, `answer`, + `metadata` (the reasoning_gym entry, with `source_dataset`) + `agent_ref`. Our packaged rows convert + one-to-one, keeping the exact system and user messages the other frameworks train on. +- **Scoring.** The Gym server scores the final message text (`` tags, then `\boxed{}`, then the + whole text), more leniently than our verifier. In MOPD that reward is diagnostic only. +- **Unpinned dependency.** The Gym server installs `reasoning-gym>=0.1.19` into its own venv at start-up. + The package pins 0.1.25 with a uv constraint file (`UV_CONSTRAINT`), without editing upstream. +- **Launch.** Multi-node launch follows upstream's `ray.sub` (`docs/cluster.md`). The package controller + runs it inside the allocation with `CONTAINER`, `MOUNTS` and `COMMAND`, and starts telemetry capture on + each host with `srun --overlap`. `ray.sub` passes `-A $SLURM_JOB_ACCOUNT` unquoted; the value here is + `root`, so it is harmless on this cluster. +- **Thinking.** Qwen3.6's chat template enables thinking unless `enable_thinking` is false, so Gym + rollouts think. +- **Final weights** would come from `examples/converters/convert_megatron_to_hf.py` (documented in the + README and `docs/about/evaluation.md`) applied to `policy/weights/iter_*`. +- Config keys were checked against v0.7.0 `examples/configs/grpo_math_1B.yaml`. + +## Attempts + +### Getting the container to run (jobs 985–996) + +| Job | Runtime | Failure | Cause | Change | +|---|---|---|---|---| +| 985 | 84 s | Driver: `uv` "Could not acquire lock ... Read-only file system" at `/root/.cache/uv` | `ray.sub` runs containers without `--container-writable` | Set `ray.sub`'s documented `UV_CACHE_DIR_OVERRIDE` (writable uv cache); put compile caches and `HF_HOME` in the campaign | +| 986 | 146 s | `uv run` tried to rebuild the editable `nemo_rl` in `/opt/nemo-rl`: "Cannot update time stamp of directory 'nemo_rl.egg-info'" | Upstream's flow runs `uv run` and assumes a writable container; `ray.sub` has no switch for it | `uv run --no-sync` (the image's venv is built from this code) | +| 987 | 31 s | `No module named 'nemo_rl'` | The image's venv does not contain the project; upstream relies on `uv run` installing it on every launch | Put `/opt/nemo-rl` on `PYTHONPATH` for the driver and all Ray processes | +| 988 | 30 s | `cannot import name 'AutoProcessor' from 'transformers' (unknown location)` | Found in 995, below | | +| 994 | cancelled at 3 min | Ray head never started | Our mistake: we set `HOME` for the whole allocation so swanlab (imported at start-up, creates `~/.swanlab`) had a writable home. Enroot keeps container root filesystems under `$HOME/.local/share/enroot`, so the 43 GB image started extracting onto Lustre (17 GB written) | Set `HOME` for the driver only | +| 995 | | Same as 988; diagnosed with a `sys.path` probe | The image's venv files are symlinks into its own uv cache (`/opt/nemo_rl_venv/.../transformers/__init__.py -> /root/.cache/uv/archive-v0/...`). `UV_CACHE_DIR_OVERRIDE` (the 985 fix) mounts a host directory over `/root/.cache/uv`, so every package becomes a dangling symlink and imports as an empty namespace package | Drop the override mount; point `UV_CACHE_DIR` at a separate writable cache | +| 996 | | Every vLLM worker: `Read-only file system: '/opt/ray_venvs/.../vllm/v1/executor/ray_executor.py.patch_lock'` | NeMo-RL patches vLLM's installed files at run time (`nemo_rl/models/generation/vllm/patches.py`), so it needs a writable root filesystem, and `ray.sub` never passes `--container-writable` | `PYXIS_CONTAINER_WRITABLE=1` (Pyxis' environment form of the flag) fixes every `ray.sub` step without editing upstream. All earlier workarounds (`--no-sync`, `PYTHONPATH`, driver `HOME`, `UV_CACHE_DIR`) were removed; the driver runs upstream's plain `uv run` | + +`ray.sub` names its containers (`--container-name=ray-head` / `ray-worker`), so Pyxis keeps the extracted +52 GB root filesystem on each node between jobs: restarts are fast, but the disk use persists. + +### Job 1000: eager mode too slow + +First clean start; cancelled at 31 min. Setup took 496 s (vLLM 161 s, teacher 290 s, NeMo Gym 167 s, +overlapping), then pre-training validation (two collections of about 6 min). Training rollouts took about +21 minutes each end to end: 179 rollouts finished in 30 minutes (126 of them validation). Trainer GPUs +showed 100% utilization at about 250 W, i.e. spin-waiting. + +Cause: vLLM 0.20.0 ran with `cudagraph_mode=none` and no compilation, because every upstream +Qwen3.5-35B-A3B recipe sets `enforce_eager: true` and we copied it. Those recipes generate at most 4k +tokens; at 30k-token thinking, eager MoE decoding is impractically slow. Upstream documents eager mode +only in connection with a DeepScaleR convergence issue; the base config default is `enforce_eager: +False`. Torch dynamo also hit its recompile limit (8) on `_prepare_qkv_for_gated_delta_rule` (one compile +per sequence length) and fell back to eager for that function. + +### Jobs 1001–1003: deadlock in the first weight sync + +- **1001** (CUDA graphs on; cancelled at 24 min). Graphs captured cleanly. Pre-training validation plus + the first training batch finished in about 15 minutes (268 rollouts, against about 50 training rollouts + in the same time in eager mode). The driver then blocked in the first weight sync + (`refit_policy_generation`, `grpo.py:1947`, called from `async_grpo_train` line 3553; an NCCL broadcast + from the Megatron trainer to vLLM) with trainer, vLLM and teacher idle for 10+ minutes. Located with + py-spy on the driver (Pyxis containers share the host PID namespace). NCCL logged nothing by default. + The image's `LD_LIBRARY_PATH` includes AWS's EFA NCCL plugin (`/opt/amazon/ofi-nccl`), which this + cluster does not have. +- **1002** (`NCCL_DEBUG=INFO`; cancelled at about 45 min). Same hang at the same point. + - NCCL chose InfiniBand (`IBext_v11`, GPUDirect RDMA on the mlx5 HCAs), not EFA, and logged no + warnings. A 16-rank, 2-node communicator (8 trainer + 8 vLLM ranks) initialized. + - Trainer ranks (py-spy through `srun --overlap`, since Kubernetes exec to that node was failing) were + all in `packed_broadcast_producer` (`nemo_rl/utils/packed_tensor.py:68`) → `stream.synchronize()`: + the broadcast was issued and waiting for receivers. + - On the vLLM node, the `VllmAsyncGenerationWorker` actors, the four `VLLM::EngineCore` processes and + the TP workers (`RayWorkerWrapper`) were all idle. We read this as the receive half + (`update_weights_from_collective` → `collective_rpc`) never reaching the engines, but engines that + had already failed and returned look the same, so this was not established. + - GPUs at idle power on all three nodes; no NCCL or Python error anywhere. + - Upstream issues searched ("refit hang", `update_weights_from_collective`, non-colocated weight + update): nothing matching. +- **1003** (8 × TP1 vLLM engines; cancelled at 33 min). Same deadlock after validation and the first batch + (273 rollouts), so vLLM's multi-GPU Ray executor is not the cause. We stopped the production attempts + here. + +Ray placed the groups differently between jobs (the teacher on gpu-nodes-2 in 1000, gpu-nodes-0 in 1001), +so node roles have to be read from the logs, not the site file. + +### Diagnosis (jobs 1008, 1010) + +- **1008**: upstream's reference MOPD recipe (Qwen3-1.7B self-distillation) passed the first weight sync + in about 2 minutes and started training. It also showed why no "Refitting" lines appeared in our logs: + the package's driver ran without `PYTHONUNBUFFERED=1`. +- **1010**: the same reference recipe with only the model changed to Qwen3.6-35B-A3B (EP8) deadlocked at + the first weight sync. With unbuffered output, every vLLM worker logs `shard_dim=0 is not a valid data + dimension for a 3D tensor (expected 1 or 2)` in `update_weights_from_collective`. NeMo-RL catches it + and returns `False` (`vllm_backend.py:466-470`), while the driver waits on the trainer's broadcast + (`grpo.py:1946`) before checking vLLM's result, so it waits forever. Our log filters had dropped lines + containing "repeated", which is how Ray tags this worker error. + +`grpo.py:3553` in jobs 1001–1003 is the same start-up refit before the first update (`🔄 Refitting policy +generation with actual model weights...`) that logged the `shard_dim` error and never returned in 1010. +Background rollout collection continues during it, which is why the first batch finished. Jobs 1001–1003 +logged no such error because their output was buffered. + +Status: unresolved upstream. No NeMo-RL training update completed. + +## Reproduction + +`frameworks/nemo-rl/repro/` holds three model-free container checks (`image`, `uv-override`, `readonly`), +each about 2 minutes on one GPU. The small-model and reference-recipe checks used in 1008 and 1010 have +been removed from `repro.sh`. The deadlock is reproduced with the production configuration by +`tools/reproduce.sh nemo-rl`, whose watchdog cancels the run after `DEADLOCK_MIN` minutes in the first +sync and writes the worker errors to `DEADLOCK`. The driver now runs with `PYTHONUNBUFFERED=1`. diff --git a/docs/mopd-2026-09-29/data/miles-1042-step10/summary.json b/docs/mopd-2026-09-29/data/miles-1042-step10/summary.json new file mode 100644 index 0000000..e538af2 --- /dev/null +++ b/docs/mopd-2026-09-29/data/miles-1042-step10/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/miles-mopd/checkpoints/miles-1042/hf/iter_9", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 383.29514169692993, + "output_tokens_per_second": 44599.05733299547, + "domains": { + "caesar_cipher": { + "avg@8": 0.7083333333333334, + "pass@8": 0.9841269841269841, + "finished_thinking": 0.9464285714285714, + "truncated": 0.05853174603174603, + "response_tokens_mean": 13718.450396825398, + "response_tokens_median": 12269.0 + }, + "simple_geometry": { + "avg@8": 0.9970703125, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 3189.84765625, + "response_tokens_median": 2735.5 + } + } +} diff --git a/docs/mopd-2026-09-29/data/miles-1042/summary.json b/docs/mopd-2026-09-29/data/miles-1042/summary.json new file mode 100644 index 0000000..e342975 --- /dev/null +++ b/docs/mopd-2026-09-29/data/miles-1042/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/miles-mopd/checkpoints/miles-1042/hf/iter_19", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 387.9073717594147, + "output_tokens_per_second": 42843.00894984641, + "domains": { + "caesar_cipher": { + "avg@8": 0.7688492063492064, + "pass@8": 1.0, + "finished_thinking": 0.939484126984127, + "truncated": 0.06448412698412699, + "response_tokens_mean": 13418.285714285714, + "response_tokens_median": 11476.5 + }, + "simple_geometry": { + "avg@8": 0.99609375, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 3020.9833984375, + "response_tokens_median": 2607.5 + } + } +} diff --git a/docs/mopd-2026-09-29/data/prime-rl-1050-step10/summary.json b/docs/mopd-2026-09-29/data/prime-rl-1050-step10/summary.json new file mode 100644 index 0000000..fc4d875 --- /dev/null +++ b/docs/mopd-2026-09-29/data/prime-rl-1050-step10/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/prime-rl-mopd/checkpoints/prime-rl-1050/checkpoints/step_10/weights", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 419.5302309989929, + "output_tokens_per_second": 43210.92655666933, + "domains": { + "caesar_cipher": { + "avg@8": 0.5317460317460317, + "pass@8": 0.9444444444444444, + "finished_thinking": 0.939484126984127, + "truncated": 0.06547619047619048, + "response_tokens_mean": 14483.602182539682, + "response_tokens_median": 12636.0 + }, + "simple_geometry": { + "avg@8": 1.0, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 3446.1123046875, + "response_tokens_median": 3023.0 + } + } +} diff --git a/docs/mopd-2026-09-29/data/prime-rl-1050/summary.json b/docs/mopd-2026-09-29/data/prime-rl-1050/summary.json new file mode 100644 index 0000000..3d93d4b --- /dev/null +++ b/docs/mopd-2026-09-29/data/prime-rl-1050/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/prime-rl-mopd/checkpoints/prime-rl-1050/checkpoints/step_20/weights", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 387.9954011440277, + "output_tokens_per_second": 44088.757107845195, + "domains": { + "caesar_cipher": { + "avg@8": 0.6488095238095238, + "pass@8": 0.9682539682539683, + "finished_thinking": 0.9325396825396826, + "truncated": 0.07242063492063493, + "response_tokens_mean": 14005.891865079366, + "response_tokens_median": 12218.0 + }, + "simple_geometry": { + "avg@8": 0.9951171875, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 2918.2578125, + "response_tokens_median": 2437.5 + } + } +} diff --git a/docs/mopd-2026-09-29/data/telemetry.json b/docs/mopd-2026-09-29/data/telemetry.json new file mode 100644 index 0000000..7fb5dc9 --- /dev/null +++ b/docs/mopd-2026-09-29/data/telemetry.json @@ -0,0 +1,1502 @@ +[ + { + "run": "miles-1042", + "job_id": "1042", + "nodes": [ + "gpu-nodes-2", + "gpu-nodes-0" + ], + "exit_code": 0, + "wall_seconds": 3721.0415003299713, + "updates": { + "updates": 20, + "per_update": [ + { + "step_s": 650.1330213546753, + "train_s": 258.5348746776581, + "wait_for_rollouts_s": 391.5981466770172, + "weight_sync_s": 4.736995697021484, + "mfu": 0.00166550800323137, + "response_tokens_mean": 3672.2890625, + "reverse_kl": 0.003982492722570896, + "train_score": 0.359375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.25, + "simple_geometry": 0.46875 + } + }, + { + "step_s": 57.066370248794556, + "train_s": 53.9838981628418, + "wait_for_rollouts_s": 3.082472085952759, + "weight_sync_s": 2.718163013458252, + "mfu": 0.013316071543986018, + "response_tokens_mean": 5667.3125, + "reverse_kl": 0.0033512248191982508, + "train_score": 0.375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.234375, + "simple_geometry": 0.515625 + } + }, + { + "step_s": 114.0905966758728, + "train_s": 110.4595594406128, + "wait_for_rollouts_s": 3.6310372352600098, + "weight_sync_s": 2.8846523761749268, + "mfu": 0.012298560779982825, + "response_tokens_mean": 8581.7421875, + "reverse_kl": 0.0027109929360449314, + "train_score": 0.3828125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.21875, + "simple_geometry": 0.546875 + } + }, + { + "step_s": 45.75210380554199, + "train_s": 42.33794856071472, + "wait_for_rollouts_s": 3.4141552448272705, + "weight_sync_s": 3.0350701808929443, + "mfu": 0.012989161309873308, + "response_tokens_mean": 4544.0546875, + "reverse_kl": 0.0027047423645853996, + "train_score": 0.5078125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.359375, + "simple_geometry": 0.65625 + } + }, + { + "step_s": 47.82393836975098, + "train_s": 18.40226101875305, + "wait_for_rollouts_s": 29.421677350997925, + "weight_sync_s": 3.0374650955200195, + "mfu": 0.03976623926245367, + "response_tokens_mean": 5605.890625, + "reverse_kl": 0.001996077597141266, + "train_score": 0.5703125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.328125, + "simple_geometry": 0.8125 + } + }, + { + "step_s": 146.11227416992188, + "train_s": 40.82797193527222, + "wait_for_rollouts_s": 105.28430223464966, + "weight_sync_s": 3.193331480026245, + "mfu": 0.015086286891166086, + "response_tokens_mean": 4869.484375, + "reverse_kl": 0.0014393492601811886, + "train_score": 0.6640625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.375, + "simple_geometry": 0.953125 + } + }, + { + "step_s": 94.6539294719696, + "train_s": 38.768635272979736, + "wait_for_rollouts_s": 55.88529419898987, + "weight_sync_s": 3.289876937866211, + "mfu": 0.02073726937943419, + "response_tokens_mean": 5934.015625, + "reverse_kl": 0.0014350214041769505, + "train_score": 0.765625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.578125, + "simple_geometry": 0.953125 + } + }, + { + "step_s": 85.12967157363892, + "train_s": 18.58953619003296, + "wait_for_rollouts_s": 66.54013538360596, + "weight_sync_s": 3.06184458732605, + "mfu": 0.03817449102228692, + "response_tokens_mean": 5357.0234375, + "reverse_kl": 0.0017237381543964148, + "train_score": 0.84375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.6875, + "simple_geometry": 1.0 + } + }, + { + "step_s": 126.53601598739624, + "train_s": 42.08498978614807, + "wait_for_rollouts_s": 84.45102620124817, + "weight_sync_s": 3.0568246841430664, + "mfu": 0.016162874581178825, + "response_tokens_mean": 5249.359375, + "reverse_kl": 0.0016534614842385054, + "train_score": 0.8125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.640625, + "simple_geometry": 0.984375 + } + }, + { + "step_s": 87.14745903015137, + "train_s": 21.87246823310852, + "wait_for_rollouts_s": 65.27499079704285, + "weight_sync_s": 3.542290210723877, + "mfu": 0.03246521076277439, + "response_tokens_mean": 5368.7890625, + "reverse_kl": 0.0015292266616597772, + "train_score": 0.8515625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.703125, + "simple_geometry": 1.0 + } + }, + { + "step_s": 451.99514746665955, + "train_s": 27.01609492301941, + "wait_for_rollouts_s": 424.97905254364014, + "weight_sync_s": 2.936904191970825, + "mfu": 0.02754502626843945, + "response_tokens_mean": 5613.171875, + "reverse_kl": 0.0013747797347605228, + "train_score": 0.84375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.6875, + "simple_geometry": 1.0 + } + }, + { + "step_s": 79.31180429458618, + "train_s": 53.32871747016907, + "wait_for_rollouts_s": 25.983086824417114, + "weight_sync_s": 4.154361009597778, + "mfu": 0.022593458491784398, + "response_tokens_mean": 7720.734375, + "reverse_kl": 0.0013867293018847704, + "train_score": 0.875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.75, + "simple_geometry": 1.0 + } + }, + { + "step_s": 35.73812246322632, + "train_s": 32.31308078765869, + "wait_for_rollouts_s": 3.425041675567627, + "weight_sync_s": 3.096125364303589, + "mfu": 0.017766988646277352, + "response_tokens_mean": 4692.6328125, + "reverse_kl": 0.0010280702263116837, + "train_score": 0.8828125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.765625, + "simple_geometry": 1.0 + } + }, + { + "step_s": 140.94706869125366, + "train_s": 49.9378662109375, + "wait_for_rollouts_s": 91.00920248031616, + "weight_sync_s": 3.558417797088623, + "mfu": 0.013222968915280043, + "response_tokens_mean": 5148.96875, + "reverse_kl": 0.001420786022208631, + "train_score": 0.921875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.84375, + "simple_geometry": 1.0 + } + }, + { + "step_s": 97.0695366859436, + "train_s": 55.76571726799011, + "wait_for_rollouts_s": 41.30381941795349, + "weight_sync_s": 3.0135838985443115, + "mfu": 0.012129283822949923, + "response_tokens_mean": 5244.6875, + "reverse_kl": 0.0015494988765567541, + "train_score": 0.9453125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.890625, + "simple_geometry": 1.0 + } + }, + { + "step_s": 86.42890214920044, + "train_s": 25.548237085342407, + "wait_for_rollouts_s": 60.88066506385803, + "weight_sync_s": 3.460808515548706, + "mfu": 0.026595589613335552, + "response_tokens_mean": 5291.109375, + "reverse_kl": 0.0012530406238511205, + "train_score": 0.953125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.921875, + "simple_geometry": 0.984375 + } + }, + { + "step_s": 95.20337295532227, + "train_s": 26.364288091659546, + "wait_for_rollouts_s": 68.83908486366272, + "weight_sync_s": 2.9037935733795166, + "mfu": 0.025932835238375928, + "response_tokens_mean": 5285.4453125, + "reverse_kl": 0.0012300864327698946, + "train_score": 0.9296875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.859375, + "simple_geometry": 1.0 + } + }, + { + "step_s": 105.27768301963806, + "train_s": 39.71882247924805, + "wait_for_rollouts_s": 65.55886054039001, + "weight_sync_s": 3.513218879699707, + "mfu": 0.015565913454802351, + "response_tokens_mean": 4852.53125, + "reverse_kl": 0.0013062767684459686, + "train_score": 0.9296875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.859375, + "simple_geometry": 1.0 + } + }, + { + "step_s": 108.97823762893677, + "train_s": 40.49566411972046, + "wait_for_rollouts_s": 68.48257350921631, + "weight_sync_s": 3.2016518115997314, + "mfu": 0.01567317848970035, + "response_tokens_mean": 5004.3984375, + "reverse_kl": 0.0011310986010357738, + "train_score": 0.9375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.875, + "simple_geometry": 1.0 + } + }, + { + "step_s": 81.34261178970337, + "train_s": 22.436383485794067, + "wait_for_rollouts_s": 58.9062283039093, + "weight_sync_s": 3.05690598487854, + "mfu": 0.03142436490411269, + "response_tokens_mean": 5346.6171875, + "reverse_kl": 0.0011946151498705149, + "train_score": 0.9375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.875, + "simple_geometry": 1.0 + } + } + ], + "step_s_median": 94.92865121364594, + "step_s_total": 2736.737867832184, + "train_s_median": 40.10724329948425, + "train_s_total": 1018.7870151996613, + "wait_for_rollouts_s_median": 63.07782793045044, + "wait_for_rollouts_s_total": 1717.9508526325226, + "weight_sync_s_median": 3.0789849758148193, + "weight_sync_s_total": 65.4522852897644, + "mfu_mean": 0.020555564069071282, + "response_tokens_mean_mean": 5452.512890625, + "reverse_kl_first": 0.003982492722570896, + "reverse_kl_last": 0.0011946151498705149, + "train_score_first": 0.359375, + "train_score_last": 0.9375 + }, + "roles": { + "generation": { + "samples": 735, + "seconds": 3670.2648618221283, + "gpus": { + "0": { + "utilization_mean_pct": 11.662585034013606, + "busy_fraction": 0.17687074829931973, + "memory_peak_gib": 157.7841796875, + "power_mean_w": 342.61114285714285, + "energy_kwh": 0.34943042855720585 + }, + "1": { + "utilization_mean_pct": 3.3700680272108845, + "busy_fraction": 0.11156462585034013, + "memory_peak_gib": 170.9248046875, + "power_mean_w": 287.43092517006806, + "energy_kwh": 0.29308509481431977 + }, + "2": { + "utilization_mean_pct": 83.58231292517007, + "busy_fraction": 0.8653061224489796, + "memory_peak_gib": 130.076171875, + "power_mean_w": 754.3675646258504, + "energy_kwh": 0.7697847173898684 + }, + "3": { + "utilization_mean_pct": 83.73605442176871, + "busy_fraction": 0.8707482993197279, + "memory_peak_gib": 130.09765625, + "power_mean_w": 767.0157823129251, + "energy_kwh": 0.7826905211695168 + }, + "4": { + "utilization_mean_pct": 81.58775510204082, + "busy_fraction": 0.8462585034013606, + "memory_peak_gib": 133.029296875, + "power_mean_w": 692.2281088435374, + "energy_kwh": 0.7063561515576284 + }, + "5": { + "utilization_mean_pct": 81.66530612244898, + "busy_fraction": 0.8476190476190476, + "memory_peak_gib": 133.029296875, + "power_mean_w": 701.4301224489797, + "energy_kwh": 0.7157368759062449 + }, + "6": { + "utilization_mean_pct": 80.57006802721088, + "busy_fraction": 0.8312925170068027, + "memory_peak_gib": 132.84765625, + "power_mean_w": 680.9558231292517, + "energy_kwh": 0.6948413266645697 + }, + "7": { + "utilization_mean_pct": 80.65306122448979, + "busy_fraction": 0.8353741496598639, + "memory_peak_gib": 132.84765625, + "power_mean_w": 703.6589115646258, + "energy_kwh": 0.718002885244092 + } + }, + "gpu_energy_kwh": 5.029928001303446, + "cpu_utilization_pct": 3.7030938101569877, + "network": { + "rx_gb": 0.397621433, + "tx_gb": 4.219588643, + "rx_gbit_s": 0.0008666871693888558, + "tx_gbit_s": 0.00919734962322072 + }, + "engines": { + "127.0.0.1:30000": { + "seconds": 3545.254227399826, + "generation_tokens": 8.0, + "generation_tokens_per_s": 0.0022565377507122782, + "prompt_tokens": 37422840.0, + "prompt_tokens_per_s": 10555.756399858185 + }, + "127.0.0.1:30001": { + "seconds": 3545.254227399826, + "generation_tokens": 8.0, + "generation_tokens_per_s": 0.0022565377507122782, + "prompt_tokens": 9557909.0, + "prompt_tokens_per_s": 2695.9728095465803 + } + } + }, + "learner": { + "samples": 730, + "seconds": 3648.7478008270264, + "gpus": { + "0": { + "utilization_mean_pct": 16.93646408839779, + "busy_fraction": 0.2292817679558011, + "memory_peak_gib": 178.1943359375, + "power_mean_w": 272.5860220994475, + "energy_kwh": 0.27560528556466035 + }, + "1": { + "utilization_mean_pct": 19.718232044198896, + "busy_fraction": 0.24861878453038674, + "memory_peak_gib": 178.2724609375, + "power_mean_w": 272.01526243093923, + "energy_kwh": 0.27508142447231876 + }, + "2": { + "utilization_mean_pct": 19.951657458563535, + "busy_fraction": 0.22375690607734808, + "memory_peak_gib": 178.1455078125, + "power_mean_w": 267.58024861878454, + "energy_kwh": 0.27050962721944255 + }, + "3": { + "utilization_mean_pct": 19.808011049723756, + "busy_fraction": 0.21823204419889503, + "memory_peak_gib": 178.0849609375, + "power_mean_w": 275.70505524861875, + "energy_kwh": 0.27869904315931665 + }, + "4": { + "utilization_mean_pct": 21.385359116022098, + "busy_fraction": 0.23342541436464087, + "memory_peak_gib": 175.9794921875, + "power_mean_w": 263.3989917127072, + "energy_kwh": 0.26629263362108535 + }, + "5": { + "utilization_mean_pct": 19.65193370165746, + "busy_fraction": 0.21408839779005526, + "memory_peak_gib": 177.5615234375, + "power_mean_w": 269.98832872928176, + "energy_kwh": 0.27293499622611733 + }, + "6": { + "utilization_mean_pct": 19.766574585635357, + "busy_fraction": 0.21961325966850828, + "memory_peak_gib": 177.7587890625, + "power_mean_w": 271.2444475138122, + "energy_kwh": 0.27417283396558495 + }, + "7": { + "utilization_mean_pct": 16.412983425414364, + "busy_fraction": 0.19060773480662985, + "memory_peak_gib": 175.0595703125, + "power_mean_w": 270.73167127071827, + "energy_kwh": 0.27385186782695586 + } + }, + "gpu_energy_kwh": 2.1871477120554816, + "cpu_utilization_pct": 2.180158543977284, + "network": { + "rx_gb": 4.214787318, + "tx_gb": 0.396716256, + "rx_gbit_s": 0.009241060326602293, + "tx_gbit_s": 0.0008698134870490751 + }, + "engines": { + "10.244.218.33:20002": { + "seconds": 3193.7063953876495, + "generation_tokens": 17091820.0, + "generation_tokens_per_s": 5351.719251551741, + "prompt_tokens": 345958.0, + "prompt_tokens_per_s": 108.32492319883647 + }, + "10.244.218.33:20037": { + "seconds": 3193.7063953876495, + "generation_tokens": 23732244.0, + "generation_tokens_per_s": 7430.941064048375, + "prompt_tokens": 249313.0, + "prompt_tokens_per_s": 78.06384467904057 + }, + "10.244.218.33:20072": { + "seconds": 3193.7063953876495, + "generation_tokens": 18114134.0, + "generation_tokens_per_s": 5671.8219389736105, + "prompt_tokens": 416174.0, + "prompt_tokens_per_s": 130.3106636856282 + } + } + } + } + }, + { + "run": "prime-rl-1050", + "job_id": "1050", + "nodes": [ + "gpu-nodes-2", + "gpu-nodes-0" + ], + "exit_code": 0, + "wall_seconds": 1932.1528022289276, + "updates": { + "updates": 20, + "per_update": [ + { + "step_s": 297.37461864599027, + "train_s": 136.19110855087638, + "wait_for_rollouts_s": 147.438710069051, + "weight_sync_s": 8.758922692853957, + "mfu": 0.0049067160332583485, + "response_tokens_mean": 1458.9765625, + "reverse_kl": -0.003125404007732868, + "train_score": 0.4765625, + "dropped_stale": 0.0, + "train_score_by_domain": { + "agg": 0.4765625, + "simple_geometry": 0.5042016806722689, + "caesar_cipher": 0.1111111111111111 + } + }, + { + "step_s": 81.72353056911379, + "train_s": 73.12696983781643, + "wait_for_rollouts_s": 3.269012086093426e-05, + "weight_sync_s": 8.447379297809675, + "mfu": 0.018245755238945654, + "response_tokens_mean": 2654.09375, + "reverse_kl": -0.0011335621820762753, + "train_score": 0.5703125, + "dropped_stale": 0.0, + "train_score_by_domain": { + "agg": 0.5703125, + "caesar_cipher": 0.34615384615384615, + "simple_geometry": 0.6274509803921569 + } + }, + { + "step_s": 24.127345885150135, + "train_s": 16.606685125967488, + "wait_for_rollouts_s": 2.7938978746533394e-05, + "weight_sync_s": 7.3877550379838794, + "mfu": 0.0402706144393272, + "response_tokens_mean": 4847.710791366906, + "reverse_kl": -0.0014577211113646626, + "train_score": 0.460431654676259, + "dropped_stale": 567.0, + "train_score_by_domain": { + "agg": 0.6484375, + "simple_geometry": 0.6779661016949152, + "caesar_cipher": 0.3 + } + }, + { + "step_s": 72.76021273783408, + "train_s": 29.936422990867868, + "wait_for_rollouts_s": 34.92963407910429, + "weight_sync_s": 7.763010489987209, + "mfu": 0.022330525217198412, + "response_tokens_mean": 1951.017316017316, + "reverse_kl": 4.0989427361637354e-05, + "train_score": 0.7489177489177489, + "dropped_stale": 103.0, + "train_score_by_domain": { + "agg": 0.875, + "simple_geometry": 0.9024390243902439, + "caesar_cipher": 0.2 + } + }, + { + "step_s": 70.68121833284386, + "train_s": 55.69134554103948, + "wait_for_rollouts_s": 6.475601650075987, + "weight_sync_s": 8.356003675842658, + "mfu": 0.011021647535770358, + "response_tokens_mean": 2603.688775510204, + "reverse_kl": 0.0005449550226330757, + "train_score": 0.8418367346938775, + "dropped_stale": 68.0, + "train_score_by_domain": { + "agg": 0.859375, + "simple_geometry": 0.9279279279279279, + "caesar_cipher": 0.4117647058823529 + } + }, + { + "step_s": 26.949786484008655, + "train_s": 17.23700745892711, + "wait_for_rollouts_s": 4.17300034314394e-05, + "weight_sync_s": 9.565543129108846, + "mfu": 0.03878971024417892, + "response_tokens_mean": 3008.737804878049, + "reverse_kl": 0.003043749835342169, + "train_score": 0.8963414634146342, + "dropped_stale": 200.0, + "train_score_by_domain": { + "agg": 0.96875, + "simple_geometry": 0.9917355371900827, + "caesar_cipher": 0.5714285714285714 + } + }, + { + "step_s": 47.38750076084398, + "train_s": 16.711925848852843, + "wait_for_rollouts_s": 21.891969391843304, + "weight_sync_s": 8.647439893102273, + "mfu": 0.039971823077660874, + "response_tokens_mean": 1508.483412322275, + "reverse_kl": -0.0019724261946976185, + "train_score": 0.966824644549763, + "dropped_stale": 83.0, + "train_score_by_domain": { + "agg": 0.96875, + "simple_geometry": 0.992, + "caesar_cipher": 0.0 + } + }, + { + "step_s": 75.6891629290767, + "train_s": 52.76680476195179, + "wait_for_rollouts_s": 12.675595352891833, + "weight_sync_s": 10.115268874913454, + "mfu": 0.01261510307355927, + "response_tokens_mean": 1453.58125, + "reverse_kl": -0.0014360127970576286, + "train_score": 0.9875, + "dropped_stale": 32.0, + "train_score_by_domain": { + "agg": 1.0, + "simple_geometry": 1.0 + } + }, + { + "step_s": 37.50508940615691, + "train_s": 30.083991391118616, + "wait_for_rollouts_s": 2.8250040486454964e-05, + "weight_sync_s": 7.288585165981203, + "mfu": 0.02216060056944051, + "response_tokens_mean": 1951.7671232876712, + "reverse_kl": -0.0010565307457000017, + "train_score": 0.9726027397260274, + "dropped_stale": 91.0, + "train_score_by_domain": { + "agg": 1.0, + "simple_geometry": 1.0 + } + }, + { + "step_s": 118.47553761792369, + "train_s": 30.2944921979215, + "wait_for_rollouts_s": 10.907322456827387, + "weight_sync_s": 9.546028025913984, + "mfu": 0.02207062988348777, + "response_tokens_mean": 2033.7926267281107, + "reverse_kl": -0.00012909715587738901, + "train_score": 0.9585253456221198, + "dropped_stale": 89.0, + "train_score_by_domain": { + "agg": 0.984375, + "simple_geometry": 1.0, + "caesar_cipher": 0.6 + } + }, + { + "step_s": 26.690888073062524, + "train_s": 16.33240217086859, + "wait_for_rollouts_s": 5.539017729461193e-05, + "weight_sync_s": 10.222453507129103, + "mfu": 0.040948160567120834, + "response_tokens_mean": 2044.8181818181818, + "reverse_kl": 5.7324075896758586e-05, + "train_score": 0.9710743801652892, + "dropped_stale": 114.0, + "train_score_by_domain": { + "agg": 0.984375, + "caesar_cipher": 0.6666666666666666, + "simple_geometry": 0.992 + } + }, + { + "step_s": 25.605685749091208, + "train_s": 16.45688864006661, + "wait_for_rollouts_s": 3.0159950256347656e-05, + "weight_sync_s": 9.007180728018284, + "mfu": 0.04061608559575782, + "response_tokens_mean": 3368.406340057637, + "reverse_kl": -0.00035442138323560357, + "train_score": 0.9596541786743515, + "dropped_stale": 219.0, + "train_score_by_domain": { + "agg": 0.9921875, + "simple_geometry": 0.9917355371900827, + "caesar_cipher": 1.0 + } + }, + { + "step_s": 48.76286663301289, + "train_s": 15.736092605162412, + "wait_for_rollouts_s": 25.489497027825564, + "weight_sync_s": 7.401062218938023, + "mfu": 0.042436303610602355, + "response_tokens_mean": 2006.1658291457286, + "reverse_kl": -0.0009322343394160271, + "train_score": 0.949748743718593, + "dropped_stale": 71.0, + "train_score_by_domain": { + "agg": 0.96875, + "simple_geometry": 0.983739837398374, + "caesar_cipher": 0.6 + } + }, + { + "step_s": 36.96956689795479, + "train_s": 16.09682928887196, + "wait_for_rollouts_s": 11.013988743070513, + "weight_sync_s": 9.717482106061652, + "mfu": 0.04154235369423416, + "response_tokens_mean": 1666.8726114649683, + "reverse_kl": -0.0004502252850215882, + "train_score": 1.0, + "dropped_stale": 29.0, + "train_score_by_domain": { + "agg": 1.0, + "simple_geometry": 1.0, + "caesar_cipher": 1.0 + } + }, + { + "step_s": 83.27362084505148, + "train_s": 53.636521321954206, + "wait_for_rollouts_s": 21.539139594882727, + "weight_sync_s": 7.961475412826985, + "mfu": 0.01246459073373506, + "response_tokens_mean": 1768.2359550561798, + "reverse_kl": -0.00037494426942430437, + "train_score": 0.9887640449438202, + "dropped_stale": 50.0, + "train_score_by_domain": { + "agg": 0.9921875, + "simple_geometry": 1.0, + "caesar_cipher": 0.75 + } + }, + { + "step_s": 24.333154729101807, + "train_s": 16.970593857113272, + "wait_for_rollouts_s": 4.0839891880750656e-05, + "weight_sync_s": 7.216402923921123, + "mfu": 0.039413086132201396, + "response_tokens_mean": 2262.878048780488, + "reverse_kl": -0.00013548543211072683, + "train_score": 0.975609756097561, + "dropped_stale": 77.0, + "train_score_by_domain": { + "agg": 0.984375, + "simple_geometry": 1.0, + "caesar_cipher": 0.6 + } + }, + { + "step_s": 94.92772321519442, + "train_s": 53.77060312894173, + "wait_for_rollouts_s": 30.275503621203825, + "weight_sync_s": 10.746734441025183, + "mfu": 0.012430472154564087, + "response_tokens_mean": 2145.436936936937, + "reverse_kl": -0.0005359856877475977, + "train_score": 0.963963963963964, + "dropped_stale": 94.0, + "train_score_by_domain": { + "agg": 0.9765625, + "simple_geometry": 0.991869918699187, + "caesar_cipher": 0.6 + } + }, + { + "step_s": 40.58343209186569, + "train_s": 32.26015897211619, + "wait_for_rollouts_s": 3.498885780572891e-05, + "weight_sync_s": 8.176488791126758, + "mfu": 0.020678372095349104, + "response_tokens_mean": 2152.7864583333335, + "reverse_kl": -0.00037460713065229356, + "train_score": 0.96875, + "dropped_stale": 64.0, + "train_score_by_domain": { + "agg": 0.9765625, + "simple_geometry": 0.9917355371900827, + "caesar_cipher": 0.7142857142857143 + } + }, + { + "step_s": 43.252023591892794, + "train_s": 16.228458323981613, + "wait_for_rollouts_s": 17.536295242141932, + "weight_sync_s": 9.351044004084542, + "mfu": 0.04121672415012727, + "response_tokens_mean": 2406.260563380282, + "reverse_kl": -0.00017596267571207136, + "train_score": 0.971830985915493, + "dropped_stale": 156.0, + "train_score_by_domain": { + "agg": 0.9765625, + "simple_geometry": 0.9836065573770492, + "caesar_cipher": 0.8333333333333334 + } + }, + { + "step_s": 89.15152556798421, + "train_s": 52.22494474006817, + "wait_for_rollouts_s": 26.09071954805404, + "weight_sync_s": 10.695798859931529, + "mfu": 0.012804229453994705, + "response_tokens_mean": 1928.8716814159293, + "reverse_kl": -0.00019766451441682875, + "train_score": 0.9867256637168141, + "dropped_stale": 98.0, + "train_score_by_domain": { + "agg": 1.0, + "simple_geometry": 1.0, + "caesar_cipher": 1.0 + } + } + ], + "step_s_median": 48.075183696928434, + "step_s_total": 1366.2244907631539, + "train_s_median": 30.010207190993242, + "train_s_total": 748.3602467544843, + "wait_for_rollouts_s_median": 10.96065559994895, + "wait_for_rollouts_s_total": 366.2642687649932, + "weight_sync_s_median": 8.703181292978115, + "weight_sync_s_total": 176.37205927656032, + "mfu_mean": 0.02684667517502571, + "response_tokens_mean_mean": 2261.12910095001, + "reverse_kl_first": -0.003125404007732868, + "reverse_kl_last": -0.00019766451441682875, + "train_score_first": 0.4765625, + "train_score_last": 0.9867256637168141, + "dropped_stale_total": 2205.0 + }, + "roles": { + "generation": { + "samples": 381, + "seconds": 1900.1600289344788, + "gpus": { + "0": { + "utilization_mean_pct": 4.078740157480315, + "busy_fraction": 0.07349081364829396, + "memory_peak_gib": 162.533203125, + "power_mean_w": 275.9556430446194, + "energy_kwh": 0.14569593868238528 + }, + "1": { + "utilization_mean_pct": 15.766404199475065, + "busy_fraction": 0.3359580052493438, + "memory_peak_gib": 164.47265625, + "power_mean_w": 350.0302887139108, + "energy_kwh": 0.18488458448460168 + }, + "2": { + "utilization_mean_pct": 73.54330708661418, + "busy_fraction": 0.7952755905511811, + "memory_peak_gib": 167.708984375, + "power_mean_w": 704.9364829396325, + "energy_kwh": 0.37270608821253376 + }, + "3": { + "utilization_mean_pct": 73.88976377952756, + "busy_fraction": 0.7926509186351706, + "memory_peak_gib": 167.712890625, + "power_mean_w": 729.7356692913386, + "energy_kwh": 0.3858239697398875 + }, + "4": { + "utilization_mean_pct": 74.46981627296589, + "busy_fraction": 0.7979002624671916, + "memory_peak_gib": 167.6953125, + "power_mean_w": 708.8944881889763, + "energy_kwh": 0.37480764913127684 + }, + "5": { + "utilization_mean_pct": 74.56955380577428, + "busy_fraction": 0.7952755905511811, + "memory_peak_gib": 167.6953125, + "power_mean_w": 732.0548556430447, + "energy_kwh": 0.38704806952739096 + }, + "6": { + "utilization_mean_pct": 74.48031496062993, + "busy_fraction": 0.8083989501312336, + "memory_peak_gib": 167.697265625, + "power_mean_w": 712.0762992125984, + "energy_kwh": 0.3764809949029876 + }, + "7": { + "utilization_mean_pct": 75.02887139107611, + "busy_fraction": 0.8031496062992126, + "memory_peak_gib": 167.693359375, + "power_mean_w": 751.4866141732284, + "energy_kwh": 0.3973159314118313 + } + }, + "gpu_energy_kwh": 2.624763226092895, + "cpu_utilization_pct": 4.432533822599538, + "network": { + "rx_gb": 0.097495148, + "tx_gb": 3.420159019, + "rx_gbit_s": 0.0004104713140594616, + "tx_gbit_s": 0.014399456748568134 + }, + "engines": { + "127.0.0.1:29000": { + "seconds": 1895.1597125530243 + }, + "127.0.0.1:8100": { + "seconds": 1680.1382446289062, + "generation_tokens": 18991508.0, + "generation_tokens_per_s": 11303.538896702321, + "prompt_tokens": 759523.0, + "prompt_tokens_per_s": 452.05982449840405 + }, + "127.0.0.1:8110": { + "seconds": 1680.1382446289062, + "generation_tokens": 19192338.0, + "generation_tokens_per_s": 11423.070727277582, + "prompt_tokens": 753563.0, + "prompt_tokens_per_s": 448.51249735490677 + }, + "127.0.0.1:8120": { + "seconds": 1680.1382446289062, + "generation_tokens": 19199360.0, + "generation_tokens_per_s": 11427.250145264434, + "prompt_tokens": 746881.0, + "prompt_tokens_per_s": 444.53544366818716 + }, + "127.0.0.1:8200": { + "seconds": 1735.144484758377, + "generation_tokens": 673.0, + "generation_tokens_per_s": 0.3878639536428673, + "prompt_tokens": 3297525.0, + "prompt_tokens_per_s": 1900.4325166956849 + }, + "127.0.0.1:8300": { + "seconds": 1740.1449596881866, + "generation_tokens": 4452.0, + "generation_tokens_per_s": 2.5584075482985886, + "prompt_tokens": 10823285.0, + "prompt_tokens_per_s": 6219.760566349257 + } + } + }, + "learner": { + "samples": 377, + "seconds": 1894.0306096076965, + "gpus": { + "0": { + "utilization_mean_pct": 28.800531914893618, + "busy_fraction": 0.48936170212765956, + "memory_peak_gib": 80.974609375, + "power_mean_w": 280.3073670212766, + "energy_kwh": 0.14744374143176608 + }, + "1": { + "utilization_mean_pct": 38.61702127659574, + "busy_fraction": 0.5904255319148937, + "memory_peak_gib": 78.58203125, + "power_mean_w": 282.00978723404256, + "energy_kwh": 0.14833395134538677 + }, + "2": { + "utilization_mean_pct": 40.034574468085104, + "busy_fraction": 0.601063829787234, + "memory_peak_gib": 78.580078125, + "power_mean_w": 274.9821010638298, + "energy_kwh": 0.1446554750184993 + }, + "3": { + "utilization_mean_pct": 37.651595744680854, + "busy_fraction": 0.5797872340425532, + "memory_peak_gib": 78.578125, + "power_mean_w": 287.61619680851067, + "energy_kwh": 0.15128732691411576 + }, + "4": { + "utilization_mean_pct": 39.72872340425532, + "busy_fraction": 0.5904255319148937, + "memory_peak_gib": 78.578125, + "power_mean_w": 271.9703989361702, + "energy_kwh": 0.14307807138007284 + }, + "5": { + "utilization_mean_pct": 38.3218085106383, + "busy_fraction": 0.5718085106382979, + "memory_peak_gib": 78.578125, + "power_mean_w": 277.08994680851066, + "energy_kwh": 0.14577761698117983 + }, + "6": { + "utilization_mean_pct": 39.180851063829785, + "busy_fraction": 0.5904255319148937, + "memory_peak_gib": 78.580078125, + "power_mean_w": 275.16914893617025, + "energy_kwh": 0.1447767609946443 + }, + "7": { + "utilization_mean_pct": 37.191489361702125, + "busy_fraction": 0.5611702127659575, + "memory_peak_gib": 78.58203125, + "power_mean_w": 275.88970744680853, + "energy_kwh": 0.14515663630388842 + } + }, + "gpu_energy_kwh": 1.1705095803695533, + "cpu_utilization_pct": 5.724327161908349, + "network": { + "rx_gb": 3.717527659, + "tx_gb": 0.098420492, + "rx_gbit_s": 0.015702080590006925, + "tx_gbit_s": 0.00041570813692556093 + }, + "engines": {} + } + } + }, + { + "run": "verl-1051", + "job_id": "1051", + "nodes": [ + "gpu-nodes-3", + "gpu-nodes-1" + ], + "exit_code": 0, + "wall_seconds": 3357.1256580352783, + "updates": { + "updates": 19, + "per_update": [ + { + "step_s": 163.68112234794535, + "train_s": 113.16815605806187, + "wait_for_rollouts_s": 41.823744534980506, + "weight_sync_s": 8.424361743964255, + "mfu": 0.0042628982218963435, + "response_tokens_mean": 2932.2578125, + "reverse_kl": 0.003643331103376113, + "train_score": 0.4609375, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 0.5730337078651685, + "caesar_cipher": 0.20512820512820512 + } + }, + { + "step_s": 109.27551374700852, + "train_s": 86.07384177809581, + "wait_for_rollouts_s": 10.982415556907654, + "weight_sync_s": 11.904187139123678, + "mfu": 0.022577542871995884, + "response_tokens_mean": 10868.34375, + "reverse_kl": 0.001609073633517255, + "train_score": 0.3828125, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 0.6470588235294118, + "caesar_cipher": 0.2872340425531915 + } + }, + { + "step_s": 44.703333143144846, + "train_s": 25.964825903996825, + "wait_for_rollouts_s": 9.8228163740132, + "weight_sync_s": 8.593307483009994, + "mfu": 0.056395093230011274, + "response_tokens_mean": 8026.3046875, + "reverse_kl": 0.0016527983048035821, + "train_score": 0.484375, + "dropped_stale": 0.0, + "train_score_by_domain": { + "caesar_cipher": 0.30434782608695654, + "simple_geometry": 0.6949152542372882 + } + }, + { + "step_s": 89.57026797602884, + "train_s": 41.057049548020586, + "wait_for_rollouts_s": 33.36260530306026, + "weight_sync_s": 14.869121566182002, + "mfu": 0.019425074560009553, + "response_tokens_mean": 4610.4609375, + "reverse_kl": 0.001959218368938309, + "train_score": 0.6015625, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 0.693069306930693, + "caesar_cipher": 0.25925925925925924 + } + }, + { + "step_s": 65.93025959189981, + "train_s": 23.672928417799994, + "wait_for_rollouts_s": 28.518038460053504, + "weight_sync_s": 13.40376951196231, + "mfu": 0.060624933694658024, + "response_tokens_mean": 8065.6171875, + "reverse_kl": 0.0012736430417135125, + "train_score": 0.625, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 0.8852459016393442, + "caesar_cipher": 0.3880597014925373 + } + }, + { + "step_s": 58.290142136858776, + "train_s": 21.37864540098235, + "wait_for_rollouts_s": 27.034399650990963, + "weight_sync_s": 9.580822207033634, + "mfu": 0.05754193755276339, + "response_tokens_mean": 6717.3046875, + "reverse_kl": 0.0012797708477592096, + "train_score": 0.765625, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 0.9571428571428572, + "caesar_cipher": 0.5344827586206896 + } + }, + { + "step_s": 82.70157731091604, + "train_s": 45.99434436787851, + "wait_for_rollouts_s": 27.172121257986873, + "weight_sync_s": 9.225600189063698, + "mfu": 0.027276676096893625, + "response_tokens_mean": 6930.9765625, + "reverse_kl": 0.0013755085337834316, + "train_score": 0.8359375, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.6181818181818182 + } + }, + { + "step_s": 61.05241607897915, + "train_s": 24.639245423022658, + "wait_for_rollouts_s": 22.706725819967687, + "weight_sync_s": 13.420536634977907, + "mfu": 0.05848962022195303, + "response_tokens_mean": 8043.2734375, + "reverse_kl": 0.0012850319462813786, + "train_score": 0.734375, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.5142857142857142 + } + }, + { + "step_s": 50.8314291338902, + "train_s": 19.999646032927558, + "wait_for_rollouts_s": 20.467740196967497, + "weight_sync_s": 10.085437720175833, + "mfu": 0.054010774910723015, + "response_tokens_mean": 5904.046875, + "reverse_kl": 0.0014304792948678369, + "train_score": 0.9140625, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.7843137254901961 + } + }, + { + "step_s": 71.88532021106221, + "train_s": 23.484106296906248, + "wait_for_rollouts_s": 37.45830306992866, + "weight_sync_s": 10.660703226923943, + "mfu": 0.052323058925796626, + "response_tokens_mean": 6720.640625, + "reverse_kl": 0.0011938333736907225, + "train_score": 0.8828125, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.6938775510204082 + } + }, + { + "step_s": 32.97425053990446, + "train_s": 22.30680564395152, + "wait_for_rollouts_s": 1.0793376150541008, + "weight_sync_s": 9.29222150798887, + "mfu": 0.059457892617337194, + "response_tokens_mean": 7364.609375, + "reverse_kl": 0.001268169364266214, + "train_score": 0.8515625, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.6885245901639344 + } + }, + { + "step_s": 65.62164144590497, + "train_s": 51.673440205166116, + "wait_for_rollouts_s": 0.7483603809960186, + "weight_sync_s": 12.808988738106564, + "mfu": 0.06966581946300494, + "response_tokens_mean": 18627.3125, + "reverse_kl": 0.0010643259547578054, + "train_score": 0.6171875, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.5882352941176471 + } + }, + { + "step_s": 40.48766777408309, + "train_s": 15.20874652499333, + "wait_for_rollouts_s": 15.520171461859718, + "weight_sync_s": 9.50544296298176, + "mfu": 0.03850649220082006, + "response_tokens_mean": 3435.4609375, + "reverse_kl": 0.001441976441128645, + "train_score": 0.9609375, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 0.98989898989899, + "caesar_cipher": 0.8620689655172413 + } + }, + { + "step_s": 65.1050968291238, + "train_s": 16.977829401846975, + "wait_for_rollouts_s": 38.402561753988266, + "weight_sync_s": 9.468254117993638, + "mfu": 0.04918835792881676, + "response_tokens_mean": 4859.3515625, + "reverse_kl": 0.001357000672214781, + "train_score": 0.9609375, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.8780487804878049 + } + }, + { + "step_s": 79.68094378011301, + "train_s": 23.60284376097843, + "wait_for_rollouts_s": 44.62459372798912, + "weight_sync_s": 11.14258997910656, + "mfu": 0.05448120349823577, + "response_tokens_mean": 7256.203125, + "reverse_kl": 0.0010941049022221705, + "train_score": 0.9296875, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.8524590163934426 + } + }, + { + "step_s": 71.62824719911441, + "train_s": 29.5394736349117, + "wait_for_rollouts_s": 32.0797269300092, + "weight_sync_s": 9.689773757942021, + "mfu": 0.05683457862502435, + "response_tokens_mean": 9108.0078125, + "reverse_kl": 0.001113153495680308, + "train_score": 0.875, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.7647058823529411 + } + }, + { + "step_s": 68.05372566683218, + "train_s": 24.138273591874167, + "wait_for_rollouts_s": 34.054053008090705, + "weight_sync_s": 9.506673542084172, + "mfu": 0.057452184659658845, + "response_tokens_mean": 7543.6640625, + "reverse_kl": 0.0011261941053817281, + "train_score": 0.921875, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.8507462686567164 + } + }, + { + "step_s": 64.7523849599529, + "train_s": 23.383745652856305, + "wait_for_rollouts_s": 32.16087406896986, + "weight_sync_s": 8.921405290951952, + "mfu": 0.062191107260580074, + "response_tokens_mean": 7860.546875, + "reverse_kl": 0.0011407502088331967, + "train_score": 0.8984375, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 0.987012987012987, + "caesar_cipher": 0.7647058823529411 + } + }, + { + "step_s": 69.09424458397552, + "train_s": 25.44133411697112, + "wait_for_rollouts_s": 33.924993091030046, + "weight_sync_s": 9.429947745986283, + "mfu": 0.05898619998200122, + "response_tokens_mean": 8213.9765625, + "reverse_kl": 0.001190158151985088, + "train_score": 0.8828125, + "dropped_stale": 0.0, + "train_score_by_domain": { + "simple_geometry": 1.0, + "caesar_cipher": 0.7794117647058824 + } + } + ], + "step_s_median": 65.93025959189981, + "step_s_total": 1355.319584456738, + "train_s_median": 24.138273591874167, + "train_s_total": 657.7052817612421, + "wait_for_rollouts_s_median": 28.518038460053504, + "wait_for_rollouts_s_total": 491.94358226284385, + "weight_sync_s_median": 9.580822207033634, + "weight_sync_s_total": 199.93314506555907, + "mfu_mean": 0.04840481297485157, + "response_tokens_mean_mean": 7530.966282894737, + "reverse_kl_first": 0.003643331103376113, + "reverse_kl_last": 0.001190158151985088, + "train_score_first": 0.4609375, + "train_score_last": 0.8828125, + "dropped_stale_total": 0.0 + }, + "roles": { + "generation": { + "samples": 669, + "seconds": 3341.782376766205, + "gpus": { + "0": { + "utilization_mean_pct": 17.86676646706587, + "busy_fraction": 0.23053892215568864, + "memory_peak_gib": 122.4892578125, + "power_mean_w": 307.13396706586826, + "energy_kwh": 0.28505555227879614 + }, + "1": { + "utilization_mean_pct": 17.396706586826348, + "busy_fraction": 0.2215568862275449, + "memory_peak_gib": 123.1572265625, + "power_mean_w": 313.47980538922155, + "energy_kwh": 0.29094793622427584 + }, + "2": { + "utilization_mean_pct": 17.98502994011976, + "busy_fraction": 0.23353293413173654, + "memory_peak_gib": 123.0185546875, + "power_mean_w": 302.2042964071856, + "energy_kwh": 0.28048325527504153 + }, + "3": { + "utilization_mean_pct": 18.18562874251497, + "busy_fraction": 0.22904191616766467, + "memory_peak_gib": 123.1572265625, + "power_mean_w": 301.2754491017964, + "energy_kwh": 0.27962060682819617 + }, + "4": { + "utilization_mean_pct": 16.809880239520957, + "busy_fraction": 0.22005988023952097, + "memory_peak_gib": 123.0419921875, + "power_mean_w": 306.42853293413174, + "energy_kwh": 0.2844035059605354 + }, + "5": { + "utilization_mean_pct": 17.169161676646706, + "busy_fraction": 0.20808383233532934, + "memory_peak_gib": 123.0283203125, + "power_mean_w": 303.24814371257486, + "energy_kwh": 0.281452491402332 + }, + "6": { + "utilization_mean_pct": 17.380239520958085, + "busy_fraction": 0.2125748502994012, + "memory_peak_gib": 122.1865234375, + "power_mean_w": 292.4883682634731, + "energy_kwh": 0.2714664277543624 + }, + "7": { + "utilization_mean_pct": 17.38622754491018, + "busy_fraction": 0.21706586826347304, + "memory_peak_gib": 123.2939453125, + "power_mean_w": 303.5594461077844, + "energy_kwh": 0.28174252198712296 + } + }, + "gpu_energy_kwh": 2.2551722977106623, + "cpu_utilization_pct": 1.9186668425835052, + "network": { + "rx_gb": 14.70530907, + "tx_gb": 12.034101441, + "rx_gbit_s": 0.03520351097004735, + "tx_gbit_s": 0.02880882136351495 + }, + "engines": {} + }, + "learner": { + "samples": 668, + "seconds": 3335.248383283615, + "gpus": { + "0": { + "utilization_mean_pct": 6.842814371257485, + "busy_fraction": 0.11826347305389222, + "memory_peak_gib": 146.8818359375, + "power_mean_w": 309.04556886227545, + "energy_kwh": 0.2863932045959737 + }, + "1": { + "utilization_mean_pct": 2.538922155688623, + "busy_fraction": 0.06287425149700598, + "memory_peak_gib": 146.8818359375, + "power_mean_w": 268.6003143712575, + "energy_kwh": 0.24885827976855504 + }, + "2": { + "utilization_mean_pct": 61.01047904191617, + "busy_fraction": 0.6362275449101796, + "memory_peak_gib": 147.181640625, + "power_mean_w": 541.1236227544911, + "energy_kwh": 0.5017111517431861 + }, + "3": { + "utilization_mean_pct": 60.79940119760479, + "busy_fraction": 0.6362275449101796, + "memory_peak_gib": 147.181640625, + "power_mean_w": 564.0933832335329, + "energy_kwh": 0.5230119488803698 + }, + "4": { + "utilization_mean_pct": 60.553892215568865, + "busy_fraction": 0.6362275449101796, + "memory_peak_gib": 147.240234375, + "power_mean_w": 540.4137874251496, + "energy_kwh": 0.5010504500694904 + }, + "5": { + "utilization_mean_pct": 61.381736526946106, + "busy_fraction": 0.6377245508982036, + "memory_peak_gib": 147.240234375, + "power_mean_w": 553.8183682634731, + "energy_kwh": 0.5134844670450655 + }, + "6": { + "utilization_mean_pct": 60.06736526946108, + "busy_fraction": 0.6287425149700598, + "memory_peak_gib": 147.2421875, + "power_mean_w": 543.4121556886228, + "energy_kwh": 0.5038254827259395 + }, + "7": { + "utilization_mean_pct": 60.67964071856287, + "busy_fraction": 0.6362275449101796, + "memory_peak_gib": 147.2421875, + "power_mean_w": 547.6597305389222, + "energy_kwh": 0.507768959205085 + } + }, + "gpu_energy_kwh": 3.586103944033665, + "cpu_utilization_pct": 3.6814619126085635, + "network": { + "rx_gb": 12.032154805, + "tx_gb": 14.709234677, + "rx_gbit_s": 0.028860590690170104, + "tx_gbit_s": 0.03528189324842663 + }, + "engines": {} + } + } + } +] diff --git a/docs/mopd-2026-09-29/data/verl-1051-step10/summary.json b/docs/mopd-2026-09-29/data/verl-1051-step10/summary.json new file mode 100644 index 0000000..f4667bf --- /dev/null +++ b/docs/mopd-2026-09-29/data/verl-1051-step10/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/verl-mopd2/checkpoints/verl-1051/global_step_10/actor/huggingface-merged", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 393.9323546886444, + "output_tokens_per_second": 42571.090697169944, + "domains": { + "caesar_cipher": { + "avg@8": 0.7400793650793651, + "pass@8": 1.0, + "finished_thinking": 0.9404761904761905, + "truncated": 0.061507936507936505, + "response_tokens_mean": 13946.602182539682, + "response_tokens_median": 12174.5 + }, + "simple_geometry": { + "avg@8": 0.9970703125, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 2648.3935546875, + "response_tokens_median": 1971.0 + } + } +} diff --git a/docs/mopd-2026-09-29/data/verl-1051/summary.json b/docs/mopd-2026-09-29/data/verl-1051/summary.json new file mode 100644 index 0000000..b3747e2 --- /dev/null +++ b/docs/mopd-2026-09-29/data/verl-1051/summary.json @@ -0,0 +1,26 @@ +{ + "model": "/shared/opd-runs/verl-mopd2/checkpoints/verl-1051/global_step_20/actor/huggingface-merged", + "samples": 8, + "seed": 0, + "gpus": 8, + "generation_seconds": 385.28420329093933, + "output_tokens_per_second": 43619.574476323265, + "domains": { + "caesar_cipher": { + "avg@8": 0.7678571428571429, + "pass@8": 1.0, + "finished_thinking": 0.9384920634920635, + "truncated": 0.06448412698412699, + "response_tokens_mean": 13545.394841269841, + "response_tokens_median": 11606.5 + }, + "simple_geometry": { + "avg@8": 0.99609375, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 3078.2958984375, + "response_tokens_median": 2806.5 + } + } +} diff --git a/docs/mopd-h100-2026-10-07/data/base/summary.json b/docs/mopd-h100-2026-10-07/data/base/summary.json new file mode 100644 index 0000000..2acd004 --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/base/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/assets/models/Qwen3.6-35B-A3B", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 849.0627772808075, + "output_tokens_per_second": 17909.520246193613, + "domains": { + "caesar_cipher": { + "avg@8": 0.2698412698412698, + "pass@8": 0.6428571428571429, + "finished_thinking": 0.9583333333333334, + "truncated": 0.04265873015873016, + "response_tokens_mean": 11439.588293650793, + "response_tokens_median": 9597.0 + }, + "simple_geometry": { + "avg@8": 0.552734375, + "pass@8": 1.0, + "finished_thinking": 0.9990234375, + "truncated": 0.0009765625, + "response_tokens_mean": 3589.064453125, + "response_tokens_median": 3233.5 + } + } +} diff --git a/docs/mopd-h100-2026-10-07/data/case-miles-535.json b/docs/mopd-h100-2026-10-07/data/case-miles-535.json new file mode 100644 index 0000000..4d224f5 --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/case-miles-535.json @@ -0,0 +1,558 @@ +{ + "meta": { + "result": "/mnt/data/opd/runs/miles-mopd-h100/results/miles-535", + "framework": "miles", + "domains": [ + "caesar_cipher", + "simple_geometry" + ], + "nodes": 3, + "policy_gpus": 12, + "teacher_gpus": 4, + "active_params": 3000000000.0, + "h100_peak_flops": 989400000000000.0, + "h100_usd_per_hour": 6.16 + }, + "updates": [ + { + "update": 1, + "window_s": 1057.6124396324158, + "step_s": 454.4655759334564, + "train_s": 354.9641637802124, + "wait_s": 99.50141215324402, + "train_tokens": 519659, + "train_tok_s_gpu": 182.99699414226072, + "mfu": 0.0033292357939768476, + "prompt_p50": 158.0, + "response_p50": 3629.0, + "rollout_e2e_p50_s": 54.848484848484844, + "rollout_requests": 623.0, + "rollout_tok_s_gpu": 359.8512578000145, + "filtered": 0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 40.94592219595842, + "teacher_p50_s": 0.15876865671641793, + "teacher_tok_s_gpu": 1077.219506226656, + "teacher_mfu": 0.006532562196644366, + "cost_usd": 43.43261752090454, + "train_score": { + "caesar_cipher": 0.28125, + "simple_geometry": 0.484375 + }, + "samples": 128 + }, + { + "update": 2, + "window_s": 140.65000009536743, + "step_s": 140.65005588531494, + "train_s": 136.06386542320251, + "wait_s": 4.586190462112427, + "train_tokens": 804841, + "train_tok_s_gpu": 739.3963466133025, + "mfu": 0.013451722497513082, + "prompt_p50": 161.0, + "response_p50": 5094.5, + "rollout_e2e_p50_s": 54.166666666666664, + "rollout_requests": 134.0, + "rollout_tok_s_gpu": 545.5065762387234, + "filtered": 0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 476.8580397288063, + "teacher_p50_s": 0.1385964912280702, + "teacher_tok_s_gpu": 1867.6395294837398, + "teacher_mfu": 0.011325891628160946, + "cost_usd": 5.776026670583089, + "train_score": { + "caesar_cipher": 0.28125, + "simple_geometry": 0.515625 + }, + "samples": 128 + }, + { + "update": 3, + "window_s": 51.53099989891052, + "step_s": 51.53161859512329, + "train_s": 46.886722564697266, + "wait_s": 4.644896030426025, + "train_tokens": 901167, + "train_tok_s_gpu": 2402.5111766889677, + "mfu": 0.04370851140125472, + "prompt_p50": 161.0, + "response_p50": 5200.5, + "rollout_e2e_p50_s": 26.086956521739133, + "rollout_requests": 56.0, + "rollout_tok_s_gpu": 369.70302738234756, + "filtered": 0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1457.3218091502183, + "teacher_p50_s": 0.08870967741935484, + "teacher_tok_s_gpu": 1125.3808409261253, + "teacher_mfu": 0.00682462608202623, + "cost_usd": 2.116206395848592, + "train_score": { + "caesar_cipher": 0.265625, + "simple_geometry": 0.515625 + }, + "samples": 128 + }, + { + "update": 4, + "window_s": 66.16100001335144, + "step_s": 66.16081881523132, + "train_s": 60.217230796813965, + "wait_s": 5.943588018417358, + "train_tokens": 774643, + "train_tok_s_gpu": 1608.0177337733571, + "mfu": 0.029254416017708135, + "prompt_p50": 159.0, + "response_p50": 4720.5, + "rollout_e2e_p50_s": 45.23809523809524, + "rollout_requests": 109.0, + "rollout_tok_s_gpu": 825.0253168631781, + "filtered": 250, + "filtered_ratio": 0.6613756613756614, + "effective_tok_s_gpu_policy": 975.7044681958597, + "teacher_p50_s": 0.12980769230769232, + "teacher_tok_s_gpu": 2352.67000149013, + "teacher_mfu": 0.014267252889570224, + "cost_usd": 2.7170117338816326, + "train_score": { + "caesar_cipher": 0.375, + "simple_geometry": 0.59375 + }, + "samples": 128 + }, + { + "update": 5, + "window_s": 139.77300000190735, + "step_s": 139.77254629135132, + "train_s": 61.7414870262146, + "wait_s": 78.03105926513672, + "train_tokens": 804687, + "train_tok_s_gpu": 1629.1456497847646, + "mfu": 0.02963879290087504, + "prompt_p50": 160.5, + "response_p50": 4608.0, + "rollout_e2e_p50_s": 57.826086956521735, + "rollout_requests": 187.0, + "rollout_tok_s_gpu": 802.0075407873501, + "filtered": 21, + "filtered_ratio": 0.14093959731543623, + "effective_tok_s_gpu_policy": 479.7582508716628, + "teacher_p50_s": 0.14347826086956522, + "teacher_tok_s_gpu": 2537.972283596683, + "teacher_mfu": 0.015390978069112694, + "cost_usd": 5.740011200078329, + "train_score": { + "caesar_cipher": 0.421875, + "simple_geometry": 0.828125 + }, + "samples": 128 + }, + { + "update": 6, + "window_s": 253.63300013542175, + "step_s": 253.63281345367432, + "train_s": 179.91235637664795, + "wait_s": 73.72045707702637, + "train_tokens": 937488, + "train_tok_s_gpu": 651.350481757185, + "mfu": 0.011849917800312644, + "prompt_p50": 160.0, + "response_p50": 5368.5, + "rollout_e2e_p50_s": 56.81818181818181, + "rollout_requests": 422.0, + "rollout_tok_s_gpu": 1152.3319777419179, + "filtered": 58, + "filtered_ratio": 0.3118279569892473, + "effective_tok_s_gpu_policy": 308.01985529598835, + "teacher_p50_s": 0.15080645161290324, + "teacher_tok_s_gpu": 3514.544635453802, + "teacher_mfu": 0.021313187601296555, + "cost_usd": 10.415861872227987, + "train_score": { + "caesar_cipher": 0.484375, + "simple_geometry": 0.890625 + }, + "samples": 128 + }, + { + "update": 7, + "window_s": 50.26599979400635, + "step_s": 50.266199350357056, + "train_s": 45.01760530471802, + "wait_s": 5.248594045639038, + "train_tokens": 836290, + "train_tok_s_gpu": 2322.1192973817333, + "mfu": 0.04224595447025591, + "prompt_p50": 161.0, + "response_p50": 4606.5, + "rollout_e2e_p50_s": 52.5, + "rollout_requests": 60.0, + "rollout_tok_s_gpu": 580.4055780493082, + "filtered": 75, + "filtered_ratio": 0.3694581280788177, + "effective_tok_s_gpu_policy": 1386.4408072838767, + "teacher_p50_s": 0.125, + "teacher_tok_s_gpu": 1903.5729994852177, + "teacher_mfu": 0.011543802301305141, + "cost_usd": 2.064257058207194, + "train_score": { + "caesar_cipher": 0.59375, + "simple_geometry": 0.890625 + }, + "samples": 128 + }, + { + "update": 8, + "window_s": 107.87600016593933, + "step_s": 107.8759081363678, + "train_s": 96.62245583534241, + "wait_s": 11.25345230102539, + "train_tokens": 690157, + "train_tok_s_gpu": 892.8527458152687, + "mfu": 0.016243530851702888, + "prompt_p50": 160.0, + "response_p50": 4147.5, + "rollout_e2e_p50_s": 55.65217391304348, + "rollout_requests": 186.0, + "rollout_tok_s_gpu": 1226.0048555430083, + "filtered": 63, + "filtered_ratio": 0.3298429319371728, + "effective_tok_s_gpu_policy": 533.140673040012, + "teacher_p50_s": 0.16181818181818183, + "teacher_tok_s_gpu": 3432.014529927823, + "teacher_mfu": 0.020812701818846712, + "cost_usd": 4.430107740147909, + "train_score": { + "caesar_cipher": 0.75, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 9, + "window_s": 210.94899988174438, + "step_s": 210.94867873191833, + "train_s": 111.44216990470886, + "wait_s": 99.50650882720947, + "train_tokens": 763241, + "train_tok_s_gpu": 856.0953639145604, + "mfu": 0.015574809531495943, + "prompt_p50": 160.0, + "response_p50": 4174.5, + "rollout_e2e_p50_s": 69.16666666666667, + "rollout_requests": 288.0, + "rollout_tok_s_gpu": 1048.445359418554, + "filtered": 172, + "filtered_ratio": 0.5733333333333334, + "effective_tok_s_gpu_policy": 301.5108708850102, + "teacher_p50_s": 0.16860465116279072, + "teacher_tok_s_gpu": 3387.948036732309, + "teacher_mfu": 0.020545470204562214, + "cost_usd": 8.662972261810303, + "train_score": { + "caesar_cipher": 0.734375, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 10, + "window_s": 122.93499994277954, + "step_s": 122.93500351905823, + "train_s": 116.31412029266357, + "wait_s": 6.620883226394653, + "train_tokens": 968232, + "train_tok_s_gpu": 1040.5357466098965, + "mfu": 0.018930304668463852, + "prompt_p50": 160.0, + "response_p50": 5408.0, + "rollout_e2e_p50_s": 62.72727272727273, + "rollout_requests": 189.0, + "rollout_tok_s_gpu": 1078.5143915758715, + "filtered": 28, + "filtered_ratio": 0.1794871794871795, + "effective_tok_s_gpu_policy": 656.330581506939, + "teacher_p50_s": 0.16228070175438597, + "teacher_tok_s_gpu": 3247.8687939630254, + "teacher_mfu": 0.019695990260539874, + "cost_usd": 5.0485306643168135, + "train_score": { + "caesar_cipher": 0.78125, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 11, + "window_s": 283.07800006866455, + "step_s": 283.0779414176941, + "train_s": 84.84011793136597, + "wait_s": 198.23782348632812, + "train_tokens": 906208, + "train_tok_s_gpu": 1335.1702326915447, + "mfu": 0.024290543954364065, + "prompt_p50": 161.0, + "response_p50": 4941.5, + "rollout_e2e_p50_s": 72.67857142857143, + "rollout_requests": 393.0, + "rollout_tok_s_gpu": 1089.5430467639792, + "filtered": 124, + "filtered_ratio": 0.49206349206349204, + "effective_tok_s_gpu_policy": 266.7721734469494, + "teacher_p50_s": 0.16551724137931034, + "teacher_tok_s_gpu": 3320.528263489206, + "teacher_mfu": 0.02013661772886116, + "cost_usd": 11.625069869486492, + "train_score": { + "caesar_cipher": 0.8125, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 12, + "window_s": 87.02100014686584, + "step_s": 87.0206401348114, + "train_s": 81.05648064613342, + "wait_s": 5.9641594886779785, + "train_tokens": 836690, + "train_tok_s_gpu": 1290.288563805157, + "mfu": 0.02347401874721329, + "prompt_p50": 160.0, + "response_p50": 5177.0, + "rollout_e2e_p50_s": 44.375, + "rollout_requests": 127.0, + "rollout_tok_s_gpu": 888.6810831425722, + "filtered": 33, + "filtered_ratio": 0.20496894409937888, + "effective_tok_s_gpu_policy": 801.2338004503831, + "teacher_p50_s": 0.14444444444444446, + "teacher_tok_s_gpu": 2693.367113736183, + "teacher_mfu": 0.016333336044488677, + "cost_usd": 3.573662406031291, + "train_score": { + "caesar_cipher": 0.84375, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 13, + "window_s": 53.014999866485596, + "step_s": 53.015042543411255, + "train_s": 42.68810558319092, + "wait_s": 10.326936960220337, + "train_tokens": 699616, + "train_tok_s_gpu": 2048.626867022076, + "mfu": 0.037270349309073546, + "prompt_p50": 161.0, + "response_p50": 4529.5, + "rollout_e2e_p50_s": 54.34782608695652, + "rollout_requests": 91.0, + "rollout_tok_s_gpu": 855.6776405591868, + "filtered": 215, + "filtered_ratio": 0.6268221574344023, + "effective_tok_s_gpu_policy": 1099.7139202142976, + "teacher_p50_s": 0.13375, + "teacher_tok_s_gpu": 2446.1803324832654, + "teacher_mfu": 0.01483432584889791, + "cost_usd": 2.177149327850342, + "train_score": { + "caesar_cipher": 0.859375, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 14, + "window_s": 122.83800005912781, + "step_s": 122.83777141571045, + "train_s": 52.6164345741272, + "wait_s": 70.22133684158325, + "train_tokens": 768957, + "train_tok_s_gpu": 1826.7985236548961, + "mfu": 0.033234660830592413, + "prompt_p50": 160.0, + "response_p50": 4313.5, + "rollout_e2e_p50_s": 52.96296296296296, + "rollout_requests": 187.0, + "rollout_tok_s_gpu": 851.9330333416946, + "filtered": 24, + "filtered_ratio": 0.15789473684210525, + "effective_tok_s_gpu_policy": 521.6606422210989, + "teacher_p50_s": 0.13125, + "teacher_tok_s_gpu": 2570.603150881697, + "teacher_mfu": 0.015588860830089127, + "cost_usd": 5.044547202428182, + "train_score": { + "caesar_cipher": 0.875, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 15, + "window_s": 200.40599989891052, + "step_s": 200.40550541877747, + "train_s": 132.22712397575378, + "wait_s": 68.17838144302368, + "train_tokens": 790512, + "train_tok_s_gpu": 747.3050689518084, + "mfu": 0.013595604650427076, + "prompt_p50": 162.0, + "response_p50": 4658.0, + "rollout_e2e_p50_s": 66.76470588235294, + "rollout_requests": 317.0, + "rollout_tok_s_gpu": 1078.6453504837166, + "filtered": 75, + "filtered_ratio": 0.3694581280788177, + "effective_tok_s_gpu_policy": 328.71271335803016, + "teacher_p50_s": 0.15202020202020203, + "teacher_tok_s_gpu": 3202.199536559328, + "teacher_mfu": 0.0194190390331069, + "cost_usd": 8.230006395848593, + "train_score": { + "caesar_cipher": 0.71875, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 16, + "window_s": 134.29500007629395, + "step_s": 134.29571843147278, + "train_s": 121.34201979637146, + "wait_s": 12.953698635101318, + "train_tokens": 721980, + "train_tok_s_gpu": 743.7448309451885, + "mfu": 0.013530833795242968, + "prompt_p50": 159.0, + "response_p50": 4292.5, + "rollout_e2e_p50_s": 63.333333333333336, + "rollout_requests": 203.0, + "rollout_tok_s_gpu": 979.6108064975983, + "filtered": 66, + "filtered_ratio": 0.3402061855670103, + "effective_tok_s_gpu_policy": 448.00625463211463, + "teacher_p50_s": 0.15066666666666667, + "teacher_tok_s_gpu": 3264.3024665918574, + "teacher_mfu": 0.019795648675511566, + "cost_usd": 5.515048003133138, + "train_score": { + "caesar_cipher": 0.828125, + "simple_geometry": 0.984375 + }, + "samples": 128 + }, + { + "update": 17, + "window_s": 88.7170000076294, + "step_s": 88.71644616127014, + "train_s": 82.4870092868805, + "wait_s": 6.229436874389648, + "train_tokens": 832002, + "train_tok_s_gpu": 1260.8076216983318, + "mfu": 0.022937676562128535, + "prompt_p50": 159.5, + "response_p50": 4860.0, + "rollout_e2e_p50_s": 53.57142857142857, + "rollout_requests": 127.0, + "rollout_tok_s_gpu": 897.73944106711, + "filtered": 64, + "filtered_ratio": 0.3333333333333333, + "effective_tok_s_gpu_policy": 781.5131259402091, + "teacher_p50_s": 0.13, + "teacher_tok_s_gpu": 2757.453475421422, + "teacher_mfu": 0.01672197377453864, + "cost_usd": 3.6433114669799807, + "train_score": { + "caesar_cipher": 0.875, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 18, + "window_s": 192.92400002479553, + "step_s": 192.92430472373962, + "train_s": 141.87546300888062, + "wait_s": 51.04884171485901, + "train_tokens": 811270, + "train_tok_s_gpu": 714.7729977357141, + "mfu": 0.013003753748982064, + "prompt_p50": 158.0, + "response_p50": 4846.0, + "rollout_e2e_p50_s": 63.333333333333336, + "rollout_requests": 299.0, + "rollout_tok_s_gpu": 1108.518552931968, + "filtered": 71, + "filtered_ratio": 0.35678391959798994, + "effective_tok_s_gpu_policy": 350.4272839286159, + "teacher_p50_s": 0.15637254901960784, + "teacher_tok_s_gpu": 3349.3137189616223, + "teacher_mfu": 0.020311180830573816, + "cost_usd": 7.92274560101827, + "train_score": { + "caesar_cipher": 0.90625, + "simple_geometry": 0.96875 + }, + "samples": 128 + }, + { + "update": 19, + "window_s": 107.51999998092651, + "step_s": 107.51968431472778, + "train_s": 96.6757264137268, + "wait_s": 10.843957901000977, + "train_tokens": 739626, + "train_tok_s_gpu": 956.3233029596635, + "mfu": 0.017398240805815587, + "prompt_p50": 161.0, + "response_p50": 3729.0, + "rollout_e2e_p50_s": 56.95652173913044, + "rollout_requests": 157.0, + "rollout_tok_s_gpu": 950.7262216468284, + "filtered": 89, + "filtered_ratio": 0.41013824884792627, + "effective_tok_s_gpu_policy": 573.24683789931, + "teacher_p50_s": 0.139622641509434, + "teacher_tok_s_gpu": 2913.2975265584705, + "teacher_mfu": 0.017667055952446758, + "cost_usd": 4.415487999216715, + "train_score": { + "caesar_cipher": 0.8125, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 20, + "window_s": 56.36500000953674, + "step_s": 56.36443829536438, + "train_s": 41.41533589363098, + "wait_s": 14.949102401733398, + "train_tokens": 710044, + "train_tok_s_gpu": 2143.05879899067, + "mfu": 0.03898833472996974, + "prompt_p50": 160.0, + "response_p50": 4314.0, + "rollout_e2e_p50_s": 69.23076923076923, + "rollout_requests": 58.0, + "rollout_tok_s_gpu": 681.2472277743834, + "filtered": 62, + "filtered_ratio": 0.3263157894736842, + "effective_tok_s_gpu_policy": 1049.7708386999368, + "teacher_p50_s": 0.14130434782608697, + "teacher_tok_s_gpu": 2084.693515126742, + "teacher_mfu": 0.01264216807232712, + "cost_usd": 2.314722667058309, + "train_score": { + "caesar_cipher": 0.921875, + "simple_geometry": 1.0 + }, + "samples": 128 + } + ] +} diff --git a/docs/mopd-h100-2026-10-07/data/case-verl-547.json b/docs/mopd-h100-2026-10-07/data/case-verl-547.json new file mode 100644 index 0000000..de5a5ab --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/case-verl-547.json @@ -0,0 +1,558 @@ +{ + "meta": { + "result": "/mnt/data/opd/runs/verl-mopd-h100x3/results/verl-547", + "framework": "verl", + "domains": [ + "caesar_cipher", + "simple_geometry" + ], + "nodes": 3, + "policy_gpus": 12, + "teacher_gpus": 4, + "active_params": 3000000000.0, + "h100_peak_flops": 989400000000000.0, + "h100_usd_per_hour": 6.16 + }, + "updates": [ + { + "update": 1, + "window_s": 1054.8999934196472, + "step_s": 157.46815395797603, + "train_s": 72.13880910514854, + "wait_s": 75.98429391812533, + "train_tokens": 432250.0, + "train_tok_s_gpu": 748.9900466924092, + "mfu": 0.013626259187854626, + "prompt_p50": 160.5, + "response_p50": 3036.5, + "rollout_e2e_p50_s": 52.35294117647059, + "rollout_requests": 256.0, + "rollout_tok_s_gpu": 144.329637200752, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 34.14620680446243, + "teacher_p50_s": 0.17539682539682538, + "teacher_tok_s_gpu": 316.9402332785843, + "teacher_mfu": 0.0019220147560860175, + "cost_usd": 43.32122639643351, + "train_score": { + "caesar_cipher": 0.22857142857142856, + "simple_geometry": 0.5483870967741935 + }, + "samples": 128 + }, + { + "update": 2, + "window_s": 151.58706712722778, + "step_s": 151.5362989641726, + "train_s": 108.55033885897137, + "wait_s": 33.06113710091449, + "train_tokens": 1155552.0, + "train_tok_s_gpu": 1330.6637410654396, + "mfu": 0.02420855805455621, + "prompt_p50": 150.5, + "response_p50": 8665.0, + "rollout_e2e_p50_s": 63.75, + "rollout_requests": 163.0, + "rollout_tok_s_gpu": 758.0813819485294, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 635.2520820208117, + "teacher_p50_s": 0.2664705882352941, + "teacher_tok_s_gpu": 2784.220039337326, + "teacher_mfu": 0.016884293749771535, + "cost_usd": 6.225175556691488, + "train_score": { + "caesar_cipher": 0.24096385542168675, + "simple_geometry": 0.5777777777777777 + }, + "samples": 128 + }, + { + "update": 3, + "window_s": 169.8319320678711, + "step_s": 169.67962916800752, + "train_s": 142.60930972499773, + "wait_s": 17.802019564900547, + "train_tokens": 1500578.0, + "train_tok_s_gpu": 1315.287552837238, + "mfu": 0.02392882145853071, + "prompt_p50": 149.0, + "response_p50": 9112.5, + "rollout_e2e_p50_s": 44.54545454545455, + "rollout_requests": 140.0, + "rollout_tok_s_gpu": 520.9218642776267, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 736.3053881804325, + "teacher_p50_s": 0.21185567010309278, + "teacher_tok_s_gpu": 1824.8526424120598, + "teacher_mfu": 0.011066419905470345, + "cost_usd": 6.9744313435872405, + "train_score": { + "caesar_cipher": 0.3855421686746988, + "simple_geometry": 0.6666666666666666 + }, + "samples": 128 + }, + { + "update": 4, + "window_s": 127.28926706314087, + "step_s": 127.13413568004034, + "train_s": 111.57209883397445, + "wait_s": 5.745203615864739, + "train_tokens": 1178884.0, + "train_tok_s_gpu": 1320.7647928115136, + "mfu": 0.02402846803174373, + "prompt_p50": 157.0, + "response_p50": 5798.0, + "rollout_e2e_p50_s": 45.666666666666664, + "rollout_requests": 131.0, + "rollout_tok_s_gpu": 617.3897072904882, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 771.7880352363248, + "teacher_p50_s": 0.17227722772277226, + "teacher_tok_s_gpu": 1522.7422898417094, + "teacher_mfu": 0.009234337718870281, + "cost_usd": 5.227345900726319, + "train_score": { + "caesar_cipher": 0.25757575757575757, + "simple_geometry": 0.7903225806451613 + }, + "samples": 128 + }, + { + "update": 5, + "window_s": 139.40089559555054, + "step_s": 139.10201638704166, + "train_s": 111.69663062901236, + "wait_s": 17.81016233493574, + "train_tokens": 847480.0, + "train_tok_s_gpu": 948.417149232111, + "mfu": 0.017254405383240346, + "prompt_p50": 160.0, + "response_p50": 4978.0, + "rollout_e2e_p50_s": 53.18181818181818, + "rollout_requests": 143.0, + "rollout_tok_s_gpu": 697.2037942662652, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 506.62037020361527, + "teacher_p50_s": 0.20833333333333334, + "teacher_tok_s_gpu": 2023.6419486031202, + "teacher_mfu": 0.012271934194075925, + "cost_usd": 5.724730112457276, + "train_score": { + "caesar_cipher": 0.47058823529411764, + "simple_geometry": 0.8441558441558441 + }, + "samples": 128 + }, + { + "update": 6, + "window_s": 113.24619054794312, + "step_s": 113.09543127985671, + "train_s": 89.40752077591605, + "wait_s": 14.375854168087244, + "train_tokens": 1111292.0, + "train_tok_s_gpu": 1553.6892063941334, + "mfu": 0.02826602558630928, + "prompt_p50": 158.5, + "response_p50": 5833.0, + "rollout_e2e_p50_s": 39.04761904761905, + "rollout_requests": 146.0, + "rollout_tok_s_gpu": 642.9627608754548, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 817.7552482656, + "teacher_p50_s": 0.19142857142857142, + "teacher_tok_s_gpu": 2281.701474899574, + "teacher_mfu": 0.013836879775012577, + "cost_usd": 4.650643558502197, + "train_score": { + "caesar_cipher": 0.41935483870967744, + "simple_geometry": 0.9090909090909091 + }, + "samples": 128 + }, + { + "update": 7, + "window_s": 94.95301008224487, + "step_s": 94.78068739897572, + "train_s": 75.11372813419439, + "wait_s": 8.124031006125733, + "train_tokens": 985744.0, + "train_tok_s_gpu": 1640.4191758378036, + "mfu": 0.029843890403356034, + "prompt_p50": 156.5, + "response_p50": 4832.5, + "rollout_e2e_p50_s": 41.111111111111114, + "rollout_requests": 140.0, + "rollout_tok_s_gpu": 746.4823910122033, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 865.1156320603424, + "teacher_p50_s": 0.18375, + "teacher_tok_s_gpu": 1460.7409483897513, + "teacher_mfu": 0.008858344138203465, + "cost_usd": 3.8994036140441892, + "train_score": { + "caesar_cipher": 0.6666666666666666, + "simple_geometry": 0.9384615384615385 + }, + "samples": 128 + }, + { + "update": 8, + "window_s": 86.06965351104736, + "step_s": 85.91620063199662, + "train_s": 54.46955477213487, + "wait_s": 22.23660610197112, + "train_tokens": 680606.0, + "train_tok_s_gpu": 1561.8954543671507, + "mfu": 0.028415320576721963, + "prompt_p50": 157.0, + "response_p50": 2741.5, + "rollout_e2e_p50_s": 47.142857142857146, + "rollout_requests": 170.0, + "rollout_tok_s_gpu": 895.1916289920213, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 658.9682234445943, + "teacher_p50_s": 0.2, + "teacher_tok_s_gpu": 2189.1019925602022, + "teacher_mfu": 0.013275330458218328, + "cost_usd": 3.5345937708536783, + "train_score": { + "caesar_cipher": 0.6410256410256411, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 9, + "window_s": 105.90701508522034, + "step_s": 105.75324994302355, + "train_s": 69.48699316591956, + "wait_s": 27.22823149501346, + "train_tokens": 839500.0, + "train_tok_s_gpu": 1510.1747135529736, + "mfu": 0.02747437319987217, + "prompt_p50": 157.0, + "response_p50": 3990.5, + "rollout_e2e_p50_s": 42.857142857142854, + "rollout_requests": 186.0, + "rollout_tok_s_gpu": 844.0179018806184, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 660.5637339230067, + "teacher_p50_s": 0.19513274336283187, + "teacher_tok_s_gpu": 2669.183432018467, + "teacher_mfu": 0.016186679393683852, + "cost_usd": 4.349248086166382, + "train_score": { + "caesar_cipher": 0.7547169811320755, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 10, + "window_s": 118.42375612258911, + "step_s": 118.2596399821341, + "train_s": 90.83887093584053, + "wait_s": 17.25522934203036, + "train_tokens": 1102893.0, + "train_tok_s_gpu": 1517.6501378729336, + "mfu": 0.027610372429465135, + "prompt_p50": 155.0, + "response_p50": 5164.5, + "rollout_e2e_p50_s": 45.0, + "rollout_requests": 167.0, + "rollout_tok_s_gpu": 755.7689684099443, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 776.0921711084687, + "teacher_p50_s": 0.21562499999999998, + "teacher_tok_s_gpu": 2694.5311519225907, + "teacher_mfu": 0.01634039509959121, + "cost_usd": 4.863268918100992, + "train_score": { + "caesar_cipher": 0.6935483870967742, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 11, + "window_s": 181.2333915233612, + "step_s": 100.26386936800554, + "train_s": 89.72285618004389, + "wait_s": 1.1828010438475758, + "train_tokens": 1204256.0, + "train_tok_s_gpu": 1677.7441825740857, + "mfu": 0.03052293843373109, + "prompt_p50": 156.0, + "response_p50": 6330.5, + "rollout_e2e_p50_s": 43.94736842105263, + "rollout_requests": 157.0, + "rollout_tok_s_gpu": 525.7226562894094, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 553.7316596193083, + "teacher_p50_s": 0.25, + "teacher_tok_s_gpu": 2179.1031259771657, + "teacher_mfu": 0.013214694517751157, + "cost_usd": 7.442651278559367, + "train_score": { + "caesar_cipher": 0.7083333333333334, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 12, + "window_s": 124.08746290206909, + "step_s": 123.90051033883356, + "train_s": 110.96198445605114, + "wait_s": 3.011363113997504, + "train_tokens": 1459774.0, + "train_tok_s_gpu": 1644.4528357572033, + "mfu": 0.02991727414961558, + "prompt_p50": 152.5, + "response_p50": 8554.0, + "rollout_e2e_p50_s": 36.875, + "rollout_requests": 130.0, + "rollout_tok_s_gpu": 595.7565328336968, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 980.3394354942922, + "teacher_p50_s": 0.17500000000000002, + "teacher_tok_s_gpu": 1276.4625554880201, + "teacher_mfu": 0.0077408281109037, + "cost_usd": 5.095858476511637, + "train_score": { + "caesar_cipher": 0.7727272727272727, + "simple_geometry": 0.975 + }, + "samples": 128 + }, + { + "update": 13, + "window_s": 77.56407570838928, + "step_s": 77.40729019814171, + "train_s": 53.547592700924724, + "wait_s": 14.265767524018884, + "train_tokens": 688585.0, + "train_tok_s_gpu": 1607.4135298805616, + "mfu": 0.02924342383045291, + "prompt_p50": 159.0, + "response_p50": 3593.0, + "rollout_e2e_p50_s": 56.15384615384615, + "rollout_requests": 146.0, + "rollout_tok_s_gpu": 868.0167468221971, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 739.8023222640803, + "teacher_p50_s": 0.1782608695652174, + "teacher_tok_s_gpu": 1499.5344034947352, + "teacher_mfu": 0.009093598565765526, + "cost_usd": 3.18529804242452, + "train_score": { + "caesar_cipher": 0.8222222222222222, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 14, + "window_s": 140.77797889709473, + "step_s": 140.6351224980317, + "train_s": 91.93189708678983, + "wait_s": 39.18968259706162, + "train_tokens": 768510.0, + "train_tok_s_gpu": 1044.9447149916793, + "mfu": 0.01901051634308695, + "prompt_p50": 159.5, + "response_p50": 3440.5, + "rollout_e2e_p50_s": 53.07692307692308, + "rollout_requests": 192.0, + "rollout_tok_s_gpu": 827.1789682280806, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 454.91845032676247, + "teacher_p50_s": 0.23883495145631065, + "teacher_tok_s_gpu": 2771.408234843267, + "teacher_mfu": 0.016806599362299984, + "cost_usd": 5.7812823333740235, + "train_score": { + "caesar_cipher": 0.85, + "simple_geometry": 0.9886363636363636 + }, + "samples": 128 + }, + { + "update": 15, + "window_s": 135.25416803359985, + "step_s": 135.09517380595207, + "train_s": 107.93955398793332, + "wait_s": 17.158166541950777, + "train_tokens": 1322467.0, + "train_tok_s_gpu": 1531.4902544296226, + "mfu": 0.027862163512970696, + "prompt_p50": 153.5, + "response_p50": 8728.5, + "rollout_e2e_p50_s": 50.476190476190474, + "rollout_requests": 156.0, + "rollout_tok_s_gpu": 672.1468771600638, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 814.8036022516959, + "teacher_p50_s": 0.23138297872340424, + "teacher_tok_s_gpu": 2692.1702694555297, + "teacher_mfu": 0.016326078043999572, + "cost_usd": 5.554437833913168, + "train_score": { + "caesar_cipher": 0.7631578947368421, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 16, + "window_s": 127.72047114372253, + "step_s": 127.55550135998055, + "train_s": 109.17334860702977, + "wait_s": 9.286360906204209, + "train_tokens": 1401481.0, + "train_tok_s_gpu": 1604.651018176424, + "mfu": 0.029193165885562596, + "prompt_p50": 154.5, + "response_p50": 6718.0, + "rollout_e2e_p50_s": 39.375, + "rollout_requests": 139.0, + "rollout_tok_s_gpu": 604.6955457366863, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 914.4194527900751, + "teacher_p50_s": 0.20380434782608695, + "teacher_tok_s_gpu": 1738.0181736896948, + "teacher_mfu": 0.010539831253424467, + "cost_usd": 5.245054014968872, + "train_score": { + "caesar_cipher": 0.65, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 17, + "window_s": 91.43886733055115, + "step_s": 91.23192371800542, + "train_s": 72.38457321701571, + "wait_s": 9.321144385030493, + "train_tokens": 890526.0, + "train_tok_s_gpu": 1537.838037205317, + "mfu": 0.027977647735694063, + "prompt_p50": 156.0, + "response_p50": 4517.5, + "rollout_e2e_p50_s": 42.142857142857146, + "rollout_requests": 142.0, + "rollout_tok_s_gpu": 744.7571474591836, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 811.5859498973159, + "teacher_p50_s": 0.1736842105263158, + "teacher_tok_s_gpu": 1968.706035577212, + "teacher_mfu": 0.011938787359473695, + "cost_usd": 3.755089485041301, + "train_score": { + "caesar_cipher": 0.8524590163934426, + "simple_geometry": 0.9850746268656716 + }, + "samples": 128 + }, + { + "update": 18, + "window_s": 102.22660803794861, + "step_s": 102.06261775107123, + "train_s": 70.66702642501332, + "wait_s": 21.698727420996875, + "train_tokens": 807348.0, + "train_tok_s_gpu": 1428.0847108670596, + "mfu": 0.025980922574901026, + "prompt_p50": 158.5, + "response_p50": 4142.0, + "rollout_e2e_p50_s": 50.0, + "rollout_requests": 160.0, + "rollout_tok_s_gpu": 817.7983365051659, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 658.1358932991757, + "teacher_p50_s": 0.18267326732673267, + "teacher_tok_s_gpu": 2108.2769362746913, + "teacher_mfu": 0.01278518457413397, + "cost_usd": 4.198106036758423, + "train_score": { + "caesar_cipher": 0.8181818181818182, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 19, + "window_s": 122.68275475502014, + "step_s": 122.52054440090433, + "train_s": 86.24118382902816, + "wait_s": 27.003234497038648, + "train_tokens": 1019850.0, + "train_tok_s_gpu": 1478.1945740996512, + "mfu": 0.02689256350696758, + "prompt_p50": 156.0, + "response_p50": 5535.5, + "rollout_e2e_p50_s": 48.05555555555556, + "rollout_requests": 165.0, + "rollout_tok_s_gpu": 777.285964304867, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 692.7420253132385, + "teacher_p50_s": 0.2154255319148936, + "teacher_tok_s_gpu": 2416.456172605376, + "teacher_mfu": 0.014654070179535333, + "cost_usd": 5.038171795272827, + "train_score": { + "caesar_cipher": 0.7777777777777778, + "simple_geometry": 1.0 + }, + "samples": 128 + }, + { + "update": 20, + "window_s": 216.9059283733368, + "step_s": null, + "train_s": null, + "wait_s": null, + "train_tokens": 1585671, + "train_tok_s_gpu": null, + "mfu": null, + "prompt_p50": 154.5, + "response_p50": 9372.5, + "rollout_e2e_p50_s": 27.5, + "rollout_requests": 30.0, + "rollout_tok_s_gpu": 72.34972068777448, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 609.2007304316872, + "teacher_p50_s": 0.5857142857142857, + "teacher_tok_s_gpu": 823.7004001683271, + "teacher_mfu": 0.00499515100162721, + "cost_usd": 8.907603458531698, + "train_score": { + "caesar_cipher": 0.7142857142857143, + "simple_geometry": 0.9772727272727273 + }, + "samples": 128 + } + ] +} diff --git a/docs/mopd-h100-2026-10-07/data/miles-535-step10/summary.json b/docs/mopd-h100-2026-10-07/data/miles-535-step10/summary.json new file mode 100644 index 0000000..1e36f39 --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/miles-535-step10/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/runs/miles-mopd-h100/checkpoints/miles-535/hf/iter_9", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 904.6952466964722, + "output_tokens_per_second": 18382.338208061297, + "domains": { + "caesar_cipher": { + "avg@8": 0.7321428571428571, + "pass@8": 0.9841269841269841, + "finished_thinking": 0.9375, + "truncated": 0.0625, + "response_tokens_mean": 13283.975198412698, + "response_tokens_median": 11330.5 + }, + "simple_geometry": { + "avg@8": 0.99609375, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 3164.2255859375, + "response_tokens_median": 2873.5 + } + } +} diff --git a/docs/mopd-h100-2026-10-07/data/miles-535/summary.json b/docs/mopd-h100-2026-10-07/data/miles-535/summary.json new file mode 100644 index 0000000..0fc3761 --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/miles-535/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/runs/miles-mopd-h100/checkpoints/miles-535/hf/iter_19", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 862.1017329692841, + "output_tokens_per_second": 18668.582122631487, + "domains": { + "caesar_cipher": { + "avg@8": 0.7728174603174603, + "pass@8": 0.9920634920634921, + "finished_thinking": 0.9494047619047619, + "truncated": 0.05357142857142857, + "response_tokens_mean": 13043.654761904761, + "response_tokens_median": 11065.0 + }, + "simple_geometry": { + "avg@8": 0.998046875, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 2877.1611328125, + "response_tokens_median": 2426.0 + } + } +} diff --git a/docs/mopd-h100-2026-10-07/data/schema-miles-535.json b/docs/mopd-h100-2026-10-07/data/schema-miles-535.json new file mode 100644 index 0000000..c82ed7a --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/schema-miles-535.json @@ -0,0 +1,342 @@ +[ + { + "update": 1, + "step_s": 454.4655759334564, + "train_s": 354.9641637802124, + "wait_s": 99.50141215324402, + "train_tok_s_gpu": 183.02453803761375, + "mfu": 0.003005292527842236, + "prompt_p50": 158.0, + "response_p50": 3629.0, + "rollout_e2e_p50_s": 54.848484848484844, + "rollout_requests_finished": 623.0, + "stale_filtered": 0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 40.94592219595842, + "teacher_p50_s": 0.14743999997153878, + "teacher_tok_s_gpu": 1102.8600423850858 + }, + { + "update": 2, + "step_s": 140.65005588531494, + "train_s": 136.06386542320251, + "wait_s": 4.586190462112427, + "train_tok_s_gpu": 739.6602012024285, + "mfu": 0.01368823748481477, + "prompt_p50": 161.0, + "response_p50": 5094.5, + "rollout_e2e_p50_s": 54.166666666666664, + "rollout_requests_finished": 134.0, + "stale_filtered": 0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 476.8580397288063, + "teacher_p50_s": 0.1164470849907957, + "teacher_tok_s_gpu": 1674.5520784948615 + }, + { + "update": 3, + "step_s": 51.53161859512329, + "train_s": 46.886722564697266, + "wait_s": 4.644896030426025, + "train_tok_s_gpu": 2407.002738343562, + "mfu": 0.04844046488615055, + "prompt_p50": 161.0, + "response_p50": 5200.5, + "rollout_e2e_p50_s": 26.086956521739133, + "rollout_requests_finished": 56.0, + "stale_filtered": 0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1457.3218091502183, + "teacher_p50_s": 0.1010839410009794, + "teacher_tok_s_gpu": 1185.6212788390258 + }, + { + "update": 4, + "step_s": 66.16081881523132, + "train_s": 60.217230796813965, + "wait_s": 5.943588018417358, + "train_tok_s_gpu": 1609.0457430853214, + "mfu": 0.03052037553908493, + "prompt_p50": 159.0, + "response_p50": 4720.5, + "rollout_e2e_p50_s": 45.23809523809524, + "rollout_requests_finished": 109.0, + "stale_filtered": 250, + "filtered_ratio": 0.6613756613756614, + "effective_tok_s_gpu_policy": 975.7044681958597, + "teacher_p50_s": 0.12048782396595925, + "teacher_tok_s_gpu": 2516.0857599855904 + }, + { + "update": 5, + "step_s": 139.77254629135132, + "train_s": 61.7414870262146, + "wait_s": 78.03105926513672, + "train_tok_s_gpu": 1629.9800873387956, + "mfu": 0.03188748272896835, + "prompt_p50": 160.5, + "response_p50": 4608.0, + "rollout_e2e_p50_s": 57.826086956521735, + "rollout_requests_finished": 187.0, + "stale_filtered": 21, + "filtered_ratio": 0.14093959731543623, + "effective_tok_s_gpu_policy": 479.7582508716628, + "teacher_p50_s": 0.1319861989468336, + "teacher_tok_s_gpu": 2539.612442997976 + }, + { + "update": 6, + "step_s": 253.63281345367432, + "train_s": 179.91235637664795, + "wait_s": 73.72045707702637, + "train_tok_s_gpu": 651.4839984570797, + "mfu": 0.013234671170088936, + "prompt_p50": 160.0, + "response_p50": 5368.5, + "rollout_e2e_p50_s": 56.81818181818181, + "rollout_requests_finished": 422.0, + "stale_filtered": 58, + "filtered_ratio": 0.3118279569892473, + "effective_tok_s_gpu_policy": 308.01985529598835, + "teacher_p50_s": 0.13818848400842398, + "teacher_tok_s_gpu": 3465.857161846635 + }, + { + "update": 7, + "step_s": 50.266199350357056, + "train_s": 45.01760530471802, + "wait_s": 5.248594045639038, + "train_tok_s_gpu": 2323.8087390906508, + "mfu": 0.04556571951149853, + "prompt_p50": 161.0, + "response_p50": 4606.5, + "rollout_e2e_p50_s": 52.5, + "rollout_requests_finished": 60.0, + "stale_filtered": 75, + "filtered_ratio": 0.3694581280788177, + "effective_tok_s_gpu_policy": 1386.4408072838767, + "teacher_p50_s": 0.11133324902039021, + "teacher_tok_s_gpu": 1931.8028169724887 + }, + { + "update": 8, + "step_s": 107.8759081363678, + "train_s": 96.62245583534241, + "wait_s": 11.25345230102539, + "train_tok_s_gpu": 893.0792374565505, + "mfu": 0.01663322525421774, + "prompt_p50": 160.0, + "response_p50": 4147.5, + "rollout_e2e_p50_s": 55.65217391304348, + "rollout_requests_finished": 186.0, + "stale_filtered": 63, + "filtered_ratio": 0.3298429319371728, + "effective_tok_s_gpu_policy": 533.140673040012, + "teacher_p50_s": 0.16745829599676654, + "teacher_tok_s_gpu": 3527.4087787335843 + }, + { + "update": 9, + "step_s": 210.94867873191833, + "train_s": 111.44216990470886, + "wait_s": 99.50650882720947, + "train_tok_s_gpu": 856.3011979658504, + "mfu": 0.0169579677177502, + "prompt_p50": 160.0, + "response_p50": 4174.5, + "rollout_e2e_p50_s": 69.16666666666667, + "rollout_requests_finished": 288.0, + "stale_filtered": 172, + "filtered_ratio": 0.5733333333333334, + "effective_tok_s_gpu_policy": 301.5108708850102, + "teacher_p50_s": 0.16717412450816482, + "teacher_tok_s_gpu": 3331.36564000755 + }, + { + "update": 10, + "step_s": 122.93500351905823, + "train_s": 116.31412029266357, + "wait_s": 6.620883226394653, + "train_tok_s_gpu": 1040.956856188867, + "mfu": 0.021993258057476407, + "prompt_p50": 160.0, + "response_p50": 5408.0, + "rollout_e2e_p50_s": 62.72727272727273, + "rollout_requests_finished": 189.0, + "stale_filtered": 28, + "filtered_ratio": 0.1794871794871795, + "effective_tok_s_gpu_policy": 656.330581506939, + "teacher_p50_s": 0.1604286614456214, + "teacher_tok_s_gpu": 3323.1199429792205 + }, + { + "update": 11, + "step_s": 283.0779414176941, + "train_s": 84.84011793136597, + "wait_s": 198.23782348632812, + "train_tok_s_gpu": 1335.7142925129465, + "mfu": 0.027430190582695597, + "prompt_p50": 161.0, + "response_p50": 4941.5, + "rollout_e2e_p50_s": 72.67857142857143, + "rollout_requests_finished": 393.0, + "stale_filtered": 124, + "filtered_ratio": 0.49206349206349204, + "effective_tok_s_gpu_policy": 266.7721734469494, + "teacher_p50_s": 0.16049630497582257, + "teacher_tok_s_gpu": 3291.133538367573 + }, + { + "update": 12, + "step_s": 87.0206401348114, + "train_s": 81.05648064613342, + "wait_s": 5.9641594886779785, + "train_tok_s_gpu": 1290.9586157715762, + "mfu": 0.02577144271137155, + "prompt_p50": 160.0, + "response_p50": 5177.0, + "rollout_e2e_p50_s": 44.375, + "rollout_requests_finished": 127.0, + "stale_filtered": 33, + "filtered_ratio": 0.20496894409937888, + "effective_tok_s_gpu_policy": 801.2338004503831, + "teacher_p50_s": 0.12049713893793523, + "teacher_tok_s_gpu": 2686.3630572558923 + }, + { + "update": 13, + "step_s": 53.015042543411255, + "train_s": 42.68810558319092, + "wait_s": 10.326936960220337, + "train_tok_s_gpu": 2050.0062400399697, + "mfu": 0.03717609736071514, + "prompt_p50": 161.0, + "response_p50": 4529.5, + "rollout_e2e_p50_s": 54.34782608695652, + "rollout_requests_finished": 91.0, + "stale_filtered": 215, + "filtered_ratio": 0.6268221574344023, + "effective_tok_s_gpu_policy": 1099.7139202142976, + "teacher_p50_s": 0.1279368325485848, + "teacher_tok_s_gpu": 2477.091395467837 + }, + { + "update": 14, + "step_s": 122.83777141571045, + "train_s": 52.6164345741272, + "wait_s": 70.22133684158325, + "train_tok_s_gpu": 1828.067920225134, + "mfu": 0.03467242470615417, + "prompt_p50": 160.0, + "response_p50": 4313.5, + "rollout_e2e_p50_s": 52.96296296296296, + "rollout_requests_finished": 187.0, + "stale_filtered": 24, + "filtered_ratio": 0.15789473684210525, + "effective_tok_s_gpu_policy": 521.6606422210989, + "teacher_p50_s": 0.11508921394124627, + "teacher_tok_s_gpu": 2626.845925891665 + }, + { + "update": 15, + "step_s": 200.40550541877747, + "train_s": 132.22712397575378, + "wait_s": 68.17838144302368, + "train_tok_s_gpu": 747.5341111507134, + "mfu": 0.014458679518432466, + "prompt_p50": 162.0, + "response_p50": 4658.0, + "rollout_e2e_p50_s": 66.76470588235294, + "rollout_requests_finished": 317.0, + "stale_filtered": 75, + "filtered_ratio": 0.3694581280788177, + "effective_tok_s_gpu_policy": 328.71271335803016, + "teacher_p50_s": 0.1426544399582781, + "teacher_tok_s_gpu": 3315.2962003889184 + }, + { + "update": 16, + "step_s": 134.29571843147278, + "train_s": 121.34201979637146, + "wait_s": 12.953698635101318, + "train_tok_s_gpu": 744.0161584962907, + "mfu": 0.013853295607187378, + "prompt_p50": 159.0, + "response_p50": 4292.5, + "rollout_e2e_p50_s": 63.333333333333336, + "rollout_requests_finished": 203.0, + "stale_filtered": 66, + "filtered_ratio": 0.3402061855670103, + "effective_tok_s_gpu_policy": 448.00625463211463, + "teacher_p50_s": 0.13606953696580604, + "teacher_tok_s_gpu": 3032.7320434016224 + }, + { + "update": 17, + "step_s": 88.71644616127014, + "train_s": 82.4870092868805, + "wait_s": 6.229436874389648, + "train_tok_s_gpu": 1261.3985739313532, + "mfu": 0.02476782405492066, + "prompt_p50": 159.5, + "response_p50": 4860.0, + "rollout_e2e_p50_s": 53.57142857142857, + "rollout_requests_finished": 127.0, + "stale_filtered": 64, + "filtered_ratio": 0.3333333333333333, + "effective_tok_s_gpu_policy": 781.5131259402091, + "teacher_p50_s": 0.12701783701777458, + "teacher_tok_s_gpu": 2750.4367818909095 + }, + { + "update": 18, + "step_s": 192.92430472373962, + "train_s": 141.87546300888062, + "wait_s": 51.04884171485901, + "train_tok_s_gpu": 714.9357763924928, + "mfu": 0.013991258510328715, + "prompt_p50": 158.0, + "response_p50": 4846.0, + "rollout_e2e_p50_s": 63.333333333333336, + "rollout_requests_finished": 299.0, + "stale_filtered": 71, + "filtered_ratio": 0.35678391959798994, + "effective_tok_s_gpu_policy": 350.4272839286159, + "teacher_p50_s": 0.15971627598628402, + "teacher_tok_s_gpu": 3354.782193593417 + }, + { + "update": 19, + "step_s": 107.51968431472778, + "train_s": 96.6757264137268, + "wait_s": 10.843957901000977, + "train_tok_s_gpu": 956.8392335095093, + "mfu": 0.018179576543788763, + "prompt_p50": 161.0, + "response_p50": 3729.0, + "rollout_e2e_p50_s": 56.95652173913044, + "rollout_requests_finished": 157.0, + "stale_filtered": 89, + "filtered_ratio": 0.41013824884792627, + "effective_tok_s_gpu_policy": 573.24683789931, + "teacher_p50_s": 0.13697821600362659, + "teacher_tok_s_gpu": 2966.8689551394023 + }, + { + "update": 20, + "step_s": 56.36443829536438, + "train_s": 41.41533589363098, + "wait_s": 14.949102401733398, + "train_tok_s_gpu": 2145.19766747417, + "mfu": 0.04005950346789854, + "prompt_p50": 160.0, + "response_p50": 4314.0, + "rollout_e2e_p50_s": 69.23076923076923, + "rollout_requests_finished": 58.0, + "stale_filtered": 62, + "filtered_ratio": 0.3263157894736842, + "effective_tok_s_gpu_policy": 1049.7708386999368, + "teacher_p50_s": 0.1467031000647694, + "teacher_tok_s_gpu": 2015.2443889076756 + } +] \ No newline at end of file diff --git a/docs/mopd-h100-2026-10-07/data/schema-verl-544.json b/docs/mopd-h100-2026-10-07/data/schema-verl-544.json new file mode 100644 index 0000000..6cbe598 --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/schema-verl-544.json @@ -0,0 +1,382 @@ +[ + { + "update": 1, + "step_s": 331.0327812922187, + "train_s": 219.9481017848011, + "wait_s": 102.1812730350066, + "train_tok_s_gpu": 366.1068195022923, + "mfu": 0.008410134588537262, + "prompt_p50": 157.0, + "response_p50": 4105.5, + "rollout_e2e_p50_s": 47.5, + "rollout_requests_finished": 160.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 42.54738004974909, + "teacher_p50_s": 0.21052631578947367, + "teacher_tok_s_gpu": 238.09037230591366, + "window_s": 1260.1265523433685, + "samples_dumped": 128 + }, + { + "update": 2, + "step_s": 277.3908871989697, + "train_s": 172.08919557393529, + "wait_s": 95.66267460607924, + "train_tok_s_gpu": 807.5332651566425, + "mfu": 0.019830302675845186, + "prompt_p50": 154.5, + "response_p50": 6769.0, + "rollout_e2e_p50_s": 52.857142857142854, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 333.65331637415323, + "teacher_p50_s": 0.2157303370786517, + "teacher_tok_s_gpu": 1133.7495453026163, + "window_s": 277.4426691532135, + "samples_dumped": 128 + }, + { + "update": 3, + "step_s": 256.5701579740271, + "train_s": 163.98876623390242, + "wait_s": 83.31280922493897, + "train_tok_s_gpu": 889.0441299622343, + "mfu": 0.022690161187340734, + "prompt_p50": 155.0, + "response_p50": 5716.0, + "rollout_e2e_p50_s": 44.705882352941174, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 378.3431757668584, + "teacher_p50_s": 0.20425531914893616, + "teacher_tok_s_gpu": 1033.1713240263273, + "window_s": 256.7122642993927, + "samples_dumped": 128 + }, + { + "update": 4, + "step_s": 257.5085908109322, + "train_s": 170.76189642003737, + "wait_s": 77.35863755992614, + "train_tok_s_gpu": 756.942870217696, + "mfu": 0.018784096047510065, + "prompt_p50": 156.0, + "response_p50": 5348.0, + "rollout_e2e_p50_s": 43.333333333333336, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 334.1782244832077, + "teacher_p50_s": 0.19591836734693877, + "teacher_tok_s_gpu": 1074.288660367763, + "window_s": 257.6511883735657, + "samples_dumped": 128 + }, + { + "update": 5, + "step_s": 238.1251734038815, + "train_s": 152.6832142099738, + "wait_s": 76.29334677709267, + "train_tok_s_gpu": 925.9760526495681, + "mfu": 0.02354885011583412, + "prompt_p50": 158.5, + "response_p50": 5658.0, + "rollout_e2e_p50_s": 45.0, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 395.2291265774918, + "teacher_p50_s": 0.192, + "teacher_tok_s_gpu": 1094.9568551295188, + "window_s": 238.26600909233093, + "samples_dumped": 128 + }, + { + "update": 6, + "step_s": 224.91772644710727, + "train_s": 135.7752734089736, + "wait_s": 80.12757786014117, + "train_tok_s_gpu": 907.7131417645487, + "mfu": 0.022638668583644685, + "prompt_p50": 159.0, + "response_p50": 4944.5, + "rollout_e2e_p50_s": 39.5, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 364.7684170486512, + "teacher_p50_s": 0.2064516129032258, + "teacher_tok_s_gpu": 1108.5405515027978, + "window_s": 225.05671954154968, + "samples_dumped": 128 + }, + { + "update": 7, + "step_s": 207.51343710604124, + "train_s": 134.80695838597603, + "wait_s": 63.829666739096865, + "train_tok_s_gpu": 886.7300058632246, + "mfu": 0.0221961574913447, + "prompt_p50": 156.5, + "response_p50": 4215.5, + "rollout_e2e_p50_s": 35.625, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 383.44290504732436, + "teacher_p50_s": 0.20210526315789473, + "teacher_tok_s_gpu": 1078.9087417451906, + "window_s": 207.64499473571777, + "samples_dumped": 128 + }, + { + "update": 8, + "step_s": 227.94323329092003, + "train_s": 136.13090053782798, + "wait_s": 82.25497572496533, + "train_tok_s_gpu": 881.1840260078687, + "mfu": 0.02186599349545001, + "prompt_p50": 156.0, + "response_p50": 4986.5, + "rollout_e2e_p50_s": 39.0, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 350.3081371482417, + "teacher_p50_s": 0.1979381443298969, + "teacher_tok_s_gpu": 1130.4410460693487, + "window_s": 228.07491898536682, + "samples_dumped": 128 + }, + { + "update": 9, + "step_s": 207.6120190178044, + "train_s": 136.08299569389783, + "wait_s": 62.69191908603534, + "train_tok_s_gpu": 888.8463572045239, + "mfu": 0.02232872727535973, + "prompt_p50": 156.0, + "response_p50": 4369.5, + "rollout_e2e_p50_s": 35.714285714285715, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 387.7832139328155, + "teacher_p50_s": 0.18823529411764706, + "teacher_tok_s_gpu": 1095.390455554909, + "window_s": 207.74601316452026, + "samples_dumped": 128 + }, + { + "update": 10, + "step_s": 192.93960647494532, + "train_s": 121.53734386898577, + "wait_s": 61.155663846060634, + "train_tok_s_gpu": 910.800306112727, + "mfu": 0.022593888564180414, + "prompt_p50": 157.0, + "response_p50": 4298.5, + "rollout_e2e_p50_s": 32.77777777777778, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 381.79327060668385, + "teacher_p50_s": 0.1846153846153846, + "teacher_tok_s_gpu": 1200.2411661074918, + "window_s": 193.10952377319336, + "samples_dumped": 128 + }, + { + "update": 11, + "step_s": 168.37801449210383, + "train_s": 151.92190787987784, + "wait_s": 7.354397712042555, + "train_tok_s_gpu": 883.5164188836405, + "mfu": 0.02209716322198431, + "prompt_p50": 155.0, + "response_p50": 6311.5, + "rollout_e2e_p50_s": 50.0, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 359.8677317669981, + "teacher_p50_s": 0.1979381443298969, + "teacher_tok_s_gpu": 1132.0254775309827, + "window_s": 248.46304750442505, + "samples_dumped": 128 + }, + { + "update": 12, + "step_s": 244.47706853295676, + "train_s": 152.43831482483074, + "wait_s": 82.56779344310053, + "train_tok_s_gpu": 893.7221600524294, + "mfu": 0.022514823414706762, + "prompt_p50": 156.0, + "response_p50": 5524.0, + "rollout_e2e_p50_s": 43.333333333333336, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 370.9860139390339, + "teacher_p50_s": 0.19999999999999998, + "teacher_tok_s_gpu": 1112.367338813155, + "window_s": 244.6237771511078, + "samples_dumped": 128 + }, + { + "update": 13, + "step_s": 249.81763369520195, + "train_s": 151.27349689090624, + "wait_s": 89.19193400908262, + "train_tok_s_gpu": 885.0583727601296, + "mfu": 0.022374732057926013, + "prompt_p50": 157.5, + "response_p50": 4804.0, + "rollout_e2e_p50_s": 37.77777777777778, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 356.72713132341113, + "teacher_p50_s": 0.2064516129032258, + "teacher_tok_s_gpu": 1108.8579864519465, + "window_s": 249.9961702823639, + "samples_dumped": 128 + }, + { + "update": 14, + "step_s": 249.78231953992508, + "train_s": 149.95460572186857, + "wait_s": 91.0141137628816, + "train_tok_s_gpu": 925.4017196202916, + "mfu": 0.02351083333310898, + "prompt_p50": 155.0, + "response_p50": 5126.0, + "rollout_e2e_p50_s": 40.0, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 369.8638961573033, + "teacher_p50_s": 0.2064516129032258, + "teacher_tok_s_gpu": 1150.3713641477327, + "window_s": 249.9284222126007, + "samples_dumped": 128 + }, + { + "update": 15, + "step_s": 238.0877565459814, + "train_s": 152.53192984987982, + "wait_s": 75.78164464305155, + "train_tok_s_gpu": 935.257458162373, + "mfu": 0.0240459958016593, + "prompt_p50": 156.0, + "response_p50": 4903.5, + "rollout_e2e_p50_s": 40.0, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 398.8513616944135, + "teacher_p50_s": 0.192, + "teacher_tok_s_gpu": 1104.79258264602, + "window_s": 238.26440739631653, + "samples_dumped": 128 + }, + { + "update": 16, + "step_s": 234.48026365903206, + "train_s": 149.89653449808247, + "wait_s": 74.72902016714215, + "train_tok_s_gpu": 870.4620853103787, + "mfu": 0.02196781718863322, + "prompt_p50": 156.0, + "response_p50": 4923.5, + "rollout_e2e_p50_s": 39.5, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 370.43391505319556, + "teacher_p50_s": 0.20425531914893616, + "teacher_tok_s_gpu": 1065.1678560452115, + "window_s": 234.6268699169159, + "samples_dumped": 128 + }, + { + "update": 17, + "step_s": 224.67240630905144, + "train_s": 137.11329072201625, + "wait_s": 78.19095233594999, + "train_tok_s_gpu": 913.3332322531139, + "mfu": 0.022800352074569298, + "prompt_p50": 155.5, + "response_p50": 5329.5, + "rollout_e2e_p50_s": 41.666666666666664, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 371.0033914065841, + "teacher_p50_s": 0.19999999999999998, + "teacher_tok_s_gpu": 1218.2342426022235, + "window_s": 224.8249888420105, + "samples_dumped": 128 + }, + { + "update": 18, + "step_s": 247.23017307603732, + "train_s": 150.11205311701633, + "wait_s": 88.01648792997003, + "train_tok_s_gpu": 932.1594908233126, + "mfu": 0.02377385759194607, + "prompt_p50": 157.0, + "response_p50": 5019.0, + "rollout_e2e_p50_s": 40.0, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 376.77709198981273, + "teacher_p50_s": 0.19999999999999998, + "teacher_tok_s_gpu": 1099.7968859575503, + "window_s": 247.38499760627747, + "samples_dumped": 128 + }, + { + "update": 19, + "step_s": 223.80606261617504, + "train_s": 150.95846942905337, + "wait_s": 64.01386622595601, + "train_tok_s_gpu": 856.3232688362228, + "mfu": 0.021661938880550076, + "prompt_p50": 156.0, + "response_p50": 4939.5, + "rollout_e2e_p50_s": 37.89473684210526, + "rollout_requests_finished": 128.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 384.1045002389141, + "teacher_p50_s": 0.192, + "teacher_tok_s_gpu": 1059.3114052153976, + "window_s": 223.94925498962402, + "samples_dumped": 128 + }, + { + "update": 20, + "step_s": null, + "train_s": null, + "wait_s": null, + "train_tok_s_gpu": null, + "mfu": null, + "prompt_p50": 154.0, + "response_p50": 7852.0, + "rollout_e2e_p50_s": 43.75, + "rollout_requests_finished": 96.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 313.8257994252811, + "teacher_p50_s": 0.20281690140845068, + "teacher_tok_s_gpu": 556.7190240792177, + "window_s": 364.88289642333984, + "samples_dumped": 128 + } +] diff --git a/docs/mopd-h100-2026-10-07/data/schema-verl-547.json b/docs/mopd-h100-2026-10-07/data/schema-verl-547.json new file mode 100644 index 0000000..62fbd5a --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/schema-verl-547.json @@ -0,0 +1,382 @@ +[ + { + "update": 1, + "step_s": 157.46815395797603, + "train_s": 72.13880910514854, + "wait_s": 75.98429391812533, + "train_tok_s_gpu": 748.9900466924092, + "mfu": 0.016801177433647334, + "prompt_p50": 160.5, + "response_p50": 3036.5, + "rollout_e2e_p50_s": 52.35294117647059, + "rollout_requests_finished": 256.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 34.08364163201794, + "teacher_p50_s": 0.17539682539682538, + "teacher_tok_s_gpu": 316.9402332785843, + "window_s": 1054.8999934196472, + "samples_dumped": 128 + }, + { + "update": 2, + "step_s": 151.5362989641726, + "train_s": 108.55033885897137, + "wait_s": 33.06113710091449, + "train_tok_s_gpu": 1330.6637410654396, + "mfu": 0.03235373365015096, + "prompt_p50": 150.5, + "response_p50": 8665.0, + "rollout_e2e_p50_s": 63.75, + "rollout_requests_finished": 163.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 634.7452225981089, + "teacher_p50_s": 0.2664705882352941, + "teacher_tok_s_gpu": 2784.220039337326, + "window_s": 151.58706712722778, + "samples_dumped": 128 + }, + { + "update": 3, + "step_s": 169.67962916800752, + "train_s": 142.60930972499773, + "wait_s": 17.802019564900547, + "train_tok_s_gpu": 1315.287552837238, + "mfu": 0.03382639595729409, + "prompt_p50": 149.0, + "response_p50": 9112.5, + "rollout_e2e_p50_s": 44.54545454545455, + "rollout_requests_finished": 140.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 735.855433535648, + "teacher_p50_s": 0.21185567010309278, + "teacher_tok_s_gpu": 1824.8526424120598, + "window_s": 169.8319320678711, + "samples_dumped": 128 + }, + { + "update": 4, + "step_s": 127.13413568004034, + "train_s": 111.57209883397445, + "wait_s": 5.745203615864739, + "train_tok_s_gpu": 1320.7647928115136, + "mfu": 0.03331142508895905, + "prompt_p50": 157.0, + "response_p50": 5798.0, + "rollout_e2e_p50_s": 45.666666666666664, + "rollout_requests_finished": 131.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 771.2263225196939, + "teacher_p50_s": 0.17227722772277226, + "teacher_tok_s_gpu": 1522.7422898417094, + "window_s": 127.28926706314087, + "samples_dumped": 128 + }, + { + "update": 5, + "step_s": 139.10201638704166, + "train_s": 111.69663062901236, + "wait_s": 17.81016233493574, + "train_tok_s_gpu": 948.417149232111, + "mfu": 0.022989644709137412, + "prompt_p50": 160.0, + "response_p50": 4978.0, + "rollout_e2e_p50_s": 53.18181818181818, + "rollout_requests_finished": 143.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 506.12360151569436, + "teacher_p50_s": 0.20833333333333334, + "teacher_tok_s_gpu": 2023.6419486031202, + "window_s": 139.40089559555054, + "samples_dumped": 128 + }, + { + "update": 6, + "step_s": 113.09543127985671, + "train_s": 89.40752077591605, + "wait_s": 14.375854168087244, + "train_tok_s_gpu": 1553.6892063941334, + "mfu": 0.0389973262949178, + "prompt_p50": 158.5, + "response_p50": 5833.0, + "rollout_e2e_p50_s": 39.04761904761905, + "rollout_requests_finished": 146.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 817.0495585970999, + "teacher_p50_s": 0.19142857142857142, + "teacher_tok_s_gpu": 2281.701474899574, + "window_s": 113.24619054794312, + "samples_dumped": 128 + }, + { + "update": 7, + "step_s": 94.78068739897572, + "train_s": 75.11372813419439, + "wait_s": 8.124031006125733, + "train_tok_s_gpu": 1640.4191758378036, + "mfu": 0.04112548315290691, + "prompt_p50": 156.5, + "response_p50": 4832.5, + "rollout_e2e_p50_s": 41.111111111111114, + "rollout_requests_finished": 140.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 864.222207689068, + "teacher_p50_s": 0.18375, + "teacher_tok_s_gpu": 1460.7409483897513, + "window_s": 94.95301008224487, + "samples_dumped": 128 + }, + { + "update": 8, + "step_s": 85.91620063199662, + "train_s": 54.46955477213487, + "wait_s": 22.23660610197112, + "train_tok_s_gpu": 1561.8954543671507, + "mfu": 0.038518620818231555, + "prompt_p50": 157.0, + "response_p50": 2741.5, + "rollout_e2e_p50_s": 47.142857142857146, + "rollout_requests_finished": 170.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 658.1617061201373, + "teacher_p50_s": 0.2, + "teacher_tok_s_gpu": 2189.1019925602022, + "window_s": 86.06965351104736, + "samples_dumped": 128 + }, + { + "update": 9, + "step_s": 105.75324994302355, + "train_s": 69.48699316591956, + "wait_s": 27.22823149501346, + "train_tok_s_gpu": 1510.1747135529736, + "mfu": 0.037286734527731645, + "prompt_p50": 157.0, + "response_p50": 3990.5, + "rollout_e2e_p50_s": 42.857142857142854, + "rollout_requests_finished": 186.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 659.892547663285, + "teacher_p50_s": 0.19513274336283187, + "teacher_tok_s_gpu": 2669.183432018467, + "window_s": 105.90701508522034, + "samples_dumped": 128 + }, + { + "update": 10, + "step_s": 118.2596399821341, + "train_s": 90.83887093584053, + "wait_s": 17.25522934203036, + "train_tok_s_gpu": 1517.6501378729336, + "mfu": 0.038940040341613066, + "prompt_p50": 155.0, + "response_p50": 5164.5, + "rollout_e2e_p50_s": 45.0, + "rollout_requests_finished": 167.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 775.5052956176419, + "teacher_p50_s": 0.21562499999999998, + "teacher_tok_s_gpu": 2694.5311519225907, + "window_s": 118.42375612258911, + "samples_dumped": 128 + }, + { + "update": 11, + "step_s": 100.26386936800554, + "train_s": 89.72285618004389, + "wait_s": 1.1828010438475758, + "train_tok_s_gpu": 1677.7441825740857, + "mfu": 0.043186395559632364, + "prompt_p50": 156.0, + "response_p50": 6330.5, + "rollout_e2e_p50_s": 43.94736842105263, + "rollout_requests_finished": 157.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 553.338979958742, + "teacher_p50_s": 0.25, + "teacher_tok_s_gpu": 2179.1031259771657, + "window_s": 181.2333915233612, + "samples_dumped": 128 + }, + { + "update": 12, + "step_s": 123.90051033883356, + "train_s": 110.96198445605114, + "wait_s": 3.011363113997504, + "train_tok_s_gpu": 1644.4528357572033, + "mfu": 0.042405367865437464, + "prompt_p50": 152.5, + "response_p50": 8554.0, + "rollout_e2e_p50_s": 36.875, + "rollout_requests_finished": 130.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 979.7733025558224, + "teacher_p50_s": 0.17500000000000002, + "teacher_tok_s_gpu": 1276.4625554880201, + "window_s": 124.08746290206909, + "samples_dumped": 128 + }, + { + "update": 13, + "step_s": 77.40729019814171, + "train_s": 53.547592700924724, + "wait_s": 14.265767524018884, + "train_tok_s_gpu": 1607.4135298805616, + "mfu": 0.03896775040085402, + "prompt_p50": 159.0, + "response_p50": 3593.0, + "rollout_e2e_p50_s": 56.15384615384615, + "rollout_requests_finished": 146.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 738.9148839402857, + "teacher_p50_s": 0.1782608695652174, + "teacher_tok_s_gpu": 1499.5344034947352, + "window_s": 77.56407570838928, + "samples_dumped": 128 + }, + { + "update": 14, + "step_s": 140.6351224980317, + "train_s": 91.93189708678983, + "wait_s": 39.18968259706162, + "train_tok_s_gpu": 1044.9447149916793, + "mfu": 0.02603640711148205, + "prompt_p50": 159.5, + "response_p50": 3440.5, + "rollout_e2e_p50_s": 53.07692307692308, + "rollout_requests_finished": 192.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 454.42476515991683, + "teacher_p50_s": 0.23883495145631065, + "teacher_tok_s_gpu": 2771.408234843267, + "window_s": 140.77797889709473, + "samples_dumped": 128 + }, + { + "update": 15, + "step_s": 135.09517380595207, + "train_s": 107.93955398793332, + "wait_s": 17.158166541950777, + "train_tok_s_gpu": 1531.4902544296226, + "mfu": 0.03890205016235665, + "prompt_p50": 153.5, + "response_p50": 8728.5, + "rollout_e2e_p50_s": 50.476190476190474, + "rollout_requests_finished": 156.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 814.2626454166467, + "teacher_p50_s": 0.23138297872340424, + "teacher_tok_s_gpu": 2692.1702694555297, + "window_s": 135.25416803359985, + "samples_dumped": 128 + }, + { + "update": 16, + "step_s": 127.55550135998055, + "train_s": 109.17334860702977, + "wait_s": 9.286360906204209, + "train_tok_s_gpu": 1604.651018176424, + "mfu": 0.04188665311845552, + "prompt_p50": 154.5, + "response_p50": 6718.0, + "rollout_e2e_p50_s": 39.375, + "rollout_requests_finished": 139.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 913.8302755084213, + "teacher_p50_s": 0.20380434782608695, + "teacher_tok_s_gpu": 1738.0181736896948, + "window_s": 127.72047114372253, + "samples_dumped": 128 + }, + { + "update": 17, + "step_s": 91.23192371800542, + "train_s": 72.38457321701571, + "wait_s": 9.321144385030493, + "train_tok_s_gpu": 1537.838037205317, + "mfu": 0.03782635952844761, + "prompt_p50": 156.0, + "response_p50": 4517.5, + "rollout_e2e_p50_s": 42.142857142857146, + "rollout_requests_finished": 142.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 810.802183991647, + "teacher_p50_s": 0.1736842105263158, + "teacher_tok_s_gpu": 1968.706035577212, + "window_s": 91.43886733055115, + "samples_dumped": 128 + }, + { + "update": 18, + "step_s": 102.06261775107123, + "train_s": 70.66702642501332, + "wait_s": 21.698727420996875, + "train_tok_s_gpu": 1428.0847108670596, + "mfu": 0.03539399104907031, + "prompt_p50": 158.5, + "response_p50": 4142.0, + "rollout_e2e_p50_s": 50.0, + "rollout_requests_finished": 160.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 657.4535856168734, + "teacher_p50_s": 0.18267326732673267, + "teacher_tok_s_gpu": 2108.2769362746913, + "window_s": 102.22660803794861, + "samples_dumped": 128 + }, + { + "update": 19, + "step_s": 122.52054440090433, + "train_s": 86.24118382902816, + "wait_s": 27.003234497038648, + "train_tok_s_gpu": 1478.1945740996512, + "mfu": 0.03643039905370572, + "prompt_p50": 156.0, + "response_p50": 5535.5, + "rollout_e2e_p50_s": 48.05555555555556, + "rollout_requests_finished": 165.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 692.1476739163182, + "teacher_p50_s": 0.2154255319148936, + "teacher_tok_s_gpu": 2416.456172605376, + "window_s": 122.68275475502014, + "samples_dumped": 128 + }, + { + "update": 20, + "step_s": null, + "train_s": null, + "wait_s": null, + "train_tok_s_gpu": null, + "mfu": null, + "prompt_p50": 154.5, + "response_p50": 9372.5, + "rollout_e2e_p50_s": 27.5, + "rollout_requests_finished": 30.0, + "stale_filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 609.2007304316872, + "teacher_p50_s": 0.5857142857142857, + "teacher_tok_s_gpu": 823.7004001683271, + "window_s": 216.9059283733368, + "samples_dumped": 128 + } +] diff --git a/docs/mopd-h100-2026-10-07/data/teacher-caesar/summary.json b/docs/mopd-h100-2026-10-07/data/teacher-caesar/summary.json new file mode 100644 index 0000000..7bb571e --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/teacher-caesar/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/assets/teachers/caesar_cipher", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 910.2614703178406, + "output_tokens_per_second": 19106.435422260813, + "domains": { + "caesar_cipher": { + "avg@8": 0.7033730158730159, + "pass@8": 0.9920634920634921, + "finished_thinking": 0.941468253968254, + "truncated": 0.060515873015873016, + "response_tokens_mean": 13412.673611111111, + "response_tokens_median": 11461.0 + }, + "simple_geometry": { + "avg@8": 0.58984375, + "pass@8": 1.0, + "finished_thinking": 0.9990234375, + "truncated": 0.0009765625, + "response_tokens_mean": 3781.1298828125, + "response_tokens_median": 3599.5 + } + } +} diff --git a/docs/mopd-h100-2026-10-07/data/teacher-geometry/summary.json b/docs/mopd-h100-2026-10-07/data/teacher-geometry/summary.json new file mode 100644 index 0000000..3a28dc1 --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/teacher-geometry/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/assets/teachers/simple_geometry", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 868.775443315506, + "output_tokens_per_second": 17547.806072671832, + "domains": { + "caesar_cipher": { + "avg@8": 0.28174603174603174, + "pass@8": 0.6746031746031746, + "finished_thinking": 0.9553571428571429, + "truncated": 0.047619047619047616, + "response_tokens_mean": 12172.739087301587, + "response_tokens_median": 10068.5 + }, + "simple_geometry": { + "avg@8": 0.99609375, + "pass@8": 1.0, + "finished_thinking": 0.998046875, + "truncated": 0.0009765625, + "response_tokens_mean": 2905.255859375, + "response_tokens_median": 2517.0 + } + } +} diff --git a/docs/mopd-h100-2026-10-07/data/telemetry.json b/docs/mopd-h100-2026-10-07/data/telemetry.json new file mode 100644 index 0000000..350c273 --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/telemetry.json @@ -0,0 +1,1120 @@ +[ + { + "run": "miles-533", + "job_id": "533", + "nodes": [ + "gds-slurm-206-111", + "h100-206-089", + "h100-206-165" + ], + "exit_code": 1, + "wall_seconds": 5768.3904757499695, + "updates": { + "updates": 14, + "per_update": [ + { + "step_s": 363.77214217185974, + "train_s": 229.9952690601349, + "wait_for_rollouts_s": 133.77687311172485, + "weight_sync_s": 9.372408151626587, + "mfu": 0.00563303341696954, + "response_tokens_mean": 4479.96875, + "reverse_kl": 0.0036524576134979725, + "train_score": 0.4375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.328125, + "simple_geometry": 0.546875 + } + }, + { + "step_s": 358.09169602394104, + "train_s": 350.43780612945557, + "wait_for_rollouts_s": 7.653889894485474, + "weight_sync_s": 7.23081111907959, + "mfu": 0.006210770851780762, + "response_tokens_mean": 6885.28125, + "reverse_kl": 0.002847725758329034, + "train_score": 0.390625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.234375, + "simple_geometry": 0.546875 + } + }, + { + "step_s": 358.0572497844696, + "train_s": 349.4730079174042, + "wait_for_rollouts_s": 8.58424186706543, + "weight_sync_s": 8.37165379524231, + "mfu": 0.006210966166977759, + "response_tokens_mean": 6517.90625, + "reverse_kl": 0.0034232502803206444, + "train_score": 0.4140625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.265625, + "simple_geometry": 0.5625 + } + }, + { + "step_s": 327.42523527145386, + "train_s": 322.77324867248535, + "wait_for_rollouts_s": 4.651986598968506, + "weight_sync_s": 4.281806468963623, + "mfu": 0.007554339217906671, + "response_tokens_mean": 7049.2890625, + "reverse_kl": 0.0022032265551388264, + "train_score": 0.484375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.234375, + "simple_geometry": 0.734375 + } + }, + { + "step_s": 155.88325595855713, + "train_s": 148.06387305259705, + "wait_for_rollouts_s": 7.819382905960083, + "weight_sync_s": 7.609277248382568, + "mfu": 0.010002288687283057, + "response_tokens_mean": 5091.7265625, + "reverse_kl": 0.0022386193741112947, + "train_score": 0.546875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.34375, + "simple_geometry": 0.75 + } + }, + { + "step_s": 308.24736285209656, + "train_s": 299.9345610141754, + "wait_for_rollouts_s": 8.312801837921143, + "weight_sync_s": 7.90252423286438, + "mfu": 0.010380828621354003, + "response_tokens_mean": 8571.171875, + "reverse_kl": 0.0014319850597530603, + "train_score": 0.6328125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.328125, + "simple_geometry": 0.9375 + } + }, + { + "step_s": 144.84795308113098, + "train_s": 136.9878420829773, + "wait_for_rollouts_s": 7.8601109981536865, + "weight_sync_s": 7.45422625541687, + "mfu": 0.009698858468299567, + "response_tokens_mean": 4555.984375, + "reverse_kl": 0.002159214112907648, + "train_score": 0.671875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.390625, + "simple_geometry": 0.953125 + } + }, + { + "step_s": 264.3724510669708, + "train_s": 255.57333636283875, + "wait_for_rollouts_s": 8.79911470413208, + "weight_sync_s": 8.427326202392578, + "mfu": 0.011673663354351741, + "response_tokens_mean": 7795.125, + "reverse_kl": 0.0022292290814220905, + "train_score": 0.78125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.5625, + "simple_geometry": 1.0 + } + }, + { + "step_s": 147.46754503250122, + "train_s": 139.45058488845825, + "wait_for_rollouts_s": 8.016960144042969, + "weight_sync_s": 7.6564624309539795, + "mfu": 0.0089899094601287, + "response_tokens_mean": 4289.9921875, + "reverse_kl": 0.002379930578172207, + "train_score": 0.859375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.734375, + "simple_geometry": 0.984375 + } + }, + { + "step_s": 217.52165126800537, + "train_s": 208.64208102226257, + "wait_for_rollouts_s": 8.879570245742798, + "weight_sync_s": 8.517735481262207, + "mfu": 0.012002494379091897, + "response_tokens_mean": 7126.90625, + "reverse_kl": 0.001414244994521141, + "train_score": 0.8203125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.640625, + "simple_geometry": 1.0 + } + }, + { + "step_s": 636.8193981647491, + "train_s": 387.1045925617218, + "wait_for_rollouts_s": 249.71480560302734, + "weight_sync_s": 4.79842734336853, + "mfu": 0.004081118707588432, + "response_tokens_mean": 5149.5078125, + "reverse_kl": 0.0017631642986088991, + "train_score": 0.84375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.6875, + "simple_geometry": 1.0 + } + }, + { + "step_s": 375.6121623516083, + "train_s": 365.95961713790894, + "wait_for_rollouts_s": 9.65254521369934, + "weight_sync_s": 9.432941913604736, + "mfu": 0.007888404875809985, + "response_tokens_mean": 7811.3203125, + "reverse_kl": 0.0012303978437557817, + "train_score": 0.90625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.828125, + "simple_geometry": 0.984375 + } + }, + { + "step_s": 415.45073890686035, + "train_s": 410.3080289363861, + "wait_for_rollouts_s": 5.142709970474243, + "weight_sync_s": 4.914021730422974, + "mfu": 0.007041074568805957, + "response_tokens_mean": 7782.9453125, + "reverse_kl": 0.001365316566079855, + "train_score": 0.84375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.6875, + "simple_geometry": 1.0 + } + }, + { + "step_s": 422.1308362483978, + "train_s": 411.89175748825073, + "wait_for_rollouts_s": 10.239078760147095, + "weight_sync_s": 9.643274068832397, + "mfu": 0.00503460508059564, + "response_tokens_mean": 6096.6484375, + "reverse_kl": 0.0013460277114063501, + "train_score": 0.8828125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.765625, + "simple_geometry": 1.0 + } + } + ], + "step_s_median": 342.74124252796173, + "step_s_total": 4495.699678182602, + "train_s_median": 311.3539048433304, + "train_s_total": 4016.595606327057, + "wait_for_rollouts_s_median": 8.448521852493286, + "wait_for_rollouts_s_total": 479.10407185554504, + "weight_sync_s_median": 7.77949333190918, + "weight_sync_s_total": 105.61289644241333, + "mfu_mean": 0.008028739704067407, + "response_tokens_mean_mean": 6371.698102678572, + "reverse_kl_first": 0.0036524576134979725, + "reverse_kl_last": 0.0013460277114063501, + "train_score_first": 0.4375, + "train_score_last": 0.8828125 + }, + "roles": { + "generation-2": { + "samples": 1142, + "seconds": 5705.88445353508, + "gpus": { + "0": { + "utilization_mean_pct": 2.8056042031523645, + "busy_fraction": 0.07880910683012259, + "memory_peak_gib": 73.63671875, + "power_mean_w": 134.15580560420315, + "energy_kwh": 0.21265254428575966 + }, + "1": { + "utilization_mean_pct": 2.765323992994746, + "busy_fraction": 0.08318739054290718, + "memory_peak_gib": 73.01171875, + "power_mean_w": 132.19023642732049, + "energy_kwh": 0.20953535675685073 + }, + "2": { + "utilization_mean_pct": 49.556042031523646, + "busy_fraction": 0.5227670753064798, + "memory_peak_gib": 73.490234375, + "power_mean_w": 259.75363397548165, + "energy_kwh": 0.4118976893853446 + }, + "3": { + "utilization_mean_pct": 49.52101576182137, + "busy_fraction": 0.5192644483362522, + "memory_peak_gib": 73.4921875, + "power_mean_w": 266.80101576182136, + "energy_kwh": 0.42307662349474034 + }, + "4": { + "utilization_mean_pct": 73.82136602451838, + "busy_fraction": 0.7644483362521891, + "memory_peak_gib": 73.474609375, + "power_mean_w": 417.88876532399297, + "energy_kwh": 0.6627556986800511 + }, + "5": { + "utilization_mean_pct": 73.72416812609457, + "busy_fraction": 0.7591943957968477, + "memory_peak_gib": 73.47265625, + "power_mean_w": 379.9784325744308, + "energy_kwh": 0.6026163216035505 + }, + "6": { + "utilization_mean_pct": 52.076182136602455, + "busy_fraction": 0.5464098073555166, + "memory_peak_gib": 73.4140625, + "power_mean_w": 264.5113747810858, + "energy_kwh": 0.4194462027881483 + }, + "7": { + "utilization_mean_pct": 51.998248686514884, + "busy_fraction": 0.542031523642732, + "memory_peak_gib": 73.4140625, + "power_mean_w": 256.61971978984235, + "energy_kwh": 0.40692597550433873 + } + }, + "gpu_energy_kwh": 3.348906412498784, + "cpu_utilization_pct": 6.820983571965044, + "network": { + "rx_gb": 0.661501107, + "tx_gb": 2.004902255, + "rx_gbit_s": 0.0009274651281662979, + "tx_gbit_s": 0.002810995941227465 + }, + "engines": { + "127.0.0.1:30001": { + "seconds": 5530.859028577805, + "generation_tokens": 8.0, + "generation_tokens_per_s": 0.001446429923211604, + "prompt_tokens": 6807812.0, + "prompt_tokens_per_s": 1230.8778735498795 + } + } + }, + "generation": { + "samples": 1142, + "seconds": 5705.839947938919, + "gpus": { + "0": { + "utilization_mean_pct": 8.1077057793345, + "busy_fraction": 0.1234676007005254, + "memory_peak_gib": 73.20703125, + "power_mean_w": 167.58359019264446, + "energy_kwh": 0.2656806372700075 + }, + "1": { + "utilization_mean_pct": 8.247810858143607, + "busy_fraction": 0.1234676007005254, + "memory_peak_gib": 72.58203125, + "power_mean_w": 164.10494746059544, + "energy_kwh": 0.2601639557024644 + }, + "2": { + "utilization_mean_pct": 53.47548161120841, + "busy_fraction": 0.5507880910683012, + "memory_peak_gib": 73.462890625, + "power_mean_w": 265.24739054290717, + "energy_kwh": 0.420613208295138 + }, + "3": { + "utilization_mean_pct": 53.52889667250438, + "busy_fraction": 0.5542907180385289, + "memory_peak_gib": 73.470703125, + "power_mean_w": 270.37746935201403, + "energy_kwh": 0.42874384731286763 + }, + "4": { + "utilization_mean_pct": 49.6970227670753, + "busy_fraction": 0.5122591943957968, + "memory_peak_gib": 73.416015625, + "power_mean_w": 265.5361646234676, + "energy_kwh": 0.42106636499279737 + }, + "5": { + "utilization_mean_pct": 49.73204903677758, + "busy_fraction": 0.5157618213660246, + "memory_peak_gib": 73.416015625, + "power_mean_w": 253.82472854640983, + "energy_kwh": 0.40249094718680445 + }, + "6": { + "utilization_mean_pct": 52.166374781085814, + "busy_fraction": 0.5385288966725044, + "memory_peak_gib": 73.41796875, + "power_mean_w": 278.6957180385289, + "energy_kwh": 0.4419430473332802 + }, + "7": { + "utilization_mean_pct": 52.388791593695274, + "busy_fraction": 0.5437828371278459, + "memory_peak_gib": 73.416015625, + "power_mean_w": 258.29959719789844, + "energy_kwh": 0.40959039148959253 + } + }, + "gpu_energy_kwh": 3.0502923995829523, + "cpu_utilization_pct": 6.790310402482036, + "network": { + "rx_gb": 0.25424189, + "tx_gb": 1.810543287, + "rx_gbit_s": 0.0003564655052644273, + "tx_gbit_s": 0.0025385125464712833 + }, + "engines": { + "127.0.0.1:30000": { + "seconds": 5530.817793369293, + "generation_tokens": 8.0, + "generation_tokens_per_s": 0.0014464407071212732, + "prompt_tokens": 27136903.0, + "prompt_tokens_per_s": 4906.490145550175 + } + } + }, + "learner": { + "samples": 996, + "seconds": 5667.745438337326, + "gpus": { + "0": { + "utilization_mean_pct": 37.71714922048998, + "busy_fraction": 0.46547884187082406, + "memory_peak_gib": 79.1591796875, + "power_mean_w": 159.3336302895323, + "energy_kwh": 0.3796435225586183 + }, + "1": { + "utilization_mean_pct": 40.512249443207125, + "busy_fraction": 0.48552338530066813, + "memory_peak_gib": 79.0048828125, + "power_mean_w": 157.71741648106902, + "energy_kwh": 0.3680851965224538 + }, + "2": { + "utilization_mean_pct": 45.02004454342985, + "busy_fraction": 0.5278396436525612, + "memory_peak_gib": 79.1513671875, + "power_mean_w": 159.45318485523384, + "energy_kwh": 0.3587665677907732 + }, + "3": { + "utilization_mean_pct": 43.26726057906459, + "busy_fraction": 0.5144766146993318, + "memory_peak_gib": 79.0712890625, + "power_mean_w": 155.94886414253895, + "energy_kwh": 0.34363129522334274 + }, + "4": { + "utilization_mean_pct": 48.74387527839644, + "busy_fraction": 0.5545657015590201, + "memory_peak_gib": 79.0927734375, + "power_mean_w": 155.219710467706, + "energy_kwh": 0.3270884253704309 + }, + "5": { + "utilization_mean_pct": 47.933184855233854, + "busy_fraction": 0.5545657015590201, + "memory_peak_gib": 79.1513671875, + "power_mean_w": 152.01447661469933, + "energy_kwh": 0.3118212352342288 + }, + "6": { + "utilization_mean_pct": 45.83518930957684, + "busy_fraction": 0.534521158129176, + "memory_peak_gib": 79.1728515625, + "power_mean_w": 157.67149220489978, + "energy_kwh": 0.32161000424948927 + }, + "7": { + "utilization_mean_pct": 44.202672605790646, + "busy_fraction": 0.5167037861915368, + "memory_peak_gib": 79.1767578125, + "power_mean_w": 152.09420935412027, + "energy_kwh": 0.3068564256740729 + } + }, + "gpu_energy_kwh": 2.71750267262341, + "cpu_utilization_pct": 9.442022124505922, + "network": { + "rx_gb": 3.326662657, + "tx_gb": 0.425704674, + "rx_gbit_s": 0.004695571024764867, + "tx_gbit_s": 0.000600880443388274 + }, + "engines": { + "10.0.11.168:20000": { + "seconds": 5217.678159713745, + "generation_tokens": 3980418.0, + "generation_tokens_per_s": 762.8715068578273, + "prompt_tokens": 93391.0, + "prompt_tokens_per_s": 17.89895757102881 + }, + "10.0.11.168:20037": { + "seconds": 5217.678159713745, + "generation_tokens": 13388806.0, + "generation_tokens_per_s": 2566.0467338473295, + "prompt_tokens": 218176.0, + "prompt_tokens_per_s": 41.814767665158115 + }, + "10.0.11.168:20072": { + "seconds": 5217.678159713745, + "generation_tokens": 3855509.0, + "generation_tokens_per_s": 738.9319314036653, + "prompt_tokens": 75316.0, + "prompt_tokens_per_s": 14.434773034014047 + }, + "10.0.11.48:20000": { + "seconds": 5192.675471544266, + "generation_tokens": 4011699.0, + "generation_tokens_per_s": 772.5687888611587, + "prompt_tokens": 65683.0, + "prompt_tokens_per_s": 12.649163299332152 + }, + "10.0.11.48:20001": { + "seconds": 5192.675471544266, + "generation_tokens": 4064826.0, + "generation_tokens_per_s": 782.7999308401126, + "prompt_tokens": 95881.0, + "prompt_tokens_per_s": 18.464662489582786 + }, + "10.0.11.48:20002": { + "seconds": 5192.675471544266, + "generation_tokens": 4005537.0, + "generation_tokens_per_s": 771.382117359393, + "prompt_tokens": 89313.0, + "prompt_tokens_per_s": 17.199803933335147 + } + } + } + } + }, + { + "run": "miles-535", + "job_id": "535", + "nodes": [ + "gds-slurm-206-111", + "h100-206-089", + "h100-206-165" + ], + "exit_code": 0, + "wall_seconds": 4184.1528124809265, + "updates": { + "updates": 20, + "per_update": [ + { + "step_s": 454.4655759334564, + "train_s": 354.9641637802124, + "wait_for_rollouts_s": 99.50141215324402, + "weight_sync_s": 5.944491147994995, + "mfu": 0.003005292527842236, + "response_tokens_mean": 3904.2734375, + "reverse_kl": 0.0038383365608751774, + "train_score": 0.3828125, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.28125, + "simple_geometry": 0.484375 + } + }, + { + "step_s": 140.65005588531494, + "train_s": 136.06386542320251, + "wait_for_rollouts_s": 4.586190462112427, + "weight_sync_s": 4.188059091567993, + "mfu": 0.01368823748481477, + "response_tokens_mean": 6130.1640625, + "reverse_kl": 0.003247650107368827, + "train_score": 0.3984375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.28125, + "simple_geometry": 0.515625 + } + }, + { + "step_s": 51.53161859512329, + "train_s": 46.886722564697266, + "wait_for_rollouts_s": 4.644896030426025, + "weight_sync_s": 4.201306104660034, + "mfu": 0.04844046488615055, + "response_tokens_mean": 6882.6171875, + "reverse_kl": 0.00294014741666615, + "train_score": 0.390625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.265625, + "simple_geometry": 0.515625 + } + }, + { + "step_s": 66.16081881523132, + "train_s": 60.217230796813965, + "wait_for_rollouts_s": 5.943588018417358, + "weight_sync_s": 4.377046823501587, + "mfu": 0.03052037553908493, + "response_tokens_mean": 5894.296875, + "reverse_kl": 0.0028153941966593266, + "train_score": 0.484375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.375, + "simple_geometry": 0.59375 + } + }, + { + "step_s": 139.77254629135132, + "train_s": 61.7414870262146, + "wait_for_rollouts_s": 78.03105926513672, + "weight_sync_s": 4.874325752258301, + "mfu": 0.03188748272896835, + "response_tokens_mean": 6129.6328125, + "reverse_kl": 0.0015889210626482964, + "train_score": 0.625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.421875, + "simple_geometry": 0.828125 + } + }, + { + "step_s": 253.63281345367432, + "train_s": 179.91235637664795, + "wait_for_rollouts_s": 73.72045707702637, + "weight_sync_s": 7.241090297698975, + "mfu": 0.013234671170088936, + "response_tokens_mean": 7167.1953125, + "reverse_kl": 0.0011802857043221593, + "train_score": 0.6875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.484375, + "simple_geometry": 0.890625 + } + }, + { + "step_s": 50.266199350357056, + "train_s": 45.01760530471802, + "wait_for_rollouts_s": 5.248594045639038, + "weight_sync_s": 4.564242362976074, + "mfu": 0.04556571951149853, + "response_tokens_mean": 6375.5, + "reverse_kl": 0.0012697380734607577, + "train_score": 0.7421875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.59375, + "simple_geometry": 0.890625 + } + }, + { + "step_s": 107.8759081363678, + "train_s": 96.62245583534241, + "wait_for_rollouts_s": 11.25345230102539, + "weight_sync_s": 4.332585096359253, + "mfu": 0.01663322525421774, + "response_tokens_mean": 5235.3125, + "reverse_kl": 0.0017686031060293317, + "train_score": 0.875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.75, + "simple_geometry": 1.0 + } + }, + { + "step_s": 210.94867873191833, + "train_s": 111.44216990470886, + "wait_for_rollouts_s": 99.50650882720947, + "weight_sync_s": 4.401268720626831, + "mfu": 0.0169579677177502, + "response_tokens_mean": 5806.4453125, + "reverse_kl": 0.0017758234171196818, + "train_score": 0.8671875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.734375, + "simple_geometry": 1.0 + } + }, + { + "step_s": 122.93500351905823, + "train_s": 116.31412029266357, + "wait_for_rollouts_s": 6.620883226394653, + "weight_sync_s": 4.2506420612335205, + "mfu": 0.021993258057476407, + "response_tokens_mean": 7407.5078125, + "reverse_kl": 0.0014976884704083204, + "train_score": 0.890625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.78125, + "simple_geometry": 1.0 + } + }, + { + "step_s": 283.0779414176941, + "train_s": 84.84011793136597, + "wait_for_rollouts_s": 198.23782348632812, + "weight_sync_s": 4.444629669189453, + "mfu": 0.027430190582695597, + "response_tokens_mean": 6922.1015625, + "reverse_kl": 0.0013543078675866127, + "train_score": 0.90625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.8125, + "simple_geometry": 1.0 + } + }, + { + "step_s": 87.0206401348114, + "train_s": 81.05648064613342, + "wait_for_rollouts_s": 5.9641594886779785, + "weight_sync_s": 5.021454572677612, + "mfu": 0.02577144271137155, + "response_tokens_mean": 6380.6484375, + "reverse_kl": 0.0013352511450648308, + "train_score": 0.921875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.84375, + "simple_geometry": 1.0 + } + }, + { + "step_s": 53.015042543411255, + "train_s": 42.68810558319092, + "wait_for_rollouts_s": 10.326936960220337, + "weight_sync_s": 9.907107591629028, + "mfu": 0.03717609736071514, + "response_tokens_mean": 5308.4140625, + "reverse_kl": 0.0012877112021669745, + "train_score": 0.9296875, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.859375, + "simple_geometry": 1.0 + } + }, + { + "step_s": 122.83777141571045, + "train_s": 52.6164345741272, + "wait_for_rollouts_s": 70.22133684158325, + "weight_sync_s": 10.200856685638428, + "mfu": 0.03467242470615417, + "response_tokens_mean": 5850.203125, + "reverse_kl": 0.001233145478181541, + "train_score": 0.9375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.875, + "simple_geometry": 1.0 + } + }, + { + "step_s": 200.40550541877747, + "train_s": 132.22712397575378, + "wait_for_rollouts_s": 68.17838144302368, + "weight_sync_s": 9.518917560577393, + "mfu": 0.014458679518432466, + "response_tokens_mean": 6016.1640625, + "reverse_kl": 0.0013089253334328532, + "train_score": 0.859375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.71875, + "simple_geometry": 1.0 + } + }, + { + "step_s": 134.29571843147278, + "train_s": 121.34201979637146, + "wait_for_rollouts_s": 12.953698635101318, + "weight_sync_s": 9.976858139038086, + "mfu": 0.013853295607187378, + "response_tokens_mean": 5483.5859375, + "reverse_kl": 0.0013256934471428394, + "train_score": 0.90625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.828125, + "simple_geometry": 0.984375 + } + }, + { + "step_s": 88.71644616127014, + "train_s": 82.4870092868805, + "wait_for_rollouts_s": 6.229436874389648, + "weight_sync_s": 5.8056559562683105, + "mfu": 0.02476782405492066, + "response_tokens_mean": 6343.5625, + "reverse_kl": 0.0012407551985234022, + "train_score": 0.9375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.875, + "simple_geometry": 1.0 + } + }, + { + "step_s": 192.92430472373962, + "train_s": 141.87546300888062, + "wait_for_rollouts_s": 51.04884171485901, + "weight_sync_s": 8.529404163360596, + "mfu": 0.013991258510328715, + "response_tokens_mean": 6181.609375, + "reverse_kl": 0.001122263609431684, + "train_score": 0.9375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.90625, + "simple_geometry": 0.96875 + } + }, + { + "step_s": 107.51968431472778, + "train_s": 96.6757264137268, + "wait_for_rollouts_s": 10.843957901000977, + "weight_sync_s": 10.305595397949219, + "mfu": 0.018179576543788763, + "response_tokens_mean": 5619.6328125, + "reverse_kl": 0.0012526132632046938, + "train_score": 0.90625, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.8125, + "simple_geometry": 1.0 + } + }, + { + "step_s": 56.36443829536438, + "train_s": 41.41533589363098, + "wait_for_rollouts_s": 14.949102401733398, + "weight_sync_s": 9.008400678634644, + "mfu": 0.04005950346789854, + "response_tokens_mean": 5391.5703125, + "reverse_kl": 0.0011565882014110684, + "train_score": 0.9609375, + "dropped_stale": null, + "train_score_by_domain": { + "caesar_cipher": 0.921875, + "simple_geometry": 1.0 + } + } + ], + "step_s_median": 122.88638746738434, + "step_s_total": 2924.4167115688324, + "train_s_median": 90.73128688335419, + "train_s_total": 2086.405994415283, + "wait_for_rollouts_s_median": 12.103575468063354, + "wait_for_rollouts_s_total": 838.0107171535492, + "weight_sync_s_median": 5.413555264472961, + "weight_sync_s_total": 131.09393787384033, + "mfu_mean": 0.02461434939706928, + "response_tokens_mean_mean": 6021.521875, + "reverse_kl_first": 0.0038383365608751774, + "reverse_kl_last": 0.0011565882014110684, + "train_score_first": 0.3828125, + "train_score_last": 0.9609375 + }, + "roles": { + "generation-2": { + "samples": 825, + "seconds": 4120.606352090836, + "gpus": { + "0": { + "utilization_mean_pct": 4.116363636363636, + "busy_fraction": 0.12, + "memory_peak_gib": 78.8984375, + "power_mean_w": 143.10112727272727, + "energy_kwh": 0.16382505751255816 + }, + "1": { + "utilization_mean_pct": 3.8266666666666667, + "busy_fraction": 0.11515151515151516, + "memory_peak_gib": 78.2734375, + "power_mean_w": 140.72172121212122, + "energy_kwh": 0.16110090991595055 + }, + "2": { + "utilization_mean_pct": 77.67515151515151, + "busy_fraction": 0.8048484848484848, + "memory_peak_gib": 73.494140625, + "power_mean_w": 364.091296969697, + "energy_kwh": 0.4170847564574487 + }, + "3": { + "utilization_mean_pct": 77.69212121212121, + "busy_fraction": 0.8084848484848485, + "memory_peak_gib": 73.501953125, + "power_mean_w": 382.1870787878788, + "energy_kwh": 0.4378217380912668 + }, + "4": { + "utilization_mean_pct": 74.39636363636363, + "busy_fraction": 0.7757575757575758, + "memory_peak_gib": 73.42578125, + "power_mean_w": 373.3276363636364, + "energy_kwh": 0.42766941003539294 + }, + "5": { + "utilization_mean_pct": 74.24484848484849, + "busy_fraction": 0.7757575757575758, + "memory_peak_gib": 73.421875, + "power_mean_w": 339.29579393939395, + "energy_kwh": 0.38866960886135565 + }, + "6": { + "utilization_mean_pct": 74.44848484848485, + "busy_fraction": 0.7745454545454545, + "memory_peak_gib": 73.421875, + "power_mean_w": 354.7247878787879, + "energy_kwh": 0.4063519643372641 + }, + "7": { + "utilization_mean_pct": 74.57818181818182, + "busy_fraction": 0.7757575757575758, + "memory_peak_gib": 73.41796875, + "power_mean_w": 340.3868363636364, + "energy_kwh": 0.38991959542090365 + } + }, + "gpu_energy_kwh": 2.7924430406321408, + "cpu_utilization_pct": 6.655772346130917, + "network": { + "rx_gb": 0.221411972, + "tx_gb": 1.195509333, + "rx_gbit_s": 0.0004298628950812609, + "tx_gbit_s": 0.002321035752213287 + }, + "engines": { + "127.0.0.1:30001": { + "seconds": 3950.584612607956, + "generation_tokens": 8.0, + "generation_tokens_per_s": 0.002025016746754057, + "prompt_tokens": 7482422.0, + "prompt_tokens_per_s": 1894.0037320351232 + } + } + }, + "generation": { + "samples": 824, + "seconds": 4115.703072547913, + "gpus": { + "0": { + "utilization_mean_pct": 12.103155339805825, + "busy_fraction": 0.1783980582524272, + "memory_peak_gib": 71.806640625, + "power_mean_w": 188.32485436893202, + "energy_kwh": 0.21539847299947076 + }, + "1": { + "utilization_mean_pct": 11.855582524271844, + "busy_fraction": 0.17354368932038836, + "memory_peak_gib": 71.181640625, + "power_mean_w": 184.55865291262134, + "energy_kwh": 0.2110916217723595 + }, + "2": { + "utilization_mean_pct": 76.91504854368932, + "busy_fraction": 0.7949029126213593, + "memory_peak_gib": 73.466796875, + "power_mean_w": 355.1497208737864, + "energy_kwh": 0.40635816453829016 + }, + "3": { + "utilization_mean_pct": 76.99878640776699, + "busy_fraction": 0.8009708737864077, + "memory_peak_gib": 73.474609375, + "power_mean_w": 361.4502305825243, + "energy_kwh": 0.4135636778844264 + }, + "4": { + "utilization_mean_pct": 82.55825242718447, + "busy_fraction": 0.8543689320388349, + "memory_peak_gib": 73.529296875, + "power_mean_w": 465.3507524271845, + "energy_kwh": 0.532488368350333 + }, + "5": { + "utilization_mean_pct": 82.51456310679612, + "busy_fraction": 0.8543689320388349, + "memory_peak_gib": 73.529296875, + "power_mean_w": 449.24247572815534, + "energy_kwh": 0.5140562494733221 + }, + "6": { + "utilization_mean_pct": 77.22936893203884, + "busy_fraction": 0.7973300970873787, + "memory_peak_gib": 73.42578125, + "power_mean_w": 387.0181067961165, + "energy_kwh": 0.44283053990727667 + }, + "7": { + "utilization_mean_pct": 77.06917475728156, + "busy_fraction": 0.8021844660194175, + "memory_peak_gib": 73.421875, + "power_mean_w": 350.8272451456311, + "energy_kwh": 0.4014093354239404 + } + }, + "gpu_energy_kwh": 3.137196430349419, + "cpu_utilization_pct": 6.985945925839876, + "network": { + "rx_gb": 0.314938471, + "tx_gb": 2.270026953, + "rx_gbit_s": 0.0006121694698544533, + "tx_gbit_s": 0.004412421232505856 + }, + "engines": { + "127.0.0.1:30000": { + "seconds": 3945.6801731586456, + "generation_tokens": 8.0, + "generation_tokens_per_s": 0.002027533821525058, + "prompt_tokens": 29610976.0, + "prompt_tokens_per_s": 7504.656916045846 + } + } + }, + "learner": { + "samples": 754, + "seconds": 4084.371591567993, + "gpus": { + "0": { + "utilization_mean_pct": 40.453007518796994, + "busy_fraction": 0.5075187969924813, + "memory_peak_gib": 73.5029296875, + "power_mean_w": 168.06665413533835, + "energy_kwh": 0.21029835639472738 + }, + "1": { + "utilization_mean_pct": 42.849624060150376, + "busy_fraction": 0.5, + "memory_peak_gib": 71.4228515625, + "power_mean_w": 164.20926691729323, + "energy_kwh": 0.2045791781640662 + }, + "2": { + "utilization_mean_pct": 38.524436090225564, + "busy_fraction": 0.4699248120300752, + "memory_peak_gib": 71.3857421875, + "power_mean_w": 166.73936090225564, + "energy_kwh": 0.2069508929164383 + }, + "3": { + "utilization_mean_pct": 38.45864661654135, + "busy_fraction": 0.47368421052631576, + "memory_peak_gib": 71.5439453125, + "power_mean_w": 164.04783834586465, + "energy_kwh": 0.20340767121909262 + }, + "4": { + "utilization_mean_pct": 39.78947368421053, + "busy_fraction": 0.4906015037593985, + "memory_peak_gib": 71.2666015625, + "power_mean_w": 172.1208082706767, + "energy_kwh": 0.21004283885988792 + }, + "5": { + "utilization_mean_pct": 40.2687969924812, + "busy_fraction": 0.49624060150375937, + "memory_peak_gib": 71.1083984375, + "power_mean_w": 167.48592105263157, + "energy_kwh": 0.20415836102967524 + }, + "6": { + "utilization_mean_pct": 25.351503759398497, + "busy_fraction": 0.34774436090225563, + "memory_peak_gib": 71.9716796875, + "power_mean_w": 172.96161654135338, + "energy_kwh": 0.2103850731110447 + }, + "7": { + "utilization_mean_pct": 28.25, + "busy_fraction": 0.3684210526315789, + "memory_peak_gib": 71.3017578125, + "power_mean_w": 167.34911654135337, + "energy_kwh": 0.20364075345412957 + } + }, + "gpu_energy_kwh": 1.653463125149062, + "cpu_utilization_pct": 9.85722184912844, + "network": { + "rx_gb": 3.46371745, + "tx_gb": 0.5334636, + "rx_gbit_s": 0.00678433364319876, + "tx_gbit_s": 0.0010448874947643107 + }, + "engines": { + "10.0.11.168:20000": { + "seconds": 3629.3050911426544, + "generation_tokens": 5511288.0, + "generation_tokens_per_s": 1518.5518609196947, + "prompt_tokens": 110696.0, + "prompt_tokens_per_s": 30.500604721866562 + }, + "10.0.11.168:20001": { + "seconds": 3629.3050911426544, + "generation_tokens": 5157567.0, + "generation_tokens_per_s": 1421.0894015460644, + "prompt_tokens": 104624.0, + "prompt_tokens_per_s": 28.82755716936987 + }, + "10.0.11.168:20033": { + "seconds": 3629.3050911426544, + "generation_tokens": 5159664.0, + "generation_tokens_per_s": 1421.6671981069317, + "prompt_tokens": 98628.0, + "prompt_tokens_per_s": 27.175450264763455 + }, + "10.0.11.48:20000": { + "seconds": 3614.3031525611877, + "generation_tokens": 13371453.0, + "generation_tokens_per_s": 3699.5936521054264, + "prompt_tokens": 225110.0, + "prompt_tokens_per_s": 62.28309870478941 + }, + "10.0.11.48:20032": { + "seconds": 3614.3031525611877, + "generation_tokens": 5361735.0, + "generation_tokens_per_s": 1483.4768345871976, + "prompt_tokens": 101844.0, + "prompt_tokens_per_s": 28.178045864202268 + }, + "10.0.11.48:20033": { + "seconds": 3614.3031525611877, + "generation_tokens": 5396994.0, + "generation_tokens_per_s": 1493.232242064574, + "prompt_tokens": 101527.0, + "prompt_tokens_per_s": 28.09033877749169 + } + } + } + } + } +] diff --git a/docs/mopd-h100-2026-10-07/data/verl-547-step10/summary.json b/docs/mopd-h100-2026-10-07/data/verl-547-step10/summary.json new file mode 100644 index 0000000..d954250 --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/verl-547-step10/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/runs/verl-mopd-h100x3/checkpoints/verl-547/global_step_10/actor/huggingface-merged", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 928.1228313446045, + "output_tokens_per_second": 18336.02560487096, + "domains": { + "caesar_cipher": { + "avg@8": 0.7410714285714286, + "pass@8": 1.0, + "finished_thinking": 0.9384920634920635, + "truncated": 0.06448412698412699, + "response_tokens_mean": 13956.647817460318, + "response_tokens_median": 12217.0 + }, + "simple_geometry": { + "avg@8": 0.998046875, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 2880.6474609375, + "response_tokens_median": 2095.5 + } + } +} diff --git a/docs/mopd-h100-2026-10-07/data/verl-547/summary.json b/docs/mopd-h100-2026-10-07/data/verl-547/summary.json new file mode 100644 index 0000000..ff5e799 --- /dev/null +++ b/docs/mopd-h100-2026-10-07/data/verl-547/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/runs/verl-mopd-h100x3/checkpoints/verl-547/global_step_20/actor/huggingface-merged", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 896.2310848236084, + "output_tokens_per_second": 19294.22031082903, + "domains": { + "caesar_cipher": { + "avg@8": 0.7648809523809523, + "pass@8": 0.9841269841269841, + "finished_thinking": 0.9454365079365079, + "truncated": 0.05853174603174603, + "response_tokens_mean": 13685.775793650793, + "response_tokens_median": 11833.5 + }, + "simple_geometry": { + "avg@8": 0.9970703125, + "pass@8": 1.0, + "finished_thinking": 1.0, + "truncated": 0.0, + "response_tokens_mean": 3414.861328125, + "response_tokens_median": 3073.0 + } + } +} diff --git a/docs/mopd-h100-2026-10-07/plots/dashboard-miles-532.png b/docs/mopd-h100-2026-10-07/plots/dashboard-miles-532.png new file mode 100644 index 0000000..bf6540f Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/dashboard-miles-532.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/dashboard.png b/docs/mopd-h100-2026-10-07/plots/dashboard.png new file mode 100644 index 0000000..2ee228a Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/dashboard.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/gpu_memory.png b/docs/mopd-h100-2026-10-07/plots/gpu_memory.png new file mode 100644 index 0000000..5eecdd6 Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/gpu_memory.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/gpu_power.png b/docs/mopd-h100-2026-10-07/plots/gpu_power.png new file mode 100644 index 0000000..27fab71 Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/gpu_power.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/gpu_util.png b/docs/mopd-h100-2026-10-07/plots/gpu_util.png new file mode 100644 index 0000000..79cb603 Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/gpu_util.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/policy_throughput.png b/docs/mopd-h100-2026-10-07/plots/policy_throughput.png new file mode 100644 index 0000000..521c768 Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/policy_throughput.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/response_length.png b/docs/mopd-h100-2026-10-07/plots/response_length.png new file mode 100644 index 0000000..16e016f Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/response_length.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/response_length_hist.png b/docs/mopd-h100-2026-10-07/plots/response_length_hist.png new file mode 100644 index 0000000..448819f Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/response_length_hist.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/reverse_kl.png b/docs/mopd-h100-2026-10-07/plots/reverse_kl.png new file mode 100644 index 0000000..445361d Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/reverse_kl.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/score.png b/docs/mopd-h100-2026-10-07/plots/score.png new file mode 100644 index 0000000..7c0e79d Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/score.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/teacher_latency.png b/docs/mopd-h100-2026-10-07/plots/teacher_latency.png new file mode 100644 index 0000000..024f592 Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/teacher_latency.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/teacher_throughput.png b/docs/mopd-h100-2026-10-07/plots/teacher_throughput.png new file mode 100644 index 0000000..ad83284 Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/teacher_throughput.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/timing.png b/docs/mopd-h100-2026-10-07/plots/timing.png new file mode 100644 index 0000000..d501f62 Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/timing.png differ diff --git a/docs/mopd-h100-2026-10-07/plots/truncation.png b/docs/mopd-h100-2026-10-07/plots/truncation.png new file mode 100644 index 0000000..25f18e3 Binary files /dev/null and b/docs/mopd-h100-2026-10-07/plots/truncation.png differ diff --git a/docs/opd-h100-2026-10-10/data/base/summary.json b/docs/opd-h100-2026-10-10/data/base/summary.json new file mode 100644 index 0000000..2acd004 --- /dev/null +++ b/docs/opd-h100-2026-10-10/data/base/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/assets/models/Qwen3.6-35B-A3B", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 849.0627772808075, + "output_tokens_per_second": 17909.520246193613, + "domains": { + "caesar_cipher": { + "avg@8": 0.2698412698412698, + "pass@8": 0.6428571428571429, + "finished_thinking": 0.9583333333333334, + "truncated": 0.04265873015873016, + "response_tokens_mean": 11439.588293650793, + "response_tokens_median": 9597.0 + }, + "simple_geometry": { + "avg@8": 0.552734375, + "pass@8": 1.0, + "finished_thinking": 0.9990234375, + "truncated": 0.0009765625, + "response_tokens_mean": 3589.064453125, + "response_tokens_median": 3233.5 + } + } +} diff --git a/docs/opd-h100-2026-10-10/data/case-verl-555.json b/docs/opd-h100-2026-10-10/data/case-verl-555.json new file mode 100644 index 0000000..893fee4 --- /dev/null +++ b/docs/opd-h100-2026-10-10/data/case-verl-555.json @@ -0,0 +1,537 @@ +{ + "meta": { + "result": "/mnt/data/opd/runs/verl-opd-h100x3/results/verl-555", + "framework": "verl", + "domains": [ + "caesar_cipher" + ], + "nodes": 3, + "policy_gpus": 12, + "teacher_gpus": 2, + "active_params": 3000000000.0, + "h100_peak_flops": 989400000000000.0, + "h100_usd_per_hour": 6.16 + }, + "updates": [ + { + "update": 1, + "window_s": 983.3573925495148, + "step_s": 228.64603980397806, + "train_s": 83.54048317717388, + "wait_s": 136.02564998390153, + "train_tokens": 804978.0, + "train_tok_s_gpu": 1204.4729234639312, + "mfu": 0.02191278817702725, + "prompt_p50": 139.0, + "response_p50": 5699.0, + "rollout_e2e_p50_s": 98.04878048780488, + "rollout_requests": 256.0, + "rollout_tok_s_gpu": 244.32698815339631, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 68.21680551572429, + "teacher_p50_s": 0.247265625, + "teacher_tok_s_gpu": 1031.3442576259813, + "teacher_mfu": 0.006254361780630571, + "cost_usd": 40.38321025403341, + "train_score": { + "caesar_cipher": 0.2734375 + }, + "samples": 128 + }, + { + "update": 2, + "window_s": 203.6893424987793, + "step_s": 203.63177325413562, + "train_s": 142.5498583209701, + "wait_s": 51.41021455312148, + "train_tokens": 1862111.0, + "train_tok_s_gpu": 1632.8593920865287, + "mfu": 0.029706356435776748, + "prompt_p50": 145.0, + "response_p50": 14327.0, + "rollout_e2e_p50_s": 86.91176470588235, + "rollout_requests": 173.0, + "rollout_tok_s_gpu": 758.4392885009476, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 761.8263909295923, + "teacher_p50_s": 0.42790697674418604, + "teacher_tok_s_gpu": 5894.878864402029, + "teacher_mfu": 0.03574820415040648, + "cost_usd": 8.36484233194987, + "train_score": { + "caesar_cipher": 0.296875 + }, + "samples": 128 + }, + { + "update": 3, + "window_s": 204.9063436985016, + "step_s": 204.74710425711237, + "train_s": 158.85415638703853, + "wait_s": 35.71765100886114, + "train_tokens": 2056644.0, + "train_tok_s_gpu": 1618.3429243968847, + "mfu": 0.029442260601520033, + "prompt_p50": 146.5, + "response_p50": 13572.5, + "rollout_e2e_p50_s": 92.3076923076923, + "rollout_requests": 142.0, + "rollout_tok_s_gpu": 713.0334800581469, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 836.4162714853678, + "teacher_p50_s": 0.33421052631578946, + "teacher_tok_s_gpu": 4624.800691355706, + "teacher_mfu": 0.028046092731083723, + "cost_usd": 8.414820514551797, + "train_score": { + "caesar_cipher": 0.28125 + }, + "samples": 128 + }, + { + "update": 4, + "window_s": 183.34074640274048, + "step_s": 183.19400741509162, + "train_s": 142.0569432231132, + "wait_s": 31.189692763844505, + "train_tokens": 1762810.0, + "train_tok_s_gpu": 1551.1473427519734, + "mfu": 0.028219781857222077, + "prompt_p50": 139.0, + "response_p50": 11428.0, + "rollout_e2e_p50_s": 111.52173913043478, + "rollout_requests": 137.0, + "rollout_tok_s_gpu": 845.7553474013176, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 801.2448744516377, + "teacher_p50_s": 0.34130434782608693, + "teacher_tok_s_gpu": 4150.844888174976, + "teacher_mfu": 0.02517189137765298, + "cost_usd": 7.529193318939209, + "train_score": { + "caesar_cipher": 0.3046875 + }, + "samples": 128 + }, + { + "update": 5, + "window_s": 193.55981516838074, + "step_s": 193.40156658296473, + "train_s": 141.893398395041, + "wait_s": 42.3260288790334, + "train_tokens": 1783218.0, + "train_tok_s_gpu": 1570.9134640600034, + "mfu": 0.028579383821588904, + "prompt_p50": 144.0, + "response_p50": 12905.0, + "rollout_e2e_p50_s": 94.41176470588235, + "rollout_requests": 154.0, + "rollout_tok_s_gpu": 784.4349468987124, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 767.7290860746544, + "teacher_p50_s": 0.38749999999999996, + "teacher_tok_s_gpu": 5201.738279839374, + "teacher_mfu": 0.03154480460787977, + "cost_usd": 7.948856409581502, + "train_score": { + "caesar_cipher": 0.40625 + }, + "samples": 128 + }, + { + "update": 6, + "window_s": 186.37790441513062, + "step_s": 186.24066243716516, + "train_s": 141.79766077594832, + "wait_s": 33.987438526935875, + "train_tokens": 1865805.0, + "train_tok_s_gpu": 1644.7776622247331, + "mfu": 0.02992318366691449, + "prompt_p50": 141.0, + "response_p50": 11509.0, + "rollout_e2e_p50_s": 81.7741935483871, + "rollout_requests": 145.0, + "rollout_tok_s_gpu": 714.8978688476435, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 834.2391791984193, + "teacher_p50_s": 0.36, + "teacher_tok_s_gpu": 4905.5090670167265, + "teacher_mfu": 0.0297483873075605, + "cost_usd": 7.653919274648031, + "train_score": { + "caesar_cipher": 0.46875 + }, + "samples": 128 + }, + { + "update": 7, + "window_s": 173.75379395484924, + "step_s": 173.6206866549328, + "train_s": 139.7115479321219, + "wait_s": 24.429785612039268, + "train_tokens": 1650035.0, + "train_tok_s_gpu": 1476.2872364724465, + "mfu": 0.026857863610778288, + "prompt_p50": 140.0, + "response_p50": 11030.0, + "rollout_e2e_p50_s": 88.125, + "rollout_requests": 136.0, + "rollout_tok_s_gpu": 724.3371811843009, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 791.3664129969874, + "teacher_p50_s": 0.3325581395348837, + "teacher_tok_s_gpu": 4603.439048978975, + "teacher_mfu": 0.027916549720915552, + "cost_usd": 7.135489138412476, + "train_score": { + "caesar_cipher": 0.6328125 + }, + "samples": 128 + }, + { + "update": 8, + "window_s": 157.11160326004028, + "step_s": 156.9639807420317, + "train_s": 126.20256746909581, + "wait_s": 21.300607747863978, + "train_tokens": 1609857.0, + "train_tok_s_gpu": 1594.5168869030913, + "mfu": 0.0290087972147318, + "prompt_p50": 140.5, + "response_p50": 10016.5, + "rollout_e2e_p50_s": 93.68421052631578, + "rollout_requests": 134.0, + "rollout_tok_s_gpu": 826.5011238650724, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 853.881872607183, + "teacher_p50_s": 0.28220338983050847, + "teacher_tok_s_gpu": 4009.477892968696, + "teacher_mfu": 0.02431460214050149, + "cost_usd": 6.452049840545654, + "train_score": { + "caesar_cipher": 0.7421875 + }, + "samples": 128 + }, + { + "update": 9, + "window_s": 156.2979187965393, + "step_s": 156.16829816391692, + "train_s": 108.29767746385187, + "wait_s": 37.295889471890405, + "train_tokens": 1469868.0, + "train_tok_s_gpu": 1696.5599291021497, + "mfu": 0.030865250377843836, + "prompt_p50": 140.0, + "response_p50": 10308.5, + "rollout_e2e_p50_s": 87.14285714285714, + "rollout_requests": 151.0, + "rollout_tok_s_gpu": 850.2107237886586, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 783.6892579449504, + "teacher_p50_s": 0.36923076923076925, + "teacher_tok_s_gpu": 5167.484034457169, + "teacher_mfu": 0.03133707722533153, + "cost_usd": 6.418634531911215, + "train_score": { + "caesar_cipher": 0.8046875 + }, + "samples": 128 + }, + { + "update": 10, + "window_s": 190.2927165031433, + "step_s": 190.1589511379134, + "train_s": 124.93277047411539, + "wait_s": 55.50364184402861, + "train_tokens": 1560530.0, + "train_tok_s_gpu": 1561.369761190203, + "mfu": 0.028405756722684108, + "prompt_p50": 140.5, + "response_p50": 10341.0, + "rollout_e2e_p50_s": 100.0, + "rollout_requests": 152.0, + "rollout_tok_s_gpu": 820.7942455017014, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 683.3901425991706, + "teacher_p50_s": 0.38, + "teacher_tok_s_gpu": 5417.995070678228, + "teacher_mfu": 0.0328562466384368, + "cost_usd": 7.814687557729086, + "train_score": { + "caesar_cipher": 0.7734375 + }, + "samples": 128 + }, + { + "update": 11, + "window_s": 236.6456470489502, + "step_s": 154.24689052789472, + "train_s": 143.28088134690188, + "wait_s": 1.230169241083786, + "train_tokens": 1960924.0, + "train_tok_s_gpu": 1710.7341725972715, + "mfu": 0.031123120180665948, + "prompt_p50": 141.5, + "response_p50": 13802.0, + "rollout_e2e_p50_s": 93.33333333333334, + "rollout_requests": 140.0, + "rollout_tok_s_gpu": 612.9065481464985, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 690.5275265829499, + "teacher_p50_s": 0.3716981132075472, + "teacher_tok_s_gpu": 4334.892751218898, + "teacher_mfu": 0.026288009407027885, + "cost_usd": 9.718247905476888, + "train_score": { + "caesar_cipher": 0.7265625 + }, + "samples": 128 + }, + { + "update": 12, + "window_s": 156.06426882743835, + "step_s": 155.9191921330057, + "train_s": 145.1196327889338, + "wait_s": 1.2156102731823921, + "train_tokens": 1891232.0, + "train_tok_s_gpu": 1629.0283778751893, + "mfu": 0.029636659391301202, + "prompt_p50": 138.0, + "response_p50": 13687.0, + "rollout_e2e_p50_s": 90.0, + "rollout_requests": 128.0, + "rollout_tok_s_gpu": 777.3715122505791, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1009.8574635359317, + "teacher_p50_s": 0.285, + "teacher_tok_s_gpu": 3801.558835072, + "teacher_mfu": 0.02305372246859915, + "cost_usd": 6.409039306513469, + "train_score": { + "caesar_cipher": 0.75 + }, + "samples": 128 + }, + { + "update": 13, + "window_s": 151.60559248924255, + "step_s": 151.4572779899463, + "train_s": 106.03791503189132, + "wait_s": 36.241084075998515, + "train_tokens": 1360469.0, + "train_tok_s_gpu": 1603.7530061662774, + "mfu": 0.029176828493018992, + "prompt_p50": 142.0, + "response_p50": 9130.0, + "rollout_e2e_p50_s": 90.9375, + "rollout_requests": 142.0, + "rollout_tok_s_gpu": 849.6536828556644, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 747.8115734728666, + "teacher_p50_s": 0.2975, + "teacher_tok_s_gpu": 4700.581873657242, + "teacher_mfu": 0.028505651144070603, + "cost_usd": 6.225936331558227, + "train_score": { + "caesar_cipher": 0.8359375 + }, + "samples": 128 + }, + { + "update": 14, + "window_s": 174.33482551574707, + "step_s": 174.20162245514803, + "train_s": 121.18467973684892, + "wait_s": 43.730492387898266, + "train_tokens": 1451886.0, + "train_tok_s_gpu": 1497.5964816187502, + "mfu": 0.027245539386635848, + "prompt_p50": 138.0, + "response_p50": 9737.0, + "rollout_e2e_p50_s": 90.0, + "rollout_requests": 151.0, + "rollout_tok_s_gpu": 819.3342872110076, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 694.0122241328731, + "teacher_p50_s": 0.34210526315789475, + "teacher_tok_s_gpu": 4986.975479098809, + "teacher_mfu": 0.03024242255366167, + "cost_usd": 7.15935016784668, + "train_score": { + "caesar_cipher": 0.8359375 + }, + "samples": 128 + }, + { + "update": 15, + "window_s": 181.21361756324768, + "step_s": 181.06817851914093, + "train_s": 128.81871459493414, + "wait_s": 42.10970986192115, + "train_tokens": 1709870.0, + "train_tok_s_gpu": 1659.1824462158172, + "mfu": 0.03018524765704943, + "prompt_p50": 142.0, + "response_p50": 10738.5, + "rollout_e2e_p50_s": 91.7910447761194, + "rollout_requests": 147.0, + "rollout_tok_s_gpu": 782.7506853000501, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 786.3049619708337, + "teacher_p50_s": 0.3836734693877551, + "teacher_tok_s_gpu": 4991.31915229445, + "teacher_mfu": 0.030268763810154337, + "cost_usd": 7.441839227930704, + "train_score": { + "caesar_cipher": 0.8203125 + }, + "samples": 128 + }, + { + "update": 16, + "window_s": 185.7009983062744, + "step_s": 185.56679539801553, + "train_s": 141.08943758509122, + "wait_s": 35.318580489838496, + "train_tokens": 1798058.0, + "train_tok_s_gpu": 1593.012587242391, + "mfu": 0.02898142972545284, + "prompt_p50": 138.0, + "response_p50": 11536.0, + "rollout_e2e_p50_s": 105.81818181818181, + "rollout_requests": 142.0, + "rollout_tok_s_gpu": 822.9326608929881, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 806.87862765035, + "teacher_p50_s": 0.38181818181818183, + "teacher_tok_s_gpu": 4626.219071709606, + "teacher_mfu": 0.028054694188657404, + "cost_usd": 7.626120997111003, + "train_score": { + "caesar_cipher": 0.765625 + }, + "samples": 128 + }, + { + "update": 17, + "window_s": 185.61707186698914, + "step_s": 185.47159188403748, + "train_s": 143.1569414981641, + "wait_s": 31.93976945616305, + "train_tokens": 1810998.0, + "train_tok_s_gpu": 1581.3047389176245, + "mfu": 0.02876843066557231, + "prompt_p50": 146.0, + "response_p50": 12635.0, + "rollout_e2e_p50_s": 86.89655172413794, + "rollout_requests": 148.0, + "rollout_tok_s_gpu": 767.3010097299997, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 813.0529076988396, + "teacher_p50_s": 0.3302325581395349, + "teacher_tok_s_gpu": 4656.742999476887, + "teacher_mfu": 0.02823979987554207, + "cost_usd": 7.622674418004355, + "train_score": { + "caesar_cipher": 0.765625 + }, + "samples": 128 + }, + { + "update": 18, + "window_s": 178.11337566375732, + "step_s": 177.95669643604197, + "train_s": 126.68978438898921, + "wait_s": 41.57034138799645, + "train_tokens": 1703337.0, + "train_tok_s_gpu": 1680.6179442713214, + "mfu": 0.030575220332407303, + "prompt_p50": 139.5, + "response_p50": 10819.0, + "rollout_e2e_p50_s": 86.19047619047619, + "rollout_requests": 147.0, + "rollout_tok_s_gpu": 775.8171790956875, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 796.9348145305129, + "teacher_p50_s": 0.31666666666666665, + "teacher_tok_s_gpu": 4828.379658715286, + "teacher_mfu": 0.02928065287274279, + "cost_usd": 7.3145226272583015, + "train_score": { + "caesar_cipher": 0.8203125 + }, + "samples": 128 + }, + { + "update": 19, + "window_s": 194.83445501327515, + "step_s": 194.66081380005926, + "train_s": 141.75438846996985, + "wait_s": 43.770514277974144, + "train_tokens": 1744979.0, + "train_tok_s_gpu": 1538.734548921626, + "mfu": 0.0279939578336257, + "prompt_p50": 138.5, + "response_p50": 11090.0, + "rollout_e2e_p50_s": 84.40677966101694, + "rollout_requests": 146.0, + "rollout_tok_s_gpu": 726.3281537670423, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 746.3511351560418, + "teacher_p50_s": 0.3738095238095238, + "teacher_tok_s_gpu": 4877.471492068746, + "teacher_mfu": 0.029578359563788632, + "cost_usd": 8.001201619211832, + "train_score": { + "caesar_cipher": 0.8046875 + }, + "samples": 128 + }, + { + "update": 20, + "window_s": 182.194828748703, + "step_s": null, + "train_s": null, + "wait_s": null, + "train_tokens": 1915136, + "train_tok_s_gpu": null, + "mfu": null, + "prompt_p50": 140.5, + "response_p50": 11545.0, + "rollout_e2e_p50_s": 16.25, + "rollout_requests": 9.0, + "rollout_tok_s_gpu": 12.205340926921508, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 875.9560727532602, + "teacher_p50_s": 0.77, + "teacher_tok_s_gpu": 736.9748138417231, + "teacher_mfu": 0.004469222643066846, + "cost_usd": 7.482134300613404, + "train_score": { + "caesar_cipher": 0.765625 + }, + "samples": 128 + } + ] +} diff --git a/docs/opd-h100-2026-10-10/data/case-verl-562.json b/docs/opd-h100-2026-10-10/data/case-verl-562.json new file mode 100644 index 0000000..3b43c36 --- /dev/null +++ b/docs/opd-h100-2026-10-10/data/case-verl-562.json @@ -0,0 +1,537 @@ +{ + "meta": { + "result": "/mnt/data/opd/runs/verl-opd-h100x2/results/verl-562", + "framework": "verl", + "domains": [ + "caesar_cipher" + ], + "nodes": 2, + "policy_gpus": 6, + "teacher_gpus": 2, + "active_params": 3000000000.0, + "h100_peak_flops": 989400000000000.0, + "h100_usd_per_hour": 6.16 + }, + "updates": [ + { + "update": 1, + "window_s": 986.0726523399353, + "step_s": 275.4030436920002, + "train_s": 104.79562207590789, + "wait_s": 161.16769961104728, + "train_tokens": 829300.0, + "train_tok_s_gpu": 989.18731476123, + "mfu": 0.017996130650598484, + "prompt_p50": 140.0, + "response_p50": 6564.5, + "rollout_e2e_p50_s": 112.34042553191489, + "rollout_requests": 256.0, + "rollout_tok_s_gpu": 457.08160103327606, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 140.16884692895664, + "teacher_p50_s": 0.21145833333333333, + "teacher_tok_s_gpu": 882.0911906804876, + "teacher_mfu": 0.005349249185448682, + "cost_usd": 26.996477948506673, + "train_score": { + "caesar_cipher": 0.203125 + }, + "samples": 128 + }, + { + "update": 2, + "window_s": 256.31589818000793, + "step_s": 256.26320623094216, + "train_s": 165.81935991486534, + "wait_s": 79.29093173495494, + "train_tokens": 1546024.0, + "train_tok_s_gpu": 1165.4429259600302, + "mfu": 0.021202721515343182, + "prompt_p50": 142.5, + "response_p50": 11292.0, + "rollout_e2e_p50_s": 116.25, + "rollout_requests": 181.0, + "rollout_tok_s_gpu": 1257.2285564602612, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1005.2855421621458, + "teacher_p50_s": 0.4173076923076923, + "teacher_tok_s_gpu": 4908.318246870757, + "teacher_mfu": 0.029765422964649832, + "cost_usd": 7.017359701283773, + "train_score": { + "caesar_cipher": 0.2890625 + }, + "samples": 128 + }, + { + "update": 3, + "window_s": 240.08418917655945, + "step_s": 239.91164597915486, + "train_s": 182.39869467401877, + "wait_s": 47.96077386196703, + "train_tokens": 2084822.0, + "train_tok_s_gpu": 1428.7533716496534, + "mfu": 0.025993087416306612, + "prompt_p50": 143.5, + "response_p50": 15071.0, + "rollout_e2e_p50_s": 113.87755102040816, + "rollout_requests": 146.0, + "rollout_tok_s_gpu": 1179.7569884608386, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1447.285364875908, + "teacher_p50_s": 0.3324324324324324, + "teacher_tok_s_gpu": 3843.303897539993, + "teacher_mfu": 0.023306876273741617, + "cost_usd": 6.572971579233806, + "train_score": { + "caesar_cipher": 0.2109375 + }, + "samples": 128 + }, + { + "update": 4, + "window_s": 194.6082944869995, + "step_s": 194.45435462798923, + "train_s": 141.20176142384298, + "wait_s": 43.59294270700775, + "train_tokens": 1809309.0, + "train_tok_s_gpu": 1601.7054087670224, + "mfu": 0.029139576872656565, + "prompt_p50": 142.0, + "response_p50": 11488.0, + "rollout_e2e_p50_s": 128.0, + "rollout_requests": 140.0, + "rollout_tok_s_gpu": 1396.825526115963, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1549.5305623786999, + "teacher_p50_s": 0.2672727272727273, + "teacher_tok_s_gpu": 3070.411266771144, + "teacher_mfu": 0.01861983788217795, + "cost_usd": 5.327942640177409, + "train_score": { + "caesar_cipher": 0.3984375 + }, + "samples": 128 + }, + { + "update": 5, + "window_s": 216.1886956691742, + "step_s": 216.02076214412227, + "train_s": 126.80879888893105, + "wait_s": 78.84452487900853, + "train_tokens": 1634971.0, + "train_tok_s_gpu": 1611.6497971012584, + "mfu": 0.02932049357976819, + "prompt_p50": 143.0, + "response_p50": 11093.0, + "rollout_e2e_p50_s": 132.6315789473684, + "rollout_requests": 170.0, + "rollout_tok_s_gpu": 1427.563849771968, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1260.4505791720778, + "teacher_p50_s": 0.38139534883720927, + "teacher_tok_s_gpu": 4742.734104696292, + "teacher_mfu": 0.028761274133998132, + "cost_usd": 5.9187660680982805, + "train_score": { + "caesar_cipher": 0.4765625 + }, + "samples": 128 + }, + { + "update": 6, + "window_s": 213.36129665374756, + "step_s": 213.19906070898287, + "train_s": 142.77159775793552, + "wait_s": 60.903837794903666, + "train_tokens": 1865070.0, + "train_tok_s_gpu": 1632.9140645695545, + "mfu": 0.029707351083739623, + "prompt_p50": 142.0, + "response_p50": 12736.0, + "rollout_e2e_p50_s": 111.35593220338984, + "rollout_requests": 159.0, + "rollout_tok_s_gpu": 1322.9554958042675, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1456.8949705271687, + "teacher_p50_s": 0.3538461538461538, + "teacher_tok_s_gpu": 3930.785541489486, + "teacher_mfu": 0.023837389578468687, + "cost_usd": 5.841358166164823, + "train_score": { + "caesar_cipher": 0.4921875 + }, + "samples": 128 + }, + { + "update": 7, + "window_s": 199.99294900894165, + "step_s": 199.8350861321669, + "train_s": 125.4230943210423, + "wait_s": 64.69409066601656, + "train_tokens": 1587563.0, + "train_tok_s_gpu": 1582.2076155452235, + "mfu": 0.02878485655934306, + "prompt_p50": 140.0, + "response_p50": 10404.0, + "rollout_e2e_p50_s": 121.5, + "rollout_requests": 158.0, + "rollout_tok_s_gpu": 1384.2413013651928, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1323.0158095298818, + "teacher_p50_s": 0.35, + "teacher_tok_s_gpu": 4632.180807327266, + "teacher_mfu": 0.028090847830971894, + "cost_usd": 5.475362515089246, + "train_score": { + "caesar_cipher": 0.6796875 + }, + "samples": 128 + }, + { + "update": 8, + "window_s": 235.2575933933258, + "step_s": 235.10581344692037, + "train_s": 164.5058148249518, + "wait_s": 61.58301124209538, + "train_tokens": 1773979.0, + "train_tok_s_gpu": 1347.9607103004723, + "mfu": 0.024523239120081364, + "prompt_p50": 139.0, + "response_p50": 12008.0, + "rollout_e2e_p50_s": 115.0, + "rollout_requests": 152.0, + "rollout_tok_s_gpu": 1183.7506680081253, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1256.763543323123, + "teacher_p50_s": 0.3326530612244898, + "teacher_tok_s_gpu": 4117.922342172033, + "teacher_mfu": 0.024972239794857687, + "cost_usd": 6.440830112457276, + "train_score": { + "caesar_cipher": 0.75 + }, + "samples": 128 + }, + { + "update": 9, + "window_s": 193.45914483070374, + "step_s": 193.31926639098674, + "train_s": 141.6433793529868, + "wait_s": 41.82529573701322, + "train_tokens": 1758251.0, + "train_tok_s_gpu": 1551.6530035073997, + "mfu": 0.028228981264537295, + "prompt_p50": 141.5, + "response_p50": 11501.0, + "rollout_e2e_p50_s": 108.23529411764706, + "rollout_requests": 136.0, + "rollout_tok_s_gpu": 1303.7929371981559, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1514.7478998202669, + "teacher_p50_s": 0.26557377049180325, + "teacher_tok_s_gpu": 3085.400798821616, + "teacher_mfu": 0.018710738622326355, + "cost_usd": 5.296481476253934, + "train_score": { + "caesar_cipher": 0.7421875 + }, + "samples": 128 + }, + { + "update": 10, + "window_s": 178.8643503189087, + "step_s": 178.72507769591175, + "train_s": 109.61572293704376, + "wait_s": 57.85768624814227, + "train_tokens": 1390524.0, + "train_tok_s_gpu": 1585.6803690454924, + "mfu": 0.028848035822537765, + "prompt_p50": 137.5, + "response_p50": 9245.0, + "rollout_e2e_p50_s": 132.41379310344828, + "rollout_requests": 156.0, + "rollout_tok_s_gpu": 1476.572420360876, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1295.6969881745074, + "teacher_p50_s": 0.33157894736842103, + "teacher_tok_s_gpu": 3826.4947642148786, + "teacher_mfu": 0.023204940959459543, + "cost_usd": 4.896908435397679, + "train_score": { + "caesar_cipher": 0.8125 + }, + "samples": 128 + }, + { + "update": 11, + "window_s": 217.63631415367126, + "step_s": 136.73373201303184, + "train_s": 124.71039526886307, + "wait_s": 1.300245586084202, + "train_tokens": 1632313.0, + "train_tok_s_gpu": 1636.1035867147416, + "mfu": 0.029765377563033504, + "prompt_p50": 139.5, + "response_p50": 10756.5, + "rollout_e2e_p50_s": 132.12121212121212, + "rollout_requests": 178.0, + "rollout_tok_s_gpu": 1472.2068231702858, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1250.0311252035485, + "teacher_p50_s": 0.34444444444444444, + "teacher_tok_s_gpu": 4556.19736005933, + "teacher_mfu": 0.02763006282631492, + "cost_usd": 5.958398645273844, + "train_score": { + "caesar_cipher": 0.7421875 + }, + "samples": 128 + }, + { + "update": 12, + "window_s": 241.16940641403198, + "step_s": 241.01047655707225, + "train_s": 145.28361166617833, + "wait_s": 85.05955668818206, + "train_tokens": 1881446.0, + "train_tok_s_gpu": 1618.7699858424535, + "mfu": 0.02945003006384088, + "prompt_p50": 139.0, + "response_p50": 12007.0, + "rollout_e2e_p50_s": 144.9056603773585, + "rollout_requests": 168.0, + "rollout_tok_s_gpu": 1418.269306014121, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1300.2243443556802, + "teacher_p50_s": 0.4228571428571428, + "teacher_tok_s_gpu": 4635.652243886572, + "teacher_mfu": 0.02811189959906957, + "cost_usd": 6.602682415601942, + "train_score": { + "caesar_cipher": 0.796875 + }, + "samples": 128 + }, + { + "update": 13, + "window_s": 232.42383813858032, + "step_s": 232.2845763049554, + "train_s": 163.4628152179066, + "wait_s": 58.39705048198812, + "train_tokens": 2099341.0, + "train_tok_s_gpu": 1605.365872661499, + "mfu": 0.02920617112179804, + "prompt_p50": 143.0, + "response_p50": 15359.5, + "rollout_e2e_p50_s": 116.0377358490566, + "rollout_requests": 155.0, + "rollout_tok_s_gpu": 1232.1061196940896, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1505.3970774634927, + "teacher_p50_s": 0.36428571428571427, + "teacher_tok_s_gpu": 4268.165898751385, + "teacher_mfu": 0.025883358997885898, + "cost_usd": 6.363248190816244, + "train_score": { + "caesar_cipher": 0.71875 + }, + "samples": 128 + }, + { + "update": 14, + "window_s": 190.86161303520203, + "step_s": 190.7127788469661, + "train_s": 141.69173830305226, + "wait_s": 39.42888449016027, + "train_tokens": 1860590.0, + "train_tok_s_gpu": 1641.4065688330256, + "mfu": 0.029861853890230907, + "prompt_p50": 139.0, + "response_p50": 12540.5, + "rollout_e2e_p50_s": 107.0, + "rollout_requests": 142.0, + "rollout_tok_s_gpu": 1306.7600622621414, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1624.7286628356198, + "teacher_p50_s": 0.2947368421052632, + "teacher_tok_s_gpu": 3327.861427446139, + "teacher_mfu": 0.020181088098521156, + "cost_usd": 5.225366827985975, + "train_score": { + "caesar_cipher": 0.78125 + }, + "samples": 128 + }, + { + "update": 15, + "window_s": 189.73096752166748, + "step_s": 189.57092022919096, + "train_s": 110.7112373940181, + "wait_s": 68.479488624027, + "train_tokens": 1458606.0, + "train_tok_s_gpu": 1646.8585691180374, + "mfu": 0.029961041281710805, + "prompt_p50": 140.0, + "response_p50": 8441.0, + "rollout_e2e_p50_s": 120.0, + "rollout_requests": 158.0, + "rollout_tok_s_gpu": 1432.8736291768146, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1281.2932078272231, + "teacher_p50_s": 0.32, + "teacher_tok_s_gpu": 4323.350640724077, + "teacher_mfu": 0.026218014801237578, + "cost_usd": 5.1944122663709855, + "train_score": { + "caesar_cipher": 0.8515625 + }, + "samples": 128 + }, + { + "update": 16, + "window_s": 222.35214972496033, + "step_s": 222.18830997496843, + "train_s": 138.99731915420853, + "wait_s": 72.89813811401837, + "train_tokens": 1663719.0, + "train_tok_s_gpu": 1496.1790361530386, + "mfu": 0.02721975202218991, + "prompt_p50": 139.0, + "response_p50": 10423.0, + "rollout_e2e_p50_s": 136.59574468085106, + "rollout_requests": 162.0, + "rollout_tok_s_gpu": 1398.3344305475082, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1247.0601266639023, + "teacher_p50_s": 0.3840909090909091, + "teacher_tok_s_gpu": 4596.368873717586, + "teacher_mfu": 0.0278736741887058, + "cost_usd": 6.087507743581137, + "train_score": { + "caesar_cipher": 0.78125 + }, + "samples": 128 + }, + { + "update": 17, + "window_s": 207.25072860717773, + "step_s": 207.10058080288582, + "train_s": 146.67118409113027, + "wait_s": 50.12097054091282, + "train_tokens": 1917361.0, + "train_tok_s_gpu": 1634.0641584449695, + "mfu": 0.029728274562370576, + "prompt_p50": 145.0, + "response_p50": 13261.0, + "rollout_e2e_p50_s": 107.06896551724138, + "rollout_requests": 149.0, + "rollout_tok_s_gpu": 1304.481139745935, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1541.9012942162428, + "teacher_p50_s": 0.3051282051282051, + "teacher_tok_s_gpu": 3547.0345746931202, + "teacher_mfu": 0.021510215734949184, + "cost_usd": 5.674064392089845, + "train_score": { + "caesar_cipher": 0.78125 + }, + "samples": 128 + }, + { + "update": 18, + "window_s": 186.1149423122406, + "step_s": 185.98569135181606, + "train_s": 124.73303614603356, + "wait_s": 51.448625684017316, + "train_tokens": 1589960.0, + "train_tok_s_gpu": 1593.3629625379722, + "mfu": 0.02898780404859864, + "prompt_p50": 140.5, + "response_p50": 10133.0, + "rollout_e2e_p50_s": 104.73684210526315, + "rollout_requests": 157.0, + "rollout_tok_s_gpu": 1372.311451695132, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1423.8154660831067, + "teacher_p50_s": 0.3, + "teacher_tok_s_gpu": 4160.466593278327, + "teacher_mfu": 0.02523024010478064, + "cost_usd": 5.095413531748454, + "train_score": { + "caesar_cipher": 0.8359375 + }, + "samples": 128 + }, + { + "update": 19, + "window_s": 197.86895513534546, + "step_s": 197.70588058698922, + "train_s": 124.16898798896, + "wait_s": 64.27982351696119, + "train_tokens": 1524674.0, + "train_tok_s_gpu": 1534.8780165377932, + "mfu": 0.027923796541015036, + "prompt_p50": 136.0, + "response_p50": 9205.5, + "rollout_e2e_p50_s": 108.28125, + "rollout_requests": 163.0, + "rollout_tok_s_gpu": 1382.7653415675152, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1284.2455915305993, + "teacher_p50_s": 0.29545454545454547, + "teacher_tok_s_gpu": 4146.532231085901, + "teacher_mfu": 0.02514573821155792, + "cost_usd": 5.417212282816569, + "train_score": { + "caesar_cipher": 0.84375 + }, + "samples": 128 + }, + { + "update": 20, + "window_s": 267.0545697212219, + "step_s": null, + "train_s": null, + "wait_s": null, + "train_tokens": 2104225, + "train_tok_s_gpu": null, + "mfu": null, + "prompt_p50": 143.0, + "response_p50": 14091.0, + "rollout_e2e_p50_s": 37.5, + "rollout_requests": 33.0, + "rollout_tok_s_gpu": 129.18895204083213, + "filtered": 0.0, + "filtered_ratio": 0.0, + "effective_tok_s_gpu_policy": 1313.230352256344, + "teacher_p50_s": 0.6950000000000001, + "teacher_tok_s_gpu": 1531.1439921318306, + "teacher_mfu": 0.009285288005650883, + "cost_usd": 7.311360664367676, + "train_score": { + "caesar_cipher": 0.734375 + }, + "samples": 128 + } + ] +} diff --git a/docs/opd-h100-2026-10-10/data/teacher-caesar/summary.json b/docs/opd-h100-2026-10-10/data/teacher-caesar/summary.json new file mode 100644 index 0000000..7bb571e --- /dev/null +++ b/docs/opd-h100-2026-10-10/data/teacher-caesar/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/assets/teachers/caesar_cipher", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 910.2614703178406, + "output_tokens_per_second": 19106.435422260813, + "domains": { + "caesar_cipher": { + "avg@8": 0.7033730158730159, + "pass@8": 0.9920634920634921, + "finished_thinking": 0.941468253968254, + "truncated": 0.060515873015873016, + "response_tokens_mean": 13412.673611111111, + "response_tokens_median": 11461.0 + }, + "simple_geometry": { + "avg@8": 0.58984375, + "pass@8": 1.0, + "finished_thinking": 0.9990234375, + "truncated": 0.0009765625, + "response_tokens_mean": 3781.1298828125, + "response_tokens_median": 3599.5 + } + } +} diff --git a/docs/opd-h100-2026-10-10/data/verl-555-step10/summary.json b/docs/opd-h100-2026-10-10/data/verl-555-step10/summary.json new file mode 100644 index 0000000..d94389a --- /dev/null +++ b/docs/opd-h100-2026-10-10/data/verl-555-step10/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/runs/verl-opd-h100x3/checkpoints/verl-555/global_step_10/actor/huggingface-merged", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 943.4778501987457, + "output_tokens_per_second": 18543.36802534854, + "domains": { + "caesar_cipher": { + "avg@8": 0.7648809523809523, + "pass@8": 1.0, + "finished_thinking": 0.9404761904761905, + "truncated": 0.061507936507936505, + "response_tokens_mean": 13245.607142857143, + "response_tokens_median": 11300.5 + }, + "simple_geometry": { + "avg@8": 0.6953125, + "pass@8": 1.0, + "finished_thinking": 0.99609375, + "truncated": 0.00390625, + "response_tokens_mean": 4046.5673828125, + "response_tokens_median": 3719.5 + } + } +} diff --git a/docs/opd-h100-2026-10-10/data/verl-555/summary.json b/docs/opd-h100-2026-10-10/data/verl-555/summary.json new file mode 100644 index 0000000..b5d9be6 --- /dev/null +++ b/docs/opd-h100-2026-10-10/data/verl-555/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/runs/verl-opd-h100x3/checkpoints/verl-555/global_step_20/actor/huggingface-merged", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 897.6359388828278, + "output_tokens_per_second": 19027.45340305441, + "domains": { + "caesar_cipher": { + "avg@8": 0.7628968253968254, + "pass@8": 1.0, + "finished_thinking": 0.9464285714285714, + "truncated": 0.05555555555555555, + "response_tokens_mean": 12973.520833333334, + "response_tokens_median": 10768.0 + }, + "simple_geometry": { + "avg@8": 0.6640625, + "pass@8": 1.0, + "finished_thinking": 0.998046875, + "truncated": 0.001953125, + "response_tokens_mean": 3908.6103515625, + "response_tokens_median": 3737.5 + } + } +} diff --git a/docs/opd-h100-2026-10-10/data/verl-562-step10/summary.json b/docs/opd-h100-2026-10-10/data/verl-562-step10/summary.json new file mode 100644 index 0000000..1a1a7bf --- /dev/null +++ b/docs/opd-h100-2026-10-10/data/verl-562-step10/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/runs/verl-opd-h100x2/checkpoints/verl-562/global_step_10/actor/huggingface-merged", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 957.9778125286102, + "output_tokens_per_second": 18717.644360332702, + "domains": { + "caesar_cipher": { + "avg@8": 0.75, + "pass@8": 0.9920634920634921, + "finished_thinking": 0.9226190476190477, + "truncated": 0.08134920634920635, + "response_tokens_mean": 13754.54365079365, + "response_tokens_median": 11722.5 + }, + "simple_geometry": { + "avg@8": 0.6484375, + "pass@8": 0.984375, + "finished_thinking": 0.9970703125, + "truncated": 0.0029296875, + "response_tokens_mean": 3971.19921875, + "response_tokens_median": 3703.0 + } + } +} diff --git a/docs/opd-h100-2026-10-10/data/verl-562/summary.json b/docs/opd-h100-2026-10-10/data/verl-562/summary.json new file mode 100644 index 0000000..220f844 --- /dev/null +++ b/docs/opd-h100-2026-10-10/data/verl-562/summary.json @@ -0,0 +1,27 @@ +{ + "model": "/mnt/data/opd/runs/verl-opd-h100x2/checkpoints/verl-562/global_step_20/actor/huggingface-merged", + "samples": 8, + "seed": 0, + "gpus": 8, + "tp": 2, + "generation_seconds": 924.1402888298035, + "output_tokens_per_second": 18583.083334399216, + "domains": { + "caesar_cipher": { + "avg@8": 0.7777777777777778, + "pass@8": 1.0, + "finished_thinking": 0.9424603174603174, + "truncated": 0.05952380952380952, + "response_tokens_mean": 13201.132936507936, + "response_tokens_median": 11318.0 + }, + "simple_geometry": { + "avg@8": 0.658203125, + "pass@8": 1.0, + "finished_thinking": 0.9990234375, + "truncated": 0.0009765625, + "response_tokens_mean": 3776.009765625, + "response_tokens_median": 3682.0 + } + } +} diff --git a/frameworks/README.md b/frameworks/README.md new file mode 100644 index 0000000..f816281 --- /dev/null +++ b/frameworks/README.md @@ -0,0 +1,82 @@ +# Frameworks + +Each directory runs the experiment defined in `shared/recipe.py` on Slurm: a Qwen3.6-35B-A3B student learns from frozen reasoning_gym GRPO teachers, one for `caesar_cipher` and one for `simple_geometry`. A directory holds only framework-specific code. `upstream/` holds the pinned revision, the patch (if any) and the license. + +The single-teacher `caesar_cipher` variant (`--domains caesar_cipher`) has run in Miles, Prime-RL, Slime and verl; NeMo-RL deadlocked before its first update. See the [benchmark report](../docs/CAESAR-OPD-2026-09-27.md). The two-teacher experiment ran in Miles, Prime-RL and verl on 16 B200 GPUs, and in Miles on 24 H100 GPUs ([H100 report](../docs/MOPD-H100-2026-10-07.md)). See [setup](../docs/SETUP.md) to run it, and [provenance](../docs/PROVENANCE.md) for the previous experiment's results. + +## Shared recipe (`shared/recipe.py`) + +| Setting | Value | +|---|---| +| Tasks | reasoning_gym 0.1.25 `caesar_cipher`, `simple_geometry` | +| Updates | 20 | +| Prompts per update | 128, split equally between tasks, one sample each | +| Learning rate | 1e-6 | +| Thinking | On | +| Context / response | 32,768 / up to 30,720 tokens | +| Maximum policy lag | 1 update | +| Checkpoints | After updates 10 and 20 (`tools/reproduce.sh` benchmarks both) | +| Evaluation | Every 10 updates, greedy, on the full dev sets (126 / 128) | +| Megatron trainer memory | No context parallelism; 32,768 max tokens per GPU; log-prob chunks of 4096 | +| Trainer node | 8 GPUs | +| Generation node | Teachers on GPUs 0-1; policy on GPUs 2-7 in 2-GPU engines | +| CPUs | 80 per node; 64 exposed to Ray | + +`shared/scoring.py` scores the text after the last `` with reasoning_gym's `score_answer`. The score is diagnostic only. + +## Implementations + +| | Miles | Prime-RL | Slime | verl | +|---|---|---|---|---| +| Objective | Sampled-token OPD (`--loss-mode legacy --candidate-top-k 0`) | Native per-source OPD through reference KL | Native OPD through an advantage correction | Sampled-token k1 reverse KL as a policy gradient (`distillation`, no task reward) | +| Scheduling | Fully async; domain-balanced source and buffer | Continuous async | `train_async`, one rollout ahead | Fully async trainer, staleness 1, weight sync after every update | +| Trainer | Megatron | Prime-RL trainer (FSDP) | Megatron | FSDP | +| Policy serving | SGLang through Ray | Three TP2 vLLM engines behind vllm-router | SGLang through Ray | Three TP2 vLLM engines in verl's rollout pool | +| Teacher serving | Prefill-only SGLang (`shared/container.py`) | Single-GPU vLLM per teacher | Prefill-only SGLang (`shared/container.py`) | Single-GPU vLLM per teacher in verl's teacher pool, routed by `data_source` | +| Runtime | Pyxis container | Host virtual environment | Pyxis container | Host virtual environment (verl's `uv.lock`) | +| Recipe in code | `MilesNode.train_command` in `node.py` | `recipe_input()` in `make_config.py` | `train.py` | `overrides()` in `train.py` | + +### Miles + +Miles uses sampled tokens because top-k candidate scoring is quadratic in response length. Its launcher arguments also disable the answer stop, set a 7200 s teacher timeout, match Slime's Adam settings (weight decay 0, beta2 0.999), drop the MTP layers and replace the puzzle-only data source. The learning rate is the pinned launcher's default. + +Throughput settings, checked against Miles' fully async examples: + +- `domain_balance.py` submits each new prompt from the task with the fewest pending samples, and keeps finished samples past a task's share of the queue in an overflow list. The earlier buffer evicted them after generation and teacher scoring: in the B200 MOPD run the teachers prefilled 47M tokens for about 14M trained. +- At most 160 samples are in flight (1.25 batches). At policy lag 1 a sample must finish within about two updates, so a deeper pipeline mostly produces stale samples. +- Greedy dev evaluation pauses generation and training (about 400 s at 30k tokens), so Miles evaluates only after the last update and skips the step-0 evaluation. `tools/benchmark.py` evaluates every saved checkpoint offline. The other frameworks still evaluate every 10 updates. +- Policy engines use 80% of GPU memory instead of the launcher's 65%, which assumed colocation. + +### Prime-RL + +Prime-RL keeps upstream defaults. Only the shared recipe values (including policy lag 1 and weight decay 0), the two-node layout and the teachers are set. The teacher engine takes the two settings from upstream's OPD example (`gpu_memory_utilization` 0.5, eager), and the router takes upstream's arguments. + +The student trains as a text model (no `model.vlm`), so the trainer keeps fp32 master weights. A one-file patch gives the frozen vision encoder SDPA, because FlashAttention 4 rejects fp32 inputs. + +### verl + +verl runs unmodified and follows upstream `examples/on_policy_distillation_trainer` on `verl.experimental.fully_async_policy`. It trains with FSDP and places its own trainer, rollout and teacher pools on the Ray cluster, so Ray chooses which node and GPUs each pool gets. It streams prompts, so each update has 128 responses but not an exact 64/64 task split. + +On `h100x3` (run 547, matched to Miles run 535): + +- Trainer: FSDP2 with `CPUOffloadPolicy`, so the Adam step runs on the CPU as in Miles, and Ulysses sequence parallelism 2. verl's `optimizer_offload` alone moves the Adam states back to the GPU for the step and ran out of memory (job 543). With the parameters on the CPU the trainer peaks at 14 GB, so each GPU takes a full 32k-token micro-batch, which halves the parameter copies per update. +- At most 160 samples in flight (27 per policy engine) and `staleness_threshold=1`, Miles' policy lag 1. verl's threshold is a fraction of a batch per weight sync; 0.25 (job 544) also capped generation at 160 samples per sync, and the trainer waited a median 78 s per update. +- Evaluation only after the last update, as in Miles. +- Compiled-kernel caches stay on each node's local disk: on CoreWeave's NFS, engines compiling into one shared cache hit "Stale file handle" and hung (job 542). +- vLLM request statistics are on (`disable_log_stats=False`) and `shared/capture.py --vllm` scrapes every policy and teacher server; `tools/schema_verl.py` turns a run into the case-study columns. + +### Memory + +Miles, Slime and verl set `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` on the trainer (verl sets it on every Ray node). + +## NeMo-RL + +`nemo-rl/` runs NeMo-RL's MOPD (async GRPO with `adv_estimator: opd`, rollouts through NeMo Gym's `reasoning_gym` server) unmodified from `nvcr.io/nvidia/nemo-rl:v0.7.0` (commit `19244a09`), launched with upstream `ray.sub`. + +The recipe is `recipe_input()` in `make_config.py`: the shared recipe on upstream's base GRPO config, the loss settings of the MOPD reference recipe, Megatron EP8 with no context parallelism, eight single-GPU vLLM engines with CUDA graphs, and the teacher on one GPU. MOPD reserves whole nodes for the teacher and for generation, so it needs three nodes (24 GPUs) and supports only one task. + +Every run so far deadlocked in the first weight sync. [`nemo-rl/repro/`](nemo-rl/repro/README.md) documents and reproduces it. + +## Hosted variants + +[Tinker](../hosted/tinker/README.md) and [Fireworks](../hosted/fireworks/README.md) use hosted training APIs instead of `tools/`. Their protocols are in their READMEs. diff --git a/frameworks/miles/campaign/campaign_async.py b/frameworks/miles/campaign/campaign_async.py deleted file mode 100644 index 837fe08..0000000 --- a/frameworks/miles/campaign/campaign_async.py +++ /dev/null @@ -1,56 +0,0 @@ -"""Record scoring attempts and retry transient reads from frozen teachers.""" -import asyncio -import json -import logging -import os -from pathlib import Path -import time - -import httpx - -from examples.mopd_puzzles.tasks import reward_func as verifier_reward -from miles.rollout.on_policy_distillation import reward_func as teacher_reward - -_stream = None -logger = logging.getLogger(__name__) - - -def _record_attempt(sample, attempt, started, total_started, success, error): - global _stream - if _stream is None: - path = Path(os.environ['MOPD_RESULT']) / f'teacher-latency-{os.getpid()}.jsonl' - _stream = path.open('a', buffering=8192) - _stream.write(json.dumps({'time_unix': time.time(), 'seconds': time.monotonic() - started, - 'total_seconds': time.monotonic() - total_started, - 'success': success, 'attempt': attempt, 'error_type': error, - 'sample_index': sample.index, 'teacher': sample.metadata.get('opd_teacher'), - 'response_tokens': sample.response_length, - 'total_tokens': len(sample.tokens)}) + '\n') - if not success: - _stream.flush() - - -async def reward_func(args, sample, **kwargs): - if sample.metadata.get('mopd_evaluation', False): - return await verifier_reward(args, sample, **kwargs) - total_started = time.monotonic() - for attempt in range(1, 4): - started = time.monotonic() - success = False - error = None - try: - result = await teacher_reward(args, sample, **kwargs) - success = True - return result - except (httpx.ReadError, httpx.RemoteProtocolError, httpx.ConnectError) as exc: - error = type(exc).__name__ - if attempt == 3: - raise - logger.warning('Frozen-teacher scoring retry: sample=%s teacher=%s attempt=%s error=%s', - sample.index, sample.metadata.get('opd_teacher'), attempt, error) - except BaseException as exc: - error = type(exc).__name__ - raise - finally: - _record_attempt(sample, attempt, started, total_started, success, error) - await asyncio.sleep(0.25 * 2 ** (attempt - 1)) diff --git a/frameworks/miles/campaign/control.py b/frameworks/miles/campaign/control.py deleted file mode 100644 index d4fadf7..0000000 --- a/frameworks/miles/campaign/control.py +++ /dev/null @@ -1,77 +0,0 @@ -"""Own one Slurm allocation and stop its two container steps after training.""" - -import json -import os -from pathlib import Path -import signal -import subprocess -import sys -import time - -ROOT = Path(__file__).resolve().parent -IMAGE = "@CONTAINER_IMAGE@" -PREVIOUS = "@ASSETS@" -BASE = "@BASE_PARENT@" - - -def stop(*_): - raise SystemExit(143) - - -def main(): - job = os.environ["SLURM_JOB_ID"] - result = ROOT / "results" / f"mopd-async-{job}" - result.mkdir(parents=True, exist_ok=False) - env = {**os.environ, "MOPD_RESULT": str(result), "MOPD_CAMPAIGN": str(ROOT), - "NVIDIA_VISIBLE_DEVICES": "all", "NVIDIA_DRIVER_CAPABILITIES": "all"} - nodes = subprocess.check_output(["scontrol", "show", "hostnames", os.environ["SLURM_JOB_NODELIST"]], text=True).split() - assert nodes == ["@GENERATION_NODE@", "@TRAINER_NODE@"], nodes - (ROOT / "active-run.json").write_text(json.dumps({"job_id": job, "result": str(result), "nodes": nodes})) - (result / "start.json").write_text(json.dumps({"time": time.time(), "job": job, "nodes": nodes})) - (result / "slurm-start.txt").write_bytes(subprocess.check_output(["scontrol", "show", "job", job])) - subprocess.run(["tar", "--exclude=__pycache__", "--exclude=.git", "--exclude=.venv", "-czf", str(result / "launch-source.tar.gz"), "-C", str(ROOT), - "source", "data", "control.py", "node.py", "observe.py", "campaign_async.py", - "serve-teachers.py", "eval.json", "teacher-provenance.json", "test-async-contract.py", - "test-scoring-retry.py", "endpoints.py", "balanced_async_buffer.py", - "test-balanced-buffer.py", "test-balanced-integration.py"], check=True) - common = ["srun", "--exclusive", "-N1", "-n1", "--gres=gpu:8", "--cpus-per-task=80", - f"--container-image={IMAGE}", "--container-writable", "--container-remap-root", - "--no-container-mount-home", f"--container-mounts={ROOT}:{ROOT},{PREVIOUS}:{PREVIOUS}:ro," - f"@MEGATRON_PARENT@:@MEGATRON_PARENT@:ro,{BASE}:{BASE}:ro"] - children = [] - code = 1 - signal.signal(signal.SIGTERM, stop) - signal.signal(signal.SIGINT, stop) - try: - for role, node in (("learner", nodes[1]), ("generation", nodes[0])): - command = common + ["-w", node, "python3", str(ROOT / "node.py"), role] - (result / f"{role}-srun.json").write_text(json.dumps(command, indent=2)) - log = (result / f"{role}-driver.out").open("wb") - children.append(subprocess.Popen(command, env=env, stdout=log, stderr=subprocess.STDOUT, start_new_session=True)) - while True: - if children[0].poll() is not None: - code = children[0].returncode - break - if children[1].poll() is not None: - raise RuntimeError(f"Generation step exited early with {children[1].returncode}") - time.sleep(5) - finally: - (result / "STOP").touch() - deadline = time.monotonic() + 40 - while any(p.poll() is None for p in children) and time.monotonic() < deadline: - time.sleep(1) - for p in children: - if p.poll() is None: - os.killpg(p.pid, signal.SIGTERM) - for p in children: - try: - p.wait(timeout=20) - except subprocess.TimeoutExpired: - os.killpg(p.pid, signal.SIGKILL) - (result / "exit-code.txt").write_text(str(code) + "\n") - (result / "end.json").write_text(json.dumps({"time": time.time(), "exit_code": code})) - return code - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/frameworks/miles/campaign/endpoints.py b/frameworks/miles/campaign/endpoints.py deleted file mode 100644 index 76e785a..0000000 --- a/frameworks/miles/campaign/endpoints.py +++ /dev/null @@ -1,48 +0,0 @@ -"""Scrape only the generation endpoints advertised by this campaign's launch log.""" - -import gzip -import json -from pathlib import Path -import re -import signal -import sys -import threading -import time -import urllib.request - -root = Path(__file__).resolve().parent -job = sys.argv[1] -assert job.isdecimal() -result = root / "results" / f"mopd-async-{job}" -stopping = threading.Event() -signal.signal(signal.SIGTERM, lambda *_: stopping.set()) -signal.signal(signal.SIGINT, lambda *_: stopping.set()) -ports = set() -offset = 0 -with gzip.open(result / "generation-endpoints.jsonl.gz", "at") as output: - while not stopping.is_set() and not (result / "end.json").exists(): - if (result / "learner-train.out").exists(): - with (result / "learner-train.out").open() as log: - log.seek(offset) - for line in log: - if "sglang.launch_server" in line and str(root / "models") in line: - ports.update(int(value) for value in re.findall(r"--port[ =]+(\d+)", line)) - offset = log.tell() - if (result / "generation-ready.json").exists(): - ip = json.loads((result / "generation-ready.json").read_text())["ip"] - assert ip == "@GENERATION_IP@" - for port in sorted(ports): - assert 1024 < port < 65536 - row = {"time_unix": time.time(), "ip": ip, "port": port} - try: - with urllib.request.urlopen(f"http://{ip}:{port}/metrics", timeout=2) as response: - row["prometheus"] = response.read().decode() - except OSError as error: - row["error"] = str(error) - output.write(json.dumps(row) + "\n") - output.flush() - (result / "generation-endpoints-status.json").write_text(json.dumps({"time_unix": time.time(), "ports": sorted(ports), "finished": False})) - if (result / "STOP").exists(): - break - stopping.wait(10) -(result / "generation-endpoints-status.json").write_text(json.dumps({"time_unix": time.time(), "ports": sorted(ports), "finished": True})) diff --git a/frameworks/miles/campaign/eval.json b/frameworks/miles/campaign/eval.json deleted file mode 100644 index 74145b6..0000000 --- a/frameworks/miles/campaign/eval.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "eval": { - "defaults": { - "temperature": 0, - "n_samples_per_eval_prompt": 1, - "max_response_len": 256 - }, - "datasets": [ - {"name": "countdown", "path": "@CAMPAIGN@/data/countdown4-dev.jsonl", "metadata_overrides": {"mopd_evaluation": true}}, - {"name": "graph_color", "path": "@CAMPAIGN@/data/graph12-dev.jsonl", "metadata_overrides": {"mopd_evaluation": true}} - ] - } -} diff --git a/frameworks/miles/campaign/node.py b/frameworks/miles/campaign/node.py deleted file mode 100644 index 3fa0790..0000000 --- a/frameworks/miles/campaign/node.py +++ /dev/null @@ -1,195 +0,0 @@ -"""Run one owned container role with lightweight measurement and bounded cleanup.""" - -import ipaddress -import json -import os -from pathlib import Path -import shutil -import shlex -import signal -import socket -import subprocess -import sys -import time - -ROOT = Path(os.environ["MOPD_CAMPAIGN"]) -RESULT = Path(os.environ["MOPD_RESULT"]) -ROLE = sys.argv[1] -CHILDREN = [] - - -def launch(command, name, extra=None): - (RESULT / f"{ROLE}-{name}-command.json").write_text(json.dumps(command, indent=2)) - process = subprocess.Popen(command, env={**os.environ, **(extra or {})}, - stdout=(RESULT / f"{ROLE}-{name}.out").open("wb"), - stderr=subprocess.STDOUT, start_new_session=True) - CHILDREN.append(process) - return process - - -def check_children(ignore=None): - if (RESULT / "STOP").exists(): - raise SystemExit(143) - if any(p is not ignore and p.poll() is not None for p in CHILDREN): - raise RuntimeError("A required process exited; inspect the per-process logs") - - -def wait_file(name, timeout=900): - deadline = time.monotonic() + timeout - while not (RESULT / name).exists(): - check_children() - if time.monotonic() > deadline: - raise TimeoutError(f"Readiness timed out for {name}") - time.sleep(2) - return json.loads((RESULT / name).read_text()) - - -def stop(*_): - raise SystemExit(143) - - -def preflight(): - tests = ["tests/fast/rollout/test_fully_async_rollout.py", - "tests/fast/backends/training_utils/loss/test_candidate_opd.py", - "tests/fast/examples/test_mopd_data_source.py", - "tests/fast/examples/test_mopd_puzzles.py", str(ROOT / "test-balanced-integration.py")] - with (RESULT / "preflight-tests.out").open("wb") as log: - subprocess.run([sys.executable, str(ROOT / "test-async-contract.py")], stdout=log, stderr=subprocess.STDOUT, check=True) - subprocess.run([sys.executable, str(ROOT / "test-scoring-retry.py")], stdout=log, stderr=subprocess.STDOUT, check=True) - subprocess.run([sys.executable, str(ROOT / "test-balanced-buffer.py")], stdout=log, stderr=subprocess.STDOUT, check=True) - subprocess.run([sys.executable, "-m", "pytest", "-q", *tests], stdout=log, stderr=subprocess.STDOUT, check=True, timeout=180) - subprocess.run([sys.executable, "-c", "from types import SimpleNamespace; from miles.utils.tracking_utils.tensorboard_utils import _TensorboardAdapter; t=_TensorboardAdapter(SimpleNamespace(use_tensorboard=True,tb_project_name='async',tb_experiment_name='preflight')); t.log({'smoke':1.0},0); t.finish()"], - env={**os.environ, "TENSORBOARD_DIR": str(RESULT / "preflight-tensorboard")}, - stdout=log, stderr=subprocess.STDOUT, check=True) - (RESULT / "preflight-passed.json").write_text(json.dumps({"time": time.time(), "tests": tests})) - - -def verify_placement(head_ip, generation_ip): - import ray - from ray.util import remove_placement_group - from miles.ray.placement_group import _create_placement_group - - ray.init(address=f"{head_ip}:6379") - deadline = time.monotonic() + 180 - while int(ray.cluster_resources().get("GPU", 0)) != 14: - check_children() - assert time.monotonic() < deadline, ray.cluster_resources() - time.sleep(2) - nodes = {n["NodeManagerAddress"]: n["Resources"].get("GPU") for n in ray.nodes() if n["Alive"]} - assert nodes == {head_ip: 8.0, generation_ip: 6.0}, nodes - assert ipaddress.ip_address(head_ip) < ipaddress.ip_address(generation_ip), nodes - group = _create_placement_group(14) - try: - actual = [int(x) for x in group.pg_reordered_gpu_ids] - assert actual == list(range(8)) + list(range(2, 8)), actual - (RESULT / "placement-verified.json").write_text(json.dumps({"nodes": nodes, "sorted_physical_gpu_ids": actual})) - finally: - remove_placement_group(group.pg) - ray.shutdown() - - -def learner(head_ip): - preflight() - launch(["ray", "start", "--head", f"--node-ip-address={head_ip}", "--num-gpus=8", "--num-cpus=64", - "--disable-usage-stats", "--block"], "ray") - (RESULT / "head-started.json").write_text(json.dumps({"ip": head_ip})) - ready = wait_file("generation-ready.json") - verify_placement(head_ip, ready["ip"]) - (RESULT / "placement-approved.json").write_text(json.dumps({"time": time.time()})) - wait_file("teachers-ready.json") - extra = ["--sglang-moe-runner-backend", "triton", "--sglang-enable-metrics", - "--custom-async-data-buffer-path", "balanced_async_buffer.BalancedAsyncBuffer", - "--custom-rm-path", "campaign_async.reward_func", "--eval-config", str(ROOT / "eval.json"), - "--async-max-concurrent-samples", "256", "--async-data-buffer-capacity-factor", "1", - "--max-weight-staleness", "2", "--update-weights-interval", "1", - "--save-debug-event-data", str(RESULT / "events"), - "--use-tensorboard", "--tb-project-name", str(RESULT / "tensorboard"), - "--tb-experiment-name", "async-mopd", "--no-offload-train", "--no-offload-rollout"] - command = [sys.executable, "scripts/run_mopd_puzzles.py", "--mode", "student", "--num-nodes", "2", - "--fully-async", "--no-colocate", "--use-rollout-logprobs", "--actor-gpus", "8", - "--megatron-path", "@MEGATRON_SOURCE@", - "--rollout-gpus", "6", "--rollout-gpus-per-engine", "2", "--model-dir", str(ROOT / "models"), - "--data-dir", str(ROOT / "data"), "--checkpoint-dir", str(ROOT / "checkpoints" / RESULT.name), - "--teacher-urls", f"countdown=http://{ready['ip']}:30000/generate graph_color=http://{ready['ip']}:30001/generate", - "--candidate-top-k", "16", "--loss-mode", "topk-candidate", "--reward-refresh", - "--domain-balance", "static", "--num-rollout", "40", "--rollout-batch-size", "128", - "--n-samples-per-prompt", "1", "--global-batch-size", "128", "--eval-interval", "10", - "--save-interval", "20", "--no-cleanup-processes", "--no-sparse-scoring", - "--extra-env-vars", json.dumps({"MOPD_RESULT": str(RESULT), "MOPD_CAMPAIGN": str(ROOT), - "TENSORBOARD_DIR": str(RESULT / "tensorboard"), - "PYTHONPATH": os.environ["PYTHONPATH"], "NCCL_DEBUG": "WARN"}), - "--extra-args", shlex.join(extra)] - training = launch(command, "train") - while training.poll() is None: - check_children(ignore=training) - time.sleep(3) - return training.returncode - - -def generation(ip): - wait_file("preflight-passed.json") - head = wait_file("head-started.json") - # The head writes its address before its GCS socket becomes ready. - deadline = time.monotonic() + 180 - while True: - try: - with socket.create_connection((head["ip"], 6379), timeout=2): - break - except OSError: - check_children() - assert time.monotonic() < deadline - time.sleep(2) - launch(["ray", "start", f"--address={head['ip']}:6379", f"--node-ip-address={ip}", - "--num-gpus=6", "--num-cpus=64", "--disable-usage-stats", "--block"], "ray", - {"CUDA_VISIBLE_DEVICES": "2,3,4,5,6,7"}) - (RESULT / "generation-ready.json").write_text(json.dumps({"ip": ip})) - wait_file("placement-approved.json") - launch([sys.executable, str(ROOT / "serve-teachers.py")], "teachers") - while not (RESULT / "STOP").exists(): - check_children() - time.sleep(3) - return 0 - - -def main(): - os.chdir(ROOT / "source") - sys.path.insert(0, str(ROOT / "source")) - ip = '@TRAINER_IP@' if ROLE == 'learner' else '@GENERATION_IP@' - ray_tmp = Path(f"/tmp/mopd-async-{os.environ['SLURM_JOB_ID']}-{ROLE}") - os.environ.update({"PYTHONPATH": f"{ROOT}:{ROOT}/source:@MEGATRON_SOURCE@", "PYTHONNOUSERSITE": "1", - "PYTHONUNBUFFERED": "1", "WANDB_MODE": "disabled", "NCCL_DEBUG": "WARN", - "MASTER_ADDR": ip, "RAY_TMPDIR": str(ray_tmp), "MILES_SCRIPT_EXTERNAL_RAY": "1", - "RAY_memory_monitor_refresh_ms": "0", "RAY_DEDUP_LOGS": "0", - "TORCHINDUCTOR_CACHE_DIR": f"/tmp/mopd-inductor-{os.environ['SLURM_JOB_ID']}-{ROLE}"}) - os.environ.pop("NETRC", None) - signal.signal(signal.SIGTERM, stop) - signal.signal(signal.SIGINT, stop) - memory = subprocess.check_output(["nvidia-smi", "--query-gpu=memory.used", "--format=csv,noheader,nounits"], text=True) - assert all(int(v) < 1024 for v in memory.split()), memory - (RESULT / f"{ROLE}-gpu-before.txt").write_bytes(subprocess.check_output(["nvidia-smi", "-q"])) - launch(["nvidia-smi", "--query-gpu=timestamp,index,uuid,utilization.gpu,utilization.memory,memory.used,power.draw,temperature.gpu,clocks.sm,clocks.mem", "--format=csv", "-l", "5"], "gpu") - launch([sys.executable, str(ROOT / "observe.py"), os.environ["SLURM_JOB_ID"], ROLE, str(os.getpid())], "observer") - if ROLE == "learner": - launch([sys.executable, str(ROOT / "endpoints.py"), os.environ["SLURM_JOB_ID"]], "endpoints") - try: - return learner(ip) if ROLE == "learner" else generation(ip) - finally: - for p in reversed(CHILDREN): - if p.poll() is None: - os.killpg(p.pid, signal.SIGTERM) - for p in CHILDREN: - try: - p.wait(timeout=8) - except subprocess.TimeoutExpired: - os.killpg(p.pid, signal.SIGKILL) - copied = [] - for session in Path("/tmp").glob(f"mopd-async-{os.environ['SLURM_JOB_ID']}-*/ray/session_*"): - if session.is_symlink() or not (session / "logs").exists(): - continue - shutil.copytree(session / "logs", RESULT / "ray-logs" / ROLE, dirs_exist_ok=True) - copied.append(str(session)) - (RESULT / f"{ROLE}-cleanup.json").write_text(json.dumps({"time": time.time(), "ray_sessions": copied})) - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/frameworks/miles/campaign/observe.py b/frameworks/miles/campaign/observe.py deleted file mode 100644 index 77484d1..0000000 --- a/frameworks/miles/campaign/observe.py +++ /dev/null @@ -1,64 +0,0 @@ -"""Collect host counters and copy logs from one identified campaign container.""" - -import gzip -import json -import os -from pathlib import Path -import shutil -import signal -import sys -import threading -import time -import urllib.request - -root = Path(__file__).resolve().parent -job, role, pid = sys.argv[1:] -result = root / "results" / f"mopd-async-{job}" -process = Path("/proc") / pid -assert (process / "cmdline").read_bytes().split(b"\0")[1:3] == [str(root / "node.py").encode(), role.encode()] -container_root = process / "root" -status = result / f"{role}-observer-status.json" -start = time.time() -last_copy = 0 -iterations = 0 -errors = [] -stopping = threading.Event() -signal.signal(signal.SIGTERM, lambda *_: stopping.set()) -signal.signal(signal.SIGINT, lambda *_: stopping.set()) -with gzip.open(result / f"{role}-host-counters.jsonl.gz", "at") as stream: - while process.exists() and not stopping.is_set() and not (result / "end.json").exists(): - now = time.time() - row = {"time_unix": now, "scope": "node procfs counters, not per-job attribution"} - for name in ("stat", "meminfo", "net/dev", "loadavg", "diskstats"): - try: - row[name] = (container_root / "proc" / name).read_text() - except OSError as error: - row[name] = str(error) - if role == "generation": - for port in (30000, 30001): - try: - with urllib.request.urlopen(f"http://127.0.0.1:{port}/metrics", timeout=1) as response: - row[f"sglang_{port}"] = response.read().decode() - except OSError as error: - row[f"sglang_{port}"] = str(error) - stream.write(json.dumps(row) + "\n") - stream.flush() - iterations += 1 - if now - last_copy >= 60 or (result / "STOP").exists(): - # Real session directories avoid absolute session_latest symlinks across mount namespaces. - for logs in (container_root / "tmp").glob(f"mopd-async-{job}-*/ray/session_*/logs"): - if logs.parent.is_symlink(): - continue - try: - shutil.copytree(logs, result / "ray-logs-live" / role, dirs_exist_ok=True, - ignore=lambda directory, names: [name for name in names if Path(directory, name).is_symlink()]) - except (OSError, shutil.Error) as error: - errors.append(str(error)[:1000]) - last_copy = now - status.write_text(json.dumps({"start": start, "last_sample": now, "iterations": iterations, - "errors": errors[-5:], "finished": False})) - if (result / "STOP").exists(): - break - stopping.wait(10) -status.write_text(json.dumps({"start": start, "end": time.time(), "iterations": iterations, - "errors": errors[-5:], "finished": True})) diff --git a/frameworks/miles/campaign/serve-teachers.py b/frameworks/miles/campaign/serve-teachers.py deleted file mode 100644 index 5b9c286..0000000 --- a/frameworks/miles/campaign/serve-teachers.py +++ /dev/null @@ -1,114 +0,0 @@ -"""Serve frozen teachers on separate GPUs and validate prefill scoring.""" -import json -import math -import os -from pathlib import Path -import signal -import socket -import subprocess -import sys -import time -import urllib.request - -campaign = Path(os.environ['MOPD_CAMPAIGN']) -result = Path(os.environ['MOPD_RESULT']) -processes = [] - -def request(url, payload=None): - body = None if payload is None else json.dumps(payload).encode() - req = urllib.request.Request(url, data=body, headers={'Content-Type': 'application/json'}) - with urllib.request.urlopen(req, timeout=60) as response: - data = response.read() - return data - -def stop(*_): - raise SystemExit(1) - -signal.signal(signal.SIGTERM, stop) -signal.signal(signal.SIGINT, stop) -try: - from sglang.srt.managers.io_struct import GenerateReqInput - (result / 'scoring-api.json').write_text(json.dumps({ - 'sparse_field_present': 'token_ids_logprob_positions' in GenerateReqInput.__dataclass_fields__, - 'selected_mode': 'dense requested IDs', - 'reason': 'Use documented compatible API and validate returned requested-token scores.' - }, indent=2)) - from transformers import AutoTokenizer - tokenizer = AutoTokenizer.from_pretrained(str(campaign / 'models/Qwen3.6-35B-A3B'), trust_remote_code=True) - for gpu, domain in enumerate(('countdown', 'graph_color')): - command = [sys.executable, '-m', 'sglang.launch_server', - '--model-path', str(campaign / 'teachers' / domain), - '--tokenizer-path', str(campaign / 'models/Qwen3.6-35B-A3B'), - '--host', '0.0.0.0', '--port', str(30000 + gpu), '--tp-size', '1', - '--context-length', '2048', '--mem-fraction-static', '0.8', - '--max-running-requests', '128', '--disable-cuda-graph', - '--chunked-prefill-size', '-1', '--disable-radix-cache', - '--prefill-max-requests', '1', '--moe-runner-backend', 'triton', - '--enable-metrics'] - (result / f'{domain}-command.json').write_text(json.dumps(command, indent=2)) - log = open(result / f'{domain}-server.out', 'wb') - processes.append(subprocess.Popen(command, env={**os.environ, 'CUDA_VISIBLE_DEVICES': str(gpu)}, - stdout=log, stderr=subprocess.STDOUT, start_new_session=True)) - deadline = time.monotonic() + 600 - for gpu, domain in enumerate(('countdown', 'graph_color')): - url = f'http://127.0.0.1:{30000 + gpu}' - while True: - if any(p.poll() is not None for p in processes): - raise RuntimeError('A teacher process exited during startup') - try: - request(url + '/health') - break - except Exception: - if time.monotonic() > deadline: - raise TimeoutError('Teacher readiness exceeded ten minutes') - time.sleep(2) - name = 'countdown4' if domain == 'countdown' else 'graph12' - row = json.loads((campaign / f'data/{name}-train.jsonl').open().readline()) - prompt = row['prompt'] - if isinstance(prompt, str): - prompt = [{'role': 'user', 'content': prompt}] - encoded = tokenizer.apply_chat_template(prompt, tokenize=True, add_generation_prompt=True, enable_thinking=False) - ids = list(encoded['input_ids'] if hasattr(encoded, 'keys') else encoded) - assert ids and all(isinstance(token, int) for token in ids), 'Expected one flat token sequence' - # This synthetic suffix checks the scoring API. It is not an accuracy or throughput result. - ids += tokenizer.encode('test', add_special_tokens=False) - selected = sorted(set(ids[-16:])) - payload = {'input_ids': ids, 'sampling_params': {'temperature': 0, 'max_new_tokens': 0}, - 'return_logprob': True, 'logprob_start_len': 0, 'token_ids_logprob': selected} - (result / f'{domain}-probe-request.json').write_text(json.dumps(payload)) - timings = [] - for attempt in range(2): - start = time.monotonic() - raw = request(url + '/generate', payload) - timings.append(time.monotonic() - start) - response = json.loads(raw) - (result / f'{domain}-probe-response-{attempt}.json').write_bytes(raw) - meta = response['meta_info'] - assert meta.get('completion_tokens', 0) == 0, meta - scores = meta['input_token_ids_logprobs'][-1] - mapping = {int(x[1]): float(x[0]) for x in scores} - assert all(t in mapping and math.isfinite(mapping[t]) for t in selected), scores - (result / f'{domain}-probe-timing.json').write_text(json.dumps({ - 'input_tokens': len(ids), 'selected_ids': len(selected), 'seconds': timings, - 'purpose': 'API smoke test; synthetic suffix; not representative throughput' - }, indent=2)) - sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) - sock.connect(('8.8.8.8', 80)) - ip = sock.getsockname()[0] - sock.close() - (result / 'teachers-ready.json').write_text(json.dumps({'ip': ip, 'ports': [30000, 30001]})) - while True: - if any(p.poll() is not None for p in processes): - raise RuntimeError('A frozen teacher exited; stop the student allocation') - time.sleep(2) -finally: - for p in processes: - try: - os.killpg(p.pid, signal.SIGTERM) - except ProcessLookupError: - pass - for p in processes: - try: - p.wait(timeout=15) - except subprocess.TimeoutExpired: - os.killpg(p.pid, signal.SIGKILL) diff --git a/frameworks/miles/campaign/teacher-provenance.json b/frameworks/miles/campaign/teacher-provenance.json deleted file mode 100644 index f7068fc..0000000 --- a/frameworks/miles/campaign/teacher-provenance.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "countdown": { - "repo": "semianalysisai/Qwen3.6-35B-A3B-countdown-GRPO-20260909", - "revision": "237a7f0345e883705026acd7f0745a3637042f73" - }, - "graph_color": { - "repo": "semianalysisai/Qwen3.6-35B-A3B-graph-color-GRPO-20260909", - "revision": "ec87f87177a4ced7256928ce67c438fafa73c28e" - } -} diff --git a/frameworks/miles/campaign/test-async-contract.py b/frameworks/miles/campaign/test-async-contract.py deleted file mode 100644 index ca882fc..0000000 --- a/frameworks/miles/campaign/test-async-contract.py +++ /dev/null @@ -1,89 +0,0 @@ -"""Execute the changed pure functions without importing the GPU runtime.""" -import ast -import dataclasses -import shlex -import types -import typing -import unittest -from pathlib import Path - -root = Path(__file__).parent / 'source' - - -def extract(path, names, namespace): - tree = ast.parse(path.read_text()) - selected = [node for node in tree.body if getattr(node, 'name', None) in names] - assert len(selected) == len(names) - tree.body = [ast.ImportFrom(module='__future__', names=[ast.alias(name='annotations')], level=0), *selected] - exec(compile(ast.fix_missing_locations(tree), str(path), 'exec'), namespace) - return namespace - - -class ContractTests(unittest.TestCase): - def setUp(self): - self.functions = extract(root / 'miles/rollout/generate_utils/generate_endpoint_utils.py', - {'_student_opd_top_k', '_retain_student_opd_scores'}, {}) - self.args = types.SimpleNamespace(use_opd=True, opd_top_k_strategy='only-student', opd_log_prob_top_k=2) - - def test_topk_request_selection(self): - fn = self.functions['_student_opd_top_k'] - self.assertEqual(fn(self.args), 2) - self.args.use_opd = False - self.assertEqual(fn(self.args), 0) - self.args.use_opd = True - self.args.opd_top_k_strategy = 'only-teacher' - self.assertEqual(fn(self.args), 0) - - def test_scores_append_without_replacing_old_behavior_scores(self): - fn = self.functions['_retain_student_opd_scores'] - sample = types.SimpleNamespace(metadata={'domain': 'countdown'}) - rows = [[[-0.3, 12], [-1.4, 13]]] - output = {'meta_info': {'output_token_logprobs': [[-0.3, 12]], 'output_top_logprobs': rows}} - fn(self.args, sample, output) - fn(self.args, sample, output) - self.assertEqual(sample.metadata['opd_student_top_logprobs'], rows + rows) - self.assertEqual(sample.metadata['domain'], 'countdown') - - def test_missing_and_misaligned_scores_fail(self): - fn = self.functions['_retain_student_opd_scores'] - for rows in (None, [], [[[-0.3, 12]]]): - sample = types.SimpleNamespace(metadata={}) - output = {'meta_info': {'output_token_logprobs': [[-0.3, 12]], 'output_top_logprobs': rows}} - with self.assertRaises(ValueError): - fn(self.args, sample, output) - - def test_empty_aborted_response_is_valid(self): - sample = types.SimpleNamespace(metadata={}) - self.functions['_retain_student_opd_scores'](self.args, sample, {'meta_info': {}}) - self.assertEqual(sample.metadata, {}) - - def test_async_launcher_contract(self): - namespace = {'dataclass': dataclasses.dataclass, 'Literal': typing.Literal, 'shlex': shlex, - 'U': types.SimpleNamespace(ExecuteTrainConfig=object, create_run_id=lambda: 'test')} - extract(root / 'scripts/run_mopd_puzzles.py', {'ScriptArgs', '_training_args'}, namespace) - cls = namespace['ScriptArgs'] - with self.assertRaises(ValueError): - cls(fully_async=True) - args = cls(mode='student', fully_async=True, colocate=False, use_rollout_logprobs=True, - rollout_gpus=6, rollout_gpus_per_engine=2, actor_gpus=8, num_rollout=40) - argv = shlex.split(namespace['_training_args'](args)) - self.assertIn('--fully-async', argv) - self.assertIn('--use-rollout-logprobs', argv) - self.assertNotIn('--colocate', argv) - self.assertNotIn('--rollout-function-path', argv) - self.assertNotIn('--eval-function-path', argv) - for flag, value in {'--rollout-num-gpus': '6', '--rollout-num-gpus-per-engine': '2', - '--sglang-ep-size': '2', '--rollout-batch-size': '128', - '--global-batch-size': '128', '--n-samples-per-prompt': '1', - '--opd-log-prob-top-k': '16', '--num-rollout': '40'}.items(): - self.assertEqual(argv[argv.index(flag) + 1], value) - - def test_endpoint_calls_retention_and_requests_topk(self): - tree = ast.parse((root / 'miles/rollout/generate_utils/generate_endpoint_utils.py').read_text()) - functions = {node.name: node for node in tree.body if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef))} - self.assertIn('_retain_student_opd_scores', ast.unparse(functions['update_sample_from_response'])) - self.assertIn('top_logprobs_num', ast.unparse(functions['compute_request_payload'])) - - -if __name__ == '__main__': - unittest.main() diff --git a/frameworks/miles/campaign/test-balanced-buffer.py b/frameworks/miles/campaign/test-balanced-buffer.py deleted file mode 100644 index 8da514e..0000000 --- a/frameworks/miles/campaign/test-balanced-buffer.py +++ /dev/null @@ -1,135 +0,0 @@ -"""Exercise the real buffer with controlled completion order and policy versions.""" - -import asyncio -import importlib.util -from pathlib import Path -import sys -from types import ModuleType, SimpleNamespace -import unittest - -ROOT = Path(__file__).resolve().parent -sys.path[:0] = [str(ROOT), str(ROOT / "source")] -if importlib.util.find_spec("torch") is None: - # Only the heavy Sample type is stubbed on the laptop; buffer code stays real. - types_stub = ModuleType("miles.utils.types") - types_stub.Sample = type("Sample", (), {"Status": SimpleNamespace(ABORTED="aborted", COMPLETED="completed")}) - sys.modules["miles.utils.types"] = types_stub - -from balanced_async_buffer import BalancedAsyncBuffer -from miles.rollout.fully_async_data_buffer import DataBufferConstructorInput, DataBufferInput -from miles.utils.types import Sample - - -def make_buffer(**overrides): - args = SimpleNamespace(async_data_buffer_capacity_factor=1, rollout_batch_size=4, - n_samples_per_prompt=1, dynamic_sampling_filter_path=None, - rollout_sample_filter_path=None, max_weight_staleness=2) - args.__dict__.update(overrides) - unused = [] - return BalancedAsyncBuffer(DataBufferConstructorInput(args=args, unused_handler_fn=unused.append)), unused - - -def entry(domain, index=0, version=1, **kwargs): - sample = SimpleNamespace(metadata={"domain": domain}, index=index, oldest_weight_version=version, - response_length=2, loss_mask=None, remove_sample=False, - status=Sample.Status.COMPLETED) - sample.__dict__.update(kwargs) - return DataBufferInput(prompt_group=[sample], group=[sample]) - - -class BufferTests(unittest.IsolatedAsyncioTestCase): - async def test_imbalanced_completion_produces_balanced_batches(self): - buffer, _ = make_buffer(rollout_batch_size=128) - - async def produce(): - for round_id in range(2): - for domain in ("countdown", "graph_color"): - for i in range(64): - await buffer.put(entry(domain, round_id * 1000 + i)) - await asyncio.sleep(0) - - async def consume(): - for _ in range(2): - rows = [await buffer.get(current_version=1) for _ in range(128)] - self.assertEqual(sum(r.group[0].metadata["domain"] == "countdown" for r in rows), 64) - metrics = buffer.get_metrics() - self.assertEqual(metrics["rollout/fully_async/domain_balance/graph_color/accepted_groups"], 64) - - await asyncio.wait_for(asyncio.gather(produce(), consume()), 3) - - async def test_missing_domain_does_not_block_producer(self): - buffer, unused = make_buffer() - for i in range(20): - await asyncio.wait_for(buffer.put(entry("countdown", i)), 0.2) - self.assertEqual(len(buffer._buffer), 2) - self.assertEqual(len(unused), 18) - await buffer.put(entry("graph_color")) - self.assertEqual((await buffer.get(current_version=1)).group[0].metadata["domain"], "countdown") - self.assertEqual((await buffer.get(current_version=1)).group[0].metadata["domain"], "graph_color") - - async def test_full_buffer_backpressure_resumes(self): - buffer, _ = make_buffer() - for domain in ("countdown", "graph_color"): - for _ in range(2): - await buffer.put(entry(domain)) - put = asyncio.create_task(buffer.put(entry("graph_color"))) - await asyncio.sleep(0) - self.assertFalse(put.done()) - await buffer.get(current_version=1) - await asyncio.wait_for(put, 0.2) - self.assertLessEqual(len(buffer._buffer), 4) - - async def test_stale_domain_is_replenished_and_accepted_lag_is_clean(self): - buffer, unused = make_buffer() - await buffer.put(entry("countdown", version=1)) - get = asyncio.create_task(buffer.get(current_version=4)) - await asyncio.sleep(0.01) - self.assertFalse(get.done()) - await buffer.put(entry("countdown", version=2)) - self.assertEqual((await get).group[0].oldest_weight_version, 2) - self.assertEqual(len(unused), 1) - metrics = buffer.get_metrics() - self.assertEqual(metrics["rollout/fully_async/max_staleness"], 2) - self.assertEqual(metrics["rollout/fully_async/stale_groups_filtered"], 1) - - async def test_inactive_removed_and_aborted_samples_are_excluded(self): - buffer, unused = make_buffer() - for kwargs in ({"loss_mask": [0, 0]}, {"remove_sample": True}, - {"response_length": 0}, {"status": Sample.Status.ABORTED}): - await buffer.put(entry("countdown", **kwargs)) - self.assertEqual(len(unused), 4) - self.assertFalse(buffer._buffer) - await buffer.put(entry("countdown", loss_mask=[0, 1])) - await buffer.get(current_version=1) - metrics = buffer.get_metrics() - self.assertEqual(metrics["rollout/fully_async/domain_balance/countdown/accepted_active_tokens"], 1) - - async def test_domain_wait_has_bounded_failure(self): - buffer, _ = make_buffer() - buffer._wait_timeout = 0.02 - with self.assertRaisesRegex(TimeoutError, "countdown"): - await buffer.get(current_version=1) - - async def test_absent_and_future_versions_fail(self): - for version in (None, 3): - buffer, _ = make_buffer() - await buffer.put(entry("countdown", version=version)) - with self.assertRaises(ValueError): - await buffer.get(current_version=1) - - async def test_metrics_reset_and_configuration_guards(self): - buffer, _ = make_buffer() - await buffer.put(entry("countdown")) - await buffer.get(current_version=1) - self.assertEqual(buffer.get_metrics()["rollout/fully_async/domain_balance/countdown/accepted_groups"], 1) - self.assertEqual(buffer.get_metrics()["rollout/fully_async/domain_balance/countdown/accepted_groups"], 0) - for kwargs in ({"n_samples_per_prompt": 2}, {"rollout_batch_size": 3}, - {"rollout_sample_filter_path": "filter"}, {"async_data_buffer_capacity_factor": 0.75}): - with self.assertRaises(ValueError): - make_buffer(**kwargs) - with self.assertRaises(ValueError): - await buffer.put(entry("unknown")) - - -if __name__ == "__main__": - unittest.main() diff --git a/frameworks/miles/campaign/test-balanced-integration.py b/frameworks/miles/campaign/test-balanced-integration.py deleted file mode 100644 index ed4202b..0000000 --- a/frameworks/miles/campaign/test-balanced-integration.py +++ /dev/null @@ -1,44 +0,0 @@ -"""Exercise the native async rollout consumer and the actual OPD balancing guard.""" - -import asyncio -from pathlib import Path -import sys - -sys.path[:0] = [str(Path(__file__).resolve().parent), str(Path(__file__).resolve().parent / "source")] - -from miles.rollout.base_types import RolloutFnTrainInput -from miles.rollout.opd_balance import set_domain_weights -from tests.fast.rollout.test_fully_async_rollout import FakeDataSource, make_args, make_fn, make_group - - -async def test_native_rollout_preserves_active_domains(monkeypatch): - class Source(FakeDataSource): - def get_samples(self, num_samples): - self.next_group_index += 1 - sample = make_group(self.next_group_index, weight_versions=["1"])[0] - sample.metadata = {"domain": "countdown" if self.next_group_index % 2 else "graph_color"} - return [[sample]] - - async def generate(state, group, **kwargs): - await asyncio.sleep(0.001 if group[0].metadata["domain"] == "countdown" else 0.01) - return group - - args = make_args(rollout_batch_size=128, n_samples_per_prompt=1, async_max_concurrent_samples=32, - async_data_buffer_capacity_factor=1, max_weight_staleness=2, - custom_async_data_buffer_path="balanced_async_buffer.BalancedAsyncBuffer", - opd_domain_balance="static", opd_domain_targets=["countdown=0.5", "graph_color=0.5"], - calculate_per_token_loss=False) - fn = make_fn(monkeypatch, args, Source(), generate=generate) - try: - for step in range(3): - output = await asyncio.wait_for(fn(RolloutFnTrainInput(rollout_id=step, weight_version=1)), 10) - samples = [group[0] for group in output.samples] - set_domain_weights(args, samples) - assert len(samples) == 128 - assert sum(s.metadata["domain"] == "countdown" for s in samples) == 64 - assert all(s.opd_loss_weights.sum().item() > 0 for s in samples) - assert output.metrics["rollout/fully_async/domain_balance/graph_color/accepted_groups"] == 64 - finally: - if fn._worker is not None: - fn._worker.cancel() - await asyncio.gather(fn._worker, return_exceptions=True) diff --git a/frameworks/miles/campaign/test-scoring-retry.py b/frameworks/miles/campaign/test-scoring-retry.py deleted file mode 100644 index b4e70ec..0000000 --- a/frameworks/miles/campaign/test-scoring-retry.py +++ /dev/null @@ -1,97 +0,0 @@ -"""Execute the real campaign adapter with controlled teacher responses.""" -import ast -import asyncio -import io -import json -import logging -from pathlib import Path -import time -import types -import unittest - - -class ReadError(Exception): - pass - - -class RemoteProtocolError(Exception): - pass - - -class ConnectError(Exception): - pass - - -class RetryTests(unittest.TestCase): - def run_adapter(self, responses, evaluation=False): - calls = [] - sleeps = [] - stream = io.StringIO() - - async def teacher(args, sample, **kwargs): - calls.append((tuple(sample.tokens), sample.metadata.copy())) - value = responses[len(calls) - 1] - if isinstance(value, BaseException): - raise value - return value - - async def verifier(*args, **kwargs): - return 'verifier' - - async def sleep(seconds): - sleeps.append(seconds) - - namespace = {'teacher_reward': teacher, 'verifier_reward': verifier, '_stream': stream, - 'json': json, 'time': time, 'logger': logging.getLogger('test'), - 'httpx': types.SimpleNamespace(ReadError=ReadError, RemoteProtocolError=RemoteProtocolError, - ConnectError=ConnectError), - 'asyncio': types.SimpleNamespace(sleep=sleep)} - tree = ast.parse(Path(__file__).with_name('campaign_async.py').read_text()) - tree.body = [node for node in tree.body if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef))] - exec(compile(tree, 'campaign_async.py', 'exec'), namespace) - sample = types.SimpleNamespace(tokens=[1, 2, 3], response_length=1, index=11, - metadata={'opd_teacher': 'graph_color', 'mopd_evaluation': evaluation}) - try: - result = asyncio.run(namespace['reward_func'](None, sample)) - except BaseException as exc: - result = exc - return result, calls, sleeps, [json.loads(line) for line in stream.getvalue().splitlines()] - - def test_success_has_no_retry(self): - result, calls, sleeps, rows = self.run_adapter([{'teacher': 'scores'}]) - self.assertEqual(result, {'teacher': 'scores'}) - self.assertEqual(len(calls), 1) - self.assertEqual(sleeps, []) - self.assertTrue(rows[0]['success']) - - def test_read_error_retries_same_tokens_and_teacher(self): - result, calls, sleeps, rows = self.run_adapter([ReadError(), RemoteProtocolError(), 'scores']) - self.assertEqual(result, 'scores') - self.assertEqual(calls, [calls[0]] * 3) - self.assertEqual(sleeps, [0.25, 0.5]) - self.assertEqual([row['success'] for row in rows], [False, False, True]) - self.assertEqual([row['attempt'] for row in rows], [1, 2, 3]) - - def test_exhausted_transport_errors_propagate(self): - result, calls, sleeps, rows = self.run_adapter([ConnectError()] * 3) - self.assertIsInstance(result, ConnectError) - self.assertEqual(len(calls), 3) - self.assertEqual(len(rows), 3) - - def test_invalid_scores_and_cancellation_are_not_retried(self): - for error in (ValueError('bad scores'), asyncio.CancelledError(), asyncio.TimeoutError()): - result, calls, sleeps, rows = self.run_adapter([error]) - self.assertIsInstance(result, type(error)) - self.assertEqual(len(calls), 1) - self.assertEqual(sleeps, []) - self.assertEqual(rows[0]['error_type'], type(error).__name__) - - def test_evaluation_uses_verifier_without_teacher(self): - result, calls, sleeps, rows = self.run_adapter([], evaluation=True) - self.assertEqual(result, 'verifier') - self.assertEqual(calls, []) - self.assertEqual(rows, []) - - -if __name__ == '__main__': - unittest.main() diff --git a/frameworks/miles/campaign/balanced_async_buffer.py b/frameworks/miles/domain_balance.py similarity index 50% rename from frameworks/miles/campaign/balanced_async_buffer.py rename to frameworks/miles/domain_balance.py index 92e758c..fee46ad 100644 --- a/frameworks/miles/campaign/balanced_async_buffer.py +++ b/frameworks/miles/domain_balance.py @@ -1,31 +1,95 @@ -"""Assemble equal-domain async batches without bypassing OPD or policy-lag guards.""" +"""Domain-balanced prompts and batches for Miles, for any `recipe.DOMAINS`. + +BalancedDataSource interleaves the domains of the mixed training file, replacing the pinned +launcher's puzzle-specific source. BalancedAsyncBuffer gives every async batch an equal share of +each domain while keeping the OPD and policy-lag checks. + +Both run in the rollout process and share `PENDING`: per domain, the prompts submitted but not yet +trained on or dropped. The source submits the domain with the fewest pending, so a fast domain +(simple_geometry) cannot fill the engines while the slow one (caesar_cipher) holds the batch. The +buffer keeps every finished sample: past a domain's share of the queue it waits in an overflow list +instead of being evicted, since by then its generation and teacher scoring are already paid for. +""" import asyncio -from collections import Counter +from collections import Counter, defaultdict, deque import time +from miles.rollout.data_source import RolloutDataSourceWithBuffer from miles.rollout.filter_hub.base_types import call_dynamic_filter from miles.rollout.fully_async_data_buffer import DefaultDataBuffer, first_sample, iter_samples from miles.utils.types import Sample +from shared import recipe + +DOMAIN_WAIT_SECONDS = 3600.0 # A 30k-token thinking rollout can take many minutes. +PENDING = Counter() # domain -> submitted prompts not yet trained on or dropped + + +class _InterleavedDataset: + def __init__(self, dataset): + self.dataset = dataset + self._interleave() + + def _interleave(self): + groups = defaultdict(list) + for sample in self.dataset.samples: + groups[sample.metadata['domain']].append(sample) + if set(groups) != set(recipe.DOMAINS) or len({len(g) for g in groups.values()}) != 1: + raise ValueError(f'Expected equally sized {recipe.DOMAINS} datasets, got {sorted(groups)}') + self.samples = [s for group in zip(*(groups[d] for d in recipe.DOMAINS), strict=True) for s in group] + + def shuffle(self, epoch_id): + self.dataset.shuffle(epoch_id) + self._interleave() + + def __len__(self): + return len(self.samples) + + +class BalancedDataSource(RolloutDataSourceWithBuffer): + def __init__(self, args): + super().__init__(args) + if self.dataset is None or args.n_samples_per_prompt != 1: + raise ValueError('Balanced OPD requires a global dataset and one response per prompt') + self.dataset = _InterleavedDataset(self.dataset) + self._deferred = defaultdict(deque) # Prompts drawn while looking for another domain, in order. + self._turn = 0 + + def get_samples(self, num_samples): + """Each prompt from the domain with the fewest pending (ties rotate), in dataset order per domain.""" + groups = [] + for _ in range(num_samples): + fewest = min(PENDING[d] for d in recipe.DOMAINS) + ties = [d for d in recipe.DOMAINS if PENDING[d] == fewest] + domain = ties[self._turn % len(ties)] + self._turn += 1 + while not self._deferred[domain]: + [group] = super().get_samples(1) + self._deferred[group[0].metadata['domain']].append(group) + groups.append(self._deferred[domain].popleft()) + PENDING[domain] += 1 + return groups class BalancedAsyncBuffer(DefaultDataBuffer): - """Reserve half of the bounded queue for each puzzle domain.""" + """Bounded queue with an equal share reserved for each domain.""" def __init__(self, input): super().__init__(input) args = input.args - self._domains = ("countdown", "graph_color") - if args.n_samples_per_prompt != 1 or args.rollout_batch_size % 2: - raise ValueError("Balanced async MOPD requires one sample per prompt and an even batch") + self._domains = recipe.DOMAINS + count = len(self._domains) + if args.n_samples_per_prompt != 1 or args.rollout_batch_size % count: + raise ValueError("Balanced async MOPD requires one sample per prompt and a batch divisible by the domains") if args.rollout_sample_filter_path: raise ValueError("A filter after batch assembly could invalidate domain balance") - if self._capacity < 2 or self._capacity % 2: - raise ValueError("Balanced async MOPD requires an even buffer capacity of at least two") - self._per_domain_capacity = self._capacity // 2 + if self._capacity < count or self._capacity % count: + raise ValueError("Balanced async MOPD requires a buffer capacity divisible by the domains") + self._per_domain_capacity = self._capacity // count + self._overflow = defaultdict(deque) # Finished samples past a domain's share of the queue. self._next_domain_index = 0 self._window = Counter() - self._wait_timeout = 180.0 + self._wait_timeout = DOMAIN_WAIT_SECONDS def _domain(self, entry): if len(entry.group) != 1 or isinstance(entry.group[0], list): @@ -46,32 +110,37 @@ def _active_tokens(entry): raise ValueError("The response and loss mask lengths differ") return sum(sample.loss_mask) + def _drop(self, domain, entry): + PENDING[domain] -= 1 + self._unused_handler_fn(entry.prompt_group) + async def put(self, input): domain = self._domain(input) if any(s.status == Sample.Status.ABORTED for s in iter_samples(input.group)): self._metric_aborted_groups += 1 - self._unused_handler_fn(input.prompt_group) + self._drop(domain, input) return if self._active_tokens(input) <= 0: self._window[f"{domain}/inactive_groups_filtered"] += 1 - self._unused_handler_fn(input.prompt_group) + self._drop(domain, input) return verdict = call_dynamic_filter(self._dynamic_filter, self._args, input.group) if not verdict.keep: self._metric_gatherer.on_dynamic_filter_drop(reason=verdict.reason) + PENDING[domain] -= 1 return async with self._cond: - # A full queue contains both domains, so the consumer can make progress. + # A full queue holds every domain, so the consumer can always make progress. while len(self._buffer) >= self._capacity: await self._cond.wait() - positions = [i for i, entry in enumerate(self._buffer) if self._domain(entry) == domain] - if len(positions) >= self._per_domain_capacity: - # Blocking here would stop the serial producer from adding the missing domain. - evicted = self._buffer.pop(positions[0]) - self._window[f"{domain}/overflow_groups_evicted"] += 1 - self._unused_handler_fn(evicted.prompt_group) - self._buffer.append(input) + if sum(self._domain(e) == domain for e in self._buffer) >= self._per_domain_capacity: + # Not blocking: the producer is serial, and blocking would stop it from ever adding + # the missing domain. The sample joins the queue when its domain's share frees up. + self._overflow[domain].append(input) + self._window[f"{domain}/overflow_groups"] += 1 + else: + self._buffer.append(input) self._cond.notify_all() async def get(self, current_version=None, **_): @@ -93,6 +162,8 @@ async def get(self, current_version=None, **_): raise TimeoutError(f"No eligible {domain} sample arrived within {self._wait_timeout}s") from exc continue entry = self._buffer.pop(position) + if self._overflow[domain]: + self._buffer.append(self._overflow[domain].popleft()) self._cond.notify_all() lag = self._staleness(entry.group, current_version) if lag is None or lag < 0: @@ -100,13 +171,14 @@ async def get(self, current_version=None, **_): if self._args.max_weight_staleness is not None and lag > self._args.max_weight_staleness: self._metric_stale_groups += 1 self._window[f"{domain}/stale_groups_filtered"] += 1 - self._unused_handler_fn(entry.prompt_group) + self._drop(domain, entry) continue active = self._active_tokens(entry) if active <= 0: self._window[f"{domain}/inactive_groups_filtered"] += 1 - self._unused_handler_fn(entry.prompt_group) + self._drop(domain, entry) continue + PENDING[domain] -= 1 self._metric_consumed_staleness.append(lag) self._window[f"{domain}/accepted_groups"] += 1 self._window[f"{domain}/accepted_active_tokens"] += active @@ -119,8 +191,10 @@ def get_metrics(self): prefix = "rollout/fully_async/domain_balance/" for domain in self._domains: metrics[f"{prefix}{domain}/queued_groups"] = sum(self._domain(e) == domain for e in self._buffer) + metrics[f"{prefix}{domain}/overflow_queued_groups"] = len(self._overflow[domain]) + metrics[f"{prefix}{domain}/pending_groups"] = PENDING[domain] for key in ("accepted_groups", "accepted_active_tokens", "wait_seconds", "inactive_groups_filtered", - "stale_groups_filtered", "overflow_groups_evicted"): + "stale_groups_filtered", "overflow_groups"): metrics[f"{prefix}{domain}/{key}"] = self._window[f"{domain}/{key}"] self._window.clear() return metrics diff --git a/frameworks/miles/node.py b/frameworks/miles/node.py new file mode 100644 index 0000000..dba5110 --- /dev/null +++ b/frameworks/miles/node.py @@ -0,0 +1,92 @@ +"""Miles entry point: `node.py` runs the controller, `node.py ROLE` runs one container role. + +The two-node runtime lives in `shared/container.py`. This file adds the Miles recipe: a fully +async, sampled-token MOPD launch of the pinned upstream script. +""" +import json +import os +import shlex +import sys + +from shared import recipe as r +from shared.container import ContainerNode, entry +from shared.slurm import DATA, ROOT, eval_config + + +class MilesNode(ContainerNode): + placement_module = 'miles.ray.placement_group' + # The upstream launcher joins the Ray cluster this role started instead of its own. + env = {'MILES_SCRIPT_EXTERNAL_RAY': '1'} + + def placement(self, group): + return [int(x) for x in group.pg_reordered_gpu_ids], group.pg + + def train_command(self, teacher_urls): + (self.result / 'eval.json').write_text(json.dumps(eval_config({'mopd_evaluation': True}), indent=2)) + dev = [x for d in r.DOMAINS for x in (d, DATA / r.data_file(d, 'dev'))] + # Appended after the launcher's own arguments, so these win. The first lines replace the + # launcher's puzzle-specific tasks, thinking setting and 2k context. + tp = ['--tensor-model-parallel-size', r.TRAINER_TENSOR_PARALLEL, '--sequence-parallel'] \ + if r.TRAINER_TENSOR_PARALLEL > 1 else [] + extra = ['--prompt-data', DATA / r.TRAIN_FILE, '--data-source-path', 'domain_balance.BalancedDataSource', + '--opd-domain-targets', *(f'{d}={1 / len(r.DOMAINS)}' for d in r.DOMAINS), + '--eval-prompt-data', *dev, '--eval-config', self.result / 'eval.json', '--skip-eval-before-train', + '--apply-chat-template-kwargs', json.dumps({'enable_thinking': r.ENABLE_THINKING}), + '--sglang-context-length', r.CONTEXT_LENGTH, '--context-parallel-size', r.TRAINER_CONTEXT_PARALLEL, + '--log-probs-chunk-size', r.LOG_PROBS_CHUNK_SIZE, + # Match the Slime and Prime-RL optimizer (the launcher uses wd 0.1, beta2 0.98). + '--weight-decay', '0', '--adam-beta2', '0.999', + # The MTP head is not trained (loss scale 0); dropping it frees memory at 32k. + '--mtp-num-layers', '0', + *tp, '--sglang-moe-runner-backend', 'triton', '--sglang-enable-metrics', + # The engines are dedicated (not colocated), so they can take more memory for the KV cache + # than the launcher's 0.65. + '--sglang-mem-fraction-static', '0.8', + '--custom-async-data-buffer-path', 'domain_balance.BalancedAsyncBuffer', + '--custom-rm-path', 'opd_reward.reward_func', '--custom-rollout-log-function-path', 'opd_reward.log_rollout', + # In flight: 1.25 batches. At policy lag 1 a sample must finish within about two updates of + # its start, so a deeper pipeline mostly generates samples that are dropped as stale. + '--async-max-concurrent-samples', '160', '--async-data-buffer-capacity-factor', '1', + '--max-weight-staleness', r.MAX_POLICY_LAG, '--update-weights-interval', '1', + '--save-debug-event-data', self.result / 'events', + # Save each checkpoint in HF format too, so it can be benchmarked without conversion. + '--save-hf', ROOT / 'checkpoints' / self.result.name / 'hf' / 'iter_{rollout_id}', + '--use-tensorboard', '--tb-project-name', self.result / 'tensorboard', + '--tb-experiment-name', 'async-mopd', '--no-offload-train', '--no-offload-rollout'] + worker_env = {'CAMPAIGN_RESULT': str(self.result), 'CAMPAIGN_ROOT': str(ROOT), + 'CAMPAIGN_PYDEPS': os.environ['CAMPAIGN_PYDEPS'], + 'TENSORBOARD_DIR': str(self.result / 'tensorboard'), + 'PYTHONPATH': os.environ['PYTHONPATH'], 'NCCL_DEBUG': 'WARN'} + if r.TRAINER_OPTIMIZER_CPU_OFFLOAD: + # Ray gives each actor OMP_NUM_THREADS=1, which runs the CPU Adam step on one core per rank + # (about 60 s per update on H100, GPUs idle). Give every trainer rank its share of the node. + worker_env['OMP_NUM_THREADS'] = str(r.CPUS_PER_NODE // r.TRAINER_GPUS) + # No --lr: the launcher's default already equals the recipe's learning rate. + command = ['scripts/run_mopd_puzzles.py', '--mode', 'student', '--num-nodes', 1 + r.GENERATION_NODES, + '--fully-async', '--no-colocate', '--use-rollout-logprobs', '--actor-gpus', r.TRAINER_GPUS, + '--megatron-path', self.site['megatron_source'], '--rollout-gpus', r.ROLLOUT_GPUS, + '--rollout-gpus-per-engine', r.POLICY_GPUS_PER_ENGINE, '--model-dir', ROOT / 'models', + '--data-dir', DATA, '--checkpoint-dir', ROOT / 'checkpoints' / self.result.name, + '--teacher-urls', ' '.join(f'{domain}={url}' for domain, url in teacher_urls.items()), + # Sampled-token OPD: top-k candidate scoring is quadratic in response length. + '--candidate-top-k', '0', '--loss-mode', 'legacy', '--reward-refresh', + '--domain-balance', 'static', '--num-rollout', r.UPDATES, '--rollout-batch-size', r.PROMPTS_PER_UPDATE, + '--n-samples-per-prompt', r.SAMPLES_PER_PROMPT, '--global-batch-size', r.PROMPTS_PER_UPDATE, + '--max-response-len', r.MAX_RESPONSE_TOKENS, + # Miles packs this times the context-parallel size into each micro-batch. + '--max-tokens-per-gpu', r.MAX_TOKENS_PER_GPU // r.TRAINER_CONTEXT_PARALLEL, + # A draft inside the thinking must not end the rollout. + '--no-stop-at-answer', + # This timeout also bounds policy generations, which take minutes at 30k tokens. + '--teacher-timeout-seconds', '7200', + # Greedy dev evaluation pauses generation and training, about 400 s each at 30k tokens, so it + # runs only after the last update; tools/benchmark.py evaluates every saved checkpoint offline. + '--eval-interval', r.UPDATES, '--save-interval', r.SAVE_INTERVAL, + '--no-cleanup-processes', '--no-sparse-scoring', '--extra-env-vars', json.dumps(worker_env), + *(['--optimizer-cpu-offload'] if r.TRAINER_OPTIMIZER_CPU_OFFLOAD else []), + '--extra-args', shlex.join(map(str, extra))] + return [sys.executable, *command], {} + + +if __name__ == '__main__': + sys.exit(entry('miles', MilesNode)) diff --git a/frameworks/miles/opd_reward.py b/frameworks/miles/opd_reward.py new file mode 100644 index 0000000..36b430c --- /dev/null +++ b/frameworks/miles/opd_reward.py @@ -0,0 +1,94 @@ +"""Miles reward hook: score with the routed frozen teacher, retrying transient transport errors. + +Evaluation samples are scored by the shared task verifier instead. Training samples also store the +verifier score in `metadata["raw_reward"]`, so Miles logs it as `rollout/raw_reward` instead of the +teacher reward's constant 0. The OPD loss ignores it. Each teacher attempt is appended to +`teacher-latency-.jsonl`. `log_rollout` appends each training batch's verifier score per task to +`train-reward.jsonl` and one row per trained sample (task, score, lengths, status) to `train-samples.jsonl`. +""" +import asyncio +import json +import logging +import os +from pathlib import Path +import time + +import httpx + +from miles.rollout.on_policy_distillation import reward_func as teacher_reward +from shared.scoring import score + +ATTEMPTS = 3 +FIRST_RETRY_SECONDS = 0.25 +_stream = None +logger = logging.getLogger(__name__) + + +def _record_attempt(sample, attempt, started, total_started, success, error): + global _stream + if _stream is None: + path = Path(os.environ['CAMPAIGN_RESULT']) / f'teacher-latency-{os.getpid()}.jsonl' + _stream = path.open('a', buffering=8192) + _stream.write(json.dumps({'time_unix': time.time(), 'seconds': time.monotonic() - started, + 'total_seconds': time.monotonic() - total_started, + 'success': success, 'attempt': attempt, 'error_type': error, + 'sample_index': sample.index, 'teacher': sample.metadata.get('opd_teacher'), + 'response_tokens': sample.response_length, + 'total_tokens': len(sample.tokens)}) + '\n') + if not success: + _stream.flush() + + +async def reward_func(args, sample, **kwargs): + if sample.metadata.get('mopd_evaluation', False): + return score(sample.response, sample.label) + sample.metadata['raw_reward'] = score(sample.response, sample.label) + total_started = time.monotonic() + for attempt in range(1, ATTEMPTS + 1): + started = time.monotonic() + success = False + error = None + try: + result = await teacher_reward(args, sample, **kwargs) + success = True + return result + except (httpx.ReadError, httpx.RemoteProtocolError, httpx.ConnectError) as exc: + error = type(exc).__name__ + if attempt == ATTEMPTS: + raise + logger.warning('Frozen-teacher scoring retry: sample=%s teacher=%s attempt=%s error=%s', + sample.index, sample.metadata.get('opd_teacher'), attempt, error) + except BaseException as exc: + error = type(exc).__name__ + raise + finally: + _record_attempt(sample, attempt, started, total_started, success, error) + await asyncio.sleep(FIRST_RETRY_SECONDS * 2 ** (attempt - 1)) + + +def log_rollout(rollout_id, args, samples, rollout_extra_metrics, rollout_time): + """Miles' custom rollout log hook: record the training batch's verifier scores and lengths. + + Returns False so Miles still logs its own rollout metrics. + """ + result = Path(os.environ['CAMPAIGN_RESULT']) + with (result / 'train-samples.jsonl').open('a') as f: + for sample in samples: + f.write(json.dumps({'rollout_id': rollout_id, 'index': sample.index, 'domain': sample.metadata.get('domain'), + 'teacher': sample.metadata.get('opd_teacher'), 'score': sample.metadata.get('raw_reward'), + 'prompt_tokens': len(sample.tokens) - sample.response_length, + 'response_tokens': sample.response_length, + 'status': getattr(sample.status, 'value', str(sample.status))}) + '\n') + by_domain, unscored = {}, 0 + for sample in samples: + value = sample.metadata.get('raw_reward') + if value is None: # logging must never stop training + unscored += 1 + continue + by_domain.setdefault(sample.metadata.get('domain'), []).append(value) + record = {'rollout_id': rollout_id, 'update': rollout_id + 1, 'unscored': unscored, + 'domains': {d: {'samples': len(v), 'score': sum(v) / len(v)} + for d, v in sorted(by_domain.items(), key=lambda kv: str(kv[0]))}} + with (result / 'train-reward.jsonl').open('a') as f: + f.write(json.dumps(record) + '\n') + return False diff --git a/frameworks/miles/UPSTREAM-LICENSE b/frameworks/miles/upstream/LICENSE similarity index 100% rename from frameworks/miles/UPSTREAM-LICENSE rename to frameworks/miles/upstream/LICENSE diff --git a/frameworks/miles/manifest.json b/frameworks/miles/upstream/manifest.json similarity index 100% rename from frameworks/miles/manifest.json rename to frameworks/miles/upstream/manifest.json diff --git a/frameworks/miles/source.patch b/frameworks/miles/upstream/source.patch similarity index 100% rename from frameworks/miles/source.patch rename to frameworks/miles/upstream/source.patch diff --git a/frameworks/nemo-rl/make_config.py b/frameworks/nemo-rl/make_config.py new file mode 100644 index 0000000..abe24e4 --- /dev/null +++ b/frameworks/nemo-rl/make_config.py @@ -0,0 +1,125 @@ +"""NeMo-RL recipe: MOPD on async GRPO with NeMo Gym rollouts, built from `shared.recipe`. + +Usage: `make_config.py RESULT_DIR` (called by `node.py`). Writes `nemo-gym-data/{train,validation}.jsonl` +and `nemo-rl.yaml` (JSON, which is valid YAML) into the result directory. + +The config extends upstream's base GRPO config. Algorithm settings come from the MOPD reference recipe, +`examples/configs/recipes/llm/mopd-qwen3-1.7b-3n8g-megatron-pack.yaml`; Qwen3.5 settings come from +`grpo-qwen3.5-35ba3b-2n8g-megatron-ep16tp2cp2.yaml`, except its eager vLLM (see `vllm_cfg`). MOPD gives the +teacher and vLLM generation a whole node each, so there are three nodes: trainer, vLLM and teacher. +""" +import json +from pathlib import Path +import sys + +from shared import recipe +from shared.slurm import DATA, ROOT, site + +UPSTREAM = Path('/opt/nemo-rl') # The NGC container's copy of the pinned release. +AGENT = 'reasoning_gym_simple_agent' # NeMo Gym's reasoning_gym agent (resources_servers/reasoning_gym). + + +def gym_rows(name): + """Convert packaged rows to NeMo Gym's reasoning_gym format, keeping the chat messages as is.""" + for line in (DATA / name).read_text().splitlines(): + row = json.loads(line) + entry = json.loads(row['label']) + yield json.dumps({'responses_create_params': {'input': row['prompt']}, 'question': entry['question'], + 'answer': entry['answer'], 'metadata': entry['metadata'], + 'agent_ref': {'type': 'responses_api_agents', 'name': AGENT}}) + + +def recipe_input(result, data, s): + r = recipe + return { + 'defaults': str(UPSTREAM / 'examples/configs/grpo_math_1B.yaml'), + 'grpo': { + 'num_prompts_per_step': r.PROMPTS_PER_UPDATE, 'num_generations_per_prompt': r.SAMPLES_PER_PROMPT, + 'max_num_steps': r.UPDATES, 'val_period': r.EVAL_INTERVAL, 'val_at_start': True, + 'max_val_samples': None, 'val_batch_size': r.EVAL_EXAMPLES[r.DOMAINS[0]], 'overlong_filtering': True, + 'async_grpo': {'enabled': True, 'max_trajectory_age_steps': r.MAX_POLICY_LAG}, + 'adv_estimator': {'name': 'opd'}, 'seq_logprob_error_threshold': 2.0, + }, + # As in the MOPD reference recipe: REINFORCE form with an ICE-POP importance-sampling gate. + 'loss_fn': { + 'reference_policy_kl_penalty': 0.0, 'kl_input_clamp_value': None, 'kl_output_clamp_value': None, + 'ratio_clip_max': 0.28, 'use_on_policy_kl_approximation': True, 'disable_ppo_ratio': True, + 'use_importance_sampling_correction': True, 'truncated_importance_sampling_ratio': 5.0, + 'truncated_importance_sampling_ratio_min': 0.2, 'truncated_importance_sampling_type': 'icepop', + }, + 'checkpointing': { + 'enabled': True, 'checkpoint_dir': str(ROOT / 'checkpoints' / result.name), 'save_period': r.SAVE_INTERVAL, + 'keep_top_k': 1, 'metric_name': None, 'save_optimizer': False, + }, + 'policy': { + 'model_name': s['base_model'], 'tokenizer': {'name': s['base_model'], + 'chat_template_kwargs': {'enable_thinking': r.ENABLE_THINKING}}, + 'train_global_batch_size': r.PROMPTS_PER_UPDATE * r.SAMPLES_PER_PROMPT, 'train_micro_batch_size': 1, + 'logprob_batch_size': 1, 'max_total_sequence_length': r.CONTEXT_LENGTH, + 'logprob_chunk_size': r.LOG_PROBS_CHUNK_SIZE, 'max_grad_norm': 1.0, 'make_sequence_length_divisible_by': 8, + 'dtensor_cfg': {'enabled': False}, + 'megatron_cfg': { + 'enabled': True, 'tensor_model_parallel_size': 1, 'pipeline_model_parallel_size': 1, + 'context_parallel_size': 1, 'expert_model_parallel_size': r.TRAINER_GPUS, 'sequence_parallel': False, + 'activation_checkpointing': True, 'bias_activation_fusion': False, 'apply_rope_fusion': False, + 'defer_fp32_logits': True, + 'optimizer': {'lr': r.LEARNING_RATE, 'min_lr': r.LEARNING_RATE, 'weight_decay': 0.0, + 'adam_beta1': 0.9, 'adam_beta2': 0.999}, + 'scheduler': {'lr_decay_style': 'constant', 'lr_decay_iters': None, 'lr_warmup_iters': 0, + 'lr_warmup_init': r.LEARNING_RATE}, + 'distributed_data_parallel_config': {'average_in_collective': False}, + }, + 'optimizer': None, 'scheduler': None, + 'generation': { + 'max_new_tokens': r.MAX_RESPONSE_TOKENS, 'temperature': 1.0, 'top_p': 1.0, + 'vllm_cfg': { + # TP1: with TP2 the first trainer-to-vLLM weight sync deadlocked twice, with the receive + # never reaching vLLM's Ray TP executor. TP1 avoids that executor, though the sync still hung (see repro/README.md). + 'async_engine': True, 'tensor_parallel_size': 1, + # CUDA graphs stay on (the base config's default). Upstream's Qwen3.5 recipes set enforce_eager, + # but they generate at most 4k tokens; at 30k thinking tokens eager decoding took ~21 min per rollout. + 'gpu_memory_utilization': 0.8, 'max_model_len': r.CONTEXT_LENGTH, 'enforce_eager': False, + 'expose_http_server': True, + 'http_server_serving_chat_kwargs': {'enable_auto_tools': True, 'tool_parser': 'hermes', + 'reasoning_parser': 'qwen3'}, + }, + # With more than one policy node, vLLM takes a whole node (grpo.py requires it). + 'colocated': {'enabled': False, 'resources': {'gpus_per_node': r.GPUS_PER_NODE, 'num_nodes': 1}}, + }, + }, + 'data': { + 'max_input_seq_length': None, + 'train': {'data_path': str(data / 'train.jsonl'), 'dataset_name': 'NemoGymDataset'}, + 'validation': {'data_path': str(data / 'validation.jsonl'), 'dataset_name': 'NemoGymDataset'}, + 'default': {'dataset_name': 'NemoGymDataset', 'env_name': 'nemo_gym', 'prompt_file': None, + 'processor': 'nemo_gym_data_processor'}, + }, + 'env': {'should_use_nemo_gym': True, 'nemo_gym': {'config_paths': [ + 'responses_api_models/vllm_model/configs/vllm_model_for_training.yaml', + 'resources_servers/reasoning_gym/configs/reasoning_gym.yaml']}}, + 'logger': {'log_dir': str(result / 'logs'), 'tensorboard_enabled': True, 'wandb_enabled': False, + 'mlflow_enabled': False, 'monitor_gpus': True}, + 'cluster': {'gpus_per_node': r.GPUS_PER_NODE, 'num_nodes': 3}, + 'on_policy_distillation': { + 'enabled': True, 'teacher_model_by_agent_name': {'default_teacher': f'{s["teachers"]}/{r.DOMAINS[0]}'}, + 'default_teacher_alias': 'default_teacher', 'strict_agent_name_match': False, + 'deduplicate_shared_teacher_checkpoints': True, + 'non_colocated_teachers': {'enabled': True, 'default_teacher_cfg': { + # One GPU, as in the other frameworks; MOPD still reserves the whole node. + 'tensor_model_parallel_size': 1, 'pipeline_model_parallel_size': 1, 'context_parallel_size': 1, + 'num_nodes': 1, 'gpus_per_node': 1, 'precision': 'bf16', 'micro_batch_size': 1}}, + }, + } + + +def main(result): + assert len(recipe.DOMAINS) == 1, 'The NeMo-RL recipe routes every sample to one teacher' + data = result / 'nemo-gym-data' + data.mkdir(exist_ok=True) + (data / 'train.jsonl').write_text('\n'.join(gym_rows(recipe.TRAIN_FILE)) + '\n') + (data / 'validation.jsonl').write_text('\n'.join(gym_rows(recipe.data_file(recipe.DOMAINS[0], 'dev'))) + '\n') + (result / 'nemo-rl.yaml').write_text(json.dumps(recipe_input(result, data, site()), indent=2) + '\n') + + +if __name__ == '__main__': + main(Path(sys.argv[1])) diff --git a/frameworks/nemo-rl/node.py b/frameworks/nemo-rl/node.py new file mode 100644 index 0000000..19d52c9 --- /dev/null +++ b/frameworks/nemo-rl/node.py @@ -0,0 +1,75 @@ +"""NeMo-RL entry point: `node.py` runs the controller inside the Slurm allocation. + +NeMo-RL starts its own Ray cluster with upstream's `ray.sub` and places the trainer, vLLM and teacher +workers itself, so there are no per-role steps. The controller writes the config (`make_config.py`), +starts telemetry capture on every node, runs `ray.sub` with the driver command, and records the +exit code like the other frameworks. +""" +import json +import os +from pathlib import Path +import sys +import time + +from shared import recipe +from shared.slurm import CACHE_ENV, PYDEPS, ROOT, STOP_FILE, Allocation, Supervisor, exit_on_signals, write_json +import make_config + +# NeMo Gym installs reasoning_gym into its server venv at run time; pin it to the version the +# packaged data and verifier use. +CONSTRAINTS = ROOT / 'nemo-gym-constraints.txt' + + +def driver_command(result): + revision = json.loads((ROOT / 'package-provenance.json').read_text())['revision'] + return ' '.join([ + # Refuse to run unless the container holds the pinned revision. + f'[ "$NEMO_RL_COMMIT" = {revision} ] && cd {make_config.UPSTREAM} &&', f'UV_CONSTRAINT={CONSTRAINTS}', 'HF_HUB_OFFLINE=1', + # Unbuffered, so progress lines (including weight syncs) reach the log immediately. + 'PYTHONUNBUFFERED=1', + # The image ships no NeMo Gym server venvs; build them once per campaign rather than per run. + f'NEMO_GYM_VENV_DIR={ROOT / "runtime-cache" / "gym_venvs"}', + 'uv run examples/nemo_gym/run_grpo_nemo_gym.py', f'--config {result / "nemo-rl.yaml"}']) + + +def control(): + allocation = Allocation('nemo-rl') + s, result = allocation.site, allocation.result + allocation.start() + make_config.main(result) + CONSTRAINTS.write_text('reasoning-gym==0.1.25\n') + nodes = [s['generation_node'], s['trainer_node'], s['teacher_node']] + mounts = sorted({str(ROOT), s['assets'], str(Path(s['base_model']).parent)}) + env = {**os.environ, 'CAMPAIGN_RESULT': str(result), 'CAMPAIGN_PYDEPS': str(PYDEPS), 'PYTHONPATH': str(ROOT), + # Slurm sets NVIDIA_VISIBLE_DEVICES=void; Pyxis needs these to expose the GPUs and driver libraries. + 'NVIDIA_VISIBLE_DEVICES': 'all', 'NVIDIA_DRIVER_CAPABILITIES': 'all', + 'CONTAINER': s['nemo_rl_container'], 'MOUNTS': ','.join(f'{m}:{m}' for m in mounts), + 'COMMAND': driver_command(result), 'BASE_LOG_DIR': str(result), 'GPUS_PER_NODE': str(recipe.GPUS_PER_NODE), + # NeMo-RL writes into its image at run time (uv syncs the project, vLLM files are patched in + # place), but ray.sub's srun steps omit --container-writable. This variable is Pyxis' env form of + # that flag. ray.sub's UV_CACHE_DIR_OVERRIDE does not work: it mounts over /root/.cache/uv, which + # the image's venv files are symlinked into, and so empties the venv. + 'PYXIS_CONTAINER_WRITABLE': '1', 'HF_HOME': str(ROOT / 'runtime-cache' / 'hf'), + **CACHE_ENV} + exit_on_signals() + processes = Supervisor(result) + code = 1 + try: + for node in nodes: # Telemetry on each host; NeMo-RL decides which node runs what. + processes.launch(['srun', '--overlap', '-N1', '-n1', '-w', node, sys.executable, '-m', 'shared.capture', + f'node-{node}'], f'capture-{node}', env=env) + ray = processes.launch(['bash', ROOT / 'source' / 'ray.sub'], 'ray-sub', env=env) + while ray.poll() is None: + processes.check(allowed=('ray-sub',)) + time.sleep(10) + code = ray.returncode + finally: + (result / STOP_FILE).touch() + processes.stop_all(grace=30, timeout=20) + (result / 'exit-code.txt').write_text(f'{code}\n') + write_json(result / 'end.json', {'time': time.time(), 'exit_code': code}) + return code + + +if __name__ == '__main__': + sys.exit(control()) diff --git a/frameworks/nemo-rl/repro/README.md b/frameworks/nemo-rl/repro/README.md new file mode 100644 index 0000000..5c8f8dc --- /dev/null +++ b/frameworks/nemo-rl/repro/README.md @@ -0,0 +1,73 @@ +# Reproducing the NeMo-RL v0.7.0 problems + +Everything here uses the benchmark's production setup (Qwen3.6-35B-A3B, the caesar_cipher teacher, +the shared recipe) or no model at all. The full log of our attempts is in +[`docs/caesar-opd-2026-09-27/nemo-rl-log.md`](../../../docs/caesar-opd-2026-09-27/nemo-rl-log.md). + +## The production run + +```bash +PARTITION=your-partition NEMO_RL_SITE=config/site-nemo-rl.local.json tools/reproduce.sh nemo-rl +``` + +This downloads the pinned models, fuses the teacher, imports `nvcr.io/nvidia/nemo-rl:v0.7.0`, prepares +the campaign with `tools/prepare.py nemo-rl --domains caesar_cipher` and submits it: MOPD on async GRPO +with NeMo Gym rollouts, launched with the container's `ray.sub` on three 8-GPU nodes. The NeMo-RL site +file is the two-node one plus `teacher_node`, `teacher_ip` and `nemo_rl_container`. + +If the first trainer-to-vLLM weight sync has not finished after `DEADLOCK_MIN` minutes (default 20), a +watchdog cancels the job, prints the vLLM workers' weight-update errors and writes them to the run's +`DEADLOCK` file. Without it, the deadlocked job holds 24 GPUs until its time limit. + +## Three container checks (one GPU, about two minutes each) + +These need Slurm with Pyxis/Enroot and network access to NGC, and nothing from this repository +except `repro.sh`. + +```bash +export PARTITION=your-partition WORKDIR=/path/to/nemo-rl-repro +# Optional: export CONTAINER=/path/to/nemo-rl-v0.7.0.sqsh (enroot import of the image, 43 GB) +./repro.sh image +./repro.sh uv-override +./repro.sh readonly +``` + +Results on 8 x B200 nodes (Slurm on Kubernetes, Pyxis, Lustre), 28 September 2026: + +| Check | What it runs | Result | +|---|---|---| +| `image` | Reads the container's own metadata | `NEMO_RL_COMMIT=19244a09`, not the `v0.7.0` tag's `81aa43dd` (the tag only adds docs and a version bump). The venv's files are symlinks into the image's uv cache (`transformers/__init__.py -> /root/.cache/uv/archive-v0/...`). All three Qwen3.5-35B-A3B recipes set `enforce_eager: true`; the base config default is `False`. | +| `uv-override` | Mounts a directory over `/root/.cache/uv`, as `ray.sub`'s documented `UV_CACHE_DIR_OVERRIDE` does, then runs the first import in `nemo_rl/algorithms/grpo.py` | `ImportError: cannot import name 'AutoProcessor' from 'transformers' (unknown location)`: hiding the cache turns every venv package into a dangling symlink. | +| `readonly` | Runs upstream's documented `uv run` in the container the way `ray.sub` starts it (no `--container-writable`, so read-only by Pyxis' default) | `uv` fails with `Read-only file system` at `/root/.cache/uv`. NeMo-RL also patches vLLM's installed files at run time (`nemo_rl/models/generation/vllm/patches.py`), so the launcher only works with writable containers. The package sets `PYXIS_CONTAINER_WRITABLE=1`. | + +## Why a failed weight update becomes a silent deadlock + +1. vLLM's weight loader rejects the fused MoE expert tensors NeMo-RL sends for Qwen3.6-35B-A3B: + `shard_dim=0 is not a valid data dimension for a 3D tensor (expected 1 or 2)`. Every vLLM worker + logged this in job 1010 (Megatron EP8). +2. NeMo-RL catches the exception on the vLLM side, prints it and returns `False` + (`nemo_rl/models/generation/vllm/vllm_backend.py`, `update_weights_from_collective`, lines 466-470). +3. The driver waits for the trainer's broadcast before it checks vLLM's result + (`nemo_rl/algorithms/grpo.py` lines 1946-1947: `ray.get(futures_train)`, then + `ray.get(futures_inference)`). The receivers have already left the collective, so the broadcast + never completes and the check that would raise never runs. The job holds its GPUs until its time + limit. + +Ray prints the worker error with a `[repeated Nx across cluster]` suffix, which log scans easily miss; +the `tools/reproduce.sh` watchdog prints it explicitly. + +## Our earlier runs + +Three clean starts of our configuration (jobs 1001-1003) got through setup and the pre-training +validation, then stopped in the first weight sync, before the first update (`async_grpo_train` → +`refit_policy_generation`, `grpo.py:3553`), with every GPU idle. This is the same call that deadlocked +in job 1010. Rollouts for the first batch still finished, because NeMo-RL collects them in the background +during that sync. + +py-spy, run from the host (Pyxis containers share the host's process table), showed the trainer ranks +waiting in `stream.synchronize()` inside `packed_broadcast_producer` (`nemo_rl/utils/packed_tensor.py:68`) +and the vLLM engines idle, which is also what engines that had already failed and returned would look +like. The driver output of those runs was buffered (no `PYTHONUNBUFFERED`) and shows no refit lines or +worker errors, so we cannot confirm they hit the `shard_dim` error. All three had vLLM CUDA graphs on, +and the hang was the same with eight single-GPU (TP1) engines. The package now runs the driver +unbuffered, so `tools/reproduce.sh nemo-rl` records the errors. diff --git a/frameworks/nemo-rl/repro/repro.sh b/frameworks/nemo-rl/repro/repro.sh new file mode 100755 index 0000000..fe74879 --- /dev/null +++ b/frameworks/nemo-rl/repro/repro.sh @@ -0,0 +1,59 @@ +#!/usr/bin/env bash +# Three NeMo-RL v0.7.0 container problems, each checked in about two minutes on one GPU with nothing but +# NVIDIA's public container. The production deadlock is reproduced by `tools/reproduce.sh nemo-rl`. +# +# ./repro.sh image container provenance, venv layout, eager-only Qwen3.5 recipes +# ./repro.sh uv-override ray.sub's UV_CACHE_DIR_OVERRIDE empties the image's venv +# ./repro.sh readonly the image cannot start read-only, the way ray.sub runs it +# +# Settings (environment): PARTITION (required), WORKDIR (shared, default $PWD/nemo-rl-repro), +# CONTAINER (default docker://nvcr.io#nvidia/nemo-rl:v0.7.0; a local .sqsh is faster), NODE (optional). +set -euo pipefail + +: "${PARTITION:?Set PARTITION to your Slurm partition}" +WORKDIR=${WORKDIR:-$PWD/nemo-rl-repro} +CONTAINER=${CONTAINER:-docker://nvcr.io#nvidia/nemo-rl:v0.7.0} +TAG_COMMIT=81aa43dda4765b0429cf31dab44441e4e4383911 # git rev-parse v0.7.0 +mkdir -p "$WORKDIR" + +# Slurm often sets NVIDIA_VISIBLE_DEVICES=void for job steps, which gives Pyxis containers no GPUs. +export NVIDIA_VISIBLE_DEVICES=all NVIDIA_DRIVER_CAPABILITIES=all + +one_node() { # one_node [extra srun args...] -- command... + local args=() + while [[ $1 != -- ]]; do args+=("$1"); shift; done; shift + srun -p "$PARTITION" -N1 -n1 ${NODE:+-w $NODE} --gres=gpu:1 --time=00:15:00 --container-image="$CONTAINER" \ + --no-container-mount-home "${args[@]}" bash -c "$*" +} + +case "${1:-}" in +image) + one_node --container-writable -- ' + cd /opt/nemo-rl + echo "EXPECT NEMO_RL_COMMIT = v0.7.0 tag '"$TAG_COMMIT"'" + echo "SAW NEMO_RL_COMMIT = $NEMO_RL_COMMIT" + [ "$NEMO_RL_COMMIT" = '"$TAG_COMMIT"' ] && echo "=> image matches its tag" || echo "=> image is NOT built from its tag" + f=$(python -c "import transformers, os; print(os.path.join(os.path.dirname(transformers.__file__), \"__init__.py\"))") + echo "venv file: $f -> $(readlink "$f" || echo "(regular file)")" + echo "Qwen3.5-35B-A3B recipes and their vLLM enforce_eager:" + grep -H "enforce_eager" examples/configs/recipes/llm/*qwen3.5-35ba3b* || true + grep -n "enforce_eager" examples/configs/grpo_math_1B.yaml' + ;; +uv-override) + # ray.sub: `MOUNTS+=",$UV_CACHE_DIR_OVERRIDE:/root/.cache/uv"`. Same mount, then import what grpo.py imports. + mkdir -p "$WORKDIR/uv-cache" + one_node --container-writable --container-mounts="$WORKDIR/uv-cache:/root/.cache/uv" -- ' + cd /opt/nemo-rl + echo "EXPECT: from transformers import AutoProcessor (first import in nemo_rl/algorithms/grpo.py)" + python -c "from transformers import AutoProcessor; print(\"=> OK\")" 2>&1 | tail -n 1' + ;; +readonly) + # ray.sub starts every container without --container-writable (Pyxis default: read-only). + one_node -- ' + cd /opt/nemo-rl + echo "EXPECT: uv run python -c \"import nemo_rl\" (the launch command upstream documents)" + uv run python -c "import nemo_rl; print(\"=> OK\")" 2>&1 | tail -n 3' + ;; +*) + sed -n 2,10p "$0"; exit 2 ;; +esac diff --git a/frameworks/nemo-rl/upstream/LICENSE b/frameworks/nemo-rl/upstream/LICENSE new file mode 100644 index 0000000..261eeb9 --- /dev/null +++ b/frameworks/nemo-rl/upstream/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/frameworks/nemo-rl/upstream/manifest.json b/frameworks/nemo-rl/upstream/manifest.json new file mode 100644 index 0000000..99d604f --- /dev/null +++ b/frameworks/nemo-rl/upstream/manifest.json @@ -0,0 +1,7 @@ +{ + "repository": "https://github.com/NVIDIA-NeMo/RL.git", + "revision": "19244a0949fb363326f71675223da75e09d14866", + "source_patch_sha256": null, + "container": "nvcr.io/nvidia/nemo-rl:v0.7.0", + "note": "Unmodified upstream, run from the NGC container nvcr.io/nvidia/nemo-rl:v0.7.0. Its /opt/nemo-rl and NEMO_RL_COMMIT are 19244a09 on the r0.7.0 branch, one commit before the v0.7.0 tag (81aa43dd), which only bumps docs and the version string. The recipe follows docs/about/algorithms/mopd.md (MOPD on async GRPO with NeMo Gym rollouts) and examples/configs/recipes/llm/mopd-qwen3-1.7b-3n8g-megatron-pack.yaml." +} diff --git a/frameworks/prime-rl/campaign/control.py b/frameworks/prime-rl/campaign/control.py deleted file mode 100644 index 77be41f..0000000 --- a/frameworks/prime-rl/campaign/control.py +++ /dev/null @@ -1,82 +0,0 @@ -"""Own the Slurm allocation and launch native Prime-RL components on two nodes.""" - -import json -import os -from pathlib import Path -import signal -import socket -import subprocess -import sys -import time - -ROOT = Path(__file__).resolve().parent -UV = '@UV@' - - -def stop(*_): - raise SystemExit(143) - - -def main(): - job = os.environ['SLURM_JOB_ID'] - result = ROOT / 'results' / f'prime-opd-{job}' - result.mkdir(parents=True, exist_ok=False) - nodes = subprocess.check_output(['scontrol', 'show', 'hostnames', os.environ['SLURM_JOB_NODELIST']], text=True).split() - assert nodes == ['@GENERATION_NODE@', '@TRAINER_NODE@'], nodes - generation_ip, learner_ip = '@GENERATION_IP@', '@TRAINER_IP@' - env = {**os.environ, 'PRL_RUN_ID': f'prime-opd-{job}', 'WANDB_MODE': 'disabled', - 'PYTHONPATH': str(ROOT / 'environments/mopd_puzzles'), - 'UV_CACHE_DIR': str(ROOT / 'uv-cache'), 'UV_NO_SYNC': '1'} - (ROOT / 'active-run.json').write_text(json.dumps({'job_id': job, 'result': str(result)})) - (result / 'start.json').write_text(json.dumps({'job_id': job, 'time': time.time(), 'nodes': nodes, - 'learner_ip': learner_ip, 'generation_ip': generation_ip, - 'physical_gpu_roles': {'@TRAINER_NODE@': {'trainer': list(range(8))}, - '@GENERATION_NODE@': {'countdown_teacher': [0], 'graph_color_teacher': [1], 'rollout': list(range(2, 8))}}})) - (result / 'slurm-start.txt').write_bytes(subprocess.check_output(['scontrol', 'show', 'job', job])) - subprocess.run(['tar', '--exclude=.venv', '--exclude=.git', '--exclude=__pycache__', '-czf', str(result / 'launch-source.tar.gz'), - '-C', str(ROOT), 'source', 'data', 'environments', 'make_config.py', 'control.py', 'node.py', - 'telemetry.py', 'preflight.py', 'teacher-provenance.json', 'source-provenance.json'], check=True) - children = [] - code = 1 - signal.signal(signal.SIGTERM, stop) - signal.signal(signal.SIGINT, stop) - try: - with (result / 'config-preflight.out').open('wb') as log: - subprocess.run([UV, 'pip', 'freeze', '--python', str(ROOT / 'source/.venv/bin/python')], - stdout=(result / 'installed-packages.txt').open('wb'), check=True) - subprocess.run([UV, 'run', '--project', str(ROOT / 'source'), '--no-sync', 'python', str(ROOT / 'make_config.py'), - str(result), learner_ip, generation_ip], env=env, stdout=log, stderr=subprocess.STDOUT, check=True, timeout=180) - subprocess.run([UV, 'run', '--project', str(ROOT / 'source'), '--no-sync', 'python', str(ROOT / 'preflight.py')], - env=env, stdout=log, stderr=subprocess.STDOUT, check=True, timeout=180) - for role, node in [('generation', nodes[0]), ('learner', nodes[1])]: - command = ['srun', '--exclusive', '-N1', '-n1', '--gres=gpu:8', '--cpus-per-task=80', '-w', node, - UV, 'run', '--project', str(ROOT / 'source'), '--no-sync', 'python', str(ROOT / 'node.py'), str(result), role] - (result / f'{role}-srun.json').write_text(json.dumps(command)) - children.append(subprocess.Popen(command, env=env, stdout=(result / f'{role}-driver.out').open('wb'), - stderr=subprocess.STDOUT, start_new_session=True)) - while True: - if children[1].poll() is not None: - code = children[1].returncode - break - if children[0].poll() is not None: - raise RuntimeError(f'Generation exited early with code {children[0].returncode}') - time.sleep(5) - finally: - (result / 'STOP').touch() - for process in children: - try: - process.wait(timeout=30) - except subprocess.TimeoutExpired: - os.killpg(process.pid, signal.SIGTERM) - try: - process.wait(timeout=15) - except subprocess.TimeoutExpired: - os.killpg(process.pid, signal.SIGKILL) - process.wait() - (result / 'end.json').write_text(json.dumps({'time': time.time(), 'exit_code': code})) - (result / 'exit-code.txt').write_text(f'{code}\n') - return code - - -if __name__ == '__main__': - sys.exit(main()) diff --git a/frameworks/prime-rl/campaign/environments/mopd_puzzles/README.md b/frameworks/prime-rl/campaign/environments/mopd_puzzles/README.md deleted file mode 100644 index c4ea6c3..0000000 --- a/frameworks/prime-rl/campaign/environments/mopd_puzzles/README.md +++ /dev/null @@ -1,19 +0,0 @@ -# mopd-puzzles - -A v1 verifiers environment, scaffolded with `init`. - -## Develop - -1. Implement `load` and the `@reward` in `mopd_puzzles/taskset.py` (see `environments/`). -2. Install + run: - -```bash -uv pip install -e . # install this package (or register it in your project) -uv run eval mopd-puzzles -n 3 # evaluate a few tasks with the bash harness -``` - -## Layout - -- `mopd_puzzles/taskset.py` — the task (`@reward` scoring + behavior) and the taskset: `load` (data + prompts). - -Tune knobs from the CLI: `--env.taskset.num-tasks 10`, `--model `, `-n`, and `-r`. diff --git a/frameworks/prime-rl/campaign/environments/mopd_puzzles/mopd_puzzles/__init__.py b/frameworks/prime-rl/campaign/environments/mopd_puzzles/mopd_puzzles/__init__.py deleted file mode 100644 index 1bf46c0..0000000 --- a/frameworks/prime-rl/campaign/environments/mopd_puzzles/mopd_puzzles/__init__.py +++ /dev/null @@ -1,3 +0,0 @@ -from mopd_puzzles.taskset import MopdPuzzlesTaskset - -__all__ = ["MopdPuzzlesTaskset"] diff --git a/frameworks/prime-rl/campaign/environments/mopd_puzzles/mopd_puzzles/scoring.py b/frameworks/prime-rl/campaign/environments/mopd_puzzles/mopd_puzzles/scoring.py deleted file mode 100644 index 17b58a6..0000000 --- a/frameworks/prime-rl/campaign/environments/mopd_puzzles/mopd_puzzles/scoring.py +++ /dev/null @@ -1,97 +0,0 @@ -import ast -import json -import re -from collections import Counter -from fractions import Fraction - -SYSTEM_PROMPT = ( - "Solve the puzzle. Output only one ... block. " "Do not include reasoning or explanation." -) - - -def extract_answer(response: str) -> str | None: - if not isinstance(response, str) or len(response) > 20000: - return None - # Miles can retain Qwen's terminal token in decoded text, while the native - # screening API omits it. Normalize that transport difference before scoring. - response = response.strip().removesuffix("<|im_end|>").rstrip() - if "" not in response and "" not in response: - return response.strip() - matches = re.findall(r"(.*?)", response, flags=re.DOTALL) - if len(matches) != 1 or response.count("") != 1 or response.count("") != 1: - return None - return matches[0].strip() - - -def check_countdown(answer: str, numbers: list[int], target: int) -> bool: - if len(answer) > 256 or not re.fullmatch(r"[\d\s()+*/-]+", answer): - return False - used = [] - - def evaluate(node): - if isinstance(node, ast.Constant) and type(node.value) is int and 0 < node.value <= 10000: - used.append(node.value) - return Fraction(node.value) - if not isinstance(node, ast.BinOp) or not isinstance(node.op, (ast.Add, ast.Sub, ast.Mult, ast.Div)): - raise ValueError("Only integer literals and +, -, *, / are allowed") - left, right = evaluate(node.left), evaluate(node.right) - if isinstance(node.op, ast.Add): - value = left + right - elif isinstance(node.op, ast.Sub): - value = left - right - elif isinstance(node.op, ast.Mult): - value = left * right - else: - value = left / right - if abs(value.numerator) > 10**12 or value.denominator > 10**12: - raise ValueError("Intermediate value too large") - return value - - try: - tree = ast.parse(answer, mode="eval") - if sum(1 for _ in ast.walk(tree)) > 64: - return False - value = evaluate(tree.body) - return value == target and Counter(used) == Counter(numbers) - except (SyntaxError, ValueError, ZeroDivisionError, RecursionError): - return False - - -def _unique_object(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("Duplicate JSON key") - result[key] = value - return result - - -def check_graph_color(answer: str, puzzle: dict) -> bool: - if len(answer) > 4096: - return False - try: - colors = json.loads(answer, object_pairs_hook=_unique_object) - if not isinstance(colors, dict) or set(colors) != {str(v) for v in puzzle["vertices"]}: - return False - if any(type(c) is not int or c not in puzzle["color_options"] for c in colors.values()): - return False - return all(colors[str(u)] != colors[str(v)] for u, v in puzzle["edges"]) - except (ValueError, TypeError, RecursionError): - return False - - -def score(response: str, label: dict | str) -> float: - if isinstance(label, str): - label = json.loads(label) - answer = extract_answer(response) - if answer is None: - return 0.0 - if label["domain"] == "countdown": - return float(check_countdown(answer, label["numbers"], label["target"])) - if label["domain"] == "graph_color": - return float(check_graph_color(answer, label["puzzle"])) - raise ValueError(f"Unknown puzzle domain {label['domain']!r}") - - -async def reward_func(args, sample, **kwargs): - return score(sample.response, sample.label) diff --git a/frameworks/prime-rl/campaign/environments/mopd_puzzles/mopd_puzzles/taskset.py b/frameworks/prime-rl/campaign/environments/mopd_puzzles/mopd_puzzles/taskset.py deleted file mode 100644 index 1b7daf8..0000000 --- a/frameworks/prime-rl/campaign/environments/mopd_puzzles/mopd_puzzles/taskset.py +++ /dev/null @@ -1,40 +0,0 @@ -import json -from pathlib import Path - -import verifiers.v1 as vf - -from mopd_puzzles.scoring import score - - -class MopdPuzzlesData(vf.TaskData): - label: str - metadata: dict - - -class MopdPuzzlesTask(vf.Task[MopdPuzzlesData]): - @vf.stop - async def single_turn(self, trace: vf.Trace) -> bool: - return trace.num_turns >= 1 - - @vf.reward(weight=1.0) - async def reward(self, trace: vf.Trace) -> float: - return score(trace.last_reply or '', self.data.label) - - -class MopdPuzzlesConfig(vf.TasksetConfig): - data_path: str - domain: str - - -class MopdPuzzlesTaskset(vf.Taskset[MopdPuzzlesTask, MopdPuzzlesConfig]): - def load(self) -> list[MopdPuzzlesTask]: - tasks = [] - with Path(self.config.data_path).open() as stream: - for i, line in enumerate(stream): - row = json.loads(line) - assert row['metadata']['domain'] == self.config.domain - assert [message['role'] for message in row['prompt']] == ['system', 'user'] - data = MopdPuzzlesData(idx=i, system_prompt=row['prompt'][0]['content'], - prompt=row['prompt'][1]['content'], label=row['label'], metadata=row['metadata']) - tasks.append(MopdPuzzlesTask(data, self.config.task)) - return tasks diff --git a/frameworks/prime-rl/campaign/environments/mopd_puzzles/pyproject.toml b/frameworks/prime-rl/campaign/environments/mopd_puzzles/pyproject.toml deleted file mode 100644 index 3d4ea19..0000000 --- a/frameworks/prime-rl/campaign/environments/mopd_puzzles/pyproject.toml +++ /dev/null @@ -1,13 +0,0 @@ -[project] -name = "mopd-puzzles" -version = "0.1.0" -description = "mopd-puzzles — ." -requires-python = ">=3.11" -dependencies = ["verifiers"] - -[build-system] -requires = ["hatchling"] -build-backend = "hatchling.build" - -[tool.hatch.build.targets.wheel] -packages = ["mopd_puzzles"] diff --git a/frameworks/prime-rl/campaign/make_config.py b/frameworks/prime-rl/campaign/make_config.py deleted file mode 100644 index 9cde4bc..0000000 --- a/frameworks/prime-rl/campaign/make_config.py +++ /dev/null @@ -1,120 +0,0 @@ -"""Resolve native Prime-RL component configs for the allocated two-node layout.""" - -import json -import os -from pathlib import Path -import sys - -from prime_rl.configs.rl import RLConfig -from prime_rl.configs.inference import InferenceConfig -from prime_rl.entrypoints.rl import write_subconfigs -from prime_rl.utils.config import dump_resolved_config -from prime_rl.utils.pathing import create_attempt_dirs - -ROOT = Path(__file__).resolve().parent -BASE = '@BASE_MODEL@' -TEACHERS = '@TEACHERS@' -result, learner_ip, generation_ip = sys.argv[1:] -result = Path(result) - - -def source(domain, split): - filename = ('countdown4' if domain == 'countdown' else 'graph12') + f'-{split}.jsonl' - value = { - 'name': domain, - 'env': { - 'taskset': {'id': 'mopd-puzzles', 'data_path': str(ROOT / 'data' / filename), 'domain': domain}, - 'agent': {'harness': {'id': 'null'}, 'runtime': {'type': 'subprocess'}}, - }, - } - if split == 'train': - port = 8200 if domain == 'countdown' else 8300 - value['ratio'] = 1.0 - value['algo'] = {'type': 'opd', 'teacher': {'name': f'{TEACHERS}/{domain}', - 'base_url': f'http://{generation_ip}:{port}/v1'}} - return value - - -# The shared resolver derives component sizes from the logical GPU budget. -# The controller launches these native components on two physical nodes. -raw = { - 'max_steps': 40, 'seq_len': 2048, 'output_dir': str(result.parent), - 'run': {'name': result.name}, 'dashboard': False, - 'model': {'name': BASE, 'vlm': {'vision_encoder_attr': 'model.visual', - 'language_model_attr': 'model.language_model', 'freeze_vision_encoder': True}}, - 'deployment': {'type': 'single_node', 'gpus_per_node': 16, 'num_train_gpus': 8, 'num_infer_gpus': 6}, - 'weight_broadcast': {'type': 'nccl', 'host': learner_ip, 'port': 29591, 'timeout': 1200}, - 'rollout_transport': {'type': 'zmq', 'host': '127.0.0.1', 'port': 15555}, - 'monitors': {'file': {'chunk_bytes': 16777216, 'compress': True}}, - 'ckpt': {'interval': 20, 'keep_last': 1, 'output_dir': str(ROOT / 'checkpoints' / result.name)}, - 'trainer': { - 'model': {'impl': 'custom', 'ep': 8, 'optim_cpu_offload': False, - 'optimization_dtype': 'bfloat16', 'reduce_dtype': 'bfloat16', - 'ac_offloading': None, 'conversion_dir': BASE}, - 'optim': {'lr': 1e-6, 'weight_decay': 0.0}, - }, - 'orchestrator': { - 'batch_size': 128, 'group_size': 1, 'max_off_policy_steps': 2, - 'concurrency': {'initial_inflight': 128, 'min_inflight': 32, 'max_inflight': 256}, - 'model': {'client': {'base_url': f'http://{generation_ip}:8000/v1', - 'admin_base_url': [f'http://{generation_ip}:{port}/v1' for port in (8100, 8110, 8120)]}}, - 'renderer': {'name': 'qwen3.6', 'enable_thinking': False}, - 'train': {'source': [source(d, 'train') for d in ('countdown', 'graph_color')], - 'sampling': {'temperature': 1.0, 'top_p': 1.0, 'max_completion_tokens': 256}}, - 'eval': {'interval': 10, 'num_examples': 512, 'group_size': 1, - 'sampling': {'temperature': 0.0, 'max_completion_tokens': 256, - 'extra_body': {'chat_template_kwargs': {'enable_thinking': False}}}, - 'source': [source(d, 'dev') for d in ('countdown', 'graph_color')]}, - }, - 'inference': { - 'server': {'host': '0.0.0.0', 'port': 8000}, 'backend_port': 8100, - 'vllm': {'tensor_parallel_size': 2, 'data_parallel_size': 3, - 'data_parallel_size_local': 3, 'api_server_count': 3, - 'gpu_memory_utilization': 0.8, 'max_model_len': 2048, - 'max_num_seqs': 128, 'max_num_batched_tokens': 8192, - 'enable_expert_parallel': False, 'dtype': 'bfloat16'}, - }, -} -raw_json = json.dumps(raw, indent=2) -config = RLConfig.model_validate(raw) -config_dir, log_dir = create_attempt_dirs(result) -(result / 'recipe-input.json').write_text(raw_json) -(config_dir / 'rl.json').write_text(json.dumps(dump_resolved_config(config), indent=2)) -write_subconfigs(config, config_dir) -# The aggregate resolver represents six workers. Each executable engine is an -# independent TP2/DP1 group because vLLM folds MoE DP into TP when EP is off. -policy_replicas = [] -for index, port in enumerate((8100, 8110, 8120)): - replica_raw = dump_resolved_config(config.inference) - replica_raw.update(router=None, server={'host': '0.0.0.0', 'port': port}, - output_dir=str(result / f'policy-{index}')) - replica_raw['vllm'].update(data_parallel_size=1, data_parallel_size_local=1, - api_server_count=1, data_parallel_rpc_port=port + 1000) - replica = InferenceConfig.model_validate(replica_raw) - assert replica.vllm.tensor_parallel_size == 2 - assert replica.vllm.data_parallel_size == 1 - (config_dir / f'policy-{index}.json').write_text(json.dumps(dump_resolved_config(replica), indent=2)) - policy_replicas.append({'name': f'policy-{index}', 'port': port, - 'gpus': [2 + index * 2, 3 + index * 2], 'tp': 2, 'dp': 1}) -(result / 'policy-topology.json').write_text(json.dumps({ - 'aggregate_inference_config_is_resolver_only': True, - 'router_port': 8000, 'replicas': policy_replicas, - 'broadcast_world_size': 6, 'broadcast_rank_offsets': [0, 2, 4]}, indent=2)) -for domain, port in [('countdown', 8200), ('graph_color', 8300)]: - teacher = InferenceConfig.model_validate({ - 'server': {'host': '0.0.0.0', 'port': port}, - 'router': None, - 'output_dir': str(result / f'teacher-{domain}'), - 'vllm': {'model': f'{TEACHERS}/{domain}', 'dtype': 'bfloat16', - 'tensor_parallel_size': 1, 'data_parallel_rpc_port': port + 1000, - 'gpu_memory_utilization': 0.8, 'max_model_len': 2048, - 'max_num_seqs': 128, 'max_num_batched_tokens': 4096, - 'enable_prefix_caching': False, 'enforce_eager': True, 'language_model_only': True}, - }) - (config_dir / f'teacher-{domain}.json').write_text(json.dumps(dump_resolved_config(teacher), indent=2)) -assert config.trainer.weight_broadcast.inference_world_size == 6 -assert config.trainer.rollout_transport.port == config.orchestrator.rollout_transport.port == 15555 -assert config.orchestrator.num_train_workers == 8 -assert all(s.algo.type == 'opd' for s in config.orchestrator.train.source) -(result / 'resolved-paths.json').write_text(json.dumps({'config': str(config_dir), 'logs': str(log_dir)})) -print(json.dumps({'config': str(config_dir), 'logs': str(log_dir), 'native_opd_sources': 2})) diff --git a/frameworks/prime-rl/campaign/node.py b/frameworks/prime-rl/campaign/node.py deleted file mode 100644 index 6aa2eae..0000000 --- a/frameworks/prime-rl/campaign/node.py +++ /dev/null @@ -1,139 +0,0 @@ -"""Supervise one node's owned processes and collect lightweight measurements.""" - -import json -import os -from pathlib import Path -import signal -import socket -import subprocess -import sys -import time -import urllib.request - -from prime_rl.utils.process import DEFAULT_COMMON_ENV_VARS, DEFAULT_INFERENCE_ENV_VARS, DEFAULT_TRAINER_ENV_VARS - -ROOT = Path(__file__).resolve().parent -result, role = Path(sys.argv[1]), sys.argv[2] -paths = json.loads((result / 'resolved-paths.json').read_text()) -config, logs = Path(paths['config']), Path(paths['logs']) -UV = '@UV@' -children = [] -env = {**os.environ, **DEFAULT_COMMON_ENV_VARS, 'WANDB_MODE': 'disabled', - 'PRL_ATTEMPT_CONFIG_DIR': str(config), 'PRL_ATTEMPT_LOG_DIR': str(logs), - 'NCCL_DEBUG': 'WARN', 'TORCHINDUCTOR_CACHE_DIR': str(ROOT / 'runtime-cache' / role / 'inductor'), - 'TRITON_CACHE_DIR': str(ROOT / 'runtime-cache' / role / 'triton')} - - -def launch(command, name, extra=None): - full = [UV, 'run', '--project', str(ROOT / 'source'), '--no-sync', *map(str, command)] - (result / f'{role}-{name}-command.json').write_text(json.dumps(full)) - process = subprocess.Popen(full, env={**env, **(extra or {})}, stdout=(logs / f'{name}.log').open('wb'), - stderr=subprocess.STDOUT, start_new_session=True) - children.append((name, process)) - return process - - -def check(allowed=()): - if (result / 'STOP').exists(): - raise SystemExit(143) - for name, process in children: - if name not in allowed and process.poll() is not None: - raise RuntimeError(f'{name} exited with {process.returncode}') - - -def stop(*_): - raise SystemExit(143) - - -def wait_ready(): - generation_ip = json.loads((result / 'start.json').read_text())['generation_ip'] - deadline = time.monotonic() + 1500 - pending = {8000, 8100, 8110, 8120, 8200, 8300} - while pending: - check() - for port in list(pending): - try: - with urllib.request.urlopen(f'http://{generation_ip}:{port}/health', timeout=2) as response: - if response.status == 200: - pending.remove(port) - except OSError: - pass - if time.monotonic() > deadline: - raise TimeoutError(f'Servers did not become ready: {pending}') - time.sleep(3) - - -signal.signal(signal.SIGTERM, stop) -signal.signal(signal.SIGINT, stop) -code = 1 -try: - # Fail before loading models if a selected campaign port is occupied. - ports = ((8000, 8100, 8110, 8120, 8200, 8300, 9100, 9110, 9120, 9200, 9300, 29000) - if role == 'generation' else (5000, 5001, 5002, 5003, 15555, 15556, 29591)) - for port in ports: - with socket.socket() as probe: - probe.bind(('0.0.0.0', port)) - (result / f'{role}-ports-preflight.json').write_text(json.dumps({'time': time.time(), 'free_ports': ports})) - launch(['python', ROOT / 'telemetry.py', result, role], f'{role}-telemetry') - (result / f'{role}-gpu-before.txt').write_bytes(subprocess.check_output(['nvidia-smi', '-q'])) - if role == 'generation': - for domain, gpu in [('countdown', '0'), ('graph_color', '1')]: - # Separate logs and config directories keep standalone teacher launchers isolated. - teacher_root = result / f'teacher-{domain}' - launch(['inference', '@', config / f'teacher-{domain}.json'], f'teacher-{domain}', - {**DEFAULT_INFERENCE_ENV_VARS, 'CUDA_VISIBLE_DEVICES': gpu, - 'PRL_ATTEMPT_CONFIG_DIR': str(teacher_root / 'configs'), - 'PRL_ATTEMPT_LOG_DIR': str(teacher_root / 'logs')}) - for index in range(3): - policy_root = result / f'policy-{index}' - launch(['inference', '@', config / f'policy-{index}.json'], f'policy-{index}', - {**DEFAULT_INFERENCE_ENV_VARS, - 'CUDA_VISIBLE_DEVICES': f'{2 + index * 2},{3 + index * 2}', - 'PRL_ATTEMPT_CONFIG_DIR': str(policy_root / 'configs'), - 'PRL_ATTEMPT_LOG_DIR': str(policy_root / 'logs')}) - # This is the native router command with three independent engine URLs. - launch(['vllm-router', '--policy', 'power_of_two', '--host', '0.0.0.0', - '--port', '8000', '--worker-urls', - 'http://127.0.0.1:8100', 'http://127.0.0.1:8110', 'http://127.0.0.1:8120', - '--intra-node-data-parallel-size', '1', '--request-id-headers', 'x-session-id', - '--worker-startup-timeout-secs', '4200', '--prometheus-port', '29000'], 'policy-router') - while True: - check() - time.sleep(5) - else: - trainer = launch(['torchrun', '--standalone', '--nproc-per-node=8', - f'--log-dir={logs / "trainer-ranks"}', '--redirect=3', '--tee=3', - '-m', 'prime_rl.trainer.rl.train', '@', config / 'trainer.json'], - 'trainer', {**DEFAULT_TRAINER_ENV_VARS, 'CUDA_VISIBLE_DEVICES': '0,1,2,3,4,5,6,7'}) - for path in sorted((config / 'envs').glob('*/*.json')): - launch(['env-server', '@', path], f'env-{path.parent.name}-{path.stem}') - wait_ready() - (result / 'servers-ready.json').write_text(json.dumps({'time': time.time()})) - orchestrator = launch(['orchestrator', '@', config / 'orchestrator.json'], 'orchestrator') - completed_since = None - while True: - check(allowed=('trainer', 'orchestrator')) - states = [trainer.poll(), orchestrator.poll()] - if any(state not in (None, 0) for state in states): - raise RuntimeError(f'Trainer/orchestrator exited unsuccessfully: {states}') - if states == [0, 0]: - code = 0 - break - if any(state == 0 for state in states): - completed_since = completed_since or time.monotonic() - if time.monotonic() - completed_since > 900: - raise TimeoutError('The remaining training component did not finish within 15 minutes') - time.sleep(5) -finally: - for name, process in reversed(children): - if process.poll() is None: - os.killpg(process.pid, signal.SIGTERM) - for name, process in children: - try: - process.wait(timeout=15) - except subprocess.TimeoutExpired: - os.killpg(process.pid, signal.SIGKILL) - process.wait() - (result / f'{role}-component-exits.json').write_text(json.dumps({name: p.returncode for name, p in children})) - (result / f'{role}-gpu-after.txt').write_bytes(subprocess.check_output(['nvidia-smi', '-q'])) -sys.exit(code) diff --git a/frameworks/prime-rl/campaign/preflight.py b/frameworks/prime-rl/campaign/preflight.py deleted file mode 100644 index 3137e73..0000000 --- a/frameworks/prime-rl/campaign/preflight.py +++ /dev/null @@ -1,70 +0,0 @@ -"""Verify puzzle identities, scoring, and tokenization before GPU training.""" - -import hashlib -import json -from pathlib import Path -import struct - -from transformers import AutoTokenizer -import torch -from prime_rl.configs.trainer import ModelConfig -from prime_rl.trainer.model import get_model -from renderers import Qwen36RendererConfig, create_renderer -from mopd_puzzles.taskset import MopdPuzzlesConfig, MopdPuzzlesTaskset -from mopd_puzzles.scoring import score - -ROOT = Path(__file__).resolve().parent -BASE = '@BASE_MODEL@' -model_config = ModelConfig(name=BASE, impl='custom', attn='flash_attention_4', - optimization_dtype='bfloat16', reduce_dtype='bfloat16', - vlm={'vision_encoder_attr': 'model.visual', 'language_model_attr': 'model.language_model', - 'freeze_vision_encoder': True}) -meta_model = get_model(model_config, device=torch.device('meta')) -assert meta_model.model.visual.config._attn_implementation == 'sdpa' -assert meta_model.model.language_model.config._attn_implementation == 'flash_attention_4' -assert all(parameter.device.type == 'meta' for parameter in meta_model.parameters()) -print(json.dumps({'model_setup': 'passed', 'vision_attention': 'sdpa', 'text_attention': 'flash_attention_4', - 'allocation': 'meta, no model weights loaded'})) -del meta_model -cache = Path(BASE) / 'prime' -assert (cache / '.prime-v1').is_file(), 'The previously completed conversion is required; do not modify the original model cache' -index = json.loads((cache / 'model.safetensors.index.json').read_text())['weight_map'] -for filename in set(index.values()): - shard = cache / filename - with shard.open('rb') as stream: - header_length = struct.unpack('1+2+3+4', {'domain': 'countdown', 'numbers': [1, 2, 3, 4], 'target': 10}) == 1 -assert score('1+2', {'domain': 'countdown', 'numbers': [1, 2, 3, 4], 'target': 10}) == 0 -puzzle = {'domain': 'graph_color', 'puzzle': {'vertices': [0, 1], 'edges': [[0, 1]], 'color_options': [1, 2]}} -assert score('{"0":1,"1":2}', puzzle) == 1 -assert score('{"0":1,"1":1}', puzzle) == 0 -summary = {} -for domain, prefix in [('countdown', 'countdown4'), ('graph_color', 'graph12')]: - for split, count in [('train', 10000), ('dev', 512)]: - path = ROOT / 'data' / f'{prefix}-{split}.jsonl' - taskset = MopdPuzzlesTaskset(MopdPuzzlesConfig(data_path=str(path), domain=domain)) - tasks = taskset.load() - rows = [json.loads(line) for line in path.read_text().splitlines()] - assert len(tasks) == len(rows) == count - for i, (task, row) in enumerate(zip(tasks, rows, strict=True)): - assert task.data.system_prompt == row['prompt'][0]['content'] - assert task.data.prompt == row['prompt'][1]['content'] - assert task.data.label == row['label'] - if i < 20: - native = renderer.render_ids(row['prompt'], add_generation_prompt=True) - expected = tokenizer.apply_chat_template(row['prompt'], tokenize=True, add_generation_prompt=True, enable_thinking=False) - if hasattr(expected, 'input_ids'): - expected = expected.input_ids - assert native == expected, (domain, split, i, native[-12:], expected[-12:]) - assert len(native) + 256 <= 2048 - summary[path.name] = {'rows': count, 'sha256': hashlib.sha256(path.read_bytes()).hexdigest(), 'tokenization_samples': 20} -print(json.dumps({'status': 'passed', 'data': summary, 'scoring_cases': 4}, indent=2)) diff --git a/frameworks/prime-rl/campaign/source-provenance.json b/frameworks/prime-rl/campaign/source-provenance.json deleted file mode 100644 index ba463cf..0000000 --- a/frameworks/prime-rl/campaign/source-provenance.json +++ /dev/null @@ -1,14 +0,0 @@ -{ - "prime_rl": "550beb6f431b44390afde6b705919765bbe33289", - "verifiers": "392d596de8cfbfca0e53ea406276653bd1510d01", - "renderers": "f91c3e7061ce50ea405cdf54fd419a45cb51a152", - "prime_envs": "1f1e050ab0cd273bca39eed5c3e5315e6a8ae9d1", - "pydantic_config": "65b15dffba82d4be19efdaf8b2b9705cc1756be8", - "prime_kernels": "3fb83eb7aa639693876034d92a8df29245372cdf", - "algorithm": "Native Prime-RL per-source OPD with the upstream ref_kl loss", - "modified_algorithm": false, - "local_runtime_change": "Use SDPA for the frozen Qwen3.5-family vision encoder with FA4 text attention; source/src/prime_rl/trainer/model.py is archived per attempt", - "base_revision": "995ad96eacd98c81ed38be0c5b274b04031597b0", - "scoring_source": "The unchanged tasks.py from the recorded Miles async campaign", - "no_publication": true -} diff --git a/frameworks/prime-rl/campaign/teacher-provenance.json b/frameworks/prime-rl/campaign/teacher-provenance.json deleted file mode 100644 index f7068fc..0000000 --- a/frameworks/prime-rl/campaign/teacher-provenance.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "countdown": { - "repo": "semianalysisai/Qwen3.6-35B-A3B-countdown-GRPO-20260909", - "revision": "237a7f0345e883705026acd7f0745a3637042f73" - }, - "graph_color": { - "repo": "semianalysisai/Qwen3.6-35B-A3B-graph-color-GRPO-20260909", - "revision": "ec87f87177a4ced7256928ce67c438fafa73c28e" - } -} diff --git a/frameworks/prime-rl/campaign/telemetry.py b/frameworks/prime-rl/campaign/telemetry.py deleted file mode 100644 index d96d7f7..0000000 --- a/frameworks/prime-rl/campaign/telemetry.py +++ /dev/null @@ -1,49 +0,0 @@ -"""Collect lightweight node and inference measurements without GPU profiling.""" - -import concurrent.futures -import gzip -import json -import os -from pathlib import Path -import signal -import subprocess -import sys -import threading -import time -import urllib.request - -result, role = Path(sys.argv[1]), sys.argv[2] -stop = threading.Event() -signal.signal(signal.SIGTERM, lambda *_: stop.set()) -signal.signal(signal.SIGINT, lambda *_: stop.set()) - - -def prometheus(port): - try: - with urllib.request.urlopen(f'http://127.0.0.1:{port}/metrics', timeout=2) as response: - return response.read().decode() - except OSError as error: - return {'error': str(error)} - - -with gzip.open(result / f'{role}-telemetry.jsonl.gz', 'at') as output: - with concurrent.futures.ThreadPoolExecutor(max_workers=6) as pool: - while not stop.is_set() and not (result / 'STOP').exists(): - started = time.monotonic() - row = {'time_unix': time.time(), 'role': role, 'host_scope': 'whole node'} - command = ['nvidia-smi', '--query-gpu=timestamp,index,uuid,utilization.gpu,utilization.memory,memory.used,memory.total,power.draw,temperature.gpu,clocks.sm,clocks.mem', '--format=csv'] - try: - row['gpu_csv'] = subprocess.check_output(command, text=True, timeout=4) - except (subprocess.SubprocessError, OSError) as error: - row['gpu_error'] = str(error) - for name in ('stat', 'meminfo', 'net/dev', 'diskstats', 'loadavg'): - try: - row[name] = Path('/proc', name).read_text() - except OSError as error: - row[name] = str(error) - if role == 'generation': - ports = (8100, 8110, 8120, 8200, 8300, 29000) - row['prometheus'] = dict(zip(map(str, ports), pool.map(prometheus, ports))) - output.write(json.dumps(row) + '\n') - output.flush() - stop.wait(max(0.0, 5 - (time.monotonic() - started))) diff --git a/frameworks/prime-rl/environments/rg_tasks/README.md b/frameworks/prime-rl/environments/rg_tasks/README.md new file mode 100644 index 0000000..3fe4298 --- /dev/null +++ b/frameworks/prime-rl/environments/rg_tasks/README.md @@ -0,0 +1,14 @@ +# rg-tasks + +A verifiers v1 taskset for the OPD reasoning_gym tasks. It loads the campaign's +`data/-.jsonl` rows (system and user prompt, JSON label) and scores +the final reply with the shared verifier (`rg_tasks/scoring.py` is a symlink to +`shared/scoring.py`). In OPD the reward is only a diagnostic. + +Install it into the prepared campaign's Prime-RL environment: + +```bash +uv pip install --python /source/.venv/bin/python --no-deps -e /environments/rg_tasks +``` + +`reasoning-gym==0.1.25` is provided by `/pydeps` (see docs/SETUP.md). diff --git a/frameworks/prime-rl/environments/rg_tasks/pyproject.toml b/frameworks/prime-rl/environments/rg_tasks/pyproject.toml new file mode 100644 index 0000000..d3e92e5 --- /dev/null +++ b/frameworks/prime-rl/environments/rg_tasks/pyproject.toml @@ -0,0 +1,13 @@ +[project] +name = "rg-tasks" +version = "0.1.0" +description = "The OPD reasoning_gym tasks as a verifiers v1 taskset, scored by the shared verifier." +requires-python = ">=3.11" +dependencies = ["verifiers", "reasoning-gym==0.1.25"] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.hatch.build.targets.wheel] +packages = ["rg_tasks"] diff --git a/frameworks/prime-rl/environments/rg_tasks/rg_tasks/__init__.py b/frameworks/prime-rl/environments/rg_tasks/rg_tasks/__init__.py new file mode 100644 index 0000000..67a2b86 --- /dev/null +++ b/frameworks/prime-rl/environments/rg_tasks/rg_tasks/__init__.py @@ -0,0 +1,3 @@ +from rg_tasks.taskset import RgTasksTaskset + +__all__ = ["RgTasksTaskset"] diff --git a/frameworks/prime-rl/environments/rg_tasks/rg_tasks/scoring.py b/frameworks/prime-rl/environments/rg_tasks/rg_tasks/scoring.py new file mode 120000 index 0000000..925d115 --- /dev/null +++ b/frameworks/prime-rl/environments/rg_tasks/rg_tasks/scoring.py @@ -0,0 +1 @@ +../../../../../shared/scoring.py \ No newline at end of file diff --git a/frameworks/prime-rl/environments/rg_tasks/rg_tasks/taskset.py b/frameworks/prime-rl/environments/rg_tasks/rg_tasks/taskset.py new file mode 100644 index 0000000..64728c5 --- /dev/null +++ b/frameworks/prime-rl/environments/rg_tasks/rg_tasks/taskset.py @@ -0,0 +1,41 @@ +import json +from pathlib import Path + +import verifiers.v1 as vf + +from rg_tasks.scoring import score_reply + + +class RgTasksData(vf.TaskData): + label: str + metadata: dict + + +class RgTasksTask(vf.Task[RgTasksData]): + @vf.stop + async def single_turn(self, trace: vf.Trace) -> bool: + return trace.num_turns >= 1 + + # last_reply excludes the thinking block, so a rollout cut off mid-thought scores 0. + @vf.reward(weight=1.0) + async def reward(self, trace: vf.Trace) -> float: + return score_reply(trace.last_reply or '', self.data.label) + + +class RgTasksConfig(vf.TasksetConfig): + data_path: str + domain: str + + +class RgTasksTaskset(vf.Taskset[RgTasksTask, RgTasksConfig]): + def load(self) -> list[RgTasksTask]: + tasks = [] + with Path(self.config.data_path).open() as stream: + for i, line in enumerate(stream): + row = json.loads(line) + assert row['metadata']['domain'] == self.config.domain + assert [message['role'] for message in row['prompt']] == ['system', 'user'] + data = RgTasksData(idx=i, system_prompt=row['prompt'][0]['content'], + prompt=row['prompt'][1]['content'], label=row['label'], metadata=row['metadata']) + tasks.append(RgTasksTask(data, self.config.task)) + return tasks diff --git a/frameworks/prime-rl/make_config.py b/frameworks/prime-rl/make_config.py new file mode 100644 index 0000000..e4224a4 --- /dev/null +++ b/frameworks/prime-rl/make_config.py @@ -0,0 +1,166 @@ +"""Prime-RL recipe and per-component configs for the two-node layout. + +Usage: `make_config.py RESULT_DIR` (called by `node.py`). `recipe_input` is the experiment in +Prime-RL's schema; `main` validates it and writes one config per component that `node.py` launches. +""" + +import json +from pathlib import Path +import sys + +from shared import recipe +from shared.slurm import DATA, ROOT, site + +# Generation node: three independent TP2 vLLM policy engines behind a router, +# and one single-GPU vLLM server per frozen teacher. +ROUTER_PORT = 8000 +ROUTER_METRICS_PORT = 29000 +POLICY_PORTS = (8100, 8110, 8120) +TEACHER_PORTS = {domain: 8200 + 100 * i for i, domain in enumerate(recipe.DOMAINS)} +RPC_PORT_OFFSET = 1000 # Each vLLM server's data-parallel RPC port is its HTTP port + 1000. +INFERENCE_PORTS = (*POLICY_PORTS, *TEACHER_PORTS.values()) +GENERATION_NODE_PORTS = (ROUTER_PORT, *INFERENCE_PORTS, *(p + RPC_PORT_OFFSET for p in INFERENCE_PORTS), + ROUTER_METRICS_PORT) +# Trainer node. +WEIGHT_BROADCAST_PORT = 29591 +ROLLOUT_TRANSPORT_PORT = 15555 +TRAINER_NODE_PORTS = (5000, 5001, 5002, 5003, ROLLOUT_TRANSPORT_PORT, ROLLOUT_TRANSPORT_PORT + 1, + WEIGHT_BROADCAST_PORT) + + +def policy_gpus(index): + """Physical GPUs of policy engine `index` on the generation node.""" + size = recipe.POLICY_GPUS_PER_ENGINE + return recipe.POLICY_GPUS[index * size:(index + 1) * size] + + +def task_source(domain, split, teachers, generation_ip): + value = { + 'name': domain, + 'env': { + 'taskset': {'id': 'rg-tasks', 'data_path': str(DATA / recipe.data_file(domain, split)), 'domain': domain}, + 'agent': {'harness': {'id': 'null'}, 'runtime': {'type': 'subprocess'}}, + }, + } + if split == 'train': + # Native per-source OPD: each domain is distilled from its own frozen teacher. + value['ratio'] = 1.0 + value['algo'] = {'type': 'opd', 'teacher': { + 'name': f'{teachers}/{domain}', + 'base_url': f'http://{generation_ip}:{TEACHER_PORTS[domain]}/v1'}} + return value + + +def recipe_input(result, s): + """The experiment in Prime-RL's schema; the resolver sizes components from the GPU budget. + + Only the shared recipe, the two-node layout (GPU budget, ports, NCCL weight broadcast, three TP2 + policy engines) and the per-domain OPD teachers are set; everything else is Prime-RL's default. + + The student is trained as a text model (no `model.vlm`; upstream's README lists Qwen3.5 MoE), so the + trainer keeps fp32 master weights and reductions, which `model.vlm` would force to bf16. The frozen + vision encoder then runs in fp32, which FlashAttention 4 rejects; the source patch switches it to + SDPA (see upstream/manifest.json). + """ + generation_ip = s['generation_ip'] + sources = {split: [task_source(d, split, s['teachers'], generation_ip) for d in recipe.DOMAINS] + for split in ('train', 'dev')} + return { + 'max_steps': recipe.UPDATES, 'seq_len': recipe.CONTEXT_LENGTH, 'output_dir': str(result.parent), + 'run': {'name': result.name}, 'dashboard': False, + 'model': {'name': s['base_model']}, + 'deployment': {'type': 'single_node', 'gpus_per_node': 2 * recipe.GPUS_PER_NODE, + 'num_train_gpus': recipe.TRAINER_GPUS, 'num_infer_gpus': len(recipe.POLICY_GPUS)}, + 'weight_broadcast': {'type': 'nccl', 'host': s['trainer_ip'], 'port': WEIGHT_BROADCAST_PORT}, + 'rollout_transport': {'type': 'zmq', 'host': '127.0.0.1', 'port': ROLLOUT_TRANSPORT_PORT}, + 'monitors': {'file': {'chunk_bytes': 16777216, 'compress': True}}, + 'ckpt': {'interval': recipe.SAVE_INTERVAL, 'output_dir': str(ROOT / 'checkpoints' / result.name)}, + 'trainer': {'optim': {'lr': recipe.LEARNING_RATE, 'weight_decay': 0.0}}, + 'orchestrator': { + 'batch_size': recipe.PROMPTS_PER_UPDATE, 'group_size': recipe.SAMPLES_PER_PROMPT, + 'max_off_policy_steps': recipe.MAX_POLICY_LAG, + 'model': {'client': {'base_url': f'http://{generation_ip}:{ROUTER_PORT}/v1', + 'admin_base_url': [f'http://{generation_ip}:{port}/v1' for port in POLICY_PORTS]}}, + 'renderer': {'name': 'qwen3.6', 'enable_thinking': recipe.ENABLE_THINKING}, + 'train': {'source': sources['train'], + 'sampling': {'temperature': 1.0, 'top_p': 1.0, 'max_completion_tokens': recipe.MAX_RESPONSE_TOKENS}}, + # num_examples -1 evaluates every dev row. + 'eval': {'interval': recipe.EVAL_INTERVAL, 'num_examples': -1, 'group_size': 1, + 'sampling': {'temperature': 0.0, 'max_completion_tokens': recipe.MAX_RESPONSE_TOKENS, + 'extra_body': {'chat_template_kwargs': {'enable_thinking': recipe.ENABLE_THINKING}}}, + 'source': sources['dev']}, + }, + 'inference': { + 'server': {'host': '0.0.0.0', 'port': ROUTER_PORT}, 'backend_port': POLICY_PORTS[0], + 'vllm': {'tensor_parallel_size': recipe.POLICY_GPUS_PER_ENGINE, 'data_parallel_size': len(POLICY_PORTS), + 'data_parallel_size_local': len(POLICY_PORTS), 'api_server_count': len(POLICY_PORTS), + 'max_model_len': recipe.CONTEXT_LENGTH}, + }, + } + + +def teacher_input(domain, result, s): + port = TEACHER_PORTS[domain] + return { + 'server': {'host': '0.0.0.0', 'port': port}, + 'router': None, + 'output_dir': str(result / f'teacher-{domain}'), + 'vllm': {'model': f'{s["teachers"]}/{domain}', 'tensor_parallel_size': 1, + 'data_parallel_rpc_port': port + RPC_PORT_OFFSET, 'max_model_len': recipe.CONTEXT_LENGTH, + # As in upstream's OPD example (configs/debug/algo/opd.toml). At the default 0.9 the teacher + # ran out of memory: fp32 prompt logprobs for one 9.8k-token sample need 8.8 GiB outside + # vLLM's profiled budget (job 1016). + 'gpu_memory_utilization': 0.5, 'enforce_eager': True}, + } + + +def main(result): + from prime_rl.configs.inference import InferenceConfig + from prime_rl.configs.rl import RLConfig + from prime_rl.entrypoints.rl import write_subconfigs + from prime_rl.utils.config import dump_resolved_config + from prime_rl.utils.pathing import create_attempt_dirs + + s = site() + raw = recipe_input(result, s) + # Written before validation, which replaces nested env dicts in `raw` with config objects. + (result / 'recipe-input.json').write_text(json.dumps(raw, indent=2)) + config = RLConfig.model_validate(raw) + config_dir, log_dir = create_attempt_dirs(result) + (config_dir / 'rl.json').write_text(json.dumps(dump_resolved_config(config), indent=2)) + write_subconfigs(config, config_dir) + + # The aggregate inference config (six GPUs) only feeds the resolver. Each engine runs as its own + # TP2/DP1 server, because vLLM folds MoE DP into TP when EP is off. + replicas = [] + for index, port in enumerate(POLICY_PORTS): + replica_raw = dump_resolved_config(config.inference) + replica_raw.update(router=None, server={'host': '0.0.0.0', 'port': port}, + output_dir=str(result / f'policy-{index}')) + replica_raw['vllm'].update(data_parallel_size=1, data_parallel_size_local=1, api_server_count=1, + data_parallel_rpc_port=port + RPC_PORT_OFFSET) + replica = InferenceConfig.model_validate(replica_raw) + assert replica.vllm.tensor_parallel_size == recipe.POLICY_GPUS_PER_ENGINE + assert replica.vllm.data_parallel_size == 1 + (config_dir / f'policy-{index}.json').write_text(json.dumps(dump_resolved_config(replica), indent=2)) + replicas.append({'name': f'policy-{index}', 'port': port, 'gpus': list(policy_gpus(index)), + 'tp': recipe.POLICY_GPUS_PER_ENGINE, 'dp': 1}) + (result / 'policy-json').write_text(json.dumps({ + 'aggregate_inference_config_is_resolver_only': True, 'router_port': ROUTER_PORT, + 'replicas': replicas, 'broadcast_world_size': len(recipe.POLICY_GPUS), + 'broadcast_rank_offsets': [i * recipe.POLICY_GPUS_PER_ENGINE for i in range(len(replicas))]}, indent=2)) + + for domain in recipe.DOMAINS: + teacher = InferenceConfig.model_validate(teacher_input(domain, result, s)) + (config_dir / f'teacher-{domain}.json').write_text(json.dumps(dump_resolved_config(teacher), indent=2)) + + assert config.trainer.weight_broadcast.inference_world_size == len(recipe.POLICY_GPUS) + assert config.trainer.rollout_transport.port == config.orchestrator.rollout_transport.port == ROLLOUT_TRANSPORT_PORT + assert config.orchestrator.num_train_workers == recipe.TRAINER_GPUS + assert all(source.algo.type == 'opd' for source in config.orchestrator.train.source) + (result / 'resolved-paths.json').write_text(json.dumps({'config': str(config_dir), 'logs': str(log_dir)})) + print(json.dumps({'config': str(config_dir), 'logs': str(log_dir), 'native_opd_sources': len(recipe.DOMAINS)})) + + +if __name__ == '__main__': + main(Path(sys.argv[1])) diff --git a/frameworks/prime-rl/node.py b/frameworks/prime-rl/node.py new file mode 100644 index 0000000..447eaf3 --- /dev/null +++ b/frameworks/prime-rl/node.py @@ -0,0 +1,148 @@ +"""Prime-RL entry point: `node.py` runs the controller, `node.py ROLE` runs one node. + +Prime-RL runs on the hosts from the campaign's `source/.venv`, not in a container. The generation +node serves the frozen teachers, three policy engines and their router; the trainer node runs the +trainer, env servers and orchestrator. `make_config.py` holds the recipe and ports. +""" + +import json +import os +from pathlib import Path +import socket +import subprocess +import sys +import time +import urllib.request + +from shared import recipe +from shared.slurm import CACHE_ENV, PYDEPS, ROOT, Allocation, Supervisor, exit_on_signals, result_dir, site +import make_config as config + +FINISH_TIMEOUT = 900 # Allowed gap between the trainer and orchestrator finishing. + + +def uv_python(s): + # --no-sync keeps a run from silently replacing the provisioned environment. + return [s['uv'], 'run', '--project', str(ROOT / 'source'), '--no-sync', 'python'] + + +def control(): + allocation = Allocation('prime-rl') + s, result = allocation.site, allocation.result + allocation.start(learner_ip=s['trainer_ip'], generation_ip=s['generation_ip']) + env = {**os.environ, 'CAMPAIGN_RESULT': str(result), 'PRL_RUN_ID': result.name, 'WANDB_MODE': 'disabled', + 'PYTHONPATH': str(ROOT / 'environments/rg_tasks'), 'CAMPAIGN_PYDEPS': str(PYDEPS), + 'UV_CACHE_DIR': str(ROOT / 'uv-cache'), + 'UV_NO_SYNC': '1'} + + def resolve_configs(): + with (result / 'installed-packages.txt').open('wb') as packages: + subprocess.run([s['uv'], 'pip', 'freeze', '--python', str(ROOT / 'source/.venv/bin/python')], + stdout=packages, check=True) + with (result / 'make-config.out').open('wb') as log: + subprocess.run([*uv_python(s), str(ROOT / 'make_config.py'), str(result)], env=env, + stdout=log, stderr=subprocess.STDOUT, check=True, timeout=180) + + srun = ['srun', '--exclusive', '-N1', '-n1', f'--gres=gpu:{recipe.GPUS_PER_NODE}', + f'--cpus-per-task={recipe.CPUS_PER_NODE}'] + steps = {role: [*srun, '-w', s[node], *uv_python(s), str(ROOT / 'node.py'), role] + for role, node in (('generation', 'generation_node'), ('learner', 'trainer_node'))} + return allocation.run(steps, env, before=resolve_configs) + + +class Node: + def __init__(self, role): + from prime_rl.utils.process import DEFAULT_COMMON_ENV_VARS + self.role, self.site, self.result = role, site(), result_dir() + paths = json.loads((self.result / 'resolved-paths.json').read_text()) + self.config, self.logs = Path(paths['config']), Path(paths['logs']) + env = {**os.environ, **DEFAULT_COMMON_ENV_VARS, 'WANDB_MODE': 'disabled', 'NCCL_DEBUG': 'WARN', + 'PRL_ATTEMPT_CONFIG_DIR': str(self.config), 'PRL_ATTEMPT_LOG_DIR': str(self.logs), **CACHE_ENV} + # Both nodes log into one directory, so component names must be unique across nodes. + self.processes = Supervisor(self.result, base_env=env) + + def launch(self, command, name, env=None): + full = [self.site['uv'], 'run', '--project', ROOT / 'source', '--no-sync', *command] + return self.processes.launch(full, name, env, log=self.logs / f'{name}.log') + + def inference(self, name, gpus): + from prime_rl.utils.process import DEFAULT_INFERENCE_ENV_VARS + # Give each standalone inference launcher its own config and log directories. + root = self.result / name + self.launch(['inference', '@', self.config / f'{name}.json'], name, + {**DEFAULT_INFERENCE_ENV_VARS, 'CUDA_VISIBLE_DEVICES': ','.join(map(str, gpus)), + 'PRL_ATTEMPT_CONFIG_DIR': str(root / 'configs'), 'PRL_ATTEMPT_LOG_DIR': str(root / 'logs')}) + + def generation(self): + for domain, gpu in recipe.TEACHER_GPU.items(): + self.inference(f'teacher-{domain}', [gpu]) + for index in range(len(config.POLICY_PORTS)): + self.inference(f'policy-{index}', config.policy_gpus(index)) + # Same arguments as upstream's `start_router` (prime_rl/entrypoints/inference.py), over our three engines. + self.launch(['vllm-router', '--policy', 'consistent_hash', '--host', '0.0.0.0', '--port', config.ROUTER_PORT, + '--worker-urls', *(f'http://127.0.0.1:{port}' for port in config.POLICY_PORTS), + '--intra-node-data-parallel-size', '1', '--request-id-headers', 'x-session-id', + '--worker-startup-timeout-secs', '4200', '--prometheus-port', config.ROUTER_METRICS_PORT], + 'policy-router') + while True: + self.processes.check() + time.sleep(5) + + def learner(self): + from prime_rl.utils.process import DEFAULT_TRAINER_ENV_VARS + trainer = self.launch(['torchrun', '--standalone', f'--nproc-per-node={recipe.TRAINER_GPUS}', + f'--log-dir={self.logs / "trainer-ranks"}', '--redirect=3', '--tee=3', + '-m', 'prime_rl.trainer.rl.train', '@', self.config / 'trainer.json'], 'trainer', + {**DEFAULT_TRAINER_ENV_VARS, + 'CUDA_VISIBLE_DEVICES': ','.join(map(str, range(recipe.TRAINER_GPUS)))}) + for path in sorted((self.config / 'envs').glob('*/*.json')): + self.launch(['env-server', '@', path], f'env-{path.parent.name}-{path.stem}') + self.wait_for_servers() + orchestrator = self.launch(['orchestrator', '@', self.config / 'orchestrator.json'], 'orchestrator') + first_finished = None + while True: + self.processes.check(allowed=('trainer', 'orchestrator')) + states = [trainer.poll(), orchestrator.poll()] + if any(state not in (None, 0) for state in states): + raise RuntimeError(f'Trainer/orchestrator exited unsuccessfully: {states}') + if states == [0, 0]: + return 0 + if 0 in states: + first_finished = first_finished or time.monotonic() + if time.monotonic() - first_finished > FINISH_TIMEOUT: + raise TimeoutError('The remaining training component did not finish in time') + time.sleep(5) + + def wait_for_servers(self, timeout=1500): + pending = {config.ROUTER_PORT, *config.INFERENCE_PORTS} + deadline = time.monotonic() + timeout + while pending: + self.processes.check() + for port in list(pending): + try: + with urllib.request.urlopen(f'http://{self.site["generation_ip"]}:{port}/health', timeout=2): + pending.remove(port) + except OSError: + pass + if time.monotonic() > deadline: + raise TimeoutError(f'Servers did not become ready: {pending}') + time.sleep(3) + + def main(self): + exit_on_signals() + # Fail before loading models if a campaign port is occupied. + for port in config.GENERATION_NODE_PORTS if self.role == 'generation' else config.TRAINER_NODE_PORTS: + with socket.socket() as probe: + probe.bind(('0.0.0.0', port)) + targets = ([f'127.0.0.1:{p}' for p in (*config.INFERENCE_PORTS, config.ROUTER_METRICS_PORT)] + if self.role == 'generation' else []) + self.launch(['python', '-m', 'shared.capture', self.role, *targets], f'{self.role}-capture', + {'PYTHONPATH': f'{ROOT}:{os.environ.get("PYTHONPATH", "")}'}) + try: + return self.generation() if self.role == 'generation' else self.learner() + finally: + self.processes.stop_all(timeout=15) + + +if __name__ == '__main__': + sys.exit(control() if len(sys.argv) == 1 else Node(sys.argv[1]).main()) diff --git a/frameworks/prime-rl/UPSTREAM-LICENSE b/frameworks/prime-rl/upstream/LICENSE similarity index 100% rename from frameworks/prime-rl/UPSTREAM-LICENSE rename to frameworks/prime-rl/upstream/LICENSE diff --git a/frameworks/prime-rl/manifest.json b/frameworks/prime-rl/upstream/manifest.json similarity index 72% rename from frameworks/prime-rl/manifest.json rename to frameworks/prime-rl/upstream/manifest.json index 05fa7ce..6ffe3b5 100644 --- a/frameworks/prime-rl/manifest.json +++ b/frameworks/prime-rl/upstream/manifest.json @@ -16,14 +16,15 @@ "teacher-provenance.json": "1cae31a1ec039a0e8a48369851fda55b69a67549d3cd179256f0cb2390a1db93", "source-provenance.json": "ea202db26ba93561871ca708868565c86a59b17c0d91b2bf101f56ebfbe5ce0c" }, - "source_patch_files": [ - "src/prime_rl/trainer/model.py" - ], - "source_patch_sha256": "018f8da92888d84f60b4e8957dcd1ab277043595a41521cb1870239bd75c2f90", + "source_patch_sha256": "6fdc891586ba560e5fdb6053d71914ab6bfba651a3e6d239d00bbd417a55e011", "recorded_run": "prime-opd-407", "runtime_validated_on_original_cluster": true, "portable_package_gpu_validated": false, + "note": "One patch, to src/prime_rl/trainer/model.py: the frozen Qwen3.5 vision encoder uses SDPA when text attention is flash_attention_4. Training this VLM checkpoint on text-only data (no model.vlm, fp32 master weights) still runs the frozen encoder on dummy pixels every step, and FA4 rejects its fp32 inputs (job 1015). Upstream switches the encoder to SDPA only under Ulysses CP. The recorded_* fields describe the previous run, prime-opd-407.", + "source_patch_files": [ + "src/prime_rl/trainer/model.py" + ], "patched_source_sha256": { - "src/prime_rl/trainer/model.py": "9f92581a18c1c526bf30b5f484e2a3b359b0a20545932fcf30165410f67f2e16" + "src/prime_rl/trainer/model.py": "d392e2f3318c2c15179cf8f1be6379d00dfff6f0591e7f3f5456e66effab8ee0" } } diff --git a/frameworks/prime-rl/recorded-packages.txt b/frameworks/prime-rl/upstream/recorded-packages.txt similarity index 100% rename from frameworks/prime-rl/recorded-packages.txt rename to frameworks/prime-rl/upstream/recorded-packages.txt diff --git a/frameworks/prime-rl/source.patch b/frameworks/prime-rl/upstream/source.patch similarity index 70% rename from frameworks/prime-rl/source.patch rename to frameworks/prime-rl/upstream/source.patch index 0ddd135..ffab345 100644 --- a/frameworks/prime-rl/source.patch +++ b/frameworks/prime-rl/upstream/source.patch @@ -1,13 +1,16 @@ +diff --git a/src/prime_rl/trainer/model.py b/src/prime_rl/trainer/model.py +index ed45d84db..c88ff417d 100644 --- a/src/prime_rl/trainer/model.py +++ b/src/prime_rl/trainer/model.py -@@ -574,10 +574,16 @@ +@@ -574,10 +574,17 @@ def get_model( _patch_qwen3_5_text_position_ids() _patch_qwen3_5_moe_conversion_mapping() _patch_qwen3_5_linear_attn_varlen() - if is_vlm_arch and config.cp > 1 and config.cp_style == "ulysses": ++ # A frozen Qwen3.5 vision encoder (VLM mode with freeze_vision_encoder, or a VLM checkpoint trained on ++ # text-only data) still runs on dummy pixels every step, and FA4 rejects its fp32 inputs in fp32 training. + frozen_qwen_vision_with_fa4 = ( -+ is_vlm_training -+ and config.vlm.freeze_vision_encoder ++ (not is_vlm_training or config.vlm.freeze_vision_encoder) + and config.attn == "flash_attention_4" + and getattr(model_config, "model_type", "").startswith("qwen3_5") + ) diff --git a/frameworks/slime/campaign/control.py b/frameworks/slime/campaign/control.py deleted file mode 100644 index 4bc938b..0000000 --- a/frameworks/slime/campaign/control.py +++ /dev/null @@ -1,76 +0,0 @@ -"""Own one Slurm allocation and stop its two container steps after training.""" - -import json -import os -from pathlib import Path -import signal -import subprocess -import sys -import time - -ROOT = Path(__file__).resolve().parent -IMAGE = "@CONTAINER_IMAGE@" -PREVIOUS = "@ASSETS@" -BASE = "@BASE_PARENT@" - - -def stop(*_): - raise SystemExit(143) - - -def main(): - job = os.environ["SLURM_JOB_ID"] - result = ROOT / "results" / f"slime-opd-{job}" - result.mkdir(parents=True, exist_ok=False) - env = {**os.environ, "MOPD_RESULT": str(result), "MOPD_CAMPAIGN": str(ROOT), - "NVIDIA_VISIBLE_DEVICES": "all", "NVIDIA_DRIVER_CAPABILITIES": "all"} - nodes = subprocess.check_output(["scontrol", "show", "hostnames", os.environ["SLURM_JOB_NODELIST"]], text=True).split() - assert nodes == ["@GENERATION_NODE@", "@TRAINER_NODE@"], nodes - (ROOT / "active-run.json").write_text(json.dumps({"job_id": job, "result": str(result), "nodes": nodes})) - (result / "start.json").write_text(json.dumps({"time": time.time(), "job": job, "nodes": nodes})) - (result / "slurm-start.txt").write_bytes(subprocess.check_output(["scontrol", "show", "job", job])) - subprocess.run(["tar", "--exclude=__pycache__", "--exclude=.git", "--exclude=.venv", "-czf", str(result / "launch-source.tar.gz"), "-C", str(ROOT), - "source", "data", "control.py", "node.py", "observe.py", "campaign.py", - "serve-teachers.py", "teacher-provenance.json", "test_campaign.py", "make_args.py", - "endpoints.py", "scoring.py", "train_entry.py", "source-provenance.json", "launch.py", "mismatch-metrics.yaml", "wheels.sha256", "test_weight_session.py", "test_collectives.py"], check=True) - common = ["srun", "--exclusive", "-N1", "-n1", "--gres=gpu:8", "--cpus-per-task=80", - f"--container-image={IMAGE}", "--container-writable", "--container-remap-root", - "--no-container-mount-home", f"--container-mounts={ROOT}:{ROOT},{PREVIOUS}:{PREVIOUS}:ro," - f"@MEGATRON_PARENT@:@MEGATRON_PARENT@:ro,{BASE}:{BASE}:ro"] - children = [] - code = 1 - signal.signal(signal.SIGTERM, stop) - signal.signal(signal.SIGINT, stop) - try: - for role, node in (("learner", nodes[1]), ("generation", nodes[0])): - command = common + ["-w", node, "python3", str(ROOT / "node.py"), role] - (result / f"{role}-srun.json").write_text(json.dumps(command, indent=2)) - log = (result / f"{role}-driver.out").open("wb") - children.append(subprocess.Popen(command, env=env, stdout=log, stderr=subprocess.STDOUT, start_new_session=True)) - while True: - if children[0].poll() is not None: - code = children[0].returncode - break - if children[1].poll() is not None: - raise RuntimeError(f"Generation step exited early with {children[1].returncode}") - time.sleep(5) - finally: - (result / "STOP").touch() - deadline = time.monotonic() + 40 - while any(p.poll() is None for p in children) and time.monotonic() < deadline: - time.sleep(1) - for p in children: - if p.poll() is None: - os.killpg(p.pid, signal.SIGTERM) - for p in children: - try: - p.wait(timeout=20) - except subprocess.TimeoutExpired: - os.killpg(p.pid, signal.SIGKILL) - (result / "exit-code.txt").write_text(str(code) + "\n") - (result / "end.json").write_text(json.dumps({"time": time.time(), "exit_code": code})) - return code - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/frameworks/slime/campaign/endpoints.py b/frameworks/slime/campaign/endpoints.py deleted file mode 100644 index e59b348..0000000 --- a/frameworks/slime/campaign/endpoints.py +++ /dev/null @@ -1,52 +0,0 @@ -"""Scrape only the generation endpoints advertised by this campaign's launch log.""" - -import gzip -import json -from pathlib import Path -import re -import signal -import sys -import threading -import time -import urllib.request - -root = Path(__file__).resolve().parent -job = sys.argv[1] -assert job.isdecimal() -result = root / "results" / f"slime-opd-{job}" -stopping = threading.Event() -signal.signal(signal.SIGTERM, lambda *_: stopping.set()) -signal.signal(signal.SIGINT, lambda *_: stopping.set()) -ports = set() -offset = 0 -with gzip.open(result / "generation-endpoints.jsonl.gz", "at") as output: - while not stopping.is_set() and not (result / "end.json").exists(): - if (result / "learner-train.out").exists(): - with (result / "learner-train.out").open() as log: - log.seek(offset) - for line in log: - if "sglang.launch_server" in line and "--model-path" in line: - ports.update(int(value) for value in re.findall(r"--port[ =]+(\d+)", line)) - ports.update(int(value) for value in re.findall( - r'Launch HttpServerEngineAdapter at: @GENERATION_IP_REGEX@:(\d+)', line)) - if 'Ports for engine' in line and "'host': '@GENERATION_IP@'" in line: - ports.update(int(value) for value in re.findall(r"'port': (\d+)",line)) - offset = log.tell() - if (result / "generation-ready.json").exists(): - ip = json.loads((result / "generation-ready.json").read_text())["ip"] - assert ip == "@GENERATION_IP@" - for port in sorted(ports): - assert 1024 < port < 65536 - row = {"time_unix": time.time(), "ip": ip, "port": port} - try: - with urllib.request.urlopen(f"http://{ip}:{port}/metrics", timeout=2) as response: - row["prometheus"] = response.read().decode() - except OSError as error: - row["error"] = str(error) - output.write(json.dumps(row) + "\n") - output.flush() - (result / "generation-endpoints-status.json").write_text(json.dumps({"time_unix": time.time(), "ports": sorted(ports), "finished": False})) - if (result / "STOP").exists(): - break - stopping.wait(10) -(result / "generation-endpoints-status.json").write_text(json.dumps({"time_unix": time.time(), "ports": sorted(ports), "finished": True})) diff --git a/frameworks/slime/campaign/make_args.py b/frameworks/slime/campaign/make_args.py deleted file mode 100644 index c838fa8..0000000 --- a/frameworks/slime/campaign/make_args.py +++ /dev/null @@ -1,63 +0,0 @@ -"""Build the exact native Slime recipe and retain all effective arguments.""" - -import json -import os -from pathlib import Path -import subprocess - -ROOT = Path(__file__).resolve().parent -BASE = '@BASE_MODEL@' -CONVERTED = '@MEGATRON_MODEL@' - - -def build(result): - model = subprocess.check_output(['bash', '-c', 'source "$1"; printf "%s\\0" "${MODEL_ARGS[@]}"', 'bash', str(ROOT/'source/scripts/models/qwen3.5-35B-A3B.sh')]).decode().rstrip('\0').split('\0') - eval_config = {'eval': {'defaults': {'temperature': 0, 'n_samples_per_eval_prompt': 1, - 'max_response_len': 256}, 'datasets': [ - {'name': domain, 'path': str(ROOT/'data'/f'{prefix}-dev.jsonl')} - for domain, prefix in [('countdown','countdown4'), ('graph_color','graph12')]]}} - (result/'eval.json').write_text(json.dumps(eval_config, indent=2)) - args = model + [ - '--hf-checkpoint', BASE, '--ref-load', CONVERTED, - '--save', str(ROOT/'checkpoints'/result.name), '--save-interval', '20', '--no-save-optim', '--no-save-rng', - '--prompt-data', str(ROOT/'data/mixed-train.jsonl'), '--input-key', 'prompt', '--label-key', 'label', - '--metadata-key', 'metadata', '--apply-chat-template', '--apply-chat-template-kwargs', '{"enable_thinking":false}', - '--rollout-skip-special-tokens', '--rollout-shuffle', '--num-rollout', '40', '--rollout-batch-size', '128', - '--n-samples-per-prompt', '1', '--global-batch-size', '128', '--rollout-max-response-len', '256', - '--rollout-temperature', '1', '--rollout-top-p', '1', - '--custom-rm-path', 'campaign.reward', '--custom-reward-post-process-path', 'campaign.postprocess', - '--reward-key', 'task_score', '--eval-reward-key', 'task_score', - '--rollout-sample-hook-path', 'campaign.stamp', '--custom-rollout-log-function-path', 'campaign.log_train', - '--custom-eval-rollout-log-function-path', 'campaign.log_eval', - '--use-opd', '--opd-type', 'sglang', '--opd-kl-coef', '1.0', '--advantage-estimator', 'grpo', - '--eps-clip', '0.2', '--eps-clip-high', '0.2', '--entropy-coef', '0', '--kl-coef', '0', '--kl-loss-coef', '0', - '--get-mismatch-metrics', '--custom-config-path', str(ROOT/'mismatch-metrics.yaml'), - '--custom-tis-function-path', 'examples.train_infer_mismatch_helper.mis.compute_mis_weights_with_cp', - '--optimizer', 'adam', '--lr', '1e-6', '--lr-decay-style', 'constant', - '--weight-decay', '0', '--adam-beta1', '0.9', '--adam-beta2', '0.999', '--clip-grad', '1', '--bf16', - '--use-precision-aware-optimizer', '--tensor-model-parallel-size', '1', '--pipeline-model-parallel-size', '1', - '--context-parallel-size', '1', '--expert-model-parallel-size', '8', '--expert-tensor-parallel-size', '1', - '--recompute-granularity', 'full', '--recompute-method', 'uniform', '--recompute-num-layers', '1', - '--use-dynamic-batch-size', '--max-tokens-per-gpu', '4096', '--balance-data', - '--attention-dropout', '0', '--hidden-dropout', '0', '--accumulate-allreduce-grads-in-fp32', - '--attention-softmax-in-fp32', '--attention-backend', 'flash', - '--actor-num-nodes', '1', '--actor-num-gpus-per-node', '8', '--num-gpus-per-node', '8', - '--rollout-num-gpus', '6', '--rollout-num-gpus-per-engine', '2', '--sglang-ep-size', '2', - '--sglang-moe-runner-backend', 'triton', '--sglang-mem-fraction-static', '0.65', - '--sglang-max-running-requests', '256', '--sglang-server-concurrency', '128', - '--sglang-context-length', '2048', '--sglang-mamba-scheduler-strategy', 'extra_buffer', - '--sglang-enable-metrics', '--no-offload-train', '--no-offload-rollout', - '--eval-interval', '10', '--eval-config', str(result/'eval.json'), '--eval-temperature', '0', - '--n-samples-per-eval-prompt', '1', '--eval-max-response-len', '256', - '--update-weights-interval', '1', '--use-tensorboard', '--tb-project-name', str(result/'tensorboard'), - '--tb-experiment-name', 'slime-mopd', '--save-debug-rollout-data', str(result/'native-rollouts'/'rollout_{rollout_id}.pt'), - ] - (result/'argv.json').write_text(json.dumps(args, indent=2)) - return args - - -if __name__ == '__main__': - import sys - result = Path(sys.argv[1]) - result.mkdir(parents=True, exist_ok=True) - build(result) diff --git a/frameworks/slime/campaign/node.py b/frameworks/slime/campaign/node.py deleted file mode 100644 index 11e6c43..0000000 --- a/frameworks/slime/campaign/node.py +++ /dev/null @@ -1,181 +0,0 @@ -"""Run one owned container role with lightweight measurement and bounded cleanup.""" - -import ipaddress -import json -import os -from pathlib import Path -import shutil -import shlex -import signal -import socket -import subprocess -import sys -import time - -ROOT = Path(os.environ["MOPD_CAMPAIGN"]) -RESULT = Path(os.environ["MOPD_RESULT"]) -ROLE = sys.argv[1] -CHILDREN = [] - - -def launch(command, name, extra=None): - (RESULT / f"{ROLE}-{name}-command.json").write_text(json.dumps(command, indent=2)) - process = subprocess.Popen(command, env={**os.environ, **(extra or {})}, - stdout=(RESULT / f"{ROLE}-{name}.out").open("wb"), - stderr=subprocess.STDOUT, start_new_session=True) - CHILDREN.append(process) - return process - - -def check_children(ignore=None): - if (RESULT / "STOP").exists(): - raise SystemExit(143) - if any(p is not ignore and p.poll() is not None for p in CHILDREN): - raise RuntimeError("A required process exited; inspect the per-process logs") - - -def wait_file(name, timeout=900): - deadline = time.monotonic() + timeout - while not (RESULT / name).exists(): - check_children() - if time.monotonic() > deadline: - raise TimeoutError(f"Readiness timed out for {name}") - time.sleep(2) - return json.loads((RESULT / name).read_text()) - - -def stop(*_): - raise SystemExit(143) - - -def preflight(): - tests = [str(ROOT / 'test_campaign.py'), str(ROOT / 'test_weight_session.py'), str(ROOT / 'train_entry.py')] - with (RESULT / "preflight-tests.out").open("wb") as log: - for test in tests: - subprocess.run([sys.executable, test], env={**os.environ, 'MOPD_PARSE_ONLY':'1'}, - stdout=log, stderr=subprocess.STDOUT, check=True, timeout=180) - subprocess.run([sys.executable, '-m', 'torch.distributed.run', '--standalone', '--nproc-per-node=2', - str(ROOT/'test_collectives.py')], stdout=log, stderr=subprocess.STDOUT, - check=True, timeout=150) - (RESULT / "preflight-passed.json").write_text(json.dumps({"time": time.time(), "tests": tests})) - - -def verify_placement(head_ip, generation_ip): - import ray - from ray.util import remove_placement_group - from slime.ray.placement_group import _create_placement_group - - ray.init(address=f"{head_ip}:6379") - deadline = time.monotonic() + 180 - while int(ray.cluster_resources().get("GPU", 0)) != 14: - check_children() - assert time.monotonic() < deadline, ray.cluster_resources() - time.sleep(2) - nodes = {n["NodeManagerAddress"]: n["Resources"].get("GPU") for n in ray.nodes() if n["Alive"]} - assert nodes == {head_ip: 8.0, generation_ip: 6.0}, nodes - assert ipaddress.ip_address(head_ip) < ipaddress.ip_address(generation_ip), nodes - group = _create_placement_group(14) - try: - actual = [int(x) for x in group[2]] - assert actual == list(range(8)) + list(range(2, 8)), actual - (RESULT / "placement-verified.json").write_text(json.dumps({"nodes": nodes, "sorted_physical_gpu_ids": actual})) - finally: - remove_placement_group(group[0]) - ray.shutdown() - - -def learner(head_ip): - preflight() - launch(["ray", "start", "--head", f"--node-ip-address={head_ip}", "--num-gpus=8", "--num-cpus=64", - "--disable-usage-stats", "--block"], "ray") - (RESULT / "head-started.json").write_text(json.dumps({"ip": head_ip})) - ready = wait_file("generation-ready.json") - verify_placement(head_ip, ready["ip"]) - (RESULT / "placement-approved.json").write_text(json.dumps({"time": time.time()})) - wait_file("teachers-ready.json") - urls = {domain: f"http://{ready['ip']}:{port}/generate" - for domain, port in [('countdown',30000), ('graph_color',30001)]} - training = launch([sys.executable, str(ROOT / 'train_entry.py')], 'train', - {'RAY_ADDRESS': f'{head_ip}:6379', 'MOPD_TEACHER_URLS': json.dumps(urls)}) - while training.poll() is None: - check_children(ignore=training) - time.sleep(3) - return training.returncode - - -def generation(ip): - wait_file("preflight-passed.json") - head = wait_file("head-started.json") - # The head writes its address before its GCS socket becomes ready. - deadline = time.monotonic() + 180 - while True: - try: - with socket.create_connection((head["ip"], 6379), timeout=2): - break - except OSError: - check_children() - assert time.monotonic() < deadline - time.sleep(2) - launch(["ray", "start", f"--address={head['ip']}:6379", f"--node-ip-address={ip}", - "--num-gpus=6", "--num-cpus=64", "--disable-usage-stats", "--block"], "ray", - {"CUDA_VISIBLE_DEVICES": "2,3,4,5,6,7"}) - (RESULT / "generation-ready.json").write_text(json.dumps({"ip": ip})) - wait_file("placement-approved.json") - launch([sys.executable, str(ROOT / "serve-teachers.py")], "teachers") - while not (RESULT / "STOP").exists(): - check_children() - time.sleep(3) - return 0 - - -def main(): - os.chdir(ROOT / "source") - # Install the pinned wheel in this ephemeral container, without changing the shared image. - with (RESULT / f'{ROLE}-numpy-install.out').open('wb') as log: - subprocess.run(['sha256sum', '-c', str(ROOT/'wheels.sha256')], cwd=ROOT, - stdout=log, stderr=subprocess.STDOUT, check=True) - subprocess.run([sys.executable, '-m', 'pip', 'install', '--no-deps', '--force-reinstall', - '--no-index', '--find-links', str(ROOT/'wheels'), 'numpy==1.26.4', 'scipy==1.15.3'], - stdout=log, stderr=subprocess.STDOUT, check=True, timeout=120) - sys.path.insert(0, str(ROOT / "source")) - ip = '@TRAINER_IP@' if ROLE == 'learner' else '@GENERATION_IP@' - ray_tmp = Path(f"/tmp/mopd-async-{os.environ['SLURM_JOB_ID']}-{ROLE}") - os.environ.update({"PYTHONPATH": f"{ROOT}:{ROOT}/source:@MEGATRON_SOURCE@", "PYTHONNOUSERSITE": "1", - "PYTHONUNBUFFERED": "1", "WANDB_MODE": "disabled", "NCCL_DEBUG": "WARN", - "MASTER_ADDR": ip, "RAY_TMPDIR": str(ray_tmp), - "TENSORBOARD_DIR": str(RESULT / 'tensorboard'), "CUDA_DEVICE_MAX_CONNECTIONS": "1", - "SLIME_NATIVE_PROCESS_GROUPS": "1", - "RAY_memory_monitor_refresh_ms": "0", "RAY_DEDUP_LOGS": "0", - "TORCHINDUCTOR_CACHE_DIR": f"/tmp/mopd-inductor-{os.environ['SLURM_JOB_ID']}-{ROLE}"}) - os.environ.pop("NETRC", None) - signal.signal(signal.SIGTERM, stop) - signal.signal(signal.SIGINT, stop) - memory = subprocess.check_output(["nvidia-smi", "--query-gpu=memory.used", "--format=csv,noheader,nounits"], text=True) - assert all(int(v) < 1024 for v in memory.split()), memory - (RESULT / f"{ROLE}-gpu-before.txt").write_bytes(subprocess.check_output(["nvidia-smi", "-q"])) - launch(["nvidia-smi", "--query-gpu=timestamp,index,uuid,utilization.gpu,utilization.memory,memory.used,power.draw,temperature.gpu,clocks.sm,clocks.mem", "--format=csv", "-l", "5"], "gpu") - launch([sys.executable, str(ROOT / "observe.py"), os.environ["SLURM_JOB_ID"], ROLE, str(os.getpid())], "observer") - if ROLE == "learner": - launch([sys.executable, str(ROOT / "endpoints.py"), os.environ["SLURM_JOB_ID"]], "endpoints") - try: - return learner(ip) if ROLE == "learner" else generation(ip) - finally: - for p in reversed(CHILDREN): - if p.poll() is None: - os.killpg(p.pid, signal.SIGTERM) - for p in CHILDREN: - try: - p.wait(timeout=8) - except subprocess.TimeoutExpired: - os.killpg(p.pid, signal.SIGKILL) - copied = [] - for session in Path("/tmp").glob(f"mopd-async-{os.environ['SLURM_JOB_ID']}-*/ray/session_*"): - if session.is_symlink() or not (session / "logs").exists(): - continue - shutil.copytree(session / "logs", RESULT / "ray-logs" / ROLE, dirs_exist_ok=True) - copied.append(str(session)) - (RESULT / f"{ROLE}-cleanup.json").write_text(json.dumps({"time": time.time(), "ray_sessions": copied})) - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/frameworks/slime/campaign/observe.py b/frameworks/slime/campaign/observe.py deleted file mode 100644 index cd76e0a..0000000 --- a/frameworks/slime/campaign/observe.py +++ /dev/null @@ -1,64 +0,0 @@ -"""Collect host counters and copy logs from one identified campaign container.""" - -import gzip -import json -import os -from pathlib import Path -import shutil -import signal -import sys -import threading -import time -import urllib.request - -root = Path(__file__).resolve().parent -job, role, pid = sys.argv[1:] -result = root / "results" / f"slime-opd-{job}" -process = Path("/proc") / pid -assert (process / "cmdline").read_bytes().split(b"\0")[1:3] == [str(root / "node.py").encode(), role.encode()] -container_root = process / "root" -status = result / f"{role}-observer-status.json" -start = time.time() -last_copy = 0 -iterations = 0 -errors = [] -stopping = threading.Event() -signal.signal(signal.SIGTERM, lambda *_: stopping.set()) -signal.signal(signal.SIGINT, lambda *_: stopping.set()) -with gzip.open(result / f"{role}-host-counters.jsonl.gz", "at") as stream: - while process.exists() and not stopping.is_set() and not (result / "end.json").exists(): - now = time.time() - row = {"time_unix": now, "scope": "node procfs counters, not per-job attribution"} - for name in ("stat", "meminfo", "net/dev", "loadavg", "diskstats"): - try: - row[name] = (container_root / "proc" / name).read_text() - except OSError as error: - row[name] = str(error) - if role == "generation": - for port in (30000, 30001): - try: - with urllib.request.urlopen(f"http://127.0.0.1:{port}/metrics", timeout=1) as response: - row[f"sglang_{port}"] = response.read().decode() - except OSError as error: - row[f"sglang_{port}"] = str(error) - stream.write(json.dumps(row) + "\n") - stream.flush() - iterations += 1 - if now - last_copy >= 60 or (result / "STOP").exists(): - # Real session directories avoid absolute session_latest symlinks across mount namespaces. - for logs in (container_root / "tmp").glob(f"mopd-async-{job}-*/ray/session_*/logs"): - if logs.parent.is_symlink(): - continue - try: - shutil.copytree(logs, result / "ray-logs-live" / role, dirs_exist_ok=True, - ignore=lambda directory, names: [name for name in names if Path(directory, name).is_symlink()]) - except (OSError, shutil.Error) as error: - errors.append(str(error)[:1000]) - last_copy = now - status.write_text(json.dumps({"start": start, "last_sample": now, "iterations": iterations, - "errors": errors[-5:], "finished": False})) - if (result / "STOP").exists(): - break - stopping.wait(10) -status.write_text(json.dumps({"start": start, "end": time.time(), "iterations": iterations, - "errors": errors[-5:], "finished": True})) diff --git a/frameworks/slime/campaign/scoring.py b/frameworks/slime/campaign/scoring.py deleted file mode 100644 index 17b58a6..0000000 --- a/frameworks/slime/campaign/scoring.py +++ /dev/null @@ -1,97 +0,0 @@ -import ast -import json -import re -from collections import Counter -from fractions import Fraction - -SYSTEM_PROMPT = ( - "Solve the puzzle. Output only one ... block. " "Do not include reasoning or explanation." -) - - -def extract_answer(response: str) -> str | None: - if not isinstance(response, str) or len(response) > 20000: - return None - # Miles can retain Qwen's terminal token in decoded text, while the native - # screening API omits it. Normalize that transport difference before scoring. - response = response.strip().removesuffix("<|im_end|>").rstrip() - if "" not in response and "" not in response: - return response.strip() - matches = re.findall(r"(.*?)", response, flags=re.DOTALL) - if len(matches) != 1 or response.count("") != 1 or response.count("") != 1: - return None - return matches[0].strip() - - -def check_countdown(answer: str, numbers: list[int], target: int) -> bool: - if len(answer) > 256 or not re.fullmatch(r"[\d\s()+*/-]+", answer): - return False - used = [] - - def evaluate(node): - if isinstance(node, ast.Constant) and type(node.value) is int and 0 < node.value <= 10000: - used.append(node.value) - return Fraction(node.value) - if not isinstance(node, ast.BinOp) or not isinstance(node.op, (ast.Add, ast.Sub, ast.Mult, ast.Div)): - raise ValueError("Only integer literals and +, -, *, / are allowed") - left, right = evaluate(node.left), evaluate(node.right) - if isinstance(node.op, ast.Add): - value = left + right - elif isinstance(node.op, ast.Sub): - value = left - right - elif isinstance(node.op, ast.Mult): - value = left * right - else: - value = left / right - if abs(value.numerator) > 10**12 or value.denominator > 10**12: - raise ValueError("Intermediate value too large") - return value - - try: - tree = ast.parse(answer, mode="eval") - if sum(1 for _ in ast.walk(tree)) > 64: - return False - value = evaluate(tree.body) - return value == target and Counter(used) == Counter(numbers) - except (SyntaxError, ValueError, ZeroDivisionError, RecursionError): - return False - - -def _unique_object(pairs): - result = {} - for key, value in pairs: - if key in result: - raise ValueError("Duplicate JSON key") - result[key] = value - return result - - -def check_graph_color(answer: str, puzzle: dict) -> bool: - if len(answer) > 4096: - return False - try: - colors = json.loads(answer, object_pairs_hook=_unique_object) - if not isinstance(colors, dict) or set(colors) != {str(v) for v in puzzle["vertices"]}: - return False - if any(type(c) is not int or c not in puzzle["color_options"] for c in colors.values()): - return False - return all(colors[str(u)] != colors[str(v)] for u, v in puzzle["edges"]) - except (ValueError, TypeError, RecursionError): - return False - - -def score(response: str, label: dict | str) -> float: - if isinstance(label, str): - label = json.loads(label) - answer = extract_answer(response) - if answer is None: - return 0.0 - if label["domain"] == "countdown": - return float(check_countdown(answer, label["numbers"], label["target"])) - if label["domain"] == "graph_color": - return float(check_graph_color(answer, label["puzzle"])) - raise ValueError(f"Unknown puzzle domain {label['domain']!r}") - - -async def reward_func(args, sample, **kwargs): - return score(sample.response, sample.label) diff --git a/frameworks/slime/campaign/serve-teachers.py b/frameworks/slime/campaign/serve-teachers.py deleted file mode 100644 index b743cd1..0000000 --- a/frameworks/slime/campaign/serve-teachers.py +++ /dev/null @@ -1,117 +0,0 @@ -"""Serve frozen teachers on separate GPUs and validate prefill scoring.""" -import json -import math -import os -from pathlib import Path -import signal -import socket -import subprocess -import sys -import time -import urllib.request - -campaign = Path(os.environ['MOPD_CAMPAIGN']) -result = Path(os.environ['MOPD_RESULT']) -processes = [] - -def request(url, payload=None): - body = None if payload is None else json.dumps(payload).encode() - req = urllib.request.Request(url, data=body, headers={'Content-Type': 'application/json'}) - with urllib.request.urlopen(req, timeout=60) as response: - data = response.read() - return data - -def stop(*_): - raise SystemExit(1) - -signal.signal(signal.SIGTERM, stop) -signal.signal(signal.SIGINT, stop) -try: - from sglang.srt.managers.io_struct import GenerateReqInput - (result / 'scoring-api.json').write_text(json.dumps({ - 'sparse_field_present': 'token_ids_logprob_positions' in GenerateReqInput.__dataclass_fields__, - 'selected_mode': 'dense requested IDs', - 'reason': 'Use documented compatible API and validate returned requested-token scores.' - }, indent=2)) - from transformers import AutoTokenizer - tokenizer = AutoTokenizer.from_pretrained(str(campaign / 'models/Qwen3.6-35B-A3B'), trust_remote_code=True) - for gpu, domain in enumerate(('countdown', 'graph_color')): - command = [sys.executable, '-m', 'sglang.launch_server', - '--model-path', str(campaign / 'teachers' / domain), - '--tokenizer-path', str(campaign / 'models/Qwen3.6-35B-A3B'), - '--host', '0.0.0.0', '--port', str(30000 + gpu), '--tp-size', '1', - '--context-length', '2048', '--mem-fraction-static', '0.8', - '--max-running-requests', '128', '--disable-cuda-graph', - '--chunked-prefill-size', '-1', '--disable-radix-cache', - '--prefill-max-requests', '1', '--moe-runner-backend', 'triton', - '--enable-metrics'] - (result / f'{domain}-command.json').write_text(json.dumps(command, indent=2)) - log = open(result / f'{domain}-server.out', 'wb') - processes.append(subprocess.Popen(command, env={**os.environ, 'CUDA_VISIBLE_DEVICES': str(gpu)}, - stdout=log, stderr=subprocess.STDOUT, start_new_session=True)) - deadline = time.monotonic() + 600 - for gpu, domain in enumerate(('countdown', 'graph_color')): - url = f'http://127.0.0.1:{30000 + gpu}' - while True: - if any(p.poll() is not None for p in processes): - raise RuntimeError('A teacher process exited during startup') - try: - request(url + '/health') - break - except Exception: - if time.monotonic() > deadline: - raise TimeoutError('Teacher readiness exceeded ten minutes') - time.sleep(2) - name = 'countdown4' if domain == 'countdown' else 'graph12' - row = json.loads((campaign / f'data/{name}-train.jsonl').open().readline()) - prompt = row['prompt'] - if isinstance(prompt, str): - prompt = [{'role': 'user', 'content': prompt}] - encoded = tokenizer.apply_chat_template(prompt, tokenize=True, add_generation_prompt=True, enable_thinking=False) - ids = list(encoded['input_ids'] if hasattr(encoded, 'keys') else encoded) - assert ids and all(isinstance(token, int) for token in ids), 'Expected one flat token sequence' - # This synthetic suffix checks the scoring API. It is not an accuracy or throughput result. - ids += tokenizer.encode('test', add_special_tokens=False) - selected = sorted(set(ids[-16:])) - payload = {'input_ids': ids, 'sampling_params': {'temperature': 0, 'max_new_tokens': 0}, - 'return_logprob': True, 'logprob_start_len': 0, 'token_ids_logprob': selected} - (result / f'{domain}-probe-request.json').write_text(json.dumps(payload)) - timings = [] - for attempt in range(2): - start = time.monotonic() - raw = request(url + '/generate', payload) - timings.append(time.monotonic() - start) - response = json.loads(raw) - (result / f'{domain}-probe-response-{attempt}.json').write_bytes(raw) - meta = response['meta_info'] - assert meta.get('completion_tokens', 0) == 0, meta - input_scores = meta['input_token_logprobs'][1:] - assert len(input_scores) == len(ids) - 1 - assert all(math.isfinite(float(x[0])) for x in input_scores) - scores = meta['input_token_ids_logprobs'][-1] - mapping = {int(x[1]): float(x[0]) for x in scores} - assert all(t in mapping and math.isfinite(mapping[t]) for t in selected), scores - (result / f'{domain}-probe-timing.json').write_text(json.dumps({ - 'input_tokens': len(ids), 'selected_ids': len(selected), 'seconds': timings, - 'purpose': 'API smoke test; synthetic suffix; not representative throughput' - }, indent=2)) - sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) - sock.connect(('8.8.8.8', 80)) - ip = sock.getsockname()[0] - sock.close() - (result / 'teachers-ready.json').write_text(json.dumps({'ip': ip, 'ports': [30000, 30001]})) - while True: - if any(p.poll() is not None for p in processes): - raise RuntimeError('A frozen teacher exited; stop the student allocation') - time.sleep(2) -finally: - for p in processes: - try: - os.killpg(p.pid, signal.SIGTERM) - except ProcessLookupError: - pass - for p in processes: - try: - p.wait(timeout=15) - except subprocess.TimeoutExpired: - os.killpg(p.pid, signal.SIGKILL) diff --git a/frameworks/slime/campaign/source-provenance.json b/frameworks/slime/campaign/source-provenance.json deleted file mode 100644 index 1fd23f1..0000000 --- a/frameworks/slime/campaign/source-provenance.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "framework": "Slime", - "repository": "https://github.com/THUDM/slime", - "revision": "4c193f1f37509cca70f0e88807a9305b70f63f4e", - "container": "@CONTAINER_IMAGE@", - "megatron_revision": "8c1e05747eb612b382df2632783df5c83a853646", - "execution": "native batched asynchronous train_async.py", - "local_source_changes": ["train_async.py adds baseline evaluation and driver timing events", "sglang_engine.py normalizes host before construction of immutable ServerArgs", "The distributed updater opens and closes the sessions required by the pinned SGLang version; HTTP methods check explicit failure responses", "ReloadableProcessGroup forwards the newer PyTorch single-tensor collective entry points to its inner NCCL group"], - "ephemeral_container_dependency_override": "numpy==1.26.4 and scipy==1.15.3; wheel digests are recorded in wheels.sha256; shared image remains unchanged", - "algorithm_modified": false, - "native_nccl_groups": "SLIME_NATIVE_PROCESS_GROUPS=1 bypasses reloadable Python process groups only when trainer offload and release are disabled; native NCCL collective tests cover asynchronous and coalesced waits", - "hooks": "Domain routing calls native OPD reward_func and native post_process_rewards. Task scores remain diagnostics, and scalar training rewards are zero." -} diff --git a/frameworks/slime/campaign/teacher-provenance.json b/frameworks/slime/campaign/teacher-provenance.json deleted file mode 100644 index f7068fc..0000000 --- a/frameworks/slime/campaign/teacher-provenance.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "countdown": { - "repo": "semianalysisai/Qwen3.6-35B-A3B-countdown-GRPO-20260909", - "revision": "237a7f0345e883705026acd7f0745a3637042f73" - }, - "graph_color": { - "repo": "semianalysisai/Qwen3.6-35B-A3B-graph-color-GRPO-20260909", - "revision": "ec87f87177a4ced7256928ce67c438fafa73c28e" - } -} diff --git a/frameworks/slime/campaign/test_campaign.py b/frameworks/slime/campaign/test_campaign.py deleted file mode 100644 index 54963af..0000000 --- a/frameworks/slime/campaign/test_campaign.py +++ /dev/null @@ -1,67 +0,0 @@ -"""Manual CPU contracts for the isolated campaign, without changes to upstream CI.""" - -import asyncio -import json -import os -from pathlib import Path -from types import SimpleNamespace -from unittest.mock import patch - -import torch -import campaign -from scoring import score - - -def main(): - import numpy - assert numpy.__version__ == '1.26.4' - import scipy - assert scipy.__version__ == '1.15.3' - root = Path(__file__).resolve().parent - labels = [json.dumps({'domain':'countdown','numbers':[1,2,3,4],'target':10}), - json.dumps({'domain':'countdown','numbers':[1,2,3,4],'target':11})] - assert [score('1+2+3+4', x) for x in labels] == [1.0, 0.0] - original = SimpleNamespace(rm_url='unchanged', reward_key='task_score') - domains = ['countdown', 'graph_color'] - os.environ['MOPD_TEACHER_URLS'] = json.dumps({d: f'http://{d}/generate' for d in domains}) - selected_urls = [] - async def fake_teacher(args, sample, **kwargs): - selected_urls.append(args.rm_url) - return {'meta_info': {'completion_tokens': 0, 'input_token_logprobs': [[None,1],[-1.0,2],[-2.0,3]]}} - samples = [] - for domain in domains: - sample = SimpleNamespace(metadata={'domain':domain, 'campaign_evaluation':False}, - response='1+2+3+4', label=labels[0], tokens=[1,2,3], - response_length=2, index=0, weight_versions=['1'], reward=None) - sample.get_reward_value = lambda args, sample=sample: sample.reward[args.reward_key] - with patch.object(campaign.opd, 'reward_func', fake_teacher): - sample.reward = asyncio.run(campaign.reward(original, sample)) - samples.append(sample) - raw, processed = campaign.postprocess(original, samples) - assert raw == [1,1] and processed == [0,0] - assert all(torch.equal(s.teacher_log_probs, torch.tensor([-1.,-2.])) for s in samples) - assert selected_urls == ['http://countdown/generate','http://graph_color/generate'] - assert original.rm_url == 'unchanged' and original.reward_key == 'task_score' - from examples.train_infer_mismatch_helper.mis import compute_mis_weights - metric_args = SimpleNamespace(use_tis=False, tis_lower_bound=0.5, tis_upper_bound=2.0, - rs_lower_bound=None, rs_upper_bound=None) - mask = [torch.ones(2)] - weights, returned_masks, metrics = compute_mis_weights(metric_args, - train_log_probs=[torch.tensor([-1.,-2.])], rollout_log_probs=[torch.tensor([-1.1,-2.1])], - loss_masks=mask) - assert weights is None and returned_masks is mask and metrics - samples[0].metadata['campaign_evaluation'] = True - assert asyncio.run(campaign.reward(original,samples[0])) == {'task_score':1.0} - counts = {} - for prefix, domain in [('countdown4','countdown'),('graph12','graph_color')]: - for split, n in [('train',10000),('dev',512)]: - path = root/'data'/f'{prefix}-{split}.jsonl' - rows = [json.loads(x) for x in path.read_text().splitlines()] - assert len(rows) == n and all(r['metadata']['domain'] == domain for r in rows) - assert all([m['role'] for m in r['prompt']] == ['system','user'] for r in rows) - counts[path.name] = n - print(json.dumps({'native_postprocess': 'passed', 'teacher_routing':'passed', 'zero_training_reward':'passed', 'datasets':counts})) - - -if __name__ == '__main__': - main() diff --git a/frameworks/slime/campaign/test_collectives.py b/frameworks/slime/campaign/test_collectives.py deleted file mode 100644 index 803e249..0000000 --- a/frameworks/slime/campaign/test_collectives.py +++ /dev/null @@ -1,43 +0,0 @@ -"""Exercise real CUDA collectives on the process groups selected for this run.""" - -from datetime import timedelta -import os -import torch -import torch.distributed as dist - -# Megatron can retain these aliases before Slime installs its module wrappers. -from torch.distributed import all_gather_into_tensor, reduce_scatter_tensor, all_to_all_single -from slime.utils.reloadable_process_group import monkey_patch_torch_dist, register_default_process_group, ReloadableProcessGroup - -rank=int(os.environ['LOCAL_RANK']) -torch.cuda.set_device(rank) -dist.init_process_group('nccl',timeout=timedelta(seconds=60)) -native_groups=os.environ.get('SLIME_NATIVE_PROCESS_GROUPS')=='1' -if not native_groups: - register_default_process_group(timedelta(seconds=60)) - monkey_patch_torch_dist() -group=dist.new_group([0,1],backend='nccl') -assert isinstance(group,ReloadableProcessGroup) is not native_groups -value=torch.tensor([float(rank)],device='cuda') -gathered=torch.empty(2,device='cuda') -all_gather_into_tensor(gathered,value,group=group) -assert gathered.tolist()==[0.,1.] -reduced=torch.empty(1,device='cuda') -reduce_scatter_tensor(reduced,torch.full((2,),float(rank+1),device='cuda'),group=group) -assert reduced.item()==3. -exchanged=torch.empty(2,device='cuda') -all_to_all_single(exchanged,torch.tensor([rank*10.,rank*10.+1],device='cuda'),group=group) -assert exchanged.tolist()==[float(rank),float(rank+10)] -coalesced=torch.empty(1,device='cuda') -with dist._coalescing_manager(group=group,async_ops=True) as manager: - reduce_scatter_tensor(coalesced,torch.full((2,),float(rank+1),device='cuda'),group=group,async_op=True) -manager.wait() -assert coalesced.item()==3. -asynchronous=torch.empty(2,device='cuda') -work=all_gather_into_tensor(asynchronous,value,group=group,async_op=True) -work.wait() -assert asynchronous.tolist()==[0.,1.] -dist.barrier(group=group) -if rank==0: - print(f'CUDA all-gather, reduce-scatter, all-to-all and asynchronous/coalesced waits passed; native_groups={native_groups}.') -dist.destroy_process_group() diff --git a/frameworks/slime/campaign/test_weight_session.py b/frameworks/slime/campaign/test_weight_session.py deleted file mode 100644 index bad1dce..0000000 --- a/frameworks/slime/campaign/test_weight_session.py +++ /dev/null @@ -1,27 +0,0 @@ -"""Execute the real updater orchestration with stubbed transport and engines.""" - -import ast -from pathlib import Path -from types import SimpleNamespace - -path=Path(__file__).resolve().parent/'source/slime/backends/megatron_utils/update_weight/update_weight_from_distributed.py' -tree=ast.parse(path.read_text()) -function=next(node for node in ast.walk(tree) if isinstance(node,ast.FunctionDef) and node.name=='update_weights') -function.decorator_list=[] -module=ast.fix_missing_locations(ast.Module(body=[function],type_ignores=[])) -for rank in (0,1): - events=[] - def remote(name): - return SimpleNamespace(remote=lambda: events.append(name)) - engine=SimpleNamespace(**{name:remote(name) for name in - ('pause_generation','flush_cache','begin_weight_update','end_weight_update','continue_generation')}) - namespace={'dist':SimpleNamespace(get_rank=lambda:rank,barrier=lambda **kwargs:events.append('barrier')), - 'ray':SimpleNamespace(get=lambda refs:None),'get_gloo_group':lambda:None,'tqdm':lambda **kwargs:None} - exec(compile(module,str(path),'exec'),namespace) - updater=SimpleNamespace(weight_version=0,rollout_engines=[engine],quantization_config=None, - _group_name='test',_is_pp_src_rank=rank==0,_send_weights=lambda progress:events.append('transfer')) - namespace['update_weights'](updater) - assert updater.weight_version==1 - expected=['pause_generation','flush_cache','begin_weight_update','barrier','transfer','end_weight_update','continue_generation','barrier'] if rank==0 else ['barrier','transfer','barrier'] - assert events==expected,(rank,events) -print('Native updater opens and closes one session per broadcast; both rank contracts passed.') diff --git a/frameworks/slime/campaign/train_entry.py b/frameworks/slime/campaign/train_entry.py deleted file mode 100644 index 6c91822..0000000 --- a/frameworks/slime/campaign/train_entry.py +++ /dev/null @@ -1,27 +0,0 @@ -"""Validate native arguments before invoking the instrumented native driver.""" - -import json -import os -from pathlib import Path -import sys - -from make_args import build -from slime.utils.arguments import parse_args -from train_async import train - -result = Path(os.environ['MOPD_RESULT']) -sys.argv = ['train_async.py', *build(result)] -args = parse_args() -assert args.num_rollout == 40 and args.global_batch_size == 128 and args.use_opd -assert args.actor_num_gpus_per_node == 8 and args.rollout_num_gpus == 6 and not args.colocate -assert args.update_weights_interval == 1 -assert not args.use_tis and not args.use_rs -assert not getattr(args, 'profile', False) and not getattr(args, 'record_memory_history', False) -(result/'resolved-args.json').write_text(json.dumps(vars(args), indent=2, default=str)) -if os.environ.get('MOPD_PARSE_ONLY') != '1': - import ray - inherited = {key: os.environ[key] for key in ( - 'MOPD_RESULT', 'MOPD_CAMPAIGN', 'MOPD_TEACHER_URLS', 'PYTHONPATH', - 'TENSORBOARD_DIR', 'NCCL_DEBUG', 'CUDA_DEVICE_MAX_CONNECTIONS', 'SLIME_NATIVE_PROCESS_GROUPS')} - ray.init(address=os.environ['RAY_ADDRESS'], runtime_env={'env_vars': inherited}) - train(args) diff --git a/frameworks/slime/campaign/mismatch-metrics.yaml b/frameworks/slime/mismatch-metrics.yaml similarity index 100% rename from frameworks/slime/campaign/mismatch-metrics.yaml rename to frameworks/slime/mismatch-metrics.yaml diff --git a/frameworks/slime/node.py b/frameworks/slime/node.py new file mode 100644 index 0000000..5bbb096 --- /dev/null +++ b/frameworks/slime/node.py @@ -0,0 +1,37 @@ +"""Slime entry point: `node.py` runs the controller, `node.py ROLE` runs one container role. + +The two-node runtime lives in `shared/container.py`. This file adds Slime's pinned wheels, +placement format and training driver (`train.py`). +""" +import json +import subprocess +import sys + +from shared.container import RAY_PORT, ContainerNode, entry +from shared.slurm import ROOT + + +class SlimeNode(ContainerNode): + placement_module = 'slime.ray.placement_group' + # Native NCCL groups are safe because trainer offload and release are disabled. + env = {'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'SLIME_NATIVE_PROCESS_GROUPS': '1'} + + def setup(self): + # Install the pinned wheels into this throwaway container only; the image stays unchanged. + with (self.result / f'{self.role}-numpy-install.out').open('wb') as log: + subprocess.run(['sha256sum', '-c', 'wheels.sha256'], cwd=ROOT, stdout=log, stderr=log, check=True) + subprocess.run([sys.executable, '-m', 'pip', 'install', '--no-deps', '--force-reinstall', '--no-index', + '--find-links', str(ROOT / 'wheels'), 'numpy==1.26.4', 'scipy==1.15.3'], + stdout=log, stderr=log, check=True, timeout=120) + + def placement(self, group): + # Slime returns (placement group, bundle indices, sorted physical GPU ids). + return [int(x) for x in group[2]], group[0] + + def train_command(self, teacher_urls): + return [sys.executable, str(ROOT / 'train.py')], { + 'RAY_ADDRESS': f'{self.ip}:{RAY_PORT}', 'CAMPAIGN_TEACHER_URLS': json.dumps(teacher_urls)} + + +if __name__ == '__main__': + sys.exit(entry('slime', SlimeNode)) diff --git a/frameworks/slime/campaign/campaign.py b/frameworks/slime/opd_hooks.py similarity index 77% rename from frameworks/slime/campaign/campaign.py rename to frameworks/slime/opd_hooks.py index ba3d1d5..a922ae5 100644 --- a/frameworks/slime/campaign/campaign.py +++ b/frameworks/slime/opd_hooks.py @@ -1,18 +1,22 @@ -"""Route puzzle domains through Slime's native OPD hooks and retain evidence.""" +"""Slime OPD hooks: route each domain to its frozen teacher and log samples. + +`train.py` registers these with Slime's native OPD path. Task scores are logged for diagnostics +only; the scalar training reward is zero. +""" import copy import json -import math import os from pathlib import Path import time -from scoring import score +from shared import recipe +from shared.scoring import score from slime.rollout import on_policy_distillation as opd def record(name, row): - target = Path(os.environ['MOPD_RESULT']) / name + target = Path(os.environ['CAMPAIGN_RESULT']) / name with target.open('a') as stream: stream.write(json.dumps({'time': time.time(), **row}, allow_nan=False) + '\n') @@ -26,14 +30,15 @@ def stamp(args, sample, *, rollout_id=None, evaluation=False): async def reward(args, sample, **kwargs): domain = sample.metadata['domain'] - if domain not in ('countdown', 'graph_color'): + if domain not in recipe.DOMAINS: raise ValueError(f'Unexpected teacher domain: {domain}') task_score = score(sample.response, sample.label) sample.metadata['task_score'] = task_score if sample.metadata['campaign_evaluation']: return {'task_score': task_score} + # Copy args so concurrent requests never overwrite each other's teacher URL. selected = copy.copy(args) - selected.rm_url = json.loads(os.environ['MOPD_TEACHER_URLS'])[domain] + selected.rm_url = json.loads(os.environ['CAMPAIGN_TEACHER_URLS'])[domain] started = time.monotonic() response = await opd.reward_func(selected, sample, **kwargs) assert response['meta_info'].get('completion_tokens', 0) == 0 @@ -44,7 +49,7 @@ async def reward(args, sample, **kwargs): def postprocess(args, samples): - # Copy args locally so concurrent requests never mutate the shared teacher URL or key. + # Copy args rather than change the shared reward key. selected = copy.copy(args) selected.reward_key = 'opd' raw, processed = opd.post_process_rewards(selected, samples) @@ -52,7 +57,7 @@ def postprocess(args, samples): for sample in samples: scores = sample.teacher_log_probs assert len(scores) == sample.response_length and scores.isfinite().all().item() - # Task scores are diagnostic only. Native OPD receives zero scalar task reward. + # Task scores are returned as raw rewards for logging; training uses the all-zero processed rewards. return [s.metadata['task_score'] for s in samples], processed @@ -65,12 +70,13 @@ def sample_record(sample): def log_train(rollout_id, args, samples, rollout_extra_metrics, rollout_time): - assert len(samples) == 128 + assert len(samples) == recipe.PROMPTS_PER_UPDATE * recipe.SAMPLES_PER_PROMPT lags = [] for sample in samples: - # Native distributed broadcast starts at version one for the initial base weights. versions = [int(v) for v in sample.weight_versions] assert versions, 'SGLang did not return a policy version' + # The base weights are version 1 and train_async generates one rollout ahead (policy lag 1), + # so this offset count is at most 2. sample_lags = [rollout_id + 2 - v for v in versions] assert all(0 <= lag <= 2 for lag in sample_lags), (rollout_id, versions) lags.extend(sample_lags) @@ -81,10 +87,11 @@ def log_train(rollout_id, args, samples, rollout_extra_metrics, rollout_time): def log_eval(rollout_id, args, data, extra_metrics): + # The baseline (rollout -1) sees the base weights (version 1); rollout r is evaluated after its broadcast. expected_version = 1 if rollout_id == -1 else rollout_id + 2 for domain, payload in data.items(): samples = payload['samples'] - assert len(samples) == 512 + assert len(samples) == recipe.EVAL_EXAMPLES[domain] versions = sorted({int(v) for s in samples for v in s.weight_versions}) assert versions == [expected_version], (rollout_id, domain, versions) for sample in samples: diff --git a/frameworks/slime/train.py b/frameworks/slime/train.py new file mode 100644 index 0000000..19a64f9 --- /dev/null +++ b/frameworks/slime/train.py @@ -0,0 +1,92 @@ +"""Slime recipe: native `train_async.py` arguments built from `shared.recipe`. + +Run by the learner role. The argument list is saved as `argv.json`. +""" +import json +import os +import subprocess +import sys + +from shared import recipe +from shared.slurm import DATA, ROOT, eval_config, result_dir, site + +# Variables Ray workers need for the hooks, logging and process groups. +RAY_WORKER_ENV = ('CAMPAIGN_RESULT', 'CAMPAIGN_ROOT', 'CAMPAIGN_TEACHER_URLS', 'CAMPAIGN_PYDEPS', 'PYTHONPATH', 'TENSORBOARD_DIR', + 'NCCL_DEBUG', 'CUDA_DEVICE_MAX_CONNECTIONS', 'SLIME_NATIVE_PROCESS_GROUPS') + + +def model_args(): + """`MODEL_ARGS` from Slime's model script for the Qwen3.5/3.6 MoE architecture.""" + script = ROOT / 'source/scripts/models/qwen3.5-35B-A3B.sh' + output = subprocess.check_output(['bash', '-c', 'source "$1"; printf "%s\\0" "${MODEL_ARGS[@]}"', 'bash', script]) + return output.decode().rstrip('\0').split('\0') + + +def build(result): + s, r = site(), recipe + (result / 'eval.json').write_text(json.dumps(eval_config(), indent=2)) + args = [*model_args(), + '--hf-checkpoint', s['base_model'], '--ref-load', s['megatron_model'], + '--save', ROOT / 'checkpoints' / result.name, '--save-interval', r.SAVE_INTERVAL, + '--no-save-optim', '--no-save-rng', + # Save each checkpoint in HF format too, so it can be benchmarked without conversion. + '--save-hf', ROOT / 'checkpoints' / result.name / 'hf' / 'iter_{rollout_id}', + # Data and rollout. + '--prompt-data', DATA / r.TRAIN_FILE, '--input-key', 'prompt', '--label-key', 'label', + '--metadata-key', 'metadata', '--apply-chat-template', '--apply-chat-template-kwargs', + json.dumps({'enable_thinking': r.ENABLE_THINKING}, separators=(',', ':')), + '--rollout-skip-special-tokens', '--rollout-shuffle', + '--num-rollout', r.UPDATES, '--rollout-batch-size', r.PROMPTS_PER_UPDATE, + '--n-samples-per-prompt', r.SAMPLES_PER_PROMPT, '--global-batch-size', r.PROMPTS_PER_UPDATE, + '--rollout-max-response-len', r.MAX_RESPONSE_TOKENS, '--rollout-temperature', '1', '--rollout-top-p', '1', + # opd_hooks.py routes each domain to its teacher; task scores are diagnostics only. + '--custom-rm-path', 'opd_hooks.reward', '--custom-reward-post-process-path', 'opd_hooks.postprocess', + '--reward-key', 'task_score', '--eval-reward-key', 'task_score', + '--rollout-sample-hook-path', 'opd_hooks.stamp', '--custom-rollout-log-function-path', 'opd_hooks.log_train', + '--custom-eval-rollout-log-function-path', 'opd_hooks.log_eval', + # Native OPD objective with mismatch metrics only (no TIS or rejection sampling). + '--use-opd', '--opd-type', 'sglang', '--opd-kl-coef', '1.0', '--advantage-estimator', 'grpo', + '--eps-clip', '0.2', '--eps-clip-high', '0.2', '--entropy-coef', '0', '--kl-coef', '0', '--kl-loss-coef', '0', + '--get-mismatch-metrics', '--custom-config-path', ROOT / 'mismatch-metrics.yaml', + '--custom-tis-function-path', 'examples.train_infer_mismatch_helper.mis.compute_mis_weights_with_cp', + # Optimizer. + '--optimizer', 'adam', '--lr', r.LEARNING_RATE, '--lr-decay-style', 'constant', '--weight-decay', '0', + '--adam-beta1', '0.9', '--adam-beta2', '0.999', '--clip-grad', '1', '--bf16', '--use-precision-aware-optimizer', + # Trainer parallelism: expert parallel over all trainer GPUs, no context parallelism. + '--tensor-model-parallel-size', '1', '--pipeline-model-parallel-size', '1', '--context-parallel-size', '1', + '--expert-model-parallel-size', r.TRAINER_GPUS, '--expert-tensor-parallel-size', '1', + '--recompute-granularity', 'full', '--recompute-method', 'uniform', '--recompute-num-layers', '1', + '--use-dynamic-batch-size', '--max-tokens-per-gpu', r.MAX_TOKENS_PER_GPU, '--balance-data', + '--log-probs-chunk-size', r.LOG_PROBS_CHUNK_SIZE, + '--attention-dropout', '0', '--hidden-dropout', '0', '--accumulate-allreduce-grads-in-fp32', + '--attention-softmax-in-fp32', '--attention-backend', 'flash', + # Placement and SGLang policy engines. + '--actor-num-nodes', '1', '--actor-num-gpus-per-node', r.TRAINER_GPUS, '--num-gpus-per-node', r.GPUS_PER_NODE, + '--rollout-num-gpus', len(r.POLICY_GPUS), '--rollout-num-gpus-per-engine', r.POLICY_GPUS_PER_ENGINE, + '--sglang-ep-size', r.POLICY_GPUS_PER_ENGINE, '--sglang-moe-runner-backend', 'triton', + '--sglang-mem-fraction-static', '0.65', '--sglang-max-running-requests', '256', + '--sglang-server-concurrency', r.PROMPTS_PER_UPDATE, '--sglang-context-length', r.CONTEXT_LENGTH, + '--sglang-mamba-scheduler-strategy', 'extra_buffer', '--sglang-enable-metrics', + '--no-offload-train', '--no-offload-rollout', + # Evaluation, weight sync after every update, and logging. + '--eval-interval', r.EVAL_INTERVAL, '--eval-config', result / 'eval.json', '--eval-temperature', '0', + '--n-samples-per-eval-prompt', '1', '--eval-max-response-len', r.MAX_RESPONSE_TOKENS, + '--update-weights-interval', '1', '--use-tensorboard', '--tb-project-name', result / 'tensorboard', + '--tb-experiment-name', 'slime-mopd', + '--save-debug-rollout-data', result / 'native-rollouts' / 'rollout_{rollout_id}.pt'] + args = [str(a) for a in args] + (result / 'argv.json').write_text(json.dumps(args, indent=2)) + return args + + +if __name__ == '__main__': + from slime.utils.arguments import parse_args + from train_async import train + import ray + + result = result_dir() + sys.argv = ['train_async.py', *build(result)] + parsed = parse_args() + (result / 'resolved-args.json').write_text(json.dumps(vars(parsed), indent=2, default=str)) + ray.init(address=os.environ['RAY_ADDRESS'], runtime_env={'env_vars': {k: os.environ[k] for k in RAY_WORKER_ENV}}) + train(parsed) diff --git a/frameworks/slime/UPSTREAM-LICENSE b/frameworks/slime/upstream/LICENSE similarity index 100% rename from frameworks/slime/UPSTREAM-LICENSE rename to frameworks/slime/upstream/LICENSE diff --git a/frameworks/slime/manifest.json b/frameworks/slime/upstream/manifest.json similarity index 93% rename from frameworks/slime/manifest.json rename to frameworks/slime/upstream/manifest.json index 50b13d7..fe20243 100644 --- a/frameworks/slime/manifest.json +++ b/frameworks/slime/upstream/manifest.json @@ -27,7 +27,7 @@ "slime/utils/reloadable_process_group.py", "train_async.py" ], - "source_patch_sha256": "3f6971278f3c951bfd4f86f565ed18aa248850d09469252ac4dbba29fd6a04b8", + "source_patch_sha256": "d2d28da61d7fadc273f07750a14b6de7b371355f473648f8f9aa61266f0588ef", "recorded_run": "slime-opd-416", "runtime_validated_on_original_cluster": true, "portable_package_gpu_validated": false, @@ -36,6 +36,6 @@ "slime/backends/megatron_utils/update_weight/update_weight_from_distributed.py": "6377c41753fbcf1fd74692869bee9528fa751f78a07535933b74fa2aee10e7fc", "slime/backends/sglang_utils/sglang_engine.py": "40455fb7fda40f64ab64e649d8a554624d81027dde1303a77ae0e786380fe418", "slime/utils/reloadable_process_group.py": "76031bb44f2abc22f330dc1f02563298cf8f82b00ad819efebbd27aa0d66ee7b", - "train_async.py": "e3c3e5a8b77572e36a1b78b707c7ace0899ace4f77ff0b59bf5bb8d6ffccd19f" + "train_async.py": "919551aab49c78d6781dcf5ba276bb48de0bb11a71be6537b338c136fe9f4d92" } } diff --git a/frameworks/slime/recorded-runtime.json b/frameworks/slime/upstream/recorded-runtime.json similarity index 100% rename from frameworks/slime/recorded-runtime.json rename to frameworks/slime/upstream/recorded-runtime.json diff --git a/frameworks/slime/source.patch b/frameworks/slime/upstream/source.patch similarity index 99% rename from frameworks/slime/source.patch rename to frameworks/slime/upstream/source.patch index 2d31ca4..ef0c8a7 100644 --- a/frameworks/slime/source.patch +++ b/frameworks/slime/upstream/source.patch @@ -120,7 +120,7 @@ @@ -1,4 +1,6 @@ import ray +import time -+from campaign import record ++from opd_hooks import record from slime.observability.logging_utils import configure_logger, finish_tracking, init_tracking from slime.ray.placement_group import create_placement_groups, create_rollout_manager, create_training_models diff --git a/frameworks/slime/campaign/wheels.sha256 b/frameworks/slime/wheels.sha256 similarity index 100% rename from frameworks/slime/campaign/wheels.sha256 rename to frameworks/slime/wheels.sha256 diff --git a/frameworks/slime/wheels/README.md b/frameworks/slime/wheels/README.md deleted file mode 100644 index e7af281..0000000 --- a/frameworks/slime/wheels/README.md +++ /dev/null @@ -1 +0,0 @@ -The setup guide specifies how to download the two pinned wheels. Their SHA-256 values are in campaign/wheels.sha256. diff --git a/frameworks/tinker/README.md b/frameworks/tinker/README.md deleted file mode 100644 index 8b771e4..0000000 --- a/frameworks/tinker/README.md +++ /dev/null @@ -1,136 +0,0 @@ -# Tinker puzzle teachers and MOPD - -Remote Qwen3.6-35B-A3B training using Tinker 0.30.0 and cookbook 0.5.7. -Only tokenizers, source, and logs are stored locally. Model and optimizer -checkpoints stay on Tinker. This directory is a standalone SDK campaign; -the existing Slurm preparation/submission tools do not launch it. - -The completed 2026-09-21 campaign is documented in [RESULTS.md](RESULTS.md), -including the 40-update student result and comparison qualifications. - -## Protocol fixed before training - -- Exact `data/` train/dev files and original standard-library verifier. -- Pinned base tokenizer revision `995ad96eacd98c81ed38be0c5b274b04031597b0`. -- Hosted base model `Qwen/Qwen3.6-35B-A3B`; hosted weight revision is provider-controlled. -- Thinking disabled; pinned HF prompt tokens must equal cookbook renderer tokens. -- Two independently initialized domain teachers: 40 GRPO updates, 32 prompts - × 8 responses, 256 response tokens, temperature 1, top-p 1. -- Rank-64 LoRA on attention, MLP, and unembedding; teacher LR `1e-5`. -- This LR applies the cookbook's documented 10× FullFT-to-LoRA conversion to - the original `1e-6`; it is a heuristic, not an equivalence guarantee. -- Teacher PPO clips at 0.8/1.2; per-group mean/std reward normalization. - Adam betas 0.9/0.98, weight decay 0.1, gradient clipping 1. -- Student: fresh base initialization, 40 optimizer updates, exactly 64 prompts - per domain per step and one response each (5,120 total trajectories), 256 - response tokens, sampled-token reverse KL coefficient 1, zero task reward. -- Student default LR `1e-5` uses the same documented LoRA conversion. A literal - `1e-6` control can be run separately and must be labeled separately. -- Student Adam betas 0.9/0.999, no weight decay, gradient clipping 1. -- Native Tinker losses sum response-token contributions; equal prompt counts - do not imply equal token counts or equal per-domain gradient weight. No - additional domain-loss renormalization is applied. -- Baseline and every 10 updates: fixed weights, all 512 dev puzzles per domain, - greedy decoding, same response cap and scorer. Test data is not used for tuning. -- Seed 20260921, independent deterministic domain shuffles; synchronous weight - publication after each update means zero accepted policy lag. -- Multi-sample teacher requests leave the sampling seed unset, as in the - cookbook: explicitly setting one seed repeated responses within the group in - the first attempts. Initialization/order are seeded; full rollout replay uses - the retained token records rather than claiming deterministic regeneration. -- Preserve all attempts. No application-level automatic retry of optimizer - mutations. Resume only from a recorded state checkpoint in a fresh directory. - -The original pinned teacher cards report their training recipe but omit their -development accuracy. Teacher-quality matching therefore remains unverified -until the original evaluation records or explicit target scores are supplied. -Do not substitute the PR's student results for teacher targets. - -## Sources - -- [Miles PR 3116](https://github.com/radixark/miles/pull/3116), pinned revision - `8f8e4dff55d80bd9d36c3490c333a119f38480ed`. -- [Tinker distillation documentation](https://tinker-docs.thinkingmachines.ai/cookbook/recipes/distillation/) -- [Tinker LoRA guidance](https://tinker-docs.thinkingmachines.ai/tinker/lora-primer/) -- [Tinker PPO loss](https://tinker-docs.thinkingmachines.ai/tinker/losses/ppo/) -- [Tinker importance sampling loss](https://tinker-docs.thinkingmachines.ai/tinker/losses/importance-sampling/) -- Cookbook `recipes/rl_loop.py`, `distillation/train_on_policy.py`, and - `hyperparam_utils.py`. Research clone revision: - `1e53aa3d1cdd6389b3290c2574641eccc0503242`; installed runtime version: 0.5.7. - -## Commands - -Use Python 3.12 with `tinker==0.30.0`, `tinker-cookbook==0.5.7` installed. -The key is read from the environment; do not put it in command arguments. - -```bash -python frameworks/tinker/run.py eval --output /absolute/run/baseline -python frameworks/tinker/run.py teacher --domain countdown --output /absolute/run/countdown -python frameworks/tinker/run.py teacher --domain graph_color --output /absolute/run/graph-color -python frameworks/tinker/run.py mopd --teachers /absolute/run/teachers.json --output /absolute/run/student -``` - -`teachers.json` maps `countdown` and `graph_color` to the selected persistent -`tinker://.../sampler_weights/...` paths in the teacher checkpoint records. - -Raw artifacts belong outside the checkout. Each run saves source, resolved -arguments, package versions, dataset hashes, sample token IDs/logprobs/text, -teacher scores, optimizer metrics, stage timing, errors, evaluation summaries, -and remote checkpoint paths. Console output should also be redirected to a log. - -### Full campaign with an account check - -`launch_campaign.py` reads one literal key assignment from `~/.zprofile`, checks -its email and organization with Tinker, and refuses to train if either differs -from the expected values. It overrides the key only in its process and children; -it does not edit the profile or fall back to another account. The key is never -written to the campaign logs or command arguments. - -```bash -python frameworks/tinker/launch_campaign.py \ - --root /absolute/new-campaign-directory \ - --credential-label SA_TINKER_API_KEY \ - --expected-email you@company.com \ - --expected-org "Your organization" -``` - -Use a fresh output directory. The controller snapshots code and data, evaluates -the base model, trains both teachers concurrently, extends unmet provisional -goals up to 80 updates, selects teachers, then trains the fresh 40-update student. -It runs the artifact audit, collects checkpoint/billing metadata, and generates -the report and plots. Install `matplotlib==3.11.2` in addition to the frozen -training requirements for plotting. The pinned tokenizer must already be cached: -this launcher sets `HF_HUB_OFFLINE=1`, and never downloads model weights. - -`status.json`, `campaign-events.jsonl`, and individual console logs show progress -and failures. Keep the controller machine awake and connected for the entire -campaign. Tinker's SDK can pause on billing errors; inspect the logs if progress -stops. The controller does not automatically restart failed optimizer updates. - -## Interpretation - -Equal steps and prompt counts do not make this a controlled framework ranking: -Tinker uses LoRA, hosted scheduling/hardware, sampled-token OPD, and independent -retrained teachers. Miles uses candidate OPD; all three recorded self-hosted -campaigns differ in scheduling and numerical details. Report specialist-training -cost separately from student consolidation, include failed attempts, and compare -measured quality alongside wall time. No GPU-utilization claim can be made for -the hosted service without provider telemetry. - -## Offline tests - -```bash -python -m unittest discover -s frameworks/tinker -p 'test_*.py' -v -``` - -## Teacher extension decision — 2026-09-21T09:36:20.465387+00:00 - -After observing graph teacher dev 70.90% at update 30 (and before the 40-update results), adopt provisional specialist goals of 50% Countdown and 90% graph, with a maximum of 80 total updates per teacher in this extension. These are operational goals informed by the existing student task strengths, NOT recovered original teacher scores and NOT evidence of matched teacher distributions. Original teacher scores remain unavailable. Keep both initial 40-update results. If needed, resume exact optimizer state at update 40, preserve data ordering at the next batch, same LR/rank/objective, evaluate every 10, and stop a teacher at its first scheduled evaluation meeting its goal. Student stays fresh base, 40 updates, 128 prompts/update. Include all teacher preparation and failed-attempt costs separately from student cost. - -### Stopping-rule qualification - -Tinker uses model end-of-turn token 248046, without an `` string stop. -The pinned Miles launcher defaults to `stop_at_answer=True`; supplied Slime and -Prime-RL campaign configs do not specify that answer stop. Exact original teacher -command lines are not in the pinned model cards. All Tinker evaluations keep the -same EOS-only rule; post-answer continuations and truncation are retained. diff --git a/frameworks/tinker/RESULTS.md b/frameworks/tinker/RESULTS.md deleted file mode 100644 index 9d5e51f..0000000 --- a/frameworks/tinker/RESULTS.md +++ /dev/null @@ -1,98 +0,0 @@ -# Tinker MOPD results — 2026-09-21 - -Two specialist teachers were trained remotely, then frozen for a fresh-base -Qwen/Qwen3.6-35B-A3B student. The student completed **40 optimizer updates, -128 responses/update, 5,120 training responses**, with 64 prompts from each -domain per update. All reported scores use the original 512 development -examples/domain, greedy decoding, thinking disabled, and a 256-token cap. -These are development results from one seed, not held-out test results. - -## Measured quality - -| Model/checkpoint | Countdown | Graph Coloring | -|---|---:|---:| -| Selected specialist (own domain), update 80 | 238/512 (46.48%) | 466/512 (91.02%) | -| Fresh student, update 0 | 59/512 (11.52%) | 130/512 (25.39%) | -| MOPD student, update 10 | 170/512 (33.20%) | 277/512 (54.10%) | -| MOPD student, update 20 | 202/512 (39.45%) | 317/512 (61.91%) | -| MOPD student, update 30 | 214/512 (41.80%) | 413/512 (80.66%) | -| MOPD student, update 40 | **215/512 (41.99%)** | **436/512 (85.16%)** | - -The specialist row represents two different teachers. Each received 80 updates -of 32 prompts × 8 responses; the first 40-update checkpoints are also retained. -The graph teacher met its provisional 90% goal. Countdown missed its provisional -50% goal and used its best scheduled checkpoint at the 80-update cap. -**Original teacher scores were not available, so teacher-quality matching is -not established.** Those provisional goals are not recovered original scores. -Selection used development scores only; the student always uses update 40. - -Final student evaluation had 49/512 Countdown responses reach the token cap -and zero graph responses reach it. All responses, including truncations, -were scored and retained without filtering. - -## Table for the service comparison - -| Execution | Countdown baseline → final | Graph baseline → final | Observed student duration | -|---|---:|---:|---:| -| Miles, self-hosted | 10.55% → 45.51% | 26.17% → 88.28% | 21m 34s allocation | -| Prime-RL, self-hosted | 10.94% → 39.65% | 25.78% → 64.65% | 24m 22s allocation | -| Slime, self-hosted | 10.94% → 44.14% | 26.37% → 89.26% | 13m 25s allocation | -| Tinker, hosted | 11.52% → 41.99% | 25.39% → 85.16% | 31m 31s client elapsed | - -Original observations and timing definitions come from -[the repository provenance](../../docs/PROVENANCE.md). All students have the -same 40-update/5,120-response budget, but this is not a controlled ranking. -The self-hosted runs used 16 B200 GPUs; Tinker hardware and exact served weight -revision are provider-controlled. Tinker used rank-64 LoRA, LR 1e-5 (the -cookbook's 10× conversion from the original full-finetuning LR 1e-6), synchronous -sampling, and sampled-token reverse-KL distillation with no task reward. -Teacher identities, objectives, scheduling, and stopping rules differ. -Tinker stops at EOS; the pinned Miles launcher defaults to an answer-tag stop. - -Tinker student elapsed time was 1,891.39 seconds, including 418.43 seconds of -evaluation. Update loops totaled 1,397.77 seconds (median 34.97 seconds/update). -This excludes specialist preparation and earlier attempts. The recorded -campaign interval including attempts, gaps, and overlapping teachers was -96.02 minutes; it excludes research/setup before the first recorded attempt. - -## Evidence and reproducibility - -Full artifacts are retained outside Git at -`/Users/joey/workspace/ssb/opd-runs/tinker-20260921/`: - -- `REPORT.md`, `summary.json`, and `learning-curves.png/.svg`: all attempts, - evaluations, timings, and cost qualifications. -- `JOURNAL.md`: decisions, errors, fixes, and teacher-selection amendments. -- Per-run source/config/package/data snapshots, console logs, `samples.jsonl`, - `learner-scores.jsonl`, `teacher-scores.jsonl`, `metrics.jsonl`, and evaluations. -- `teacher-selection.json`, `teachers.json`, and per-run `checkpoints.jsonl`: - exact private remote model and optimizer-state paths. -- `final-audit.json`: all 40 updates, 5,120 training sequences, and 5,120 - evaluation sequences verified; every task score recomputed; every routed - distillation advantage checked. Seven Tinker tests and six package tests pass. -- `service-metadata-20260921T104053Z/`: run/checkpoint metadata and billing snapshot. - -The final student sampler is -`tinker://e7a10aea-05ff-554d-bbe6-a1ba0511c480:train:0/sampler_weights/step-040`. -The optimizer state uses the same run prefix with `weights/step-040`. -No model weights were downloaded; checkpoints remain private on Tinker. - -The main execution issue was explicit seeding of multi-response teacher calls, -which produced repeated responses within groups. Both initial teacher attempts -were invalidated and retained; corrected teachers restarted from fresh base -with the multi-response seed unset, following the cookbook. Other fixes were -explicit tokenizer `return_dict=False`, a floating-point test tolerance, and -disabling scorer-import bytecode in the original campaign template directory. -The corrected teachers, extensions, and student all exited successfully. - -Billing had no settled usage rows at collection time; this does not mean zero -cost. Recorded-token scenarios are $2.57/$3.74 for the student and $20.77/$25.86 -across recorded attempts with all-cached/all-uncached prefill. They are not -invoices or total-cost bounds and exclude unlogged retries, unfinished mutations, -probes, and storage. Forty retained remote checkpoints total 357.67 decimal GB, -approximately $35.77/month at the documented storage rate if kept for a full -month, subject to billing units and retention time. - -See [the protocol and official documentation links](README.md) for the complete -recipe and [run.py](run.py) for execution. Original self-hosted recipes were not -changed by this campaign. diff --git a/frameworks/tinker/analyze.py b/frameworks/tinker/analyze.py deleted file mode 100644 index e7d232b..0000000 --- a/frameworks/tinker/analyze.py +++ /dev/null @@ -1,152 +0,0 @@ -"""Summarize retained Tinker runs, including invalidated and failed attempts.""" -import argparse -from collections import Counter -from datetime import datetime -import json -from pathlib import Path -import statistics - - -def read_jsonl(path): - if not path.exists(): - return [] - lines = path.read_text().splitlines() - # A live writer may be midway through the final record. - result = [] - for i, line in enumerate(lines): - try: - result.append(json.loads(line)) - except json.JSONDecodeError: - if i != len(lines)-1: - raise - return result - - -def summarize(path): - config = json.loads((path/'config.json').read_text()) - events = read_jsonl(path/'events.jsonl') - steps = read_jsonl(path/'metrics.jsonl') - evaluations = read_jsonl(path/'evaluations.jsonl') - counters = Counter() - for row in read_jsonl(path/'samples.jsonl'): - prefix = row['split'] - counters[f'{prefix}_samples'] += 1 - counters[f'{prefix}_prompt_tokens'] += len(row['prompt_tokens']) - counters[f'{prefix}_response_tokens'] += len(row['tokens']) - sample_prompt_tokens = counters['train_prompt_tokens'] + counters['dev_prompt_tokens'] - sample_response_tokens = counters['train_response_tokens'] + counters['dev_response_tokens'] - train_tokens = sum(s['prompt_tokens'] + s['response_tokens'] - s['samples'] for s in steps) - teacher_tokens = sum(len(r['prompt_tokens']) + len(r['tokens']) for r in read_jsonl(path/'teacher-scores.jsonl')) - # Public prices observed 2026-09-21; not settled billing. Counts exclude - # unfinished mutations, unlogged retries, standalone probes, and storage. - common_cost = (sample_response_tokens*1.335 + train_tokens*1.177)/1e6 - estimates = {'all_prefill_cached_usd': common_cost+(sample_prompt_tokens+teacher_tokens)*0.108/1e6, - 'all_prefill_uncached_usd': common_cost+(sample_prompt_tokens+teacher_tokens)*0.54/1e6, - 'training_tokens_completed_steps': train_tokens, 'teacher_scoring_tokens': teacher_tokens, - 'status': 'partial recorded-token estimate; not an invoice or full campaign cost'} - invalid = path/'INVALIDATED.json' - status = 'invalidated' if invalid.exists() else events[-1]['event'] if events else 'unknown' - info_path = path/'model-info.json' - model_info = json.loads(info_path.read_text()) if info_path.exists() else None - return {'name': path.name, 'status': status, 'mode': config['mode'], - 'domain': config['domain'], 'rank': config['rank'], 'learning_rate': config['learning_rate'], - 'completed_updates': len(steps), 'sample_counts': dict(counters), 'token_cost_estimate': estimates, - 'elapsed_seconds': events[-1]['elapsed_seconds'] if events else None, - 'started_utc': events[0]['utc'] if events else None, - 'last_event_utc': events[-1]['utc'] if events else None, - 'sum_step_seconds': sum(s['step_seconds'] for s in steps), - 'sum_sampling_seconds': sum(s['sampling_seconds'] for s in steps), - 'sum_teacher_scoring_seconds': sum(s['teacher_seconds'] for s in steps), - 'sum_train_seconds': sum(s['train_seconds'] for s in steps), - 'sum_evaluation_seconds': sum(e['seconds'] for e in evaluations), - 'median_step_seconds': statistics.median(s['step_seconds'] for s in steps) if steps else None, - 'evaluations': evaluations, 'checkpoints': read_jsonl(path/'checkpoints.jsonl'), - 'model_info': model_info, 'path': str(path)} - - -def main(): - p = argparse.ArgumentParser(description=__doc__) - p.add_argument('root', type=Path) - args = p.parse_args() - runs = [summarize(x.parent) for x in sorted(args.root.glob('*/config.json'))] - (args.root/'summary.json').write_text(json.dumps(runs,indent=2)) - lines = ['# Tinker MOPD execution report', '', - 'Live summary generated from retained artifacts. Invalidated attempts must not be used as benchmark results.', '', - '| Attempt | Status | Updates | LR | Median step (s) | Elapsed (s) |', - '|---|---|---:|---:|---:|---:|'] - for r in runs: - median = f"{r['median_step_seconds']:.2f}" if r['median_step_seconds'] else '—' - lines.append(f"| {r['name']} | {r['status']} | {r['completed_updates']} | {r['learning_rate']:g} | {median} | {r['elapsed_seconds']:.2f} |") - starts=[datetime.fromisoformat(r['started_utc']) for r in runs if r['started_utc']] - ends=[datetime.fromisoformat(r['last_event_utc']) for r in runs if r['last_event_utc']] - if starts and ends: - lines += ['', f"Observed campaign interval from the first recorded attempt to the latest event: {(max(ends)-min(starts)).total_seconds()/60:.2f} minutes. This includes gaps and overlapping teacher runs, and excludes research/setup before the first attempt.", ''] - students=[r for r in runs if r['mode']=='mopd'] - for r in students: - lines += ['', f"## Student execution: {r['name']}", '', - f"Observed interval: {r['started_utc']} to {r['last_event_utc']}. Status: **{r['status']}**.", '', - f"Client elapsed: {r['elapsed_seconds']:.2f} s; sum of update loops: {r['sum_step_seconds']:.2f} s; evaluations: {r['sum_evaluation_seconds']:.2f} s.", '', - f"Within the update loops, sampling took {r['sum_sampling_seconds']:.2f} s, frozen-teacher scoring {r['sum_teacher_scoring_seconds']:.2f} s, and learner requests/logging {r['sum_train_seconds']:.2f} s. Remaining loop time includes weight publication. These are client-observed wall times, not isolated GPU compute times.", ''] - selection_path=args.root/'teacher-selection.json' - if selection_path.exists(): - selection=json.loads(selection_path.read_text()) - lines += ['', '## Selected frozen teachers', '', - '| Domain | Selected update | Own-domain dev accuracy | Provisional goal | Goal met |', - '|---|---:|---:|---:|---|'] - for domain, s in selection['selected'].items(): - lines.append(f"| {domain} | {s['step']} | {s['evaluation'][domain]['accuracy']:.2%} | {s['provisional_target']:.0%} | {s['target_met']} |") - lines += ['', 'These goals are operational surrogates, not the missing original teacher scores. The teacher-selection JSON retains exact private remote checkpoint paths and evaluations. The student is initialized independently from the base model, not from either teacher.', ''] - lines += ['', '## Development evaluation', '', - 'All scores below use 512 examples per domain unless the sample count says otherwise. Temperature 0, response cap 256, thinking disabled.', '', - '| Attempt | Update | Countdown | Graph Coloring | N/domain |', - '|---|---:|---:|---:|---:|'] - for r in runs: - for e in r['evaluations']: - lines.append(f"| {r['name']} | {e['step']} | {e['countdown']['accuracy']:.2%} | {e['graph_color']['accuracy']:.2%} | {e['countdown']['n']}/{e['graph_color']['n']} |") - lines += ['', '## Recorded-token cost estimates', '', - 'Illustrative all-cached versus all-uncached prefill cases at the [published model prices](https://tinker-docs.thinkingmachines.ai/tinker/models/): $0.108/$0.54 per million prefill tokens, $1.335 per million generated tokens, $1.177 per million training tokens. These are not invoices or bounds on total spend. They exclude unlogged retries, unfinished mutations, standalone probes, and checkpoint storage. Settled billing is authoritative.', '', - '| Attempt | All prefill cached | All prefill uncached |', '|---|---:|---:|'] - for r in runs: - c = r['token_cost_estimate'] - lines.append(f"| {r['name']} | ${c['all_prefill_cached_usd']:.3f} | ${c['all_prefill_uncached_usd']:.3f} |") - cached=sum(r['token_cost_estimate']['all_prefill_cached_usd'] for r in runs) - uncached=sum(r['token_cost_estimate']['all_prefill_uncached_usd'] for r in runs) - lines.append(f"| Total recorded attempts | ${cached:.3f} | ${uncached:.3f} |") - lines += ['', '## Original self-hosted observations', '', - 'From `docs/PROVENANCE.md` in the OPD repo; one run each, different objectives and scheduling. Durations include successful allocation startup/evaluation/checkpoint/cleanup, and exclude earlier failures.', '', - '| Framework | Countdown baseline → final | Graph Coloring baseline → final | Successful allocation |', - '|---|---:|---:|---:|', - '| Miles | 10.55% → 45.51% | 26.17% → 88.28% | 21m 34s |', - '| Prime-RL | 10.94% → 39.65% | 25.78% → 64.65% | 24m 22s |', - '| Slime | 10.94% → 44.14% | 26.37% → 89.26% | 13m 25s |', '', - 'The exact original teacher development scores are missing from the supplied sources; teacher-quality matching is not established. Tinker is rank-64 LoRA with a documented 10× LR adjustment, remote provider-controlled weights/hardware, synchronous rollouts, and sampled-token distillation. This is a workload-matched service observation, not an isolated framework ranking.', '', - 'Teacher runs may overlap in wall time. Report their individual elapsed times and total campaign elapsed time separately; do not sum overlapping durations as elapsed wall time. Hosted cost must include invalidated attempts and specialist training, with consolidation cost reported separately. Billing data can lag by hours.', '', - 'See `JOURNAL.md`, `protocol.md`, per-run JSONL, console logs, and remote checkpoint records for evidence. No model weights were downloaded.', ''] - audit_path=args.root/'final-audit.json' - if audit_path.exists(): - audit=json.loads(audit_path.read_text()) - lines += ['## Final artifact audit', '', - f"Passed: **{audit['passed']}**. Verified {audit['optimizer_updates']} student updates, {audit['training_sequences']} training sequences and {audit['development_sequences']} development sequences; recomputed all task scores and teacher-minus-student advantages. See `final-audit.json` and `frameworks/tinker/audit.py`.", ''] - metadata_dirs=sorted(args.root.glob('service-metadata-*')) - if metadata_dirs: - metadata=metadata_dirs[-1] - checkpoints={} - for path in metadata.glob('*-checkpoints.json'): - for checkpoint in json.loads(path.read_text()).get('checkpoints',[]): - checkpoints[checkpoint['tinker_path']]=checkpoint - size=sum(c['size_bytes'] for c in checkpoints.values()) - lines += ['## Remote retention and billing', '', - f"Metadata snapshot: `{metadata.name}`. It lists {len(checkpoints)} persistent remote checkpoints totaling {size:,} bytes ({size/1e9:.2f} decimal GB). No checkpoint files were downloaded. At the documented $0.10/GB-month storage rate, retaining this amount for a full month would be approximately ${size/1e9*.1:.2f}, subject to provider billing units and retention time. Storage is separate from token usage.", ''] - billing_path=metadata/'billing.json' - if billing_path.exists(): - billing=json.loads(billing_path.read_text()) - if billing.get('data')==[]: - lines += ['The billing snapshot contains no settled usage rows. This does **not** mean zero spend; billing can lag. Re-query the retained billing command later for the invoice evidence.', ''] - else: - lines += ['Billing rows are retained in the snapshot. Use their settled provider amounts, not the illustrative token estimates above, for a cost claim.', ''] - (args.root/'REPORT.md').write_text('\n'.join(lines)) - print(json.dumps([{k:r[k] for k in ('name','status','completed_updates','sample_counts')} for r in runs],indent=2)) - - -if __name__ == '__main__': - main() diff --git a/frameworks/tinker/audit.py b/frameworks/tinker/audit.py deleted file mode 100644 index 0740432..0000000 --- a/frameworks/tinker/audit.py +++ /dev/null @@ -1,138 +0,0 @@ -"""Verify the completed student's workload, scores, routing, and OPD signal.""" -import argparse -from collections import Counter, defaultdict -import gzip -import hashlib -import importlib.util -import json -import math -from pathlib import Path - - -def rows(path): - with path.open() as stream: - for line in stream: - yield json.loads(line) - - -def key(r): - return (r['step'],r['domain'],r['index'],r['sample']) - - -def main(): - p=argparse.ArgumentParser(description=__doc__) - p.add_argument('root',type=Path) - args=p.parse_args() - root=args.root.resolve() - run=root/'student' - config=json.loads((run/'config.json').read_text()) - assert config['mode']=='mopd' and config['checkpoint'] is None - assert config['start_step']==0 and config['steps']==40 - assert config['prompts']==128 and config['group_size']==1 - assert config['max_tokens']==256 and config['eval_size']==512 - events=list(rows(run/'events.jsonl')) - assert events[-1]['event']=='complete' - assert events[-1]['optimizer_updates']==40 - metrics=list(rows(run/'metrics.jsonl')) - assert [r['step'] for r in metrics]==list(range(1,41)) - assert all(r['samples']==128 and r['accepted_policy_lag']==0 for r in metrics) - manifest=json.loads((run/'data-manifest.json').read_text()) - dataset={} - for domain,prefix in [('countdown','countdown4'),('graph_color','graph12')]: - for split in ('train','dev'): - name=f'{prefix}-{split}.jsonl' - raw=gzip.decompress((root/'research/data'/f'{name}.gz').read_bytes()) - assert hashlib.sha256(raw).hexdigest()==manifest[name]['sha256'] - dataset[domain,split]=[json.loads(line) for line in raw.splitlines()] - spec=importlib.util.spec_from_file_location('retained_scorer',run/'scoring.py') - scorer=importlib.util.module_from_spec(spec) - spec.loader.exec_module(scorer) - counts=Counter() - correct=Counter() - train_keys=set() - train_signatures={} - train_lengths={} - eval_keys=set() - def signature(r): - return hashlib.sha256(json.dumps({k:r[k] for k in - ('prompt_tokens','tokens','logprobs','response','score')},sort_keys=True).encode()).hexdigest() - for r in rows(run/'samples.jsonl'): - label=dataset[r['domain'],r['split']][r['index']]['label'] - assert scorer.score(r['response'],label)==r['score'] - assert 0 < len(r['tokens']) <= 256 - assert len(r['tokens'])==len(r['logprobs']) - assert all(math.isfinite(x) for x in r['logprobs']) - assert r['sample']==0 - k=(r['split'],r['step'],r['domain']) - counts[k]+=1 - correct[k]+=r['score'] - if r['split']=='train': - assert key(r) not in train_keys - train_keys.add(key(r)) - train_signatures[key(r)]=signature(r) - train_lengths[key(r)]=len(r['tokens']) - else: - assert key(r) not in eval_keys - eval_keys.add(key(r)) - assert len(train_keys)==5120 - for step in range(40): - for domain in ('countdown','graph_color'): - assert counts['train',step,domain]==64 - for step in (0,10,20,30,40): - for domain in ('countdown','graph_color'): - assert counts['dev',step,domain]==512 - assert sum(counts.values())==10240 - assert eval_keys=={(step,domain,index,0) for step in (0,10,20,30,40) - for domain in ('countdown','graph_color') for index in range(512)} - evals=list(rows(run/'evaluations.jsonl')) - assert [e['step'] for e in evals]==[0,10,20,30,40] - for e in evals: - for domain in ('countdown','graph_color'): - assert e[domain]['correct']==correct['dev',e['step'],domain] - assert e[domain]['accuracy']==correct['dev',e['step'],domain]/512 - score_keys=set() - advantage_signatures={} - gap_sums=defaultdict(float) - gap_counts=Counter() - for r in rows(run/'teacher-scores.jsonl'): - k=key(r) - assert k in train_keys and k not in score_keys - assert signature(r)==train_signatures[k] - score_keys.add(k) - advantage_signatures[k]=hashlib.sha256(json.dumps(r['advantages']).encode()).hexdigest() - assert len(r['tokens'])==len(r['teacher_logprobs'])==len(r['advantages']) - for t,s,a in zip(r['teacher_logprobs'],r['logprobs'],r['advantages']): - assert math.isfinite(a) and math.isclose(a,t-s,rel_tol=0,abs_tol=1e-12) - gap_sums[r['step'],r['domain']]-=a - gap_counts[r['step'],r['domain']]+=1 - assert score_keys==train_keys - for m in metrics: - for domain in ('countdown','graph_color'): - k=(m['step']-1,domain) - assert math.isclose(m['sampled_reverse_kl_by_domain'][domain],gap_sums[k]/gap_counts[k],abs_tol=1e-10) - learner_rows=list(rows(run/'learner-scores.jsonl')) - assert len(learner_rows)==5120 - learner_keys={key(r) for r in learner_rows} - assert learner_keys==train_keys - for r in learner_rows: - assert len(r['logprobs'])==len(r['advantages'])==train_lengths[key(r)] - assert all(math.isfinite(x) for x in r['logprobs']) - assert hashlib.sha256(json.dumps(r['advantages']).encode()).hexdigest()==advantage_signatures[key(r)] - routes=json.loads((run/'teachers.json').read_text()) - selection=json.loads((root/'teacher-selection.json').read_text()) - assert routes=={d:v['checkpoint']['sampler_path'] for d,v in selection['selected'].items()} - student_id=json.loads((run/'model-info.json').read_text())['model_id'] - assert all(student_id not in path for path in routes.values()) - report={'passed':True,'optimizer_updates':40,'training_sequences':5120, - 'training_sequences_per_domain':2560,'development_sequences':5120, - 'exact_data_hashes':True,'all_sample_scores_recomputed':True, - 'exact_teacher_minus_student_advantages':True,'balanced_routing_each_update':True, - 'learner_and_teacher_scores_cover_all_training_sequences':True, - 'fresh_student':True,'zero_accepted_policy_lag':True, - 'source_sha256':hashlib.sha256((run/'source.py').read_bytes()).hexdigest()} - (root/'final-audit.json').write_text(json.dumps(report,indent=2)) - print(json.dumps(report,indent=2)) - - -if __name__=='__main__': - main() diff --git a/frameworks/tinker/collect_metadata.py b/frameworks/tinker/collect_metadata.py deleted file mode 100644 index d80ceb7..0000000 --- a/frameworks/tinker/collect_metadata.py +++ /dev/null @@ -1,42 +0,0 @@ -"""Retain remote run/checkpoint metadata and a billing snapshot, without weights.""" -from concurrent.futures import ThreadPoolExecutor -from datetime import datetime, timezone, timedelta -import argparse -import json -from pathlib import Path -import subprocess -import sys - - -def main(): - p=argparse.ArgumentParser(description=__doc__) - p.add_argument('root',type=Path) - args=p.parse_args() - root=args.root.resolve() - stamp=datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ') - out=root/f'service-metadata-{stamp}' - out.mkdir() - cli=str(Path(sys.executable).parent/'tinker') - jobs=[] - for path in sorted(root.glob('*/model-info.json')): - model_id=json.loads(path.read_text())['model_id'] - for kind,command in [('run',['run','info',model_id]),('checkpoints',['checkpoint','list','--run-id',model_id])]: - jobs.append((f'{path.parent.name}-{kind}',[cli,'-f','json',*command])) - times=[datetime.fromisoformat(json.loads(line)['utc']) - for path in root.glob('*/events.jsonl') for line in path.read_text().splitlines()] - start=min(times).replace(minute=0,second=0,microsecond=0).isoformat().replace('+00:00','Z') - end=(max(times).replace(minute=0,second=0,microsecond=0)+timedelta(hours=1)).isoformat().replace('+00:00','Z') - jobs.append(('billing',[cli,'-f','json','billing','usage',start,end])) - def run(job): - name,command=job - with (out/f'{name}.json').open('w') as stdout, (out/f'{name}.error.log').open('w') as stderr: - result=subprocess.run(command,stdout=stdout,stderr=stderr) - return {'name':name,'command':command,'exit_code':result.returncode} - with ThreadPoolExecutor(max_workers=4) as pool: - results=list(pool.map(run,jobs)) - (out/'commands.json').write_text(json.dumps(results,indent=2)) - print(json.dumps({'directory':str(out),'results':[{k:r[k] for k in ('name','exit_code')} for r in results]},indent=2)) - - -if __name__=='__main__': - main() diff --git a/frameworks/tinker/launch_campaign.py b/frameworks/tinker/launch_campaign.py deleted file mode 100644 index 3941afb..0000000 --- a/frameworks/tinker/launch_campaign.py +++ /dev/null @@ -1,143 +0,0 @@ -"""Run an account-verified teacher → MOPD campaign, keeping all remote weights remote.""" -from concurrent.futures import ThreadPoolExecutor -from datetime import datetime, timezone -import argparse -import json -import os -from pathlib import Path -import re -import shlex -import shutil -import subprocess -import sys -import threading -import traceback - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument('--root', type=Path, required=True) - parser.add_argument('--credential-label', required=True) - parser.add_argument('--expected-email', required=True) - parser.add_argument('--expected-org', required=True) - args = parser.parse_args() - # Read a literal assignment without executing the user's shell profile or - # printing any credential. Hyphenated labels are supported here explicitly. - pattern = re.compile(r'^\s*(?:export\s+)?' + re.escape(args.credential_label) + r'\s*=\s*(.*)$') - values = [shlex.split(m.group(1), comments=True) - for line in (Path.home()/'.zprofile').read_text().splitlines() - if (m := pattern.match(line))] - if len(values) != 1 or len(values[0]) != 1: - raise RuntimeError('Expected exactly one literal credential assignment') - key = values[0][0] - import httpx - from tinker.cli.auth_api import TinkerAuthApi - with httpx.Client(timeout=30) as client: - identity = TinkerAuthApi(client).get_self_api_key(key) - email = identity.details.user_details.email - organization = identity.details.org_details.name - if email != args.expected_email or organization != args.expected_org: - raise RuntimeError(f'Account guard rejected {email} / {organization}') - os.environ['TINKER_API_KEY'] = key - os.environ.pop('TINKER_CREDENTIAL_CMD', None) - os.environ['HF_HUB_OFFLINE'] = '1' # Reuse the already cached tokenizer only. - root = args.root.resolve() - root.mkdir(parents=True, exist_ok=False) - repo = Path(__file__).resolve().parents[2] - snapshot = root/'source-repo' - shutil.copytree(repo/'frameworks/tinker', snapshot/'frameworks/tinker', - ignore=shutil.ignore_patterns('__pycache__')) - scorer = snapshot/'frameworks/slime/campaign/scoring.py' - scorer.parent.mkdir(parents=True) - shutil.copyfile(repo/'frameworks/slime/campaign/scoring.py', scorer) - shutil.copytree(repo/'data', snapshot/'data') - shutil.copytree(repo/'data', root/'research/data') - shutil.copyfile(repo/'frameworks/tinker/README.md', root/'protocol.md') - (root/'account.json').write_text(json.dumps({ - 'email': email, 'organization': organization, 'key_name': identity.name, - 'credential_label': args.credential_label, 'verified_utc': datetime.now(timezone.utc).isoformat(), - 'account_guard_passed': True}, indent=2)) - (root/'JOURNAL.md').write_text( - '# SemiAnalysis Tinker campaign\n\n' - f'Account verified before any model creation: {email} / {organization}. ' - 'The personal-account campaign remains separate and unchanged.\n\n' - 'Repeat the corrected recipe: baseline; independent 40-update teachers; ' - 'continue unmet provisional goals to at most 80 updates; choose frozen ' - 'teachers; fresh-base 40-update, 128-response/update MOPD student. ' - 'Goals: Countdown 50%, graph 90%; original teacher-quality matching remains ' - 'unverified. Teacher multi-response seeds stay unset. No optimizer mutation ' - 'retries or model-weight downloads. Sources are snapshotted before execution.\n') - scripts = snapshot/'frameworks/tinker' - lock = threading.Lock() - - def record(event, **fields): - row = {'utc': datetime.now(timezone.utc).isoformat(), 'event': event, **fields} - with lock: - with (root/'campaign-events.jsonl').open('a') as stream: - stream.write(json.dumps(row)+'\n') - print(json.dumps(row), flush=True) - - def status(stage, **fields): - temp = root/'status.tmp' - temp.write_text(json.dumps({'stage': stage, **fields}, indent=2)) - temp.replace(root/'status.json') - - def execute(name, script, *arguments): - command = [sys.executable, '-u', str(scripts/script), *map(str, arguments)] - record('process_start', name=name, command=command) - with (root/f'{name}-console.log').open('w') as log: - result = subprocess.run(command, stdout=log, stderr=subprocess.STDOUT) - record('process_exit', name=name, exit_code=result.returncode) - if result.returncode: - raise RuntimeError(f'{name} exited {result.returncode}; see retained console log') - - def teachers(domain): - name = 'teacher-'+domain.replace('_', '-') - initial = root/(name+'-v2') - execute(name+'-v2', 'run.py', 'teacher', '--domain', domain, - '--output', initial, '--skip-baseline') - evaluations = [json.loads(x) for x in (initial/'evaluations.jsonl').read_text().splitlines()] - target = {'countdown': 0.5, 'graph_color': 0.9}[domain] - if not any(e[domain]['accuracy'] >= target for e in evaluations): - checkpoints = [json.loads(x) for x in (initial/'checkpoints.jsonl').read_text().splitlines()] - checkpoint = next(c['state_path'] for c in checkpoints if c['step'] == 40) - execute(name+'-extension', 'run.py', 'teacher', '--domain', domain, - '--output', root/(name+'-extension'), '--checkpoint', checkpoint, - '--start-step', 40, '--steps', 80, '--target-score', target, '--skip-baseline') - - record('account_verified', email=email, organization=organization) - try: - status('baseline', pid=os.getpid(), account=email, organization=organization) - execute('baseline', 'run.py', 'eval', '--output', root/'baseline') - status('teachers', pid=os.getpid(), account=email, organization=organization) - with ThreadPoolExecutor(max_workers=2) as pool: - futures = [pool.submit(teachers, domain) for domain in ('countdown', 'graph_color')] - failures = [] - for future in futures: - try: - future.result() - except Exception as error: - failures.append(str(error)) - if failures: - raise RuntimeError('; '.join(failures)) - execute('teacher-selection', 'select_teachers.py', root) - status('student', pid=os.getpid(), account=email, organization=organization) - execute('student', 'run.py', 'mopd', '--teachers', root/'teachers.json', '--output', root/'student') - execute('audit', 'audit.py', root) - execute('metadata', 'collect_metadata.py', root) - execute('analysis', 'analyze.py', root) - execute('plot', 'plot.py', root) - record('complete') - status('complete', account=email, organization=organization, audit_passed=True) - with (root/'JOURNAL.md').open('a') as stream: - stream.write('\nCampaign completed; see campaign-events.jsonl for all process exit statuses, ' - 'REPORT.md for results, and final-audit.json for verification.\n') - except BaseException: - (root/'campaign-error.txt').write_text(traceback.format_exc()) - record('failed') - status('failed', account=email, organization=organization, error_file='campaign-error.txt') - raise - - -if __name__ == '__main__': - main() diff --git a/frameworks/tinker/plot.py b/frameworks/tinker/plot.py deleted file mode 100644 index 43e286f..0000000 --- a/frameworks/tinker/plot.py +++ /dev/null @@ -1,46 +0,0 @@ -"""Plot measured development checkpoints; requires matplotlib (3.11.2 used).""" -import argparse -import json -from pathlib import Path - -import matplotlib -matplotlib.use('Agg') -import matplotlib.pyplot as plt - - -def main(): - p = argparse.ArgumentParser(description=__doc__) - p.add_argument('root', type=Path) - args = p.parse_args() - runs = json.loads((args.root/'summary.json').read_text()) - baseline = next(r for r in runs if r['mode']=='eval' and r['status']=='complete')['evaluations'][0] - plt.rcParams.update({'font.family':'DejaVu Sans', 'font.size':10, - 'axes.spines.top':False, 'axes.spines.right':False}) - fig, axes = plt.subplots(1,2,figsize=(11,4.6),sharey=True) - for ax, domain, title in zip(axes, ('countdown','graph_color'), ('Countdown','Graph Coloring')): - teacher_points = {0: baseline[domain]['accuracy']*100} - for r in runs: - if r['mode']=='teacher' and r['domain']==domain and r['status']!='invalidated': - teacher_points.update({e['step']:e[domain]['accuracy']*100 for e in r['evaluations']}) - xs=sorted(teacher_points) - ax.plot(xs,[teacher_points[x] for x in xs],'-o',color='#156f63',label='Specialist teacher',linewidth=2) - for r in runs: - if r['mode']=='mopd' and r['status']!='invalidated' and r['evaluations']: - es=r['evaluations'] - ax.plot([e['step'] for e in es],[e[domain]['accuracy']*100 for e in es],'-o',color='#a047a5',label=r['name'],linewidth=2) - ax.set_title(title,loc='left',fontweight='bold') - ax.set_xlabel('Optimizer updates') - ax.set_ylim(0,100) - ax.grid(axis='y',alpha=.2) - ax.legend(frameon=False,loc='lower right') - axes[0].set_ylabel('Development accuracy (%)') - fig.suptitle('Tinker: independently trained teachers and routed MOPD student',fontsize=14,fontweight='bold',y=1.01) - fig.text(.01,-.03,'512 fixed development puzzles/domain · 256-token cap · thinking disabled · single training seed\nTeacher and student updates are separate stages. Lines connect measured checkpoints; original teacher targets unavailable.',fontsize=9,color='#555555') - fig.tight_layout() - for ext in ('png','svg'): - fig.savefig(args.root/f'learning-curves.{ext}',dpi=180,bbox_inches='tight') - plt.close(fig) - - -if __name__=='__main__': - main() diff --git a/frameworks/tinker/run.py b/frameworks/tinker/run.py deleted file mode 100644 index a3da091..0000000 --- a/frameworks/tinker/run.py +++ /dev/null @@ -1,337 +0,0 @@ -"""Remote puzzle GRPO and routed sampled-token OPD using the Tinker SDK. - -Adapted from the first-party cookbook's recipes/rl_loop.py and -distillation/train_on_policy.py. No model weights are downloaded. -""" -from __future__ import annotations - -import argparse -from concurrent.futures import ThreadPoolExecutor, as_completed -from datetime import datetime, timezone -import gzip -import hashlib -import importlib.metadata -import importlib.util -import json -import math -from pathlib import Path -import random -import statistics -import sys -import time -import traceback - -import tinker -from transformers import AutoTokenizer -from tinker_cookbook import renderers - -ROOT = Path(__file__).resolve().parents[2] -MODEL = 'Qwen/Qwen3.6-35B-A3B' -REVISION = '995ad96eacd98c81ed38be0c5b274b04031597b0' -DOMAINS = ('countdown', 'graph_color') -spec = importlib.util.spec_from_file_location('puzzle_scoring', ROOT/'frameworks/slime/campaign/scoring.py') -scoring = importlib.util.module_from_spec(spec) -# Keep bytecode out of the archived campaign-template directory. -_prior_bytecode_setting = sys.dont_write_bytecode -sys.dont_write_bytecode = True -try: - spec.loader.exec_module(scoring) -finally: - sys.dont_write_bytecode = _prior_bytecode_setting - - -def load_data(domain, split): - name = ('countdown4' if domain == 'countdown' else 'graph12') + f'-{split}.jsonl' - raw = gzip.decompress((ROOT/'data'/f'{name}.gz').read_bytes()) - manifest = json.loads((ROOT/'data/manifest.json').read_text())[name] - assert hashlib.sha256(raw).hexdigest() == manifest['sha256'] - rows = [json.loads(line) for line in raw.splitlines()] - assert len(rows) == manifest['rows'] - for i, row in enumerate(rows): - assert row['metadata']['domain'] == domain - row['_index'] = i - return rows - - -def centered_advantages(rewards): - # Match verifier GRPO: normalize within each prompt's response group. - mean = statistics.mean(rewards) - std = statistics.stdev(rewards) if len(rewards) > 1 else 0.0 - return [(v - mean) / (std + 1e-6) for v in rewards] - - -def routed_advantages(prompt_length, response_logprobs, full_teacher_logprobs): - """Teacher logprob index j predicts full_sequence[j], including prompt slots.""" - teacher = full_teacher_logprobs[prompt_length:] - assert len(teacher) == len(response_logprobs) - assert all(v is not None and math.isfinite(v) for v in teacher) - return teacher, [t-s for t,s in zip(teacher, response_logprobs)] - - -def datum(prompt_tokens, response_tokens, logprobs, advantages): - assert response_tokens and len(response_tokens) == len(logprobs) == len(advantages) - assert all(math.isfinite(x) for x in logprobs + advantages) - prefix = len(prompt_tokens) - 1 - return tinker.Datum( - model_input=tinker.ModelInput.from_ints(prompt_tokens + response_tokens[:-1]), - loss_fn_inputs={ - 'target_tokens': tinker.TensorData(data=[0] * prefix + response_tokens, dtype='int64', shape=[prefix + len(response_tokens)]), - 'logprobs': tinker.TensorData(data=[0.0] * prefix + logprobs, dtype='float32', shape=[prefix + len(response_tokens)]), - 'advantages': tinker.TensorData(data=[0.0] * prefix + advantages, dtype='float32', shape=[prefix + len(response_tokens)]), - }, - ) - - -class Run: - def __init__(self, args): - self.args = args - self.out = args.output.resolve() - self.out.mkdir(parents=True, exist_ok=False) - self.started = time.monotonic() - self.pool = ThreadPoolExecutor(max_workers=args.concurrency) - self.event('start', config={k: str(v) if isinstance(v, Path) else v for k,v in vars(args).items()}) - (self.out/'config.json').write_text(json.dumps({k: str(v) if isinstance(v, Path) else v for k,v in vars(args).items()}, indent=2)) - (self.out/'versions.json').write_text(json.dumps({n: importlib.metadata.version(n) for n in ('tinker', 'tinker-cookbook', 'transformers', 'torch', 'numpy')}, indent=2)) - (self.out/'source.py').write_text(Path(__file__).read_text()) - (self.out/'scoring.py').write_text((ROOT/'frameworks/slime/campaign/scoring.py').read_text()) - (self.out/'data-manifest.json').write_text((ROOT/'data/manifest.json').read_text()) - self.tokenizer = AutoTokenizer.from_pretrained(MODEL, revision=REVISION) - self.renderer = renderers.get_renderer('qwen3_5_disable_thinking', self.tokenizer, model_name=MODEL) - self.stop = self.renderer.get_stop_sequences() - self.service = tinker.ServiceClient(user_metadata={'experiment': 'opd-puzzles-20260921', 'stage': args.mode}) - caps = self.service.get_server_capabilities() - assert MODEL in [m.model_name for m in caps.supported_models] - (self.out/'capabilities.json').write_text(caps.model_dump_json(indent=2)) - self.rows = {d: {s: load_data(d, s) for s in ('train', 'dev')} for d in DOMAINS} - self.order = {} - for d in DOMAINS: - self.order[d] = list(range(len(self.rows[d]['train']))) - random.Random(args.seed + DOMAINS.index(d)).shuffle(self.order[d]) - self.event('ready', stop=self.stop, tokenizer_revision=REVISION) - - def write(self, filename, row): - with (self.out/filename).open('a') as f: - f.write(json.dumps(row, allow_nan=False) + '\n') - - def event(self, name, **data): - row = {'event': name, 'utc': datetime.now(timezone.utc).isoformat(), 'elapsed_seconds': time.monotonic()-self.started, **data} - self.write('events.jsonl', row) - print(json.dumps(row), flush=True) - - def prompt(self, row): - # Use the pinned original HF template; fail if cookbook rendering differs. - ids = self.tokenizer.apply_chat_template(row['prompt'], tokenize=True, return_dict=False, add_generation_prompt=True, enable_thinking=False) - cooked = self.renderer.build_generation_prompt(row['prompt']).to_ints() - assert ids == cooked, 'Pinned HF and cookbook prompt tokens differ' - assert len(ids) + self.args.max_tokens <= 2048 - return ids - - def sample_one(self, client, domain, row, group, temperature, seed): - tokens = self.prompt(row) - start = time.monotonic() - result = client.sample(tinker.ModelInput.from_ints(tokens), num_samples=group, - sampling_params=tinker.SamplingParams(max_tokens=self.args.max_tokens, - temperature=temperature, top_p=1.0, top_k=-1, stop=self.stop, - seed=seed if group == 1 else None)).result() - assert len(result.sequences) == group - records = [] - for j, seq in enumerate(result.sequences): - response = self.tokenizer.decode(seq.tokens, skip_special_tokens=True) - assert seq.logprobs is not None and len(seq.tokens) == len(seq.logprobs) - records.append({'domain': domain, 'index': row['_index'], 'sample': j, - 'prompt_tokens': tokens, 'tokens': seq.tokens, 'logprobs': seq.logprobs, - 'response': response, 'score': scoring.score(response, row['label']), - 'stop_reason': seq.stop_reason, 'request_seconds': time.monotonic()-start, - 'seed': seed if group == 1 else None}) - return records - - def sample_batch(self, client, work, group, temperature, step, split): - futures = {self.pool.submit(self.sample_one, client, d, row, group, temperature, - self.args.seed + step*100000 + i): i for i, (d,row) in enumerate(work)} - gathered = {} - for f in as_completed(futures): - i = futures[f] - records = f.result() - for r in records: - self.write('samples.jsonl', {'step': step, 'split': split, **r}) - gathered[i] = records - return [gathered[i] for i in range(len(work))] - - def evaluate(self, client, step): - start = time.monotonic() - work = [(d,r) for d in DOMAINS for r in self.rows[d]['dev'][:self.args.eval_size]] - groups = self.sample_batch(client, work, 1, 0.0, step, 'dev') - metrics = {} - for d in DOMAINS: - rs = [r for g in groups for r in g if r['domain'] == d] - n = len(rs); correct = sum(r['score'] for r in rs) - metrics[d] = {'n': n, 'correct': correct, 'accuracy': correct/n, - 'mean_tokens': statistics.mean(len(r['tokens']) for r in rs), - 'truncated': sum(r['stop_reason'] == 'length' for r in rs)} - self.write('evaluations.jsonl', {'step': step, 'seconds': time.monotonic()-start, **metrics}) - self.event('evaluation', step=step, metrics=metrics) - return metrics - - def checkpoint(self, train, step): - name = f'step-{step:03d}' - state = train.save_state(name=name).result() - sampler = train.save_weights_for_sampler(name=name).result() - row = {'step': step, 'state_path': state.path, 'sampler_path': sampler.path} - self.write('checkpoints.jsonl', row) - self.event('checkpoint', **row) - return row - - def execute(self): - a = self.args - if a.mode == 'eval': - client = self.service.create_sampling_client(model_path=a.checkpoint) if a.checkpoint else self.service.create_sampling_client(base_model=MODEL) - self.evaluate(client, 0) - self.event('complete') - return - if a.checkpoint: - train = self.service.create_training_client_from_state_with_optimizer(a.checkpoint) - else: - train = self.service.create_lora_training_client(MODEL, rank=a.rank, seed=a.seed, - user_metadata={'domain': a.domain or 'mixed', 'run': self.out.name}) - (self.out/'model-info.json').write_text(train.get_info().model_dump_json(indent=2)) - self.event('training_client_created') - teachers = {} - if a.mode == 'mopd': - routes = json.loads(a.teachers.read_text()) - assert set(routes) == set(DOMAINS) - (self.out/'teachers.json').write_text(json.dumps(routes, indent=2)) - teachers = {d: self.service.create_sampling_client(model_path=routes[d]) for d in DOMAINS} - sample = train.save_weights_and_get_sampling_client() - if not a.skip_baseline: - self.evaluate(sample, a.start_step) - completed_updates = 0 - completion_reason = 'step_budget_reached' - for step in range(a.start_step, a.steps): - start = time.monotonic() - domains = (a.domain,) if a.mode == 'teacher' else DOMAINS - count = a.prompts if a.mode == 'teacher' else a.prompts//2 - work = [(d,self.rows[d]['train'][self.order[d][(step*count+i)%len(self.order[d])]]) for d in domains for i in range(count)] - self.event('step_start', step=step+1, prompts=len(work)) - groups = self.sample_batch(sample, work, a.group_size, 1.0, step, 'train') - sampling_seconds = time.monotonic()-start - flat = [r for g in groups for r in g] - teacher_start = time.monotonic() - if a.mode == 'teacher': - for g in groups: - for r, advantage in zip(g, centered_advantages([r['score'] for r in g])): - r['advantages'] = [advantage] * len(r['tokens']) - else: - def teacher_score(r): - lp = teachers[r['domain']].compute_logprobs(tinker.ModelInput.from_ints(r['prompt_tokens']+r['tokens'])).result() - return routed_advantages(len(r['prompt_tokens']), r['logprobs'], lp) - futures = {self.pool.submit(teacher_score,r): r for r in flat} - for f in as_completed(futures): - r = futures[f] - r['teacher_logprobs'], r['advantages'] = f.result() - self.write('teacher-scores.jsonl', {'step': step, **r}) - teacher_seconds = time.monotonic()-teacher_start - datums = [datum(r['prompt_tokens'],r['tokens'],r['logprobs'],r['advantages']) for r in flat] - self.event('forward_backward_start', step=step+1, samples=len(datums)) - learn_start = time.monotonic() - # Never application-retry a mutation: an ambiguous failure stops the run. - fb_future = train.forward_backward(datums, - loss_fn='ppo' if a.mode == 'teacher' else 'importance_sampling', - loss_fn_config={'clip_low_threshold': 0.8, 'clip_high_threshold': 1.2} if a.mode == 'teacher' else None) - opt_future = train.optim_step(tinker.AdamParams(learning_rate=a.learning_rate, - beta1=0.9, beta2=0.98 if a.mode == 'teacher' else 0.999, eps=1e-8, - weight_decay=0.1 if a.mode == 'teacher' else 0.0, grad_clip_norm=1.0)) - fb = fb_future.result() - opt = opt_future.result() - response_deltas = [] - for r, output in zip(flat, fb.loss_fn_outputs, strict=True): - learner_lp = output['logprobs'].data[len(r['prompt_tokens'])-1:] - assert len(learner_lp) == len(r['tokens']) - differences = [float(x)-y for x,y in zip(learner_lp,r['logprobs'])] - assert all(math.isfinite(x) for x in differences) - response_deltas.extend(differences) - self.write('learner-scores.jsonl', {'step': step, 'domain': r['domain'], - 'index': r['index'], 'sample': r['sample'], 'logprobs': learner_lp, - 'advantages': r['advantages']}) - learn_seconds = time.monotonic()-learn_start - sample = train.save_weights_and_get_sampling_client() - row = {'step': step+1, 'samples': len(flat), 'learning_rate': a.learning_rate, - 'sampling_seconds': sampling_seconds, 'teacher_seconds': teacher_seconds, - 'train_seconds': learn_seconds, 'step_seconds': time.monotonic()-start, - 'prompt_tokens': sum(len(r['prompt_tokens']) for r in flat), - 'response_tokens': sum(len(r['tokens']) for r in flat), - 'response_train_sample_logprob_delta_mean': statistics.mean(response_deltas), - 'response_train_sample_logprob_delta_abs_mean': statistics.mean(abs(x) for x in response_deltas), - 'mean_unique_responses_per_group': statistics.mean(len(set(r['response'] for r in g)) for g in groups), - 'scores': {d: statistics.mean(r['score'] for r in flat if r['domain']==d) for d in domains}, - 'nonzero_advantage_samples': sum(any(v != 0 for v in r['advantages']) for r in flat), - 'forward_backward_metrics': fb.metrics, 'optimizer_metrics': opt.metrics} - if a.mode == 'mopd': - row['sampled_reverse_kl_by_domain'] = { - d: -statistics.mean(x for r in flat if r['domain'] == d for x in r['advantages']) - for d in domains} - row['accepted_policy_lag'] = 0 - self.write('metrics.jsonl', row) - self.event('step_complete', **row) - completed_updates += 1 - if (step+1)%a.save_every == 0 or step+1 == a.steps: - self.checkpoint(train,step+1) - if (step+1)%a.eval_every == 0 or step+1 == a.steps: - evaluation = self.evaluate(sample,step+1) - if a.mode == 'teacher' and a.target_score is not None and evaluation[a.domain]['accuracy'] >= a.target_score: - if (step+1)%a.save_every != 0 and step+1 != a.steps: - self.checkpoint(train,step+1) - completion_reason = 'teacher_target_reached' - break - self.event('complete', optimizer_updates=completed_updates, - total_optimizer_updates=a.start_step+completed_updates, reason=completion_reason) - - -def main(): - p = argparse.ArgumentParser(description=__doc__) - p.add_argument('mode', choices=('eval','teacher','mopd')) - p.add_argument('--output',type=Path,required=True) - p.add_argument('--domain',choices=DOMAINS) - p.add_argument('--teachers',type=Path) - p.add_argument('--checkpoint') - p.add_argument('--start-step',type=int,default=0) - p.add_argument('--steps',type=int,default=40) - p.add_argument('--prompts',type=int) - p.add_argument('--group-size',type=int) - p.add_argument('--learning-rate',type=float,default=1e-5) - p.add_argument('--target-score',type=float,help='Optional teacher-only dev threshold; stops after a scheduled evaluation') - p.add_argument('--rank',type=int,default=64) - p.add_argument('--seed',type=int,default=20260921) - p.add_argument('--max-tokens',type=int,default=256) - p.add_argument('--eval-size',type=int,default=512) - p.add_argument('--eval-every',type=int,default=10) - p.add_argument('--save-every',type=int,default=10) - p.add_argument('--concurrency',type=int,default=32) - p.add_argument('--skip-baseline',action='store_true') - args = p.parse_args() - args.prompts = args.prompts or (32 if args.mode == 'teacher' else 128) - args.group_size = args.group_size or (8 if args.mode == 'teacher' else 1) - for key in ('prompts', 'group_size', 'max_tokens', 'rank', 'eval_every', 'save_every', 'concurrency'): - if getattr(args, key) <= 0: p.error(f'{key} must be positive') - if not 1 <= args.eval_size <= 512: p.error('--eval-size must be between 1 and 512') - if not math.isfinite(args.learning_rate) or args.learning_rate <= 0: p.error('learning rate must be finite and positive') - if args.mode != 'eval' and not 0 <= args.start_step < args.steps: p.error('require 0 <= start-step < steps') - if args.start_step and not args.checkpoint: p.error('--start-step requires an optimizer-state checkpoint') - if args.mode == 'teacher' and not args.domain: p.error('teacher requires --domain') - if args.target_score is not None and (args.mode != 'teacher' or not 0 < args.target_score <= 1): - p.error('--target-score is a teacher-only fraction in (0, 1]') - if args.mode == 'mopd' and (not args.teachers or args.prompts%2 or args.group_size != 1): p.error('MOPD requires --teachers, even prompt count, and group size 1') - run = None - try: - run = Run(args) - run.execute() - except BaseException as exc: - if run: - run.event('failed', error_type=type(exc).__name__, message=str(exc)) - (run.out/'error.txt').write_text(traceback.format_exc()) - raise - finally: - if run: run.pool.shutdown(wait=False, cancel_futures=True) - - -if __name__ == '__main__': - main() diff --git a/frameworks/tinker/select_teachers.py b/frameworks/tinker/select_teachers.py deleted file mode 100644 index 28a1fbc..0000000 --- a/frameworks/tinker/select_teachers.py +++ /dev/null @@ -1,57 +0,0 @@ -"""Select completed, valid teacher checkpoints using recorded dev evaluations.""" -import argparse -import json -from pathlib import Path - - -def read(path): - return [json.loads(x) for x in path.read_text().splitlines()] - - -def choose(rows, domain, target): - qualifying = [r for r in rows if r['evaluation'][domain]['accuracy'] >= target] - chosen = min(qualifying,key=lambda r:r['step']) if qualifying else max(rows,key=lambda r:(r['evaluation'][domain]['accuracy'],-r['step'])) - return {**chosen, 'provisional_target':target, 'target_met':bool(qualifying)} - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument('root',type=Path) - args = parser.parse_args() - root = args.root.resolve() - targets = {'countdown':0.5, 'graph_color':0.9} - candidates = {d:[] for d in targets} - for config_path in sorted(root.glob('*/config.json')): - path = config_path.parent - config = json.loads(config_path.read_text()) - if config['mode'] != 'teacher' or (path/'INVALIDATED.json').exists(): - continue - events = read(path/'events.jsonl') - if events[-1]['event'] != 'complete': - raise RuntimeError(f'Teacher attempt is not complete: {path.name}') - checkpoints = {r['step']:r for r in read(path/'checkpoints.jsonl')} - domain = config['domain'] - for evaluation in read(path/'evaluations.jsonl'): - if evaluation['step'] not in checkpoints: - continue - candidates[domain].append({'source_run':path.name, 'step':evaluation['step'], - 'evaluation':evaluation, 'checkpoint':checkpoints[evaluation['step']]}) - selected = {} - for domain, rows in candidates.items(): - if not rows: - raise RuntimeError(f'No saved, evaluated teacher for {domain}') - selected[domain] = choose(rows,domain,targets[domain]) - paths = {d:r['checkpoint']['sampler_path'] for d,r in selected.items()} - assert len(set(paths.values()))==2 - report = {'rule':'First scheduled checkpoint reaching provisional goal; otherwise highest own-domain dev score, earliest on ties.', - 'original_teacher_quality_match_verified':False, - 'target_source':'Operational goals; original teacher scores unavailable. Not claimed as original teacher benchmarks.', - 'selected':selected} - for filename, value in [('teacher-selection.json',report),('teachers.json',paths)]: - with (root/filename).open('x') as f: - json.dump(value,f,indent=2) - print(json.dumps(report,indent=2)) - - -if __name__ == '__main__': - main() diff --git a/frameworks/tinker/test_run.py b/frameworks/tinker/test_run.py deleted file mode 100644 index 2b4a914..0000000 --- a/frameworks/tinker/test_run.py +++ /dev/null @@ -1,59 +0,0 @@ -"""Offline checks for the train/eval boundary and token alignment.""" -import unittest - -from run import centered_advantages, datum, load_data, routed_advantages, scoring -from select_teachers import choose - - -class TrainingTests(unittest.TestCase): - def test_teacher_selection_goal_and_fallback(self): - rows = [{'step':step, 'evaluation':{'countdown':{'accuracy':score}}} - for step,score in [(10,.3),(20,.5),(30,.6),(40,.5)]] - self.assertEqual(choose(rows,'countdown',.5)['step'],20) - selected = choose(rows,'countdown',.7) - self.assertEqual(selected['step'],30) - self.assertFalse(selected['target_met']) - - def test_next_token_alignment_and_prompt_mask(self): - d = datum([10, 11, 12], [20, 21], [-0.5, -0.7], [1.5, -0.2]) - self.assertEqual(d.model_input.to_ints(), [10, 11, 12, 20]) - self.assertEqual(d.loss_fn_inputs['target_tokens'].data, [0, 0, 20, 21]) - for actual, expected in zip(d.loss_fn_inputs['advantages'].data, [0, 0, 1.5, -0.2]): - self.assertAlmostEqual(actual, expected) - - def test_group_advantages(self): - self.assertEqual(centered_advantages([0]*8), [0]*8) - self.assertEqual(centered_advantages([1]*8), [0]*8) - values = centered_advantages([1, 0, 0, 0, 0, 0, 0, 0]) - self.assertAlmostEqual(sum(values), 0) - self.assertGreater(values[0], 0) - self.assertLess(values[1], 0) - - def test_nonfinite_supervision_rejected(self): - with self.assertRaises(AssertionError): - datum([10], [20], [-0.5], [float('nan')]) - - def test_teacher_alignment_and_reverse_kl_sign(self): - teacher, adv = routed_advantages(3, [-2.0, -0.1], [None, -5, -8, -1.0, -0.5]) - self.assertEqual(teacher, [-1.0, -0.5]) - self.assertEqual(adv, [1.0, -0.4]) - with self.assertRaises(AssertionError): - routed_advantages(2, [-1], [None, -5, None]) - - def test_exact_data_and_no_dev_training_overlap(self): - for domain in ('countdown', 'graph_color'): - train = load_data(domain, 'train') - dev = load_data(domain, 'dev') - self.assertEqual(len(train), 10000) - self.assertEqual(len(dev), 512) - train_prompts = {str(r['prompt']) for r in train} - self.assertFalse(train_prompts.intersection(str(r['prompt']) for r in dev)) - - def test_countdown_uses_all_numbers(self): - label = {'domain': 'countdown', 'numbers': [2, 3, 4, 5], 'target': 14} - self.assertEqual(scoring.score('2+3+4+5', label), 1) - self.assertEqual(scoring.score('14', label), 0) - - -if __name__ == '__main__': - unittest.main() diff --git a/frameworks/verl/node.py b/frameworks/verl/node.py new file mode 100644 index 0000000..8c38f51 --- /dev/null +++ b/frameworks/verl/node.py @@ -0,0 +1,142 @@ +"""verl entry point: `node.py` runs the controller, `node.py ROLE` runs one node. + +verl runs on the hosts from the campaign's `source/.venv`. Every role joins one Ray cluster and verl +places its own resource pools on it: the trainer on one node, the policy engines and frozen teachers +on the generation node (b200x2) or nodes (h100x3, one teacher per node). The learner role runs the +driver (`train.py`). Each node's capture also scrapes the vLLM servers Ray starts there. +""" + +import os +from pathlib import Path +import shutil +import socket +import subprocess +import sys +import time + +from shared import recipe +from shared.slurm import (CACHE, CACHE_ENV, PYDEPS, ROOT, Allocation, Supervisor, STOP_FILE, exit_on_signals, generation_hosts, + result_dir, site, write_json) + +RAY_PORT = 6379 + + +def generation_roles(s): + """{role: (node, IP)} for the generation nodes: `generation`, then `generation_2` (h100x3).""" + return {f'generation{"_" + str(i + 1) if i else ""}': host for i, host in enumerate(generation_hosts(s))} + + +def uv_python(s): + # --no-sync keeps a run from silently replacing the provisioned environment. + return [s['uv'], 'run', '--project', str(ROOT / 'source'), '--no-sync', 'python'] + + +def control(): + allocation = Allocation('verl') + s, result = allocation.site, allocation.result + allocation.start() + env = {**os.environ, 'CAMPAIGN_RESULT': str(result), 'CAMPAIGN_PYDEPS': str(PYDEPS), + 'PYTHONPATH': str(ROOT), 'UV_CACHE_DIR': str(ROOT / 'uv-cache'), 'UV_NO_SYNC': '1', + 'WANDB_MODE': 'disabled', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', + # Ray workers already use the venv's Python via `ray start`. Ray's `uv run` hook would require + # pyproject.toml in the working directory and ship it to every worker. + 'RAY_ENABLE_UV_RUN_RUNTIME_ENV': '0'} + + def freeze_packages(): + with (result / 'installed-packages.txt').open('wb') as packages: + subprocess.run([s['uv'], 'pip', 'freeze', '--python', str(ROOT / 'source/.venv/bin/python')], + stdout=packages, check=True) + + srun = ['srun', '--exclusive', '-N1', '-n1', f'--gres=gpu:{recipe.GPUS_PER_NODE}', + f'--cpus-per-task={recipe.CPUS_PER_NODE}'] + nodes = {'learner': s['trainer_node'], **{role: node for role, (node, _) in generation_roles(s).items()}} + steps = {role: [*srun, '-w', node, *uv_python(s), str(ROOT / 'node.py'), role] for role, node in nodes.items()} + return allocation.run(steps, env, before=freeze_packages) + + +def wait_for_gpus(head, processes, timeout=300): + """Start the driver only once every node's GPUs are in the Ray cluster.""" + import ray + ray.init(address=f'{head}:{RAY_PORT}') + deadline = time.monotonic() + timeout + while int(ray.cluster_resources().get('GPU', 0)) != (1 + recipe.GENERATION_NODES) * recipe.GPUS_PER_NODE: + processes.check() + if time.monotonic() > deadline: + raise TimeoutError(f'Ray has {ray.cluster_resources().get("GPU", 0)} GPUs') + time.sleep(2) + ray.shutdown() + + +def keep_trainer_log(result, role): + """Copy the trainer actor's Ray worker log, which holds verl's per-step console metrics, into the result. + + Ray keeps worker logs on the node that ran them and does not always forward them to the driver. + """ + try: + for log in Path('/tmp/ray/session_latest/logs').glob('worker-*.out'): + if b'training/global_step' in log.read_bytes(): + shutil.copyfile(log, result / f'{role}-trainer-worker.out') + except OSError as error: + print(f'could not copy the trainer log: {error}', file=sys.stderr) + + +def main(role): + s, result = site(), result_dir() + generation = generation_roles(s) + ip = s['trainer_ip'] if role == 'learner' else generation[role][1] + processes = Supervisor(result, prefix=f'{role}-') + exit_on_signals() + # Slurm's GPU plugin also sets the AMD variable, and verl workers refuse to start if both are set. + os.environ.pop('ROCR_VISIBLE_DEVICES', None) + ray = ['ray', 'start', f'--node-ip-address={ip}', f'--num-gpus={recipe.GPUS_PER_NODE}', + f'--num-cpus={recipe.RAY_CPUS_PER_NODE}', '--disable-usage-stats', '--block'] + # `--vllm` scrapes the Prometheus endpoint of each vLLM server (policy or teacher) on this node. + capture = [sys.executable, '-m', 'shared.capture', role, '--vllm'] + # Ray workers inherit this. Without it the trainer can run out of memory from fragmentation at + # 32k, and verl already enables it for its vLLM servers. + ray_env = {'PYTORCH_CUDA_ALLOC_CONF': 'expandable_segments:True', **CACHE_ENV} + if recipe.H100: + # On the CoreWeave NFS share, vLLM engines on different nodes compiling into one shared cache hit + # "Stale file handle"; the failing TP rank's partner then waited forever (job 542). Keep the + # compiled-kernel caches on each node's local disk instead. + local = Path('/tmp/opd-runtime-cache') / ROOT.name + ray_env.update({key: str(local / Path(value).relative_to(CACHE)) if Path(value) != CACHE else str(local) + for key, value in CACHE_ENV.items()}) + if recipe.TRAINER_OPTIMIZER_CPU_OFFLOAD: + # Ray gives each worker OMP_NUM_THREADS=1 unless it is set, which runs the CPU optimizer step on one + # core per rank (as in Miles run 532). Ray places verl's trainer pool on any node, so set it everywhere. + ray_env['OMP_NUM_THREADS'] = str(recipe.CPUS_PER_NODE // recipe.TRAINER_GPUS) + try: + processes.launch(capture, 'capture') + if role == 'learner': + processes.launch([*ray, '--head'], 'ray', ray_env) + write_json(result / 'head-started.json', {'ip': ip}) + for name in generation: + processes.wait_for_file(f'{name}-ready.json') + wait_for_gpus(ip, processes) + processes.launch([sys.executable, str(ROOT / 'train.py')], 'train', {'RAY_ADDRESS': f'{ip}:{RAY_PORT}'}) + return processes.run_until_exit('train') + head = processes.wait_for_file('head-started.json')['ip'] + deadline = time.monotonic() + 180 + while True: # The head writes its address before its GCS socket accepts connections. + try: + socket.create_connection((head, RAY_PORT), timeout=2).close() + break + except OSError: + processes.check() + if time.monotonic() > deadline: + raise TimeoutError('The Ray head did not accept connections') + time.sleep(2) + processes.launch([*ray, f'--address={head}:{RAY_PORT}'], 'ray', ray_env) + write_json(result / f'{role}-ready.json', {'ip': ip}) + while not (result / STOP_FILE).exists(): + processes.check() + time.sleep(3) + return 0 + finally: + keep_trainer_log(result, role) + processes.stop_all() + + +if __name__ == '__main__': + sys.exit(control() if len(sys.argv) == 1 else main(sys.argv[1])) diff --git a/frameworks/verl/train.py b/frameworks/verl/train.py new file mode 100644 index 0000000..9f9ac6b --- /dev/null +++ b/frameworks/verl/train.py @@ -0,0 +1,163 @@ +"""verl recipe: multi-teacher OPD on verl's fully async trainer, built from `shared.recipe`. + +Follows upstream `examples/on_policy_distillation_trainer` (teachers routed by `data_source`, +sampled-token k1 reverse KL applied as a policy gradient, no task reward), run on +`verl.experimental.fully_async_policy` with FSDP training and vLLM serving. The learner role runs +this from the campaign's `source/` and saves the overrides as `argv.json`. `compute_score` is verl's +reward hook; its score is diagnostic only. Each trained batch, with its scores, is written to +`/rollouts/.jsonl`. +""" +import json +import os +import subprocess +import sys + +from shared import recipe +from shared.slurm import DATA, ROOT, result_dir, site + +# Concurrent policy requests. b200x2: 256, as in the other frameworks. h100x3: 160 (1.25 batches), as in +# Miles run 535; at policy lag 1 a deeper pipeline mostly produces samples that are dropped as stale. +MAX_IN_FLIGHT = 160 if recipe.H100 else 256 +# verl's staleness_threshold is a fraction of a batch: the rollouter generates at most (1 + threshold) +# batches per weight sync. 1 matches Miles' policy lag 1 with a one-batch buffer; concurrency is capped +# separately by MAX_IN_FLIGHT. (0.25 in job 544 also capped generation at 160 samples per sync, so the +# rollouter stopped every update and the trainer waited a median 78 s for rollouts.) +STALENESS_THRESHOLD = recipe.MAX_POLICY_LAG +# Tokens per GPU per micro-batch. h100x3 offloads the parameters to the CPU (below), so every micro-batch +# copies them back to the GPU; job 544 peaked at 14 GB of 80 GB with MAX_TOKENS_PER_GPU split over the two +# sequence-parallel GPUs, so each GPU takes a full MAX_TOKENS_PER_GPU instead (half the micro-batches). +# This changes only the batching, not the gradient. +MICRO_BATCH_TOKENS_PER_GPU = recipe.MAX_TOKENS_PER_GPU if recipe.H100 else recipe.MAX_TOKENS_PER_GPU // recipe.TRAINER_CONTEXT_PARALLEL +# Teacher pool: b200x2 puts every teacher on the generation node, one GPU each; h100x3 puts one TP2 +# teacher on each generation node. verl splits the pool per teacher, in DOMAINS order. +TEACHER_NODES = len({node for node, _ in recipe.TEACHER_PLACEMENT.values()}) +TEACHER_GPUS_PER_NODE = len(recipe.DOMAINS) * recipe.TEACHER_GPUS_PER_ENGINE // TEACHER_NODES + + +def write_parquet(result): + """Write parquet rows for verl: chat prompt, teacher routing key and scoring label.""" + import pandas + + def rows(name): + for line in (DATA / name).read_text().splitlines(): + row = json.loads(line) + yield {'prompt': row['prompt'], 'data_source': row['metadata']['domain'], + 'reward_model': {'style': 'rule', 'ground_truth': row['label']}, + 'extra_info': {'index': row['metadata']['source_index']}} + + out = result / 'verl-data' + out.mkdir(exist_ok=True) + pandas.DataFrame(rows(recipe.TRAIN_FILE)).to_parquet(out / 'train.parquet') + dev = [out / f'{d}-dev.parquet' for d in recipe.DOMAINS] + for domain, path in zip(recipe.DOMAINS, dev): + pandas.DataFrame(rows(recipe.data_file(domain, 'dev'))).to_parquet(path) + return out / 'train.parquet', dev + + +def overrides(result, train, dev): + s, r = site(), recipe + prompt_tokens = r.CONTEXT_LENGTH - r.MAX_RESPONSE_TOKENS + policy_engines = r.ROLLOUT_GPUS // r.POLICY_GPUS_PER_ENGINE # across every generation node + # As in upstream's example, one teacher uses the default `teacher_model` entry; several are added + # as named entries routed by `data_source`. The +1 in max_model_len fits the student's prompt and + # full response plus one generated teacher token. + # disable_log_stats=False turns on vLLM's Prometheus request metrics (latency, tokens), which + # shared/capture.py scrapes; verl turns them off by default. + inference = ['inference.name=vllm', f'inference.tensor_model_parallel_size={r.TEACHER_GPUS_PER_ENGINE}', + 'inference.gpu_memory_utilization=0.8', f'inference.max_model_len={r.CONTEXT_LENGTH + 1}', + 'inference.disable_log_stats=False'] + if len(r.DOMAINS) == 1: + entry = 'distillation.teacher_models.teacher_model' + teachers = [f'{entry}.model_path={s["teachers"]}/{r.DOMAINS[0]}', *(f'{entry}.{x}' for x in inference)] + else: + # Named entries do not inherit the `teacher_model` block of distillation.yaml; they start from + # RolloutConfig, whose load_format is "dummy" (random weights) and whose prompt and response + # lengths are 512. Set the values the default entry would have. + named = [*inference, 'inference.load_format=auto', 'inference.enforce_eager=True', + f'inference.prompt_length={prompt_tokens}', f'inference.response_length={r.MAX_RESPONSE_TOKENS}'] + teachers = [value for d in r.DOMAINS for value in ( + f'+distillation.teacher_models.{d}.key={d}', + f'+distillation.teacher_models.{d}.model_path={s["teachers"]}/{d}', + f'+distillation.teacher_models.{d}.num_replicas=1', + *(f'+distillation.teacher_models.{d}.{x}' for x in named))] + return [ + # Data: one prompt at a time; with two domains, the mixed file interleaves them. + f'data.train_files=["{train}"]', f'data.val_files={json.dumps([str(p) for p in dev])}', + 'data.prompt_key=prompt', 'data.return_raw_chat=True', 'data.train_batch_size=0', 'data.gen_batch_size=1', + f'data.max_prompt_length={prompt_tokens}', f'data.max_response_length={r.MAX_RESPONSE_TOKENS}', + 'data.filter_overlong_prompts=True', 'data.truncation=error', 'data.shuffle=True', + f'+data.apply_chat_template_kwargs.enable_thinking={r.ENABLE_THINKING}', + # Objective: sampled-token reverse KL to the routed teacher as the only reward. + 'algorithm.adv_estimator=grpo', 'algorithm.use_kl_in_reward=False', + 'distillation.enabled=True', f'distillation.n_gpus_per_node={TEACHER_GPUS_PER_NODE}', + f'distillation.nnodes={TEACHER_NODES}', + 'distillation.teacher_key=data_source', *teachers, + 'distillation.distillation_loss.loss_mode=k1', 'distillation.distillation_loss.use_policy_gradient=True', + 'distillation.distillation_loss.use_task_rewards=False', + 'distillation.distillation_loss.loss_max_clamp=10.0', 'distillation.distillation_loss.log_prob_min_clamp=-10.0', + f'reward.custom_reward_function.path={ROOT / "train.py"}', 'reward.custom_reward_function.name=compute_score', + # Student model and FSDP trainer, as in upstream run_qwen3_8b_mopd_fsdp.sh; no sequence parallelism. + f'actor_rollout_ref.model.path={s["base_model"]}', 'actor_rollout_ref.model.trust_remote_code=True', + 'actor_rollout_ref.model.use_remove_padding=True', 'actor_rollout_ref.model.enable_gradient_checkpointing=True', + 'actor_rollout_ref.actor.use_torch_compile=True', 'actor_rollout_ref.hybrid_engine=False', + f'actor_rollout_ref.actor.optim.lr={r.LEARNING_RATE}', 'actor_rollout_ref.actor.optim.weight_decay=0', + 'actor_rollout_ref.actor.optim.clip_grad=1.0', 'actor_rollout_ref.actor.entropy_coeff=0', + 'actor_rollout_ref.actor.use_kl_loss=False', + f'actor_rollout_ref.actor.ppo_mini_batch_size={r.PROMPTS_PER_UPDATE}', + 'actor_rollout_ref.actor.use_dynamic_bsz=True', + # verl packs this times the sequence-parallel size into each micro-batch; see MICRO_BATCH_TOKENS_PER_GPU. + f'actor_rollout_ref.actor.ppo_max_token_len_per_gpu={MICRO_BATCH_TOKENS_PER_GPU}', + # h100x3: two GPUs share each sequence (Ulysses), the counterpart of Miles' context parallelism 2. + f'actor_rollout_ref.actor.fsdp_config.ulysses_sequence_parallel_size={r.TRAINER_CONTEXT_PARALLEL}', + # h100x3: the counterpart of Miles' CPU Adam is FSDP2's CPUOffloadPolicy: fp32 parameters, gradients + # and Adam states stay in pinned CPU memory and the optimizer step runs on the CPU. verl's + # `optimizer_offload` is not enough: it moves the Adam states back to the GPU for the step, which ran + # out of memory in update 1 (job 543). Entropy checkpointing as in upstream's Qwen3.5 OPD examples. + *([f'actor_rollout_ref.actor.strategy=fsdp2', 'actor_rollout_ref.actor.fsdp_config.offload_policy=True', + 'actor_rollout_ref.actor.fsdp_config.entropy_checkpointing=True'] + if r.TRAINER_OPTIMIZER_CPU_OFFLOAD else []), + # Policy engines: vLLM on their own GPUs, weights synced after every update. + 'actor_rollout_ref.rollout.name=vllm', 'actor_rollout_ref.rollout.mode=async', + f'actor_rollout_ref.rollout.n={r.SAMPLES_PER_PROMPT}', + f'actor_rollout_ref.rollout.tensor_model_parallel_size={r.POLICY_GPUS_PER_ENGINE}', + 'actor_rollout_ref.rollout.gpu_memory_utilization=0.8', 'actor_rollout_ref.rollout.disable_log_stats=False', + f'actor_rollout_ref.rollout.max_model_len={r.CONTEXT_LENGTH + 1}', + 'actor_rollout_ref.rollout.temperature=1.0', 'actor_rollout_ref.rollout.top_p=1.0', + f'actor_rollout_ref.rollout.prompt_length={prompt_tokens}', + f'actor_rollout_ref.rollout.response_length={r.MAX_RESPONSE_TOKENS}', + 'actor_rollout_ref.rollout.log_prob_use_dynamic_bsz=True', + f'actor_rollout_ref.rollout.log_prob_max_token_len_per_gpu={MICRO_BATCH_TOKENS_PER_GPU}', + 'actor_rollout_ref.rollout.val_kwargs.do_sample=False', 'actor_rollout_ref.rollout.val_kwargs.n=1', + f'rollout.nnodes={r.GENERATION_NODES}', f'rollout.n_gpus_per_node={len(r.POLICY_GPUS)}', + f'rollout.n={r.SAMPLES_PER_PROMPT}', + f'rollout.total_rollout_steps={r.UPDATES * r.PROMPTS_PER_UPDATE}', + # A weight sync after every update; see STALENESS_THRESHOLD. + f'async_training.staleness_threshold={STALENESS_THRESHOLD}', 'async_training.trigger_parameter_sync_step=1', + # Each weight sync aborts in-flight requests; partial rollout (the trainer's default) resumes + # them instead of returning them truncated. + 'async_training.require_batches=1', 'async_training.partial_rollout=True', + f'async_training.concurrent_samples_per_replica={-(-MAX_IN_FLIGHT // policy_engines)}', + # Trainer, evaluation and checkpoints. + 'trainer.nnodes=1', f'trainer.n_gpus_per_node={r.TRAINER_GPUS}', 'trainer.total_epochs=1', 'trainer.balance_batch=True', + # h100x3 evaluates only after the last update, as Miles run 535 does; greedy 30k-token evaluation + # pauses the rollouter, and tools/benchmark.py evaluates every saved checkpoint offline. + f'trainer.val_before_train={not r.H100}', f'trainer.test_freq={r.UPDATES if r.H100 else r.EVAL_INTERVAL}', + f'trainer.save_freq={r.SAVE_INTERVAL}', + f'trainer.default_local_dir={ROOT / "checkpoints" / result.name}', 'trainer.logger=["console","tensorboard"]', + f'trainer.rollout_data_dir={result / "rollouts"}', + 'trainer.project_name=opd', f'trainer.experiment_name={result.name}'] + + +def compute_score(data_source, solution_str, ground_truth, extra_info=None): + from shared.scoring import score + return score(solution_str, ground_truth) + + +if __name__ == '__main__': + result = result_dir() + args = overrides(result, *write_parquet(result)) + (result / 'argv.json').write_text(json.dumps(args, indent=2)) + command = [sys.executable, '-m', 'verl.experimental.fully_async_policy.fully_async_main', + '--config-name=fully_async_ppo_trainer', *args] + # verl's config search path is relative to the source root. + sys.exit(subprocess.run(command, cwd=ROOT / 'source', env={**os.environ, 'TENSORBOARD_DIR': str(result / 'tensorboard')}).returncode) diff --git a/frameworks/verl/upstream/LICENSE b/frameworks/verl/upstream/LICENSE new file mode 100644 index 0000000..d645695 --- /dev/null +++ b/frameworks/verl/upstream/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/frameworks/verl/upstream/manifest.json b/frameworks/verl/upstream/manifest.json new file mode 100644 index 0000000..6e65e5b --- /dev/null +++ b/frameworks/verl/upstream/manifest.json @@ -0,0 +1,6 @@ +{ + "repository": "https://github.com/verl-project/verl.git", + "revision": "6093e007cc341973c9d9a6fb3867a85976c7c458", + "source_patch_sha256": null, + "note": "Unmodified upstream. The recipe follows examples/on_policy_distillation_trainer (multi-teacher OPD), run on the fully async trainer in verl/experimental/fully_async_policy." +} diff --git a/hosted/fireworks/README.md b/hosted/fireworks/README.md new file mode 100644 index 0000000..d30137b --- /dev/null +++ b/hosted/fireworks/README.md @@ -0,0 +1,46 @@ +# Fireworks: hosted teacher RL, then MOPD + +`campaign.py` trains one RL teacher per domain, then trains a fresh student on both domains by multi-teacher on-policy distillation (MOPD). It uses the shared recipe in `shared/recipe.py` (reasoning_gym `caesar_cipher` and `simple_geometry`, thinking on). Unlike the self-hosted frameworks, which distill the frozen `recipe.TEACHERS`, this campaign trains its own teachers. Fireworks settings are at the top of `campaign.py`; budget settings are at the top of `lifecycle.py`. + +## Setup + +- **Billing.** Uses the dedicated Training API, billed per GPU time: one four-B200 trainer plus a one-replica rollout deployment. It does not use the per-token Serverless API. All weights and checkpoints stay on Fireworks. +- **Base model.** Qwen3.5-35B-A3B, not the Qwen3.6 base used elsewhere, so comparisons with other runs are not same-base. Rollouts use FP8 inference without router replay; the trainer-versus-sampler logprob gap is logged. +- **Prompts.** Qwen3.5 chat template with `enable_thinking=True`, up to 30,720 response tokens, `max_context_length` 32,768, stop at EOS. A response scores 1 only if the text after its last `` contains an `` that reasoning_gym accepts. +- **Teachers.** Rank-64 LoRAs trained with PPO on group-normalized rewards: 32 prompts × 8 responses per update, LR 1e-5 (10× the recipe's full-finetuning 1e-6), no weight decay, at most 200 updates. Every 25 updates they are evaluated greedily on the full dev split (126 caesar_cipher, 128 simple_geometry problems). The best evaluated checkpoint is kept, even if it misses the target. +- **Teacher targets.** 70% on caesar_cipher and 95% on simple_geometry. These are operational goals set just below the held-out scores of the GRPO teachers in `recipe.TEACHERS` (73.3% and 99.8%, sampled at temperature 1). Reaching them does not mean the teachers match `recipe.TEACHERS`. +- **Student.** 20 updates × 128 responses (64 per domain), LR 1e-5, with teacher-minus-student token advantages. Training is synchronous (policy lag 0, within `recipe.MAX_POLICY_LAG`). +- **Single session.** The backend allows one resident LoRA session. At each MOPD update the controller saves the student, loads each frozen teacher checkpoint to score that teacher's domain, then restores the student's weights and optimizer state before the training step. This checkpoint I/O is counted as teacher time. + +## Prerequisites + +- `fireworks-ai[training]==1.2.14` +- `reasoning-gym==0.1.25`, installed or with its site-packages directory in `$CAMPAIGN_PYDEPS` +- `FIREWORKS_API_KEY` exported, or an `api_key_profile_label` in `config/fireworks.local.json` so the key is read from `~/.zprofile` +- The account ID, from `--account`, `config/fireworks.local.json`, or `FIREWORKS_ACCOUNT_ID` +- A tokenizer info JSON (`model`, `revision`, `path`, `eos_token_id`) for a locally cached `Qwen/Qwen3.5-35B-A3B` tokenizer + +## Run + +``` +python hosted/fireworks/campaign.py --root NEW_RUN_ROOT --tokenizer-info TOKENIZER_JSON [--account ACCOUNT] +python hosted/fireworks/report.py RUN_ROOT # Also runs automatically after verified cleanup. +cd hosted/fireworks && python test_campaign.py # Offline regression tests. +``` + +The launcher verifies the account, copies `hosted/fireworks`, `shared` and `data` into `RUN_ROOT/source-repo/`, and runs that frozen copy. + +- `--teacher-config JSON` overrides the teacher `prompts`, `learning_rate`, `eval_every`, `max_steps` and `target_tolerance`. Any override changes the protocol and must be reported as such. +- `--resume-plan JSON` maps each domain to `{source_root, step, restore_reference, best, [prompt_offset], [complete]}`. Use `cross_job://` references, not sampler snapshots. The source run must match on account, tokenizer revision and data hashes. If the teacher batch size changed, carry `prompt_offset` over. + +`report.py` writes `audit.json` and `REPORT.md`, including a check of the student protocol (20 updates, 2,560 responses, 64 per domain per update) and an audit of the checkpoint swaps. A partial run does not count as a completed protocol. + +## Cost + +Thinking at 32k tokens costs far more than the earlier 256-token protocol. GRPO teacher responses averaged about 11k tokens on caesar_cipher and 4k on simple_geometry, and one teacher can run 200 updates of 256 responses. The default $200 budget will very likely stop the campaign during the first teacher, so set `--budget-usd` deliberately. + +## Budget guard + +An independent watchdog starts before provisioning. It estimates spend from the start of provisioning at $117/hour (six B200s at the public $13/hour, plus 50% headroom), plus `--prior-spend-usd`. It stops the run when the estimate reaches `--budget-usd` minus a $25 cleanup reserve (175 with the default budget of 200), when training ends, or when the controller dies. It then keeps deleting this run's named trainer and deployment (the deployment with `ignoreChecks`) until their release is observed. + +This is a conservative client-side estimate, not an invoice or a provider-enforced cap. Network failures can delay cleanup, and the timings do not measure GPU utilization. diff --git a/hosted/fireworks/campaign.py b/hosted/fireworks/campaign.py new file mode 100644 index 0000000..9629a94 --- /dev/null +++ b/hosted/fireworks/campaign.py @@ -0,0 +1,422 @@ +"""Hosted teacher RL, then routed multi-teacher OPD, on the Fireworks dedicated Training API. Thinking is on. + + python campaign.py --root NEW_RUN_ROOT --tokenizer-info TOKENIZER_JSON --account ACCOUNT +""" +from concurrent.futures import ThreadPoolExecutor, as_completed +from datetime import datetime, timezone +from pathlib import Path +import argparse, importlib.metadata, json, logging, math, os, random, shutil, statistics, subprocess, sys, time +import traceback + +import tinker +from transformers import AutoTokenizer +from fireworks.training.sdk import FiretitanServiceClient, TrainerJobManager, TrainerJobConfig +from fireworks.training.sdk.concurrency import FixedConcurrencyController + +from lifecycle import ( # Also makes the shared package importable. + CLEANUP_RESERVE_USD, DEFAULT_BUDGET_USD, RATE_USD_HOUR, REPO, api_key, check, cleanup, event, read_json, utc_now, + verify_account, write) +from shared import credentials, puzzles, scoring +from shared.recipe import (CONTEXT_LENGTH, DOMAINS, ENABLE_THINKING, EVAL_EXAMPLES, EVAL_INTERVAL, LEARNING_RATE, + MAX_RESPONSE_TOKENS, PROMPTS_PER_UPDATE, SAMPLES_PER_PROMPT, UPDATES) + +MODEL = 'accounts/fireworks/models/qwen3p5-35b-a3b' +SHAPE = 'accounts/fireworks/trainingShapes/qwen3p5-35b-a3b-256k-lora/versions/e3dzebwt' +LORA_RANK = 64 +SEED = 20260921 +SEED_STRIDE = 100_000 # Sampling seed: SEED + step * SEED_STRIDE + request index. +CONCURRENCY = 32 +# Teacher GRPO settings of the recipe.TEACHERS runs: 32 prompts x 8 responses, at most 200 updates, +# evaluation every 25, no weight decay. Their full-finetuning LR of 1e-6 is scaled 10x for LoRA. +TEACHER_DEFAULTS = {'prompts': 32, 'learning_rate': 1e-5, 'eval_every': 25, 'max_steps': 200, 'target_tolerance': 0} +TEACHER_GROUP_SIZE = 8 +# Operational greedy dev-accuracy targets, just below the held-out scores of recipe.TEACHERS +# (caesar_cipher 73.3%, simple_geometry 99.8%). Meeting them does not show a match with recipe.TEACHERS. +TEACHER_TARGETS = {'caesar_cipher': 0.70, 'simple_geometry': 0.95} +PPO_CLIP = {'clip_low_threshold': 0.8, 'clip_high_threshold': 1.2} +TEACHER_ADAM = {'beta1': 0.9, 'beta2': 0.98, 'eps': 1e-8, 'weight_decay': 0.0, 'grad_clip_norm': 1.0} +STUDENT_LEARNING_RATE = 10 * LEARNING_RATE # The recipe's full-finetuning LR, times 10 for LoRA. +STUDENT_ADAM = {**TEACHER_ADAM, 'beta2': 0.999, 'weight_decay': 0.0} +SCORING_NOTE = 'single-session checkpoint swaps; frozen teacher forwards; exact student optimizer restore' + + +def append(path, row): + with Path(path).open('a') as file: + file.write(json.dumps(row, allow_nan=False, default=str) + '\n') + + +def tensor(values, dtype): + return tinker.TensorData(data=values, dtype=dtype, shape=[len(values)]) + + +def scoring_datum(r): + """Forward-only datum whose output log-probabilities score the sampled response.""" + full = r['prompt_tokens'] + r['tokens'] + weights = [0.0] * (len(r['prompt_tokens']) - 1) + [1.0] * len(r['tokens']) + return tinker.Datum(model_input=tinker.ModelInput.from_ints(full[:-1]), + loss_fn_inputs={'target_tokens': tensor(full[1:], 'int64'), 'weights': tensor(weights, 'float32')}) + + +def training_datum(r): + """Policy-gradient datum; prompt positions carry zero advantage.""" + pad = len(r['prompt_tokens']) - 1 + return tinker.Datum(model_input=tinker.ModelInput.from_ints(r['prompt_tokens'] + r['tokens'][:-1]), loss_fn_inputs={ + 'target_tokens': tensor([0] * pad + r['tokens'], 'int64'), + 'logprobs': tensor([0.0] * pad + r['logprobs'], 'float32'), + 'advantages': tensor([0.0] * pad + r['advantages'], 'float32')}) + + +def response_logprobs(r, output): + values = [float(x) for x in output['logprobs'].data[len(r['prompt_tokens']) - 1:]] + assert len(values) == len(r['tokens']) and all(math.isfinite(x) for x in values) + return values + + +class Campaign: + def __init__(self, root, tokenizer, rows, teacher_settings=None, resume_plan=None): + self.root, self.tokenizer, self.data = root, tokenizer, rows + self.teacher_settings = s = {**TEACHER_DEFAULTS, **(teacher_settings or {})} + assert set(s) == set(TEACHER_DEFAULTS) and isinstance(s['prompts'], int) and s['prompts'] > 0 + assert 0 < s['learning_rate'] < 1 and s['eval_every'] > 0 and s['max_steps'] > 0 + assert 0 <= s['target_tolerance'] < min(TEACHER_TARGETS.values()) + self.resume_plan = resume_plan or {} + self.order = {d: list(range(len(rows[d]['train']))) for d in DOMAINS} # Fixed shuffled training order. + for d in DOMAINS: + random.Random(SEED + DOMAINS.index(d)).shuffle(self.order[d]) + self.started = time.monotonic() + self.pool = ThreadPoolExecutor(max_workers=CONCURRENCY) + self.prompts, self.selected = {}, {} + self.service = self.sampler = self.train = None + + def render_prompts(self): # Before paying for GPUs. + for d in DOMAINS: + for split in ('train', 'dev'): + for row in self.data[d][split]: + ids = self.tokenizer.apply_chat_template(row['prompt'], tokenize=True, add_generation_prompt=True, + enable_thinking=ENABLE_THINKING, return_dict=False) + assert len(ids) + MAX_RESPONSE_TOKENS <= CONTEXT_LENGTH + self.prompts[d, split, row['_index']] = ids + + def log(self, event_name, **fields): + row = {'event': event_name, 'utc': utc_now(), 'elapsed_seconds': time.monotonic() - self.started, **fields} + append(self.root / 'events.jsonl', row) + print(json.dumps(row, default=str), flush=True) + + def publish(self, name): # Snapshot the weights for sampling and point the sampler at them. + check(self.root) + start = time.monotonic() + path = self.train.save_weights_for_sampler(name).result().path + if self.sampler is not None: + self.sampler.close() + self.sampler = None + self.sampler = self.service.create_sampling_client( + model_path=path, tokenizer=self.tokenizer, training_client=self.train, + concurrency_controller=FixedConcurrencyController(CONCURRENCY)) + self.log('published', path=path, name=name, seconds=time.monotonic() - start) + return path + + def sample_one(self, domain, row, split, group, temperature, seed): + check(self.root) + ids = self.prompts[domain, split, row['_index']] + start = time.monotonic() + seed = seed if group == 1 else None # A group needs distinct samples. + params = tinker.SamplingParams(max_tokens=MAX_RESPONSE_TOKENS, temperature=temperature, top_p=1, top_k=-1, + stop=[self.tokenizer.eos_token_id], seed=seed) + result = self.sampler.sample(tinker.ModelInput.from_ints(ids), num_samples=group, sampling_params=params).result() + records = [] + for sample, sequence in enumerate(result.sequences): + response = self.tokenizer.decode(list(sequence.tokens), skip_special_tokens=True) + records.append({'domain': domain, 'index': row['_index'], 'sample': sample, 'prompt_tokens': ids, + 'tokens': list(sequence.tokens), 'logprobs': list(sequence.logprobs), 'response': response, + 'score': scoring.score(response, row['label']), 'stop_reason': sequence.stop_reason, + 'request_seconds': time.monotonic() - start, 'seed': seed}) + return records + + def sample_batch(self, out, work, step, split, group=1, temperature=1.0): # Groups return in `work` order. + check(self.root) + futures = {self.pool.submit(self.sample_one, domain, row, split, group, temperature, + SEED + step * SEED_STRIDE + i): i for i, (domain, row) in enumerate(work)} + groups = {} + for future in as_completed(futures): + groups[futures[future]] = future.result() + for record in groups[futures[future]]: + append(out / 'samples.jsonl', {'step': step, 'split': split, **record}) + return [groups[i] for i in range(len(work))] + + def evaluate(self, out, step): # Greedy accuracy on every domain's full dev split. + start = time.monotonic() + groups = self.sample_batch(out, [(d, row) for d in DOMAINS for row in self.data[d]['dev']], step, 'dev', 1, 0) + result = {'step': step, 'seconds': time.monotonic() - start} + for d in DOMAINS: + rows = [r for group in groups for r in group if r['domain'] == d] + result[d] = {'n': len(rows), 'correct': sum(r['score'] for r in rows), + 'accuracy': statistics.mean(r['score'] for r in rows), + 'mean_tokens': statistics.mean(len(r['tokens']) for r in rows), + 'truncated': sum(r['stop_reason'] == 'length' for r in rows)} + append(out / 'evaluations.jsonl', result) + self.log('evaluation', run=out.name, **result) + return result + + def score_with_teachers(self, out, step, records): + """MOPD advantages: teacher minus student token log-probabilities. The one session holds the + student, so save it with its optimizer, score each domain with its frozen teacher checkpoint, + and restore the student before learning. Sampler snapshots are immutable meanwhile.""" + student_state = self.train.save_state(f'student-pre-{step:03d}').result().path + self.log('student_state_saved', step=step, path=student_state) + for d in DOMAINS: + rows = [r for r in records if r['domain'] == d] + teacher_state = self.selected[d]['checkpoint']['state_path'] + self.train.load_state(teacher_state).result() + self.log('teacher_state_loaded', step=step, domain=d, path=teacher_state) + result = self.train.forward([scoring_datum(r) for r in rows], loss_fn='cross_entropy').result() + append(out / 'teacher-forward-metrics.jsonl', {'step': step, 'domain': d, 'metrics': result.metrics}) + for r, output in zip(rows, result.loss_fn_outputs, strict=True): + r['teacher_logprobs'] = response_logprobs(r, output) + r['advantages'] = [t - s for t, s in zip(r['teacher_logprobs'], r['logprobs'])] + append(out / 'teacher-scores.jsonl', {'step': step, **r}) + self.train.load_state_with_optimizer(student_state).result() + self.log('student_state_restored', step=step, path=student_state) + + def train_stage(self, domain=None): + """Train one teacher (`domain` given) or the student (`domain` None).""" + teacher = domain is not None + name = 'teacher-' + domain if teacher else 'student' + out = self.root / name + out.mkdir() + start = time.monotonic() + s = self.teacher_settings + resume = self.resume_plan.get(domain, {}) if teacher else {} + domains = (domain,) if teacher else DOMAINS + per_domain = s['prompts'] if teacher else PROMPTS_PER_UPDATE // len(DOMAINS) + start_step, steps = resume.get('step', 0), s['max_steps'] if teacher else UPDATES + # A resume plan carries prompt_offset when the teacher batch size changed. + cfg = {'start_step': start_step, 'resume': resume, 'mode': 'teacher' if teacher else 'mopd', 'domain': domain, + 'steps': steps, 'group_size': TEACHER_GROUP_SIZE if teacher else SAMPLES_PER_PROMPT, + 'prompts': per_domain * len(domains), 'prompt_offset': resume.get('prompt_offset', start_step * per_domain), + 'rank': LORA_RANK, 'learning_rate': s['learning_rate'] if teacher else STUDENT_LEARNING_RATE, + 'seed': SEED, 'max_tokens': MAX_RESPONSE_TOKENS, 'eval_examples': EVAL_EXAMPLES, + 'eval_every': s['eval_every'] if teacher else EVAL_INTERVAL, 'target': TEACHER_TARGETS.get(domain), + 'stopping_threshold': TEACHER_TARGETS[domain] - s['target_tolerance'] if teacher else None, + 'fresh_base': not resume, 'teacher_scoring': SCORING_NOTE, 'pipeline_depth': 0, 'max_policy_lag': 0} + assert 0 <= start_step <= steps + write(out / 'config.json', cfg) + best = resume.get('best') + if resume.get('complete'): # Reuse a finished teacher without touching the session. + self.selected[domain] = {**best, 'source_root': resume['source_root']} + write(self.root / 'teacher-selection.json', self.selected) + write(out / 'complete.json', {'updates': start_step, 'reused': True, 'source_root': resume['source_root']}) + self.log('stage_restored', run=name, updates=start_step) + return + # The dedicated backend permits only one resident LoRA session. Reuse it, + # restoring a pristine base weights/optimizer checkpoint for fresh stages. + if self.train is None: + self.train = self.service.create_training_client( + base_model=MODEL, lora_rank=LORA_RANK, seed=SEED, + user_metadata={'experiment': self.root.name, 'stage': 'shared-session'}) + self.initial_state = self.train.save_state('initial-base').result().path + write(self.root / 'initial-state.json', {'path': self.initial_state}) + self.train.load_state_with_optimizer(resume['restore_reference'] if resume else self.initial_state).result() + write(out / 'model-info.json', self.train.get_info().model_dump()) + assert start_step < steps + self.publish(f'{name}-{start_step:03d}'.replace('_', '-')) + if start_step == 0 and (not teacher or domain == DOMAINS[0]): # Evaluate the shared base once. + self.evaluate(out, 0) + for step in range(start_step, steps): + update = step + 1 + check(self.root) + begin = time.monotonic() + offset = cfg['prompt_offset'] + (step - start_step) * per_domain + work = [(d, self.data[d]['train'][self.order[d][(offset + i) % len(self.order[d])]]) + for d in domains for i in range(per_domain)] + self.log('step_start', run=name, step=update) + groups = self.sample_batch(out, work, step, 'train', cfg['group_size']) + sample_seconds = time.monotonic() - begin + records = [r for group in groups for r in group] + score_start = time.monotonic() + if teacher: # Group-normalized rewards on every token. + for group in groups: + scores = [r['score'] for r in group] + mean, std = statistics.mean(scores), statistics.stdev(scores) if len(scores) > 1 else 0 + for r in group: + r['advantages'] = [(r['score'] - mean) / (std + 1e-6)] * len(r['tokens']) + else: + self.score_with_teachers(out, step, records) + teacher_seconds = time.monotonic() - score_start + learn_start = time.monotonic() + # Keep the documented Fireworks operation order; never retry optimizer mutations. + backward = self.train.forward_backward([training_datum(r) for r in records], + loss_fn='ppo' if teacher else 'importance_sampling', + loss_fn_config=PPO_CLIP if teacher else None).result() + optimizer = self.train.optim_step(tinker.AdamParams( + learning_rate=cfg['learning_rate'], **(TEACHER_ADAM if teacher else STUDENT_ADAM))).result() + deltas = [] + for r, output in zip(records, backward.loss_fn_outputs, strict=True): + logprobs = response_logprobs(r, output) + deltas.extend(x - y for x, y in zip(logprobs, r['logprobs'])) + append(out / 'learner-scores.jsonl', {'step': step, 'domain': r['domain'], 'index': r['index'], + 'sample': r['sample'], 'logprobs': logprobs, + 'advantages': r['advantages']}) + train_seconds = time.monotonic() - learn_start + sampler_path = self.publish(f'{name}-{update:03d}'.replace('_', '-')) + metric = {'step': update, 'samples': len(records), 'sampling_seconds': sample_seconds, + 'teacher_seconds': teacher_seconds, 'train_seconds': train_seconds, + 'step_seconds': time.monotonic() - begin, 'accepted_policy_lag': 0, + 'prompt_tokens': sum(len(r['prompt_tokens']) for r in records), + 'response_tokens': sum(len(r['tokens']) for r in records), + 'scores': {d: statistics.mean(r['score'] for r in records if r['domain'] == d) for d in domains}, + 'nonzero_advantage_samples': sum(any(v != 0 for v in r['advantages']) for r in records), + 'response_train_sample_logprob_delta_abs_mean': statistics.mean(abs(x) for x in deltas), + 'forward_backward_metrics': backward.metrics, 'optimizer_metrics': optimizer.metrics} + append(out / 'metrics.jsonl', metric) + self.log('step_complete', run=name, **metric) + if update % cfg['eval_every'] == 0 or update == steps: + state = self.train.save_state(f'{name}-{update:03d}'.replace('_', '-')).result().path + checkpoint = {'step': update, 'prompt_offset': offset + per_domain, 'sampler_path': sampler_path, + 'state_path': state} + append(out / 'checkpoints.jsonl', checkpoint) + evaluation = self.evaluate(out, update) + if teacher: + accuracy = evaluation[domain]['accuracy'] + if best is None or accuracy > best['evaluation'][domain]['accuracy']: + best = {'step': update, 'evaluation': evaluation, 'checkpoint': checkpoint, + 'target': TEACHER_TARGETS[domain], 'stopping_threshold': cfg['stopping_threshold']} + if accuracy >= cfg['stopping_threshold']: + break + if teacher: # Keep the best evaluated checkpoint, even below target. + self.train.load_state_with_optimizer(best['checkpoint']['state_path']).result() + self.selected[domain] = {**best, 'model_id': self.train.get_info().model_id} + write(self.root / 'teacher-selection.json', self.selected) + self.log('stage_complete', run=name, updates=update, elapsed_seconds_stage=time.monotonic() - start) + write(out / 'complete.json', {'updates': update, 'elapsed_seconds': time.monotonic() - start}) + + +def prepare(root, args): + """Record the inputs, verify the account, freeze the source, and re-execute the frozen copy.""" + os.environ['FIREWORKS_API_KEY'] = credentials.api_key('FIREWORKS_API_KEY', args.settings.get('api_key_profile_label')) + root.mkdir(parents=True, exist_ok=False) + write(root / 'teacher-config.json', read_json(args.teacher_config) if args.teacher_config else {}) + write(root / 'resume-plan.json', read_json(args.resume_plan) if args.resume_plan else {}) + write(root / 'tokenizer.json', read_json(args.tokenizer_info)) + write(root / 'account.json', verify_account(args.account)) + for package in ['fireworks-ai', 'tinker', 'transformers', 'torch']: + append(root / 'versions.jsonl', {'package': package, 'version': importlib.metadata.version(package)}) + for part in ['hosted/fireworks', 'shared', 'data']: + shutil.copytree(REPO / part, root / 'source-repo' / part, ignore=shutil.ignore_patterns('__pycache__')) + os.execv(sys.executable, [ + sys.executable, str(root / 'source-repo/hosted/fireworks/campaign.py'), '--root', str(root), + '--tokenizer-info', str(root / 'tokenizer.json'), '--prior-spend-usd', str(args.prior_spend_usd), + '--budget-usd', str(args.budget_usd), '--rate-usd-hour', str(args.rate_usd_hour), '--prepared']) + + +def run(root, args): + info = read_json(root / 'tokenizer.json') + account = read_json(root / 'account.json')['account'] + rows = {d: {split: puzzles.load_split(d, split) for split in ('train', 'dev')} for d in DOMAINS} + campaign = Campaign(root, AutoTokenizer.from_pretrained(info['path'], local_files_only=True), rows, + read_json(root / 'teacher-config.json'), read_json(root / 'resume-plan.json')) + campaign.render_prompts() + write(root / 'data-manifest.json', puzzles.manifest()) + # Resumed teachers must come from a run with the same account, tokenizer and data. + from fireworks.training.sdk.client import _validate_checkpoint_ref + for domain, plan in campaign.resume_plan.items(): + assert domain in DOMAINS and 0 <= plan['step'] <= campaign.teacher_settings['max_steps'] + _validate_checkpoint_ref(plan['restore_reference']) + if plan.get('best'): + _validate_checkpoint_ref(plan['best']['checkpoint']['state_path']) + source = Path(plan['source_root']) + assert read_json(source / 'account.json')['account'] == account + assert read_json(source / 'tokenizer.json')['revision'] == info['revision'] + assert read_json(source / 'data-manifest.json') == read_json(root / 'data-manifest.json') + run_id = 'opd-' + datetime.now(timezone.utc).strftime('%m%d%H%M%S') + resources = {'account': account, 'trainer_id': run_id + '-train', 'deployment_id': run_id + '-sample', + 'budget_start_unix': time.time(), 'prior_spend_estimate_usd': args.prior_spend_usd, + 'budget_usd': args.budget_usd, 'stop_estimate_usd': args.budget_usd - CLEANUP_RESERVE_USD, + 'conservative_usd_hour': args.rate_usd_hour, 'controller_pid': os.getpid()} + write(root / 'resources.json', resources) + # The watchdog runs in its own session, so it outlives a killed controller. + with (root / 'watchdog.log').open('w') as log: + watchdog = subprocess.Popen( + [sys.executable, str(root / 'source-repo/hosted/fireworks/lifecycle.py'), str(root), '--pid', str(os.getpid())], + stdout=log, stderr=subprocess.STDOUT, start_new_session=True) + write(root / 'watchdog.json', {'pid': watchdog.pid}) + for _ in range(60): + if (root / 'watchdog-ready.json').exists(): + break + if watchdog.poll() is not None: + raise RuntimeError('Independent watchdog failed to start') + time.sleep(1) + else: + raise RuntimeError('Independent watchdog did not become ready') + secret = api_key() + os.environ['FIREWORKS_API_KEY'] = secret + + class Redact(logging.Filter): + def filter(self, record): + record.msg, record.args = record.getMessage().replace(secret, '[REDACTED]'), () + return True + + logging.basicConfig(level=logging.INFO, format='%(asctime)s %(levelname)s %(name)s %(message)s') + for handler in logging.getLogger().handlers: + handler.addFilter(Redact()) + for noisy in ['httpx', 'httpcore']: + logging.getLogger(noisy).setLevel(logging.WARNING) + try: + write(root / 'status.json', {'stage': 'provisioning'}) + TrainerJobManager(api_key=secret).create(TrainerJobConfig( + base_model=MODEL, lora_rank=LORA_RANK, learning_rate=campaign.teacher_settings['learning_rate'], + training_shape_ref=SHAPE, trainer_replica_count=1, requested_job_id=resources['trainer_id'], + inactivity_timeout='1800s', display_name=root.name)) + check(root) + campaign.service = FiretitanServiceClient.from_firetitan_config( + api_key=secret, base_model=MODEL, tokenizer_model=info['path'], max_lora_rank=LORA_RANK, + training_shape_id=SHAPE, trainer_job_id=resources['trainer_id'], deployment_id=resources['deployment_id'], + max_context_length=CONTEXT_LENGTH, replica_count=1, trainer_replica_count=1, + hot_load_transition_type='SYNC', disable_speculative_decoding=True, cleanup_trainer_on_close=False, + cleanup_deployment_on_close=None, trainer_timeout_s=900, trainer_pending_timeout_s=900, + deployment_timeout_s=900, wait_for_trainer_before_deployment=False) + for domain in [*DOMAINS, None]: + write(root / 'status.json', {'stage': 'teacher', 'domain': domain} if domain else {'stage': 'student'}) + campaign.train_stage(domain) + write(root / 'status.json', {'stage': 'training_complete'}) + except BaseException: + (root / 'error.txt').write_text(traceback.format_exc()) + write(root / 'status.json', {'stage': 'budget_stopped' if (root / 'STOP').exists() else 'failed'}) + raise + finally: + (root / 'TRAINING_FINISHED').write_text(utc_now()) + try: # The independent watchdog also handles abrupt controller exits. + cleanup(root, resources) + except Exception as error: + event(root, 'controller_cleanup_error', error=str(error)) + for client in (campaign.sampler, campaign.service): + if client is not None: + client.close() + campaign.pool.shutdown(wait=False, cancel_futures=True) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument('--root', type=Path, required=True) + parser.add_argument('--tokenizer-info', type=Path, required=True) + parser.add_argument('--account', help='Default: config/fireworks.local.json, then $FIREWORKS_ACCOUNT_ID') + parser.add_argument('--config', type=Path, help='Account settings (default: config/fireworks.local.json)') + parser.add_argument('--teacher-config', type=Path) + parser.add_argument('--resume-plan', type=Path) + parser.add_argument('--budget-usd', type=float, default=DEFAULT_BUDGET_USD) + parser.add_argument('--prior-spend-usd', type=float, default=0) + parser.add_argument('--rate-usd-hour', type=float, default=RATE_USD_HOUR) + parser.add_argument('--prepared', action='store_true', help=argparse.SUPPRESS) + args = parser.parse_args() + args.settings = credentials.local_config('fireworks', args.config) + args.account = args.account or args.settings.get('account') or os.environ.get('FIREWORKS_ACCOUNT_ID') + if not (args.prepared or args.account): + parser.error('Set --account, config/fireworks.local.json, or $FIREWORKS_ACCOUNT_ID') + assert args.budget_usd > CLEANUP_RESERVE_USD and args.rate_usd_hour > 0 + root = args.root.resolve() + if not args.prepared: + prepare(root, args) # Does not return. + run(root, args) + + +if __name__ == '__main__': + main() diff --git a/hosted/fireworks/lifecycle.py b/hosted/fireworks/lifecycle.py new file mode 100644 index 0000000..775641c --- /dev/null +++ b/hosted/fireworks/lifecycle.py @@ -0,0 +1,182 @@ +"""Credentials, the budget estimate, and the independent fail-closed watchdog. + +`python lifecycle.py RUN_ROOT --pid CONTROLLER_PID` requests a stop when the spend estimate reaches the +run's threshold, training finishes, or the controller dies, then deletes this run's two named resources +(account and names from `resources.json`) until their release is observed. +""" +from datetime import datetime, timezone +from pathlib import Path +import argparse, json, os, sys, time + +import httpx + +REPO = Path(__file__).resolve().parents[2] # The frozen `source-repo/` copy in a launched run. +sys.path.insert(0, str(REPO)) # Makes the shared package importable. + +API_URL = 'https://api.fireworks.ai' +# Conservative client-side estimate, not an invoice or a provider cap: six B200s (the four-GPU +# trainer plus the rollout replica) at the public $13/hour, plus 50% headroom. +GPU_USD_HOUR, ALLOCATED_GPUS, HEADROOM = 13, 6, 1.5 +RATE_USD_HOUR = GPU_USD_HOUR * ALLOCATED_GPUS * HEADROOM # 117 +DEFAULT_BUDGET_USD = 200 +CLEANUP_RESERVE_USD = 25 # Kept back so shutdown can finish within the budget. +STOP_USD = DEFAULT_BUDGET_USD - CLEANUP_RESERVE_USD # For a run that records no stop. + + +def api_key(): + # campaign.py resolves the key once (environment or ~/.zprofile label) and passes it on + # to its frozen copy and the watchdog through the environment. + if not os.environ.get('FIREWORKS_API_KEY'): + raise RuntimeError('FIREWORKS_API_KEY is not set') + return os.environ['FIREWORKS_API_KEY'] + + +def headers(): + return {'Authorization': 'Bearer ' + api_key()} + + +def utc_now(): + return datetime.now(timezone.utc).isoformat() + + +def read_json(path): + return json.loads(Path(path).read_text()) + + +def write(path, data): + Path(path).write_text(json.dumps(data, indent=2, default=str) + '\n') + + +def event(root, name, **fields): + with (Path(root) / 'lifecycle.jsonl').open('a') as file: + file.write(json.dumps(dict(utc=utc_now(), event=name, **fields), default=str) + '\n') + + +def resource_estimate(resources, now=None): + """Prior attempts' spend plus allocation time since provisioning at the run's rate.""" + elapsed = max(0, (time.time() if now is None else now) - resources['budget_start_unix']) + return (resources.get('prior_spend_estimate_usd', 0) + + elapsed * resources.get('conservative_usd_hour', RATE_USD_HOUR) / 3600) + + +def verify_account(account): + with httpx.Client(headers=headers(), timeout=30) as client: + response = client.get(API_URL + '/v1/accounts') + response.raise_for_status() + assert any(x['name'] == 'accounts/' + account for x in response.json()['accounts']), f'No access to {account}' + return {'account': account, 'verified_utc': utc_now()} + + +def paths(resources): + """This run's named resources. `extra_deployments` (hosted/opd teacher deployments) is optional, and a + teacher-only lease has no trainer or main deployment.""" + account = f'/v1/accounts/{resources["account"]}' + named = {} + if resources.get('trainer_id'): + named['trainer'] = f'{account}/rlorTrainerJobs/{resources["trainer_id"]}' + if resources.get('deployment_id'): + named['deployment'] = f'{account}/deployments/{resources["deployment_id"]}' + for deployment_id in resources.get('extra_deployments', []): + named['deployment:' + deployment_id] = f'{account}/deployments/{deployment_id}' + return named + + +def snapshot(root, resources): + result = {} + with httpx.Client(headers=headers(), timeout=20) as client: + for kind, path in paths(resources).items(): + response = client.get(API_URL + path) + result[kind] = {'http_status': response.status_code, 'data': response.json() if response.content else {}} + with (Path(root) / 'resource-metrics.jsonl').open('a') as file: + file.write(json.dumps({'utc': utc_now(), 'resources': result}) + '\n') + return result + + +def released(state): + trainer = state.get('trainer', {'http_status': 404, 'data': {}}) + + def deployment_released(deployment): + data = deployment['data'] + # A scale-down must be observed, not only requested. + scaled_to_zero = deployment['http_status'] == 200 and ( + data.get('minReplicaCount', 0) == 0 and data.get('maxReplicaCount', -1) == 0 + and data.get('currentReplicaCount', -1) == 0) + return deployment['http_status'] == 404 or data.get('state') == 'DELETED' or scaled_to_zero + + return ((trainer['http_status'] == 404 or trainer['data'].get('state') in ('DELETED', 'JOB_STATE_DELETED')) + and all(deployment_released(v) for k, v in state.items() if k.startswith('deployment'))) + + +def cleanup(root, resources): + """Delete only this run's named trainer and deployment; checkpoints remain server-side.""" + with httpx.Client(headers=headers(), timeout=30) as client: + for kind, path in paths(resources).items(): + # The run owns its deployment; recent rollout traffic must not block shutdown (as in the SDK). + response = client.delete(API_URL + path, params={'ignoreChecks': 'true'} if kind.startswith('deployment') else None) + event(root, 'cleanup_request', resource=kind, status=response.status_code, + response=response.json() if response.content else {}) + + +def check(root): + """Raise if a stop was requested or the estimate reached the stop threshold.""" + root = Path(root) + if (root / 'STOP').exists(): + raise RuntimeError('Budget/lifecycle watchdog requested stop') + if (root / 'resources.json').exists(): + resources = read_json(root / 'resources.json') + if resource_estimate(resources) >= resources.get('stop_estimate_usd', STOP_USD): + (root / 'STOP').write_text('Conservative spend reserve reached\n') + raise RuntimeError('Conservative spend reserve reached') + + +def watchdog(root, pid): + root = Path(root) + stop_at = None + while True: + try: + os.kill(pid, 0) + alive = True + except ProcessLookupError: + alive = False + resources = read_json(root / 'resources.json') # Re-read, so budget corrections apply. + cost = resource_estimate(resources) + must_stop = (cost >= resources.get('stop_estimate_usd', STOP_USD) or not alive + or (root / 'TRAINING_FINISHED').exists() or (root / 'STOP').exists()) + if must_stop: + if stop_at is None: + stop_at = time.time() + (root / 'STOP').write_text('Watchdog stop/cleanup requested\n') + event(root, 'stop_requested', estimated_usd=cost, parent_alive=alive) + # Give the controller 30 s to finish its current request and save state. + if not alive or (root / 'TRAINING_FINISHED').exists() or time.time() - stop_at >= 30: + try: + cleanup(root, resources) + except Exception as error: + event(root, 'cleanup_error', error=str(error)) + try: + state = snapshot(root, resources) + if not (root / 'watchdog-ready.json').exists(): + write(root / 'watchdog-ready.json', {'pid': os.getpid(), 'utc': utc_now()}) + event(root, 'budget_observation', conservative_estimate_usd=cost, + rate_usd_hour=resources.get('conservative_usd_hour', RATE_USD_HOUR)) + if must_stop and released(state): + write(root / 'cleanup.json', {'verified_released': True, 'utc': utc_now(), + 'conservative_estimate_usd': cost, + 'billing': 'Estimate only; not an invoice', 'resources': state}) + try: + from report import report + report(root) + except Exception as error: + event(root, 'audit_error', error=str(error)) + return + except Exception as error: + event(root, 'observation_error', error=str(error)) + time.sleep(15) + + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('root', type=Path) + parser.add_argument('--pid', type=int, required=True) + args = parser.parse_args() + watchdog(args.root, args.pid) diff --git a/hosted/fireworks/report.py b/hosted/fireworks/report.py new file mode 100644 index 0000000..79a6b4c --- /dev/null +++ b/hosted/fireworks/report.py @@ -0,0 +1,82 @@ +"""Read-only final audit of a run root: `python report.py RUN_ROOT` writes `audit.json` and `REPORT.md`.""" +from pathlib import Path +import argparse +import json +import math + +from lifecycle import read_json # Also makes `shared` importable. +from shared.recipe import DOMAINS, PROMPTS_PER_UPDATE, UPDATES + +LIMITATIONS = ['Qwen3.5 base differs from the Qwen3.6 base used elsewhere.', + 'Teachers are retrained here; their targets are operational goals, not the frozen recipe teachers.', + 'FP8 rollout inference versus trainer numerical precision; no router replay.', + 'Client wall times and resource allocation are not measured hardware utilization.', + 'Conservative allocation cost estimate is not an invoice or provider billing cap.'] +SWAP = ['student_state_saved', *['teacher_state_loaded'] * len(DOMAINS), 'student_state_restored'] + + +def rows(path): + return [json.loads(line) for line in path.read_text().splitlines()] if path.exists() else [] + + +def stage(root, directory): + config = read_json(directory / 'config.json') + metrics = rows(directory / 'metrics.jsonl') + evaluations = rows(directory / 'evaluations.jsonl') + train = [s for s in rows(directory / 'samples.jsonl') if s['split'] == 'train'] + domains = DOMAINS if directory.name == 'student' else (directory.name[len('teacher-'):],) + per_step = config['prompts'] * config['group_size'] // len(domains) + first = config.get('start_step', 0) + 1 + item = {'completed_updates': len(metrics), 'logged_training_responses': len(train), + 'last_evaluation': evaluations[-1] if evaluations else None, 'checks': { + 'unique_samples': len({(s['step'], s['domain'], s['index'], s['sample']) for s in train}) == len(train), + 'aligned_finite_logprobs': all(s['tokens'] and len(s['tokens']) == len(s['logprobs']) + and all(map(math.isfinite, s['logprobs'])) for s in train), + 'sequential_updates': [m['step'] for m in metrics] == list(range(first, first + len(metrics))), + 'responses_per_update': all(sum(s['step'] == m['step'] - 1 and s['domain'] == d for s in train) + == per_step for m in metrics for d in domains)}} + if (directory / 'complete.json').exists(): + item['completion'] = read_json(directory / 'complete.json') + if directory.name == 'student': + events = [e for e in rows(root / 'events.jsonl') if e['event'] in SWAP] + swaps = [[e for e in events if e['step'] == m['step'] - 1] for m in metrics] + counts = {d: sum(s['domain'] == d for s in train) for d in DOMAINS} + item['checks'].update( + checkpoint_swaps=all([e['event'] for e in swap] == SWAP and swap[0]['path'] == swap[-1]['path'] + and [e['domain'] for e in swap[1:-1]] == list(DOMAINS) for swap in swaps), + protocol_complete=(len(metrics) == UPDATES and len(train) == UPDATES * PROMPTS_PER_UPDATE + and set(counts.values()) == {UPDATES * PROMPTS_PER_UPDATE // len(DOMAINS)} + and all(m['accepted_policy_lag'] == 0 for m in metrics))) + item['domain_counts'] = counts + return item + + +def report(root): + root = Path(root) + result = {'status': read_json(root / 'status.json'), 'limitations': LIMITATIONS, 'stages': { + name: stage(root, root / name) for name in [*('teacher-' + d for d in DOMAINS), 'student'] if (root / name).exists()}} + if (root / 'teacher-selection.json').exists(): + result['teachers'] = {d: {'step': s['step'], 'accuracy': s['evaluation'][d]['accuracy'], 'target': s['target'], + 'target_reached': s['evaluation'][d]['accuracy'] >= s['target']} + for d, s in read_json(root / 'teacher-selection.json').items()} + result['cleanup_verified'] = (root / 'cleanup.json').exists() and read_json(root / 'cleanup.json')['verified_released'] + (root / 'audit.json').write_text(json.dumps(result, indent=2) + '\n') + lines = ['# Fireworks hosted RL and MOPD audit', '', f"Status: {result['status']['stage']}", ''] + for name, item in result['stages'].items(): + lines.append(f"{name}: {item['completed_updates']} completed updates; " + f"{item['logged_training_responses']} logged training responses.") + if item.get('completion', {}).get('reused'): + lines.append(f"Reused trained teacher from {item['completion']['source_root']}, " + f"logical update {item['completion']['updates']}. No new updates in this run.") + if item['last_evaluation']: + last = item['last_evaluation'] + scores = '; '.join(f"{d} {last[d]['correct']:g}/{last[d]['n']} ({last[d]['accuracy']:.2%})" for d in DOMAINS) + lines.append(f"Last evaluation at update {last['step']}: {scores}.") + (root / 'REPORT.md').write_text('\n\n'.join([*lines, '', *LIMITATIONS]) + '\n') + return result + + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('root', type=Path) + report(parser.parse_args().root) diff --git a/hosted/fireworks/test_campaign.py b/hosted/fireworks/test_campaign.py new file mode 100644 index 0000000..7eab8cc --- /dev/null +++ b/hosted/fireworks/test_campaign.py @@ -0,0 +1,148 @@ +"""No-network regressions: `python test_campaign.py` (importing shared.scoring needs reasoning-gym==0.1.25).""" +import contextlib, io, json, tempfile, time, unittest +from pathlib import Path +from types import SimpleNamespace as NS +from unittest.mock import patch + +import httpx + +from campaign import TEACHER_DEFAULTS, Campaign, DOMAINS +from shared import recipe +from lifecycle import STOP_USD, check, cleanup, write + + +def done(value=None): + return NS(result=lambda: value) + + +class Session: + """The one training session. `weight` and `optimizer` count the updates each reflects.""" + def __init__(self): + self.updates = self.weight = self.optimizer = 0 + self.states, self.scored, self.restored = {}, [], [] + + def get_info(self): + return NS(model_id='0', model_dump=lambda: {}) + + def save_weights_for_sampler(self, name): + return done(NS(path=name)) + + def save_state(self, name): + self.states[name] = (self.weight, self.optimizer) + return done(NS(path=name)) + + def load_state_with_optimizer(self, path): + self.restored.append(path) + self.weight, self.optimizer = self.states.get(path, (self.weight, self.optimizer)) + return done() + + def load_state(self, path): + self.weight = self.states[path][0] + self.scored.append(self.weight) + return done() + + def forward(self, data, **kwargs): + outputs = [{'logprobs': NS(data=[-0.5] * len(d.loss_fn_inputs['target_tokens'].data))} for d in data] + return done(NS(metrics={}, loss_fn_outputs=outputs)) + + forward_backward = forward + + def optim_step(self, params): + assert self.weight == self.optimizer, 'Learned without the matching optimizer state' + self.learning_rate = params.learning_rate + self.updates, self.weight, self.optimizer = self.updates + 1, self.weight + 1, self.optimizer + 1 + return done(NS(metrics={})) + + +def campaign(root, dev_score=lambda index: 1, **settings): + """A campaign on fake rows. Training rewards alternate 0, 1 within each group.""" + rows = {d: {'train': [{'_index': i} for i in range(recipe.TRAIN_EXAMPLES_PER_DOMAIN)], + 'dev': [{'_index': i} for i in range(recipe.EVAL_EXAMPLES[d])]} for d in DOMAINS} + c = Campaign(root, None, rows, **settings) + c.batches = [] + c.service = NS(sessions=[], create_sampling_client=lambda **kwargs: NS(close=lambda: None)) + + def create_training_client(**kwargs): + assert not c.service.sessions, 'The backend permits only one session' + c.service.sessions.append(Session()) + return c.service.sessions[0] + + def sample_batch(out, work, step, split, group=1, temperature=1): + c.batches.append((out.name, step, split, [row['_index'] for _, row in work])) + return [[{'domain': d, 'index': row['_index'], 'sample': j, 'prompt_tokens': [1, 2, 3], 'tokens': [4, 5], + 'logprobs': [-1.0, -1.0], 'score': j % 2 if split == 'train' else dev_score(row['_index']), + 'stop_reason': 'stop'} for j in range(group)] for d, row in work] + + c.service.create_training_client, c.sample_batch = create_training_client, sample_batch + return c + + +def lines(path): + return [json.loads(line) for line in path.read_text().splitlines()] + + +class CampaignTest(unittest.TestCase): + def setUp(self): + self.root = Path(self.enterContext(tempfile.TemporaryDirectory())) + self.enterContext(contextlib.redirect_stdout(io.StringIO())) + + def test_full_campaign_on_one_session(self): + c = campaign(self.root) + for domain in (*DOMAINS, None): + c.train_stage(domain) + session, = c.service.sessions + # Teachers stop at their first evaluation (25 updates each). The student starts from the + # pristine base and scores every update with both frozen 25-update teachers. + first, updates = TEACHER_DEFAULTS['eval_every'], recipe.UPDATES + self.assertEqual((session.updates, session.weight, session.optimizer), (2 * first + updates, updates, updates)) + self.assertEqual(session.scored, [first] * (2 * updates)) + self.assertEqual(sum(m['samples'] for m in lines(self.root / 'student/metrics.jsonl')), 2560) + # Prior spend counts against the budget. + write(self.root / 'resources.json', {'budget_start_unix': time.time(), 'prior_spend_estimate_usd': STOP_USD + 1}) + self.assertRaises(RuntimeError, check, self.root) + + def test_resume_keeps_update_and_prompt_offset(self): + c = campaign(self.root, resume_plan={'caesar_cipher': {'step': 10, 'restore_reference': 'cross_job://a/step-10'}}) + c.train_stage('caesar_cipher') + self.assertEqual(c.service.sessions[0].restored[0], 'cross_job://a/step-10') + self.assertEqual(c.batches[0], ('teacher-caesar_cipher', 10, 'train', c.order['caesar_cipher'][320:352])) + self.assertEqual(lines(self.root / 'teacher-caesar_cipher/checkpoints.jsonl')[-1]['prompt_offset'], 800) + self.assertEqual(json.loads((self.root / 'teacher-caesar_cipher/complete.json').read_text())['updates'], 25) + + def test_teacher_settings_override(self): + # 85/126 correct is below the 0.70 target but within the 0.03 tolerance. The earlier + # run used 32-prompt batches, so the plan carries offset 640 rather than 20 * 16. + settings = {'prompts': 16, 'learning_rate': 5e-5, 'eval_every': 5, 'max_steps': 10000, 'target_tolerance': .03} + plan = {'caesar_cipher': {'step': 20, 'prompt_offset': 640, 'restore_reference': 'cross_job://a/step-20'}} + c = campaign(self.root, lambda index: float(index < 85), teacher_settings=settings, resume_plan=plan) + c.train_stage('caesar_cipher') + train = [batch[3] for batch in c.batches if batch[2] == 'train'] + self.assertEqual((train[0], train[-1]), (c.order['caesar_cipher'][640:656], c.order['caesar_cipher'][704:720])) + self.assertEqual((c.service.sessions[0].learning_rate, c.service.sessions[0].updates), (5e-5, 5)) + self.assertEqual([m['samples'] for m in lines(self.root / 'teacher-caesar_cipher/metrics.jsonl')], [128] * 5) + + def test_prompts_render_with_thinking_and_fit_context(self): + calls = [] + rows = {d: {s: [{'_index': 0, 'prompt': []}] for s in ('train', 'dev')} for d in DOMAINS} + tokenizer = NS(apply_chat_template=lambda prompt, **kwargs: calls.append(kwargs) or [0] * 200) + Campaign(self.root, tokenizer, rows).render_prompts() + self.assertTrue(calls and all(kwargs['enable_thinking'] for kwargs in calls)) + tokenizer.apply_chat_template = lambda prompt, **kwargs: [0] * (recipe.CONTEXT_LENGTH - recipe.MAX_RESPONSE_TOKENS + 1) + self.assertRaises(AssertionError, Campaign(self.root, tokenizer, rows).render_prompts) + + def test_cleanup_deletes_only_named_resources(self): + seen = [] + + def handler(request): + seen.append((request.url.path, dict(request.url.params))) + return httpx.Response(200, json={}) + + with patch('lifecycle.httpx.Client', return_value=httpx.Client(transport=httpx.MockTransport(handler))), \ + patch('lifecycle.headers', return_value={}): + cleanup(self.root, {'account': 'acct', 'trainer_id': 'owned-trainer', 'deployment_id': 'owned-rollout'}) + self.assertEqual(seen, [('/v1/accounts/acct/rlorTrainerJobs/owned-trainer', {}), + ('/v1/accounts/acct/deployments/owned-rollout', {'ignoreChecks': 'true'})]) + + +if __name__ == '__main__': + unittest.main() diff --git a/hosted/opd/README.md b/hosted/opd/README.md new file mode 100644 index 0000000..8f8e9ad --- /dev/null +++ b/hosted/opd/README.md @@ -0,0 +1,100 @@ +# OPD and MOPD on a hosted training API + +Two experiments that share one training loop: + +| File | Purpose | +|---|---| +| `opd.py` | On-policy distillation. A fresh student learns one domain from one frozen teacher. | +| `mopd.py` | Multi-teacher OPD. A fresh student learns several domains at once; each response is scored by its own domain's teacher. | +| `common.py` | Shared code: prompts, rollouts, the distillation update, evaluation, records. | +| `backend.py` | The service. Only the Tinker client API is used, so another Tinker-compatible provider is one more class (Tinker, Fireworks dedicated, Fireworks serverless). | +| `sft.py` | SFT teachers from teacher traces, and the teacher benchmark. | +| `report.py` | Tables, figures, throughput and a cost estimate from saved records. | +| `fireworks_lease.py`, `check_teachers.py`, `run_fireworks.sh`, `run_serverless.sh` | Fireworks: provision and release resources, check uploaded teachers against the Tinker ones, and run the campaign end to end. See the script headers. | +| `test_opd.py`, `test_sft.py` | Offline tests. | + +Results of the Tinker runs are in [RESULTS.md](RESULTS.md). + +## One update + +1. The student samples one response per training prompt (temperature 1, thinking on, up to 30,720 tokens). +2. The prompt's teacher scores every response token. This is a prefill with no generation, and it starts as + soon as a response finishes, while other responses are still generating. +3. The per-token advantage is teacher logprob minus student logprob, i.e. the negative sampled-token reverse + KL. There is no task reward. +4. One importance-sampling policy-gradient step on those advantages (`loss_fn='importance_sampling'`). +5. The new weights are saved and published to the sampler, so the next update samples from them. + +The loop is asynchronous by default, matching `shared/recipe.py` (`MAX_POLICY_LAG = 1`) and +`hosted/tinker/run.py --pipeline-depth 1`. Once update n's responses are in, update n+1's prompts are sampled +and scored on the same weights while update n trains and publishes. Each batch is at most one optimizer step +stale. Every rollout and metrics row records its `policy_lag`, and the loss uses the logprobs of the weights +that sampled it. + +`--policy-lag 0` is the synchronous variant, where every response comes from the current weights, as in the +cookbook's `tinker_cookbook/distillation/train_on_policy.py`. The Tinker and Fireworks runs of 2026-09-27/28 +used `--policy-lag 0`, which was the only mode at the time. + +## Protocol + +Defaults come from `shared/recipe.py`, so runs are comparable with the self-hosted frameworks. + +- Student: fresh rank-64 LoRA on `Qwen/Qwen3.6-35B-A3B` (tokenizer pinned at + `995ad96eacd98c81ed38be0c5b274b04031597b0`), LR 1e-5 (the recipe's 1e-6 times the cookbook's 10× for + LoRA), Adam betas 0.9/0.999, gradient clip 1, no weight decay. +- 20 updates × 128 prompts × 1 response. OPD takes all 128 from its domain; MOPD takes 64 from each. Each + domain's training split is shuffled once with a fixed seed, so OPD and MOPD see the same caesar prompts in + the same order (OPD goes through them twice as fast). +- Evaluation at updates 0, 10 and 20 on every domain, so OPD also reports the domain it does not train on: + the first 100 dev problems, 3 samples each at temperature 1. This is the teacher benchmark from + `sft.py`, so student, teacher and base-model numbers are directly comparable. +- Teachers: the SFT teachers from `hosted/tinker/launch.py --teachers sft` (see + [hosted/tinker/README.md](../tinker/README.md)). + +## Run + +Requires Python 3.12 with `hosted/tinker/requirements.txt`, `reasoning-gym==0.1.25` (or its site-packages +in `$CAMPAIGN_PYDEPS`), the pinned tokenizer in the local cache, and `TINKER_API_KEY` exported (or a +`config/tinker.local.json`, which can also pin the account). + +```bash +TEACHERS=/abs/sft-teachers/teachers.json # {domain: sampler path} + +python hosted/opd/opd.py --domain caesar_cipher --output /abs/opd-caesar \ + --teacher "$(python -c "import json; print(json.load(open('$TEACHERS'))['caesar_cipher'])")" +python hosted/opd/mopd.py --teachers $TEACHERS --output /abs/mopd + +python hosted/opd/report.py /abs/opd-caesar /abs/mopd --output /abs/report --teachers-campaign /abs/sft-teachers +python -B -m unittest discover -s hosted/opd -p 'test_*.py' +``` + +`--help` lists every flag (updates, prompts, LR, rank, evaluation size, concurrency, backend). To resume +from an optimizer-state checkpoint into a new directory, pass `--resume STATE --start-update N`. + +## Saved records + +| File | Contents | +|---|---| +| `config.json`, `teachers.json`, `versions.json`, `source/`, `data-manifest.json` | Exact inputs: settings, teacher paths, package versions, a copy of the code, data hashes | +| `account.json`, `capabilities.json`, `model-info.json` | The verified account and the service's view of the model and LoRA | +| `rollouts/update-NNN.jsonl.gz` | Every training response: prompt and response tokens, student and teacher logprobs, advantages, text, task score, stop reason, and submit/sample/score times | +| `rollouts/eval-NNN.jsonl.gz` | Every evaluation response with its tokens, text and score | +| `metrics.jsonl` | Per update: tokens, truncations, per-domain reverse KL and score, timings, throughput, trainer-versus-sampler logprob gap, sampler path, service metrics | +| `evaluations.jsonl` | Per evaluation: avg@3, pass@3, mean tokens and truncations per domain | +| `checkpoints.jsonl` | Optimizer-state paths at each evaluation, for resuming or further training | +| `events.jsonl` | Event log with UTC and elapsed time; a failure also writes `error.txt` | + +Weights stay on the provider; only their paths are recorded. + +## Timing definitions + +All times are client wall time on a shared hosted service, not GPU utilization. + +- **rollout**: from submitting the update's prompts until every response is sampled and teacher-scored. +- **median / p90 / last response**: when those responses finished sampling, measured from the update's + start. The gap between median and last is the straggler cost of long thinking responses in a synchronous + update. +- **teacher tail**: scoring still running after the last response finished. +- **train**: `forward_backward` plus `optim_step`. +- **sync**: saving weights and pointing the sampler at them. +- **sampled tokens/s**: the update's response tokens divided by the time to its last response. diff --git a/hosted/opd/RESULTS.md b/hosted/opd/RESULTS.md new file mode 100644 index 0000000..9821c0f --- /dev/null +++ b/hosted/opd/RESULTS.md @@ -0,0 +1,132 @@ +# OPD and MOPD on Tinker: results (2026-09-27) + +All runs used the Tinker API (SA RL Infra Research account) with thinking on and a 32k context. The full +records are kept outside Git, in `opd-runs/`: + +| Directory | Contents | +|---|---| +| `tinker-sa-sft-teachers-20260927T115503Z/` | The two SFT teachers and the base-model benchmark (`sft.py`) | +| `tinker-sa-opd-caesar-20260927T123742Z/` | OPD on caesar_cipher (`opd.py`), 853 MB of records | +| `tinker-sa-mopd-20260927T123742Z/` | MOPD on caesar_cipher + simple_geometry (`mopd.py`), 538 MB of records | +| `tinker-sa-opd-mopd-report-20260927/` | `REPORT.md`, `summary.json`, and the figures (PNG and SVG) from `report.py` | + +Weights stay on Tinker; the records hold only their paths. Each run used one seed, so these are single +observations, not a controlled ranking. + +## Pipeline + +1. Teachers by SFT (`sft.py`): one rank-64 LoRA per domain, one epoch over the verified traces of the frozen + GRPO teachers + ([dataset](https://huggingface.co/datasets/semianalysisai/Qwen3.6-35B-A3B-teacher-sft-caesar-geometry)). +2. OPD (`opd.py`): a fresh rank-64 student, 20 updates × 128 caesar_cipher prompts, from the caesar teacher. +3. MOPD (`mopd.py`): a fresh rank-64 student, 20 updates × (64 caesar_cipher + 64 simple_geometry) prompts, + each response scored by its own domain's teacher. + +Both students use the shared recipe: LR 1e-5, one response per prompt, up to 30,720 response tokens, and +sampled-token reverse KL as the only signal. These runs were synchronous (policy lag 0). The recipe default, +and now `common.py`'s default, is asynchronous with policy lag 1 (`--policy-lag 1`). + +## Quality + +Same benchmark for every model: the first 100 dev problems of each domain (never trained on), 3 samples each +at temperature 1, avg@3. + +| Model | caesar_cipher | simple_geometry | +|---|---|---| +| Base Qwen3.6-35B-A3B | 27.3% | 51.3% | +| SFT teacher (own domain) | 79.3% | 99.7% | +| OPD, update 10 | 65.3% | 64.0% (not trained) | +| OPD, update 20 | 74.7% | 70.7% (not trained) | +| MOPD, update 10 | 59.0% | 98.0% | +| MOPD, update 20 | 75.3% | 99.3% | + +- MOPD matches OPD on caesar_cipher (75.3% vs 74.7%) with half as many caesar prompts (1,280 vs 2,560), and + also reaches the geometry teacher (99.3% vs 99.7%). The second teacher cost nothing measurable on the + first domain. +- Both students close most of the gap between the base model and the caesar teacher (27.3% to 79.3%) in 20 + updates: OPD 91%, MOPD 92%. Geometry is easier; MOPD is at 98% by update 10. +- OPD improves geometry without training on it (53% to 71%), so distilling the caesar teacher transfers + partially. MOPD's geometry teacher closes the rest of the gap. +- Noise: the base model was benchmarked three times (caesar 27.3 / 28.0 / 27.0%, geometry 51.3 / 53.3 / + 56.0%). Differences of 2–3 points are within sampling noise at 300 samples. + +## Throughput and time + +Client wall time on a shared service, not GPU utilization. OPD and MOPD ran at the same time on one account, +and the SFT teachers also trained concurrently, so neither run had the service to itself. + +| | OPD | MOPD | +|---|---|---| +| Wall time (20 updates + 3 evaluations) | 67.4 min | 59.7 min | +| Median update | 173 s | 152 s | +| … rollout (sample + teacher score) | 142 s (83%) | 127 s (83%) | +| … train step (`forward_backward` + `optim_step`) | 16 s (9%) | 14 s (9%) | +| … weight sync (save + new sampler) | 4 s (3%) | 5 s (4%) | +| … client-side (writing rollouts, building the batch) | 6% | 4% | +| Median / p90 / last response finishes | 65 / 115 / 140 s | 33 / 88 / 125 s | +| Teacher scoring left after the last response | 2 s | 2 s | +| Sampled tokens/s (aggregate, median update) | 11,790 | 8,355 | +| Trained tokens/s (median update) | 103,907 | 74,876 | +| Evaluation (600 samples) | ~2.5 min each | ~2.5 min each | + +- Long thinking responses set the pace. A single response decodes at a median of 183 tokens/s (p10 152, + p90 232), so a response that hits the 30,720-token cap takes about 170 s. Every update had some truncated + responses (122 OPD and 73 MOPD training responses in total), so a synchronous update cannot go much below + 2.5 minutes regardless of parallelism. The median response finished in 65 s (OPD) or 33 s (MOPD); the rest + of the rollout is waiting for stragglers. +- Teacher scoring costs almost no wall time. Scoring a response is one prefill (`compute_logprobs`), about + 2 s, and it starts as soon as the response finishes. It overlaps the remaining generation, leaving only 2 s + after the last response. +- Training is fast relative to sampling. One update's 1.4M (OPD) or 0.9M (MOPD) tokens train in about 15 s + (75–104k tokens/s). SFT, which only trains, ran at 196k tokens/s on caesar (1.5M-token batches, 7.7 s per + update, pipelined) and 66k tokens/s on geometry (0.4M-token batches, 6.1 s per update). +- Ways to cut time: asynchronous (off-policy) sampling, so update n+1 samples while n trains; a lower token + cap; or dropping and resampling stragglers. Each changes the protocol and would be reported as a separate + variant. +- Trainer and sampler agree closely: the mean absolute logprob gap on sampled tokens is 0.015 nats. + +## Cost + +Estimates from token counts at Tinker's published prices (sampling $1.335/M, prefill $0.54/M, training +$1.177/M), not invoices. Prefill caching is ignored. + +| Stage | Tokens | Estimated cost | +|---|---|---| +| SFT teachers (both) | 292M trained | ~$345 | +| Base and teacher benchmarks | ~5M sampled | ~$7 | +| OPD student | 48M sampled, 34M prefill, 34M trained | ~$122 | +| MOPD student | 35M sampled, 22M prefill, 21M trained | ~$83 | + +The SFT teachers cost more than both students combined: one epoch covers 292M tokens of thinking traces, +while each student needs only 20 × 128 responses. + +## Observations + +- The API surface is small. The loop uses only `sample`, `compute_logprobs`, `forward_backward`, + `optim_step`, `save_weights_for_sampler` and `save_state` (`backend.py`). The loss is the built-in + `importance_sampling` with per-token advantages. Multi-teacher routing is a dict of sampling clients. No + GPUs, serving or weight transfer are managed client-side. +- Weights never leave the service. Teachers and students are addressed by `tinker://` paths. Every update's + student weights were saved as sampler checkpoints, so any update can be re-evaluated later. +- The SFT loss barely moved (caesar 0.316 to 0.308 nats/token, geometry 0.342 to 0.331) while accuracy + jumped (27% to 79%, 51% to 99.7%). The traces come from a GRPO-trained copy of the same base, so they are + nearly on-distribution and the useful signal is a small per-token shift. OPD shows the same pattern: the + reverse KL starts at only 0.003–0.005 nats/token and still drives large accuracy gains. +- For large `forward_backward` requests that the service chunks internally, its `loss:sum` metric came back + 2–3× the true sum. The per-token logprobs it returns were exact (checked against `forward` on the same + batches), so the NLL is computed from those. +- The first SFT step spikes (caesar NLL 0.305 to 0.350, geometry 0.339 to 0.421 after update 1) and recovers + by update 3. This is Adam's first step at the cookbook's recommended LR of about 5e-4. +- Thinking at 32k dominates response length (11–13k tokens median on caesar), step time (the straggler + floor), and cost (sampling is the largest cost for both students). +- No stage logged an error, warning or retry (the SDK may retry internally without logging). The runs used + about 1% CPU and a few MB/min of network on the laptop that drove them. + +## Reproduce + +```bash +python hosted/tinker/launch.py --root /abs/sft-teachers --teachers sft --teachers-only +python hosted/opd/opd.py --domain caesar_cipher --teacher --output /abs/opd +python hosted/opd/mopd.py --teachers /abs/sft-teachers/teachers.json --output /abs/mopd +python hosted/opd/report.py /abs/opd /abs/mopd --output /abs/report --teachers-campaign /abs/sft-teachers +``` diff --git a/hosted/opd/backend.py b/hosted/opd/backend.py new file mode 100644 index 0000000..8321854 --- /dev/null +++ b/hosted/opd/backend.py @@ -0,0 +1,258 @@ +"""The service behind sft.py, opd.py and mopd.py. + +The scripts use only the Tinker client API: a training client (forward_backward, optim_step, +save_state, save_weights_for_sampler) and sampling clients (sample, compute_logprobs). +A backend connects and hands out those clients: + +- Tinker: the hosted Tinker service. +- Fireworks: the dedicated Training API (Tinker-compatible), attached to a lease that + fireworks_lease.py provisions (a trainer and a sampling deployment, under a budget watchdog). +- FireworksServerless: the per-token serverless Training API for the student; uploaded teachers + run on their own deployments from a teacher-only lease (fireworks_lease.py up --teachers-only). +""" +import json +import os +from pathlib import Path +import sys + +import tinker + +sys.path.insert(0, str(Path(__file__).resolve().parents[2])) +from shared.credentials import api_key, local_config # noqa: E402 + + +class Tinker: + """Tinker's hosted service. The key comes from $TINKER_API_KEY, or from the `~/.zprofile` label in + config/tinker.local.json; if that file sets expected_email and expected_org, other accounts are refused.""" + name = 'tinker' + + def __init__(self, experiment): + settings = local_config('tinker') + os.environ['TINKER_API_KEY'] = api_key('TINKER_API_KEY', settings.get('api_key_profile_label')) + self.account = self.verify(settings) + self.service = tinker.ServiceClient(user_metadata={'experiment': experiment}) + + @staticmethod + def verify(settings): + import httpx + from tinker.cli.auth_api import TinkerAuthApi + with httpx.Client(timeout=30) as client: + identity = TinkerAuthApi(client).get_self_api_key(os.environ['TINKER_API_KEY']) + account = {'email': identity.details.user_details.email, 'organization': identity.details.org_details.name} + expected = settings.get('expected_email'), settings.get('expected_org') + if any(expected) and (account['email'], account['organization']) != expected: + raise RuntimeError(f'Account guard rejected {account["email"]} / {account["organization"]}') + return {**account, 'account_guard_passed': any(expected)} + + def capabilities(self, model): + caps = self.service.get_server_capabilities() + assert model in [m.model_name for m in caps.supported_models], f'{model} is not served' + return caps.model_dump() + + def new_student(self, model, rank, seed, metadata): + """A fresh LoRA on the base model.""" + return self.service.create_lora_training_client(model, rank=rank, seed=seed, user_metadata=metadata) + + def resume_student(self, state_path): + """A student restored from an optimizer-state checkpoint.""" + return self.service.create_training_client_from_state_with_optimizer(state_path) + + def sampler(self, model=None, path=None): + """A sampling client for the base model or for saved sampler weights (a frozen teacher).""" + return self.service.create_sampling_client(**({'model_path': path} if path else {'base_model': model})) + + def sync(self, train, name): + """Publish the student's current weights and return a sampling client for them, plus their path.""" + path = train.save_weights_for_sampler(name=name).result().path + return self.sampler(path=path), path + + def use_tokenizers(self, student, teacher): + pass + + def close(self): + pass + + +class Fireworks: + """Fireworks' dedicated Training API on the lease in $FIREWORKS_LEASE (see fireworks_lease.py). + + Differences from Tinker that the scripts do not see: + - One LoRA session per trainer, so runs on one lease go one after another. + - Sampling hot-loads a saved snapshot into the lease's single deployment, so one process + samples from one set of weights at a time. + - A base-model sampler is the snapshot of a fresh LoRA (its B matrices are zero). + - With --policy-lag 1 the next batch is in flight during the hot load. The deployment's SYNC transition lets + in-flight requests finish on the old weights, but requests still queued client-side (beyond the concurrency + window) are sent after it and sample the new weights, so their true lag is 0, not the recorded 1. The loss + stays correct: it uses the logprobs the deployment returned. + - A frozen teacher is an uploaded model on its own deployment: teachers.json holds + `fireworks-deployment://accounts//deployments/` paths (fireworks_lease.py --teacher). + """ + TEACHER_PREFIX = 'fireworks-deployment://' + INFERENCE_URL = 'https://api.fireworks.ai' + # Requests in flight per sampler, per deployment replica. The SDK's adaptive window grows while the deployment + # keeps up and halves on overload (HTTP 429), but its default start of 8 barely grows during multi-minute + # thinking responses; a fixed 256 on one 2-GPU replica was refused with 429s. Start at 32 per replica instead. + WINDOW_PER_REPLICA = 32 + name = 'fireworks' + MODELS = {'Qwen/Qwen3.6-35B-A3B': 'accounts/fireworks/models/qwen3p6-35b-a3b'} + SHAPE = 'accounts/fireworks/trainingShapes/qwen3p6-35b-a3b-256k-lora/versions/a1k9h54n' + MAX_LORA_RANK = 64 + CONTEXT_LENGTH = 32_768 + + def __init__(self, experiment): + from fireworks.training.sdk import FiretitanServiceClient + sys.path.insert(0, str(Path(__file__).resolve().parents[1]/'fireworks')) + from lifecycle import check, verify_account # The budget check and account check of hosted/fireworks. + self.check = check + self.lease = Path(os.environ['FIREWORKS_LEASE']).resolve() + self.resources = json.loads((self.lease/'resources.json').read_text()) + self.check(self.lease) + settings = local_config('fireworks') + os.environ['FIREWORKS_API_KEY'] = api_key('FIREWORKS_API_KEY', settings.get('api_key_profile_label')) + self.account = {**verify_account(self.resources['account']), 'lease': str(self.lease), + 'trainer_id': self.resources['trainer_id'], 'deployment_id': self.resources['deployment_id']} + self.service = FiretitanServiceClient.from_firetitan_config(**service_config(self.resources, experiment)) + # The deployment's completions API takes text stop sequences; the sampler converts token ids with this. + from transformers import AutoTokenizer + from common import MODEL, REVISION + self.tokenizer = AutoTokenizer.from_pretrained(MODEL, revision=REVISION) + + def capabilities(self, model): + assert model in self.MODELS, f'{model} has no Fireworks mapping' + return {'model': self.MODELS[model], 'training_shape': self.SHAPE, 'max_lora_rank': self.MAX_LORA_RANK, + 'max_context_length': self.CONTEXT_LENGTH, 'resources': self.resources} + + def new_student(self, model, rank, seed, metadata): + self.check(self.lease) + assert rank <= self.MAX_LORA_RANK + return self.service.create_lora_training_client(base_model=self.MODELS[model], rank=rank, seed=seed, + alpha=32, user_metadata=metadata) + + def resume_student(self, state_path): + self.check(self.lease) + return self.service.create_training_client_from_state_with_optimizer(state_path) + + def sampler(self, model=None, path=None): + """A sampler for saved weights (hot-loaded into the deployment), or for the base model.""" + self.check(self.lease) + if path and path.startswith(self.TEACHER_PREFIX): + from fireworks.training.sdk.client import FiretitanSamplingClient + from fireworks.training.sdk.sampling import DeploymentSampler + return FiretitanSamplingClient(DeploymentSampler( + inference_url=self.INFERENCE_URL, model=path[len(self.TEACHER_PREFIX):], + api_key=os.environ['FIREWORKS_API_KEY'], tokenizer=self.tokenizer, + concurrency_controller=self.concurrency(replicas=1))) + if path: + return self.service.create_sampling_client(model_path=path, tokenizer=self.tokenizer, + concurrency_controller=self.concurrency()) + client, _ = self.sync(self.new_student(model, self.MAX_LORA_RANK, 0, {'purpose': 'base-model sampler'}), 'base') + return client + + def sync(self, train, name): + self.check(self.lease) + path = train.save_weights_for_sampler(name=name).result().path + return self.service.create_sampling_client(model_path=path, tokenizer=self.tokenizer, training_client=train, + concurrency_controller=self.concurrency()), path + + def concurrency(self, replicas=None): + from fireworks.training.sdk.concurrency import AdaptiveConcurrencyController + replicas = replicas or self.resources.get('sampler_replicas', 1) + return AdaptiveConcurrencyController(initial_window=self.WINDOW_PER_REPLICA * replicas, + max_window=4 * self.WINDOW_PER_REPLICA * replicas) + + def use_tokenizers(self, student, teacher): + pass # This backend serves the recipe's model and loads its tokenizer itself. + + def close(self): + self.service.close() + + +class FireworksServerless: + """Fireworks' serverless Training API: a pooled, per-token LoRA trainer and sampler for the student. + + Teachers are uploaded models on their own deployments (`fireworks-deployment://...` paths in + teachers.json), from a teacher-only lease in $FIREWORKS_LEASE whose budget check applies here too. + Serverless snapshots can only be sampled while this session is alive, and checkpoint names must be + at most 17 characters. + """ + name = 'fireworks-serverless' + URL = 'https://api.fireworks.ai/training/v1/serverless' + MODELS = {'Qwen/Qwen3.8-27B': 'accounts/fireworks/models/qwen3p8-27b'} + WINDOW = 64 # Adaptive concurrency start; it halves on 429s. + + def __init__(self, experiment): + from fireworks.training.sdk import FiretitanServiceClient + sys.path.insert(0, str(Path(__file__).resolve().parents[1]/'fireworks')) + from lifecycle import check, verify_account + self.check = check + self.lease = Path(os.environ['FIREWORKS_LEASE']).resolve() if os.environ.get('FIREWORKS_LEASE') else None + settings = local_config('fireworks') + os.environ['FIREWORKS_API_KEY'] = api_key('FIREWORKS_API_KEY', settings.get('api_key_profile_label')) + account = settings.get('account') or os.environ.get('FIREWORKS_ACCOUNT_ID') + self.account = {**verify_account(account), 'endpoint': self.URL, 'teacher_lease': str(self.lease)} + self.service = FiretitanServiceClient(api_key=os.environ['FIREWORKS_API_KEY'], base_url=self.URL) + self.tokenizer = self.teacher_tokenizer = None + + def guard(self): + if self.lease: + self.check(self.lease) + + def use_tokenizers(self, student, teacher): + self.tokenizer, self.teacher_tokenizer = student, teacher # Samplers turn stop-token ids into text with them. + + def capabilities(self, model): + assert model in self.MODELS, f'{model} is not mapped to a serverless model' + return {'model': self.MODELS[model], 'endpoint': self.URL, 'teacher_lease': str(self.lease)} + + def new_student(self, model, rank, seed, metadata): + self.guard() + return self.service.create_lora_training_client(base_model=self.MODELS[model], rank=rank, seed=seed, + user_metadata=metadata) + + def resume_student(self, state_path): + self.guard() + return self.service.create_training_client_from_state_with_optimizer(state_path) + + def sampler(self, model=None, path=None): + self.guard() + from fireworks.training.sdk.concurrency import AdaptiveConcurrencyController + if path and path.startswith(Fireworks.TEACHER_PREFIX): + from fireworks.training.sdk.client import FiretitanSamplingClient + from fireworks.training.sdk.sampling import DeploymentSampler + return FiretitanSamplingClient(DeploymentSampler( + inference_url=Fireworks.INFERENCE_URL, model=path[len(Fireworks.TEACHER_PREFIX):], + api_key=os.environ['FIREWORKS_API_KEY'], tokenizer=self.teacher_tokenizer, + concurrency_controller=AdaptiveConcurrencyController(initial_window=32, max_window=128))) + assert path, 'Serverless samples saved snapshots; save a fresh LoRA for the base model' + return self.service.create_sampling_client( + model_path=path, tokenizer=self.tokenizer, + concurrency_controller=AdaptiveConcurrencyController(initial_window=self.WINDOW, max_window=8 * self.WINDOW)) + + def sync(self, train, name): + self.guard() + assert len(name) <= 17, 'Serverless checkpoint names are at most 17 characters' + path = train.save_weights_for_sampler(name=name).result().path + return self.sampler(path=path), path + + def close(self): + self.service.close() + + +def service_config(resources, experiment=None): + """from_firetitan_config arguments for the lease's trainer and deployment; closing never deletes them.""" + return dict( + api_key=os.environ['FIREWORKS_API_KEY'], base_model=Fireworks.MODELS['Qwen/Qwen3.6-35B-A3B'], + tokenizer_model='Qwen/Qwen3.6-35B-A3B', max_lora_rank=Fireworks.MAX_LORA_RANK, + training_shape_id=Fireworks.SHAPE, trainer_job_id=resources['trainer_id'], + deployment_id=resources['deployment_id'], max_context_length=Fireworks.CONTEXT_LENGTH, + replica_count=resources.get('sampler_replicas', 1), + **({'deployment_shape': resources['sampler_shape']} if resources.get('sampler_shape') else {}), + trainer_replica_count=1, hot_load_transition_type=resources.get('hot_load_transition_type', 'SYNC'), + disable_speculative_decoding=True, + cleanup_trainer_on_close=False, cleanup_deployment_on_close=None, trainer_timeout_s=1800, + trainer_pending_timeout_s=1800, deployment_timeout_s=1800, wait_for_trainer_before_deployment=False, + **({'user_metadata': {'experiment': experiment}} if experiment else {})) + + +BACKENDS = {'tinker': Tinker, 'fireworks': Fireworks, 'fireworks-serverless': FireworksServerless} diff --git a/hosted/opd/check_teachers.py b/hosted/opd/check_teachers.py new file mode 100644 index 0000000..47ef936 --- /dev/null +++ b/hosted/opd/check_teachers.py @@ -0,0 +1,61 @@ +"""Check that a backend's teachers match the Tinker teachers on recorded rollouts, before paying for a run. + +Each reference rollout holds its prompt and response tokens, the Tinker teacher's logprobs, and the +near-base student's logprobs on the same tokens. A teacher passes when its logprobs are much closer +to the Tinker teacher's than the student's are (the ratio is at most --max-ratio). + + FIREWORKS_LEASE=/abs/lease python hosted/opd/check_teachers.py --backend fireworks \\ + --teachers /abs/lease/teachers.json --rollouts reference-rollouts.json --output /abs/lease/teacher-check.json +""" +import argparse +import json +from pathlib import Path +import statistics +import time + +import tinker + +from backend import BACKENDS + + +def mean_abs(a, b): + assert len(a) == len(b) + return statistics.mean(abs(x - y) for x, y in zip(a, b)) + + +def main(): + p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + p.add_argument('--backend', choices=sorted(BACKENDS), required=True) + p.add_argument('--teachers', type=Path, required=True, help='JSON: {domain: teacher path}') + p.add_argument('--rollouts', type=Path, required=True, help='JSON: {domain: [recorded rollouts]}') + p.add_argument('--output', type=Path, required=True) + p.add_argument('--max-ratio', type=float, default=0.6) + args = p.parse_args() + teachers = json.loads(args.teachers.read_text()) + rollouts = json.loads(args.rollouts.read_text()) + backend = BACKENDS[args.backend](experiment='check-teachers') + results, passed = [], True + try: + for domain, path in teachers.items(): + sampler = backend.sampler(path=path) + for r in rollouts[domain]: + start = time.monotonic() + full = sampler.compute_logprobs(tinker.ModelInput.from_ints(r['prompt_tokens'] + r['tokens'])).result() + teacher = full[len(r['prompt_tokens']):] + gap, student_gap = mean_abs(teacher, r['teacher_logprobs']), mean_abs(r['student_logprobs'], r['teacher_logprobs']) + ok = gap <= args.max_ratio * student_gap + passed &= ok + results.append({'domain': domain, 'index': r['index'], 'tokens': len(r['tokens']), 'passed': ok, + 'mean_abs_teacher_minus_tinker_teacher': round(gap, 4), + 'mean_abs_student_minus_tinker_teacher': round(student_gap, 4), + 'seconds': round(time.monotonic() - start, 1)}) + print(json.dumps(results[-1]), flush=True) + finally: + backend.close() + args.output.write_text(json.dumps({'passed': passed, 'max_ratio': args.max_ratio, 'results': results}, indent=2)) + if not passed: + raise SystemExit('A teacher does not match its Tinker reference') + + +if __name__ == '__main__': + main() diff --git a/hosted/opd/common.py b/hosted/opd/common.py new file mode 100644 index 0000000..c2236e9 --- /dev/null +++ b/hosted/opd/common.py @@ -0,0 +1,408 @@ +"""The pieces opd.py and mopd.py share: prompts, rollouts, the distillation update, evaluation, records. + +One update (Experiment.update): + 1. The student samples one response per training prompt. + 2. The frozen teacher of that prompt's domain scores every response token (a prefill, no + generation). Scoring starts as soon as a response finishes, while others still generate. + 3. The per-token advantage is teacher logprob minus student logprob: the negative + sampled-token reverse KL. There is no task reward. + 4. One importance-sampling policy-gradient step on those advantages. + 5. The new weights are saved and published to the sampler for the next update. + +With --policy-lag 1 (the recipe's default, recipe.MAX_POLICY_LAG), the next update's prompts are sampled +and scored on the current weights while this update trains and publishes: each batch is at most one +optimizer step stale, and the importance-sampling loss uses the logprobs of the weights that sampled it. +--policy-lag 0 is the synchronous variant: every response comes from the current weights. +The protocol defaults are shared/recipe.py's. Everything goes to the run directory: config and +sources, every rollout with its tokens and logprobs, per-update metrics and timings, +evaluations, checkpoints, and events. report.py turns a run into tables and plots. +""" +from concurrent.futures import ThreadPoolExecutor +from datetime import datetime, timezone +import gzip +import importlib.metadata +import json +import math +from pathlib import Path +import random +import shutil +import statistics +import sys +import threading +import time +import traceback + +REPO = Path(__file__).resolve().parents[2] +sys.path.insert(0, str(REPO)) +import tinker # noqa: E402 +from transformers import AutoTokenizer # noqa: E402 +from tinker_cookbook import renderers # noqa: E402 + +from backend import BACKENDS # noqa: E402 +from shared import recipe, scoring # noqa: E402 +from shared.puzzles import REPO_DATA, load_split # noqa: E402 + +# Models a run can use: HF repo, the pinned tokenizer revision, and the cookbook renderer (thinking on). +# Hosted weights are provider-controlled; the revision pins the tokenizer and chat template. +MODELS = { + 'qwen3.6-35b-a3b': (*recipe.BASE_MODEL, 'qwen3_5'), + # Qwen3.8's template adds a reasoning-effort system preamble; xhigh is the HF default. + 'qwen3.8-27b': ('Qwen/Qwen3.8-27B', '1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0', 'qwen3_8_xhigh_reasoning'), +} +DEFAULT_MODEL = 'qwen3.6-35b-a3b' +MODEL, REVISION, RENDERER = MODELS[DEFAULT_MODEL] # The recipe's model, used by sft.py and the Fireworks backend. +SEED_STRIDE = 100_000 # Training prompt i of update u samples with seed + u*SEED_STRIDE + i. +ADAM = {'beta1': 0.9, 'beta2': 0.999, 'eps': 1e-8, 'weight_decay': 0.0, 'grad_clip_norm': 1.0} + + +def add_protocol_arguments(parser): + """The flags opd.py and mopd.py share. Defaults are shared/recipe.py's.""" + parser.add_argument('--output', type=Path, required=True, help='A new run directory') + parser.add_argument('--backend', choices=sorted(BACKENDS), default='tinker') + parser.add_argument('--student-model', choices=sorted(MODELS), default=DEFAULT_MODEL) + parser.add_argument('--teacher-model', choices=sorted(MODELS), default=None, + help='The teachers\' base model, for their prompt template (default: the student\'s). They must share ' + 'the student\'s vocabulary: they score the student\'s tokens after a prompt in their own template.') + parser.add_argument('--updates', type=int, default=recipe.UPDATES) + parser.add_argument('--prompts', type=int, default=recipe.PROMPTS_PER_UPDATE, help='Training prompts per update') + parser.add_argument('--learning-rate', type=float, default=10 * recipe.LEARNING_RATE, + help="The recipe's full-finetuning LR times the cookbook's 10x for LoRA") + parser.add_argument('--rank', type=int, default=64) + parser.add_argument('--seed', type=int, default=20260921) + parser.add_argument('--max-tokens', type=int, default=recipe.MAX_RESPONSE_TOKENS) + parser.add_argument('--eval-every', type=int, default=recipe.EVAL_INTERVAL) + parser.add_argument('--eval-examples', type=int, default=100, help='First N dev problems of every domain') + parser.add_argument('--eval-samples', type=int, default=3) + parser.add_argument('--eval-temperature', type=float, default=1.0) + parser.add_argument('--skip-initial-eval', action='store_true', + help='Skip the update-0 evaluation (a protocol change; the base model is measured elsewhere)') + parser.add_argument('--no-eval', action='store_true', help='Run no evaluations at all (a protocol change)') + parser.add_argument('--save-every', type=int, help='Optimizer-state checkpoint interval (default: --eval-every); ' + 'the last update is always saved') + parser.add_argument('--policy-lag', type=int, choices=(0, 1), default=recipe.MAX_POLICY_LAG, + help='1: sample the next batch while this update trains (async, one step stale); 0: synchronous') + parser.add_argument('--concurrency', type=int, default=256) + parser.add_argument('--resume', help='Optimizer-state path from checkpoints.jsonl; needs --start-update') + parser.add_argument('--start-update', type=int, default=0) + + +def check_protocol_arguments(parser, args): + if bool(args.resume) != bool(args.start_update): parser.error('--resume and --start-update go together') + if not 0 <= args.start_update < args.updates: parser.error('require 0 <= start-update < updates') + if args.save_every is not None and args.save_every <= 0: parser.error('--save-every must be positive') + for key in ('updates', 'prompts', 'rank', 'max_tokens', 'eval_every', 'eval_examples', 'eval_samples', 'concurrency'): + if getattr(args, key) <= 0: parser.error(f'--{key.replace("_", "-")} must be positive') + if args.eval_examples > min(recipe.EVAL_EXAMPLES.values()): parser.error('--eval-examples exceeds a dev split') + + +def load_renderer(model): + """The pinned tokenizer and cookbook renderer of a MODELS entry.""" + repo, revision, renderer = model + tokenizer = AutoTokenizer.from_pretrained(repo, revision=revision) + return tokenizer, renderers.get_renderer(renderer, tokenizer, model_name=repo) + + +def utc(): + return datetime.now(timezone.utc).isoformat() + + +def advantages(teacher_logprobs, student_logprobs): + """Per-token teacher minus student logprob: the negative sampled-token reverse KL.""" + assert len(teacher_logprobs) == len(student_logprobs) + values = [t - s for t, s in zip(teacher_logprobs, student_logprobs)] + assert all(math.isfinite(v) for v in values) + return values + + +def policy_datum(prompt, tokens, logprobs, advantage): + """An importance-sampling datum; prompt positions carry zero advantage.""" + assert tokens and len(tokens) == len(logprobs) == len(advantage) + pad = len(prompt) - 1 + shape = [pad + len(tokens)] + return tinker.Datum(model_input=tinker.ModelInput.from_ints(prompt + tokens[:-1]), loss_fn_inputs={ + 'target_tokens': tinker.TensorData(data=[0] * pad + tokens, dtype='int64', shape=shape), + 'logprobs': tinker.TensorData(data=[0.0] * pad + logprobs, dtype='float32', shape=shape), + 'advantages': tinker.TensorData(data=[0.0] * pad + advantage, dtype='float32', shape=shape)}) + + +def percentile(values, q): + values = sorted(values) + return values[min(len(values) - 1, int(q * len(values)))] + + +class Experiment: + """One run directory: a student distilled from `teachers` ({domain: teacher sampler path}).""" + + def __init__(self, args, kind, teachers): + self.args, self.kind, self.teacher_paths = args, kind, teachers + self.domains = list(teachers) + self.out = args.output.resolve() + self.out.mkdir(parents=True, exist_ok=False) + (self.out/'rollouts').mkdir() + self.started = time.monotonic() + self.lock = threading.Lock() + self.pool = ThreadPoolExecutor(max_workers=args.concurrency) + self.record_inputs() + self.backend = BACKENDS[args.backend](experiment=f'{kind}-{self.out.name}') + (self.out/'account.json').write_text(json.dumps(self.backend.account, indent=2)) + self.student_model = MODELS[args.student_model] + (self.out/'capabilities.json').write_text(json.dumps(self.backend.capabilities(self.student_model[0]), indent=2, default=str)) + self.tokenizer, self.renderer = load_renderer(self.student_model) + self.stop = self.renderer.get_stop_sequences() + teacher_model = MODELS[args.teacher_model or args.student_model] + # A teacher on another base model reads the prompt in its own template, then scores the student's tokens. + self.teacher_tokenizer, self.teacher_renderer = ( + (self.tokenizer, self.renderer) if teacher_model == self.student_model else load_renderer(teacher_model)) + self.backend.use_tokenizers(self.tokenizer, self.teacher_tokenizer) + self.data = {d: {s: load_split(d, s) for s in ('train', 'dev')} for d in recipe.DOMAINS} + self.order = {} # An independent seeded shuffle per domain: OPD and MOPD see one prompt order. + for d in self.domains: + self.order[d] = list(range(len(self.data[d]['train']))) + random.Random(args.seed + recipe.DOMAINS.index(d)).shuffle(self.order[d]) + self.log('ready', kind=kind, backend=args.backend, account=self.backend.account) + + def record_inputs(self): + config = {k: str(v) if isinstance(v, Path) else v for k, v in vars(self.args).items()} + student = MODELS[self.args.student_model] + teacher = MODELS[self.args.teacher_model or self.args.student_model] + config.update(kind=self.kind, domains=self.domains, model=student[0], tokenizer_revision=student[1], renderer=student[2], + teacher_model=teacher[0], teacher_tokenizer_revision=teacher[1], teacher_renderer=teacher[2], + adam=ADAM, loss='importance_sampling', advantage='teacher_logprob - student_logprob', + started_utc=utc()) + (self.out/'config.json').write_text(json.dumps(config, indent=2)) + (self.out/'teachers.json').write_text(json.dumps(self.teacher_paths, indent=2)) + packages = ('tinker', 'tinker-cookbook', 'transformers', 'torch') + (self.out/'versions.json').write_text(json.dumps({p: importlib.metadata.version(p) for p in packages}, indent=2)) + (self.out/'source').mkdir() + here = Path(__file__).parent + for path in (*here.glob('*.py'), REPO/'shared/recipe.py', REPO/'shared/scoring.py'): + shutil.copyfile(path, self.out/'source'/path.name) + (self.out/'data-manifest.json').write_text((REPO_DATA/'manifest.json').read_text()) + + # Records. + + def append(self, filename, row): + with self.lock, (self.out/filename).open('a') as f: + f.write(json.dumps(row, allow_nan=False) + '\n') + + def log(self, event, **fields): + row = {'event': event, 'utc': utc(), 'elapsed_seconds': time.monotonic() - self.started, **fields} + self.append('events.jsonl', row) + print(json.dumps(row), flush=True) + + def save_rollouts(self, name, rows): + with gzip.open(self.out/'rollouts'/f'{name}.jsonl.gz', 'wt') as f: + f.writelines(json.dumps(r) + '\n' for r in rows) + + # Prompts and sampling. + + def prompt(self, row, teacher=False): + """The pinned HF chat template with thinking on; it must equal the cookbook renderer's tokens. + With teacher=True, the teachers' template (the same unless the teachers are another base model).""" + tokenizer, renderer = (self.teacher_tokenizer, self.teacher_renderer) if teacher else (self.tokenizer, self.renderer) + ids = tokenizer.apply_chat_template(row['prompt'], tokenize=True, return_dict=False, + add_generation_prompt=True, enable_thinking=recipe.ENABLE_THINKING) + assert ids == renderer.build_generation_prompt(row['prompt']).to_ints() + assert len(ids) + self.args.max_tokens <= recipe.CONTEXT_LENGTH + return ids + + def params(self, temperature, seed=None): + return tinker.SamplingParams(max_tokens=self.args.max_tokens, temperature=temperature, top_p=1.0, top_k=-1, + stop=self.stop, seed=seed) + + def train_prompts(self, domain, update, count): + """Update `update`'s `count` prompts of one domain: the next rows of its shuffled training split.""" + order = self.order[domain] + return [(domain, self.data[domain]['train'][order[(update*count + i) % len(order)]]) for i in range(count)] + + def rollout(self, student, domain, row, seed, t0): + """Sample one response, then score it with its domain's teacher. Times are seconds after t0.""" + prompt = self.prompt(row) + begin = time.monotonic() + sequence = student.sample(tinker.ModelInput.from_ints(prompt), num_samples=1, + sampling_params=self.params(1.0, seed)).result().sequences[0] + sampled = time.monotonic() + tokens, logprobs = list(sequence.tokens), list(sequence.logprobs) + teacher_prompt = self.prompt(row, teacher=True) + full = self.teachers[domain].compute_logprobs(tinker.ModelInput.from_ints(teacher_prompt + tokens)).result() + scored = time.monotonic() + teacher = full[len(teacher_prompt):] # Entry j is the teacher's logprob of token j of the full sequence. + response = self.tokenizer.decode(tokens, skip_special_tokens=True) + return {'domain': domain, 'index': row['_index'], 'seed': seed, 'prompt_tokens': prompt, 'tokens': tokens, + 'logprobs': logprobs, 'teacher_logprobs': teacher, 'advantages': advantages(teacher, logprobs), + 'response': response, 'score': scoring.score(response, row['label']), + 'stop_reason': sequence.stop_reason, 'submitted': begin - t0, 'sampled': sampled - t0, + 'scored': scored - t0} + + # Evaluation: the teacher benchmark protocol (hosted/opd/sft.py) on every recipe domain, + # including domains this run does not train on. + + def evaluate(self, student, update): + a = self.args + start = time.monotonic() + + def one(domain, row): + result = student.sample(tinker.ModelInput.from_ints(self.prompt(row)), num_samples=a.eval_samples, + sampling_params=self.params(a.eval_temperature)).result() + records = [] + for j, sequence in enumerate(result.sequences): + text = self.tokenizer.decode(sequence.tokens, skip_special_tokens=True) + records.append({'update': update, 'domain': domain, 'index': row['_index'], 'sample': j, + 'tokens': list(sequence.tokens), 'response': text, + 'score': scoring.score(text, row['label']), 'stop_reason': sequence.stop_reason}) + return records + + futures = [self.pool.submit(one, d, row) for d in recipe.DOMAINS for row in self.data[d]['dev'][:a.eval_examples]] + records = [r for f in futures for r in f.result()] + self.save_rollouts(f'eval-{update:03d}', records) + result = {'update': update, 'seconds': time.monotonic() - start} + for d in recipe.DOMAINS: + rows = [r for r in records if r['domain'] == d] + by_problem = {} + for r in rows: + by_problem.setdefault(r['index'], []).append(r['score']) + result[d] = {'problems': len(by_problem), 'samples': len(rows), 'accuracy': statistics.mean(r['score'] for r in rows), + f'pass_at_{a.eval_samples}': statistics.mean(max(v) for v in by_problem.values()), + 'mean_tokens': statistics.mean(len(r['tokens']) for r in rows), + 'truncated': sum(r['stop_reason'] == 'length' for r in rows)} + self.append('evaluations.jsonl', result) + self.log('evaluation', **result) + + # Training. + + def start(self): + """The training client, the frozen teachers, and a sampler of the starting weights.""" + a = self.args + self.teachers = {d: self.backend.sampler(path=p) for d, p in self.teacher_paths.items()} + if a.resume: + train = self.backend.resume_student(a.resume) + else: + train = self.backend.new_student(self.student_model[0], a.rank, a.seed, {'run': self.out.name, 'kind': self.kind}) + try: + (self.out/'model-info.json').write_text(train.get_info().model_dump_json(indent=2)) + except Exception as error: # Not every service implements get_info. + (self.out/'model-info.json').write_text(json.dumps({'unavailable': str(error)[:300]})) + student, _ = self.backend.sync(train, f'update-{a.start_update:03d}') + return train, student + + def submit(self, student, update, work, version): + """Start sampling and scoring update `update`'s `work`, a list of (domain, row) prompts, on `student`: + the weights after `version` optimizer steps. Rollout times are seconds after this submission.""" + t0 = time.monotonic() + self.log('rollouts_submitted', update=update + 1, prompts=len(work), policy_version=version) + futures = [self.pool.submit(self.rollout, student, d, row, self.args.seed + update*SEED_STRIDE + i, t0) + for i, (d, row) in enumerate(work)] + return {'update': update, 'version': version, 'futures': futures} + + def update(self, train, batch, prefetch): + """One distillation update on a submitted batch. Returns the new student sampler and the next batch: + prefetch() submits it on the weights that sampled this one once this batch is in (async), or returns + None (synchronous, or the last update) for the caller to submit on the new sampler.""" + a = self.args + update = batch['update'] + t0 = time.monotonic() + self.log('update_start', update=update + 1, prompts=len(batch['futures'])) + rollouts = [f.result() for f in batch['futures']] + rollout_seconds = time.monotonic() - t0 # The exposed wait; with lookahead, sampling began during the last update. + lag = update - batch['version'] + assert 0 <= lag <= a.policy_lag, 'Refuse a stale or future-policy batch' + upcoming = prefetch() + self.save_rollouts(f'update-{update+1:03d}', [{'update': update + 1, 'policy_lag': lag, **r} for r in rollouts]) + + train_start = time.monotonic() + datums = [policy_datum(r['prompt_tokens'], r['tokens'], r['logprobs'], r['advantages']) for r in rollouts] + fb = train.forward_backward(datums, loss_fn='importance_sampling') + opt = train.optim_step(tinker.AdamParams(learning_rate=a.learning_rate, **ADAM)) + fb, opt = fb.result(), opt.result() + train_seconds = time.monotonic() - train_start + # Trainer-versus-sampler logprob mismatch on the sampled tokens, before this update's step, when the + # provider returns per-token logprobs from forward_backward (Tinker does; Fireworks returns a loss). + mismatch = [float(x) - y for r, o in zip(rollouts, fb.loss_fn_outputs, strict=True) if 'logprobs' in o + for x, y in zip(o['logprobs'].data[len(r['prompt_tokens']) - 1:], r['logprobs'], strict=True)] + + sync_start = time.monotonic() + student, sampler_path = self.backend.sync(train, f'update-{update+1:03d}') + sync_seconds = time.monotonic() - sync_start + self.append('metrics.jsonl', self.update_metrics(update, rollouts, t0, rollout_seconds, train_seconds, + sync_seconds, mismatch, sampler_path, fb, opt, lag)) + return student, upcoming + + def update_metrics(self, update, rollouts, t0, rollout_seconds, train_seconds, sync_seconds, mismatch, + sampler_path, fb, opt, lag): + sampled, scored = [r['sampled'] for r in rollouts], [r['scored'] for r in rollouts] + response_tokens = sum(len(r['tokens']) for r in rollouts) + prompt_tokens = sum(len(r['prompt_tokens']) for r in rollouts) + by_domain = {} + for d in self.domains: + rs = [r for r in rollouts if r['domain'] == d] + by_domain[d] = {'responses': len(rs), 'score': statistics.mean(r['score'] for r in rs), + 'reverse_kl': -statistics.mean(v for r in rs for v in r['advantages']), + 'mean_response_tokens': statistics.mean(len(r['tokens']) for r in rs)} + update_seconds = time.monotonic() - t0 + metric = { + 'update': update + 1, 'policy_lag': lag, 'responses': len(rollouts), 'learning_rate': self.args.learning_rate, + 'prompt_tokens': prompt_tokens, 'response_tokens': response_tokens, + 'max_response_tokens': max(len(r['tokens']) for r in rollouts), + 'truncated': sum(r['stop_reason'] == 'length' for r in rollouts), 'by_domain': by_domain, + 'timing': { # Seconds. Rollout is the wait for sampling plus the overlapped teacher scoring; response + # times are from the batch's submission, which is earlier than this update with lookahead. + 'rollout': rollout_seconds, 'train': train_seconds, 'sync': sync_seconds, 'update': update_seconds, + 'first_response': min(sampled), 'median_response': statistics.median(sampled), + 'p90_response': percentile(sampled, 0.9), 'last_response': max(sampled), + 'teacher_tail': max(scored) - max(sampled), + 'teacher_request_median': statistics.median(r['scored'] - r['sampled'] for r in rollouts)}, + 'throughput': { # Tokens per second of wall time. + 'sampled_tokens_per_second': response_tokens / max(sampled), + 'trained_tokens_per_second': (prompt_tokens + response_tokens) / train_seconds, + 'update_response_tokens_per_second': response_tokens / update_seconds}, + 'train_sampler_logprob_delta_mean': statistics.mean(mismatch) if mismatch else None, + 'train_sampler_logprob_delta_abs_mean': statistics.mean(abs(x) for x in mismatch) if mismatch else None, + 'sampler_path': sampler_path, 'forward_backward_metrics': fb.metrics, 'optimizer_metrics': opt.metrics} + self.log('update_complete', **{k: v for k, v in metric.items() if k != 'forward_backward_metrics'}) + return metric + + def checkpoint(self, train, update): + state = train.save_state(name=f'state-{update:03d}').result().path + self.append('checkpoints.jsonl', {'update': update, 'state_path': state}) + self.log('checkpoint', update=update, state_path=state) + + def train_loop(self, prompts_for_update): + """Evaluate, then run every update on prompts_for_update(update), with scheduled checkpoints and evaluations.""" + a = self.args + save_every = a.save_every or a.eval_every + train, student = self.start() + if a.start_update == 0 and not (a.skip_initial_eval or a.no_eval): + self.evaluate(student, 0) + batch = None + for update in range(a.start_update, a.updates): + batch = batch or self.submit(student, update, prompts_for_update(update), version=update) + + def prefetch(): # Async: the next batch samples on the weights that sampled this one while it trains. + if a.policy_lag and update + 1 < a.updates: + return self.submit(student, update + 1, prompts_for_update(update + 1), version=update) + + student, batch = self.update(train, batch, prefetch) + done = update + 1 + if done % save_every == 0 or done == a.updates: + self.checkpoint(train, done) + if not a.no_eval and (done % a.eval_every == 0 or done == a.updates): + self.evaluate(student, done) + self.log('complete', updates=a.updates - a.start_update) + + +def run(experiment_factory, prompts_for_update): + """Run an experiment, recording any failure in its directory.""" + experiment = None + try: + experiment = experiment_factory() + experiment.train_loop(lambda update: prompts_for_update(experiment, update)) + except BaseException as error: + if experiment: + experiment.log('failed', error_type=type(error).__name__, message=str(error)) + (experiment.out/'error.txt').write_text(traceback.format_exc()) + raise + finally: + if experiment: + experiment.pool.shutdown(wait=True, cancel_futures=True) + if hasattr(experiment, 'backend'): + experiment.backend.close() diff --git a/hosted/opd/fireworks_lease.py b/hosted/opd/fireworks_lease.py new file mode 100644 index 0000000..ebdaf0a --- /dev/null +++ b/hosted/opd/fireworks_lease.py @@ -0,0 +1,266 @@ +"""Provision, hold, and release the Fireworks resources that `--backend fireworks` runs attach to. + +A lease is one dedicated LoRA trainer plus one sampling deployment for Qwen3.6-35B-A3B, +billed per GPU-hour while it exists. For OPD/MOPD, each `--teacher DOMAIN=MODEL_ID` (an uploaded +Fireworks model) also gets its own one-GPU deployment, and the lease writes `teachers.json` +pointing at them. Run the scripts one at a time against it (one LoRA session per trainer), +then release it: + + python hosted/opd/fireworks_lease.py up --root /abs/lease --budget-usd 400 & # holds the lease + python hosted/opd/fireworks_lease.py up --root /abs/lease --budget-usd 400 \ + --teacher caesar_cipher=qwen3p6-35b-a3b-caesar-cipher-sft-tinker & # with a teacher + FIREWORKS_LEASE=/abs/lease python hosted/opd/sft.py train --backend fireworks ... + python hosted/opd/fireworks_lease.py down --root /abs/lease # deletes both + +The independent watchdog of hosted/fireworks/lifecycle.py starts before anything is created. +It deletes the trainer and deployment when the conservative spend estimate reaches the budget +minus a cleanup reserve, when `down` is called, or when the `up` process dies, and it keeps +retrying until their release is observed. The estimate is client-side (elapsed time x +--rate-usd-hour), not an invoice or a provider-enforced cap. +""" +import argparse +from datetime import datetime, timezone +import json +import os +from pathlib import Path +import subprocess +import sys +import time + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE)) +sys.path.insert(0, str(HERE.parent/'fireworks')) +from backend import Fireworks, service_config # noqa: E402 +from lifecycle import CLEANUP_RESERVE_USD, check, read_json, verify_account, write # noqa: E402 +from shared.credentials import api_key, local_config # noqa: E402 + +GPU_USD_HOUR = 13 # Public on-demand B200 price. +TEACHER_GPU = 'NVIDIA_B200_180GB' +TEACHER_DEPLOYMENT_PREFIX = 'fireworks-deployment://' # teachers.json paths that backend.Fireworks samples from. + + +def utc(): + return datetime.now(timezone.utc).isoformat() + + +def log(root, event, **fields): + row = {'utc': utc(), 'event': event, **fields} + with (root/'lease-events.jsonl').open('a') as f: + f.write(json.dumps(row, default=str) + '\n') + print(json.dumps(row, default=str), flush=True) + + +BILLING_EVERY_SECONDS = 600 + + +def billing(root, account, label): + """Fireworks' own billed usage for today (UTC, aggregated daily, lagging real time), appended to billing.jsonl.""" + import httpx + today = datetime.now(timezone.utc).date() + params = {'startTime': f'{today}T00:00:00Z', 'endTime': f'{today.fromordinal(today.toordinal() + 1)}T00:00:00Z'} + try: + response = httpx.get(f'https://api.fireworks.ai/v1/accounts/{account}/billing/summary', params=params, + headers={'Authorization': 'Bearer ' + os.environ['FIREWORKS_API_KEY']}, timeout=30) + response.raise_for_status() + items = response.json().get('lineItems', []) + cost = lambda i: int(i['totalCost'].get('units', 0)) + i['totalCost'].get('nanos', 0) / 1e9 + row = {'utc': utc(), 'label': label, 'day': str(today), 'total_usd': round(sum(cost(i) for i in items), 4), + 'items': [{'category': i.get('category'), 'gpu': i.get('groupingValue'), 'quantity': i.get('quantity'), + 'usd': round(cost(i), 4)} for i in items]} + except Exception as error: # Billing is a record, never a reason to stop the lease. + row = {'utc': utc(), 'label': label, 'error': str(error)[:300]} + with (root/'billing.jsonl').open('a') as f: + f.write(json.dumps(row) + '\n') + return row + + +def up(root, account, budget, rate, teachers, sampler_replicas, sampler_shape=None, teacher_shape=None, + teachers_only=False, hot_load_transition='SYNC'): + root.mkdir(parents=True, exist_ok=False) + run_id = 'opd-lease-' + datetime.now(timezone.utc).strftime('%m%d%H%M%S') + teacher_deployments = {d: f'{run_id}-teacher-{d.split("_")[0]}' for d in teachers} + resources = {'account': account, 'teachers_only': teachers_only, + 'trainer_id': None if teachers_only else run_id + '-train', + 'deployment_id': None if teachers_only else run_id + '-sample', + 'teacher_models': teachers, 'teacher_deployments': teacher_deployments, + 'sampler_replicas': sampler_replicas, 'sampler_shape': sampler_shape, 'teacher_shape': teacher_shape, + 'hot_load_transition_type': hot_load_transition, + 'extra_deployments': list(teacher_deployments.values()), # The watchdog deletes these too. + 'budget_start_unix': time.time(), 'prior_spend_estimate_usd': 0, 'budget_usd': budget, + 'stop_estimate_usd': budget - CLEANUP_RESERVE_USD, 'conservative_usd_hour': rate, + 'controller_pid': os.getpid(), 'training_shape': Fireworks.SHAPE, 'created_utc': utc()} + write(root/'resources.json', resources) + write(root/'account.json', verify_account(account)) + baseline = billing(root, account, 'baseline') # Today's billed total before this lease. + log(root, 'billing_baseline', total_usd=baseline.get('total_usd')) + with (root/'watchdog.log').open('w') as out: # Its own session, so it outlives this process. + watchdog = subprocess.Popen([sys.executable, str(HERE.parent/'fireworks/lifecycle.py'), str(root), + '--pid', str(os.getpid())], stdout=out, stderr=subprocess.STDOUT, + start_new_session=True) + write(root/'watchdog.json', {'pid': watchdog.pid}) + for _ in range(60): + if (root/'watchdog-ready.json').exists(): + break + if watchdog.poll() is not None: + raise RuntimeError('The watchdog failed to start; see watchdog.log') + time.sleep(1) + else: + raise RuntimeError('The watchdog did not become ready') + log(root, 'watchdog_ready', pid=watchdog.pid, budget_usd=budget, stop_estimate_usd=resources['stop_estimate_usd'], + rate_usd_hour=rate) + + from fireworks.training.sdk import FiretitanServiceClient, TrainerJobConfig, TrainerJobManager + try: + start = time.monotonic() + log(root, 'provisioning', trainer_id=resources['trainer_id'], deployment_id=resources['deployment_id']) + if not teachers_only: # A teacher-only lease serves teachers for a serverless student. + TrainerJobManager(api_key=os.environ['FIREWORKS_API_KEY']).create(TrainerJobConfig( + base_model=Fireworks.MODELS['Qwen/Qwen3.6-35B-A3B'], lora_rank=Fireworks.MAX_LORA_RANK, + learning_rate=1e-5, training_shape_ref=Fireworks.SHAPE, trainer_replica_count=1, + requested_job_id=resources['trainer_id'], inactivity_timeout='1800s', display_name=root.name)) + log(root, 'trainer_requested', seconds=time.monotonic() - start) + request_teachers(root, resources) + check(root) + wait_ready(root, resources, start) + last_billing = time.monotonic() + while not (root/'RELEASE').exists(): # Hold until `down`; the watchdog may also stop the lease. + check(root) + if time.monotonic() - last_billing >= BILLING_EVERY_SECONDS: + row = billing(root, account, 'during') + log(root, 'billing', total_usd=row.get('total_usd'), + lease_usd=round(row['total_usd'] - baseline['total_usd'], 2) if 'total_usd' in row and 'total_usd' in baseline else None) + last_billing = time.monotonic() + time.sleep(15) + log(root, 'release_requested') + finally: + (root/'TRAINING_FINISHED').write_text(utc()) # The watchdog deletes both resources and verifies it. + for _ in range(240): + if (root/'cleanup.json').exists(): + log(root, 'released', cleanup=read_json(root/'cleanup.json').get('verified_released')) + break + time.sleep(5) + else: + log(root, 'release_not_yet_verified', note='The watchdog keeps retrying; check lifecycle.jsonl') + billing(root, account, 'after_release') # Billing lags; `fireworks_lease.py billing` records it again later. + + +def request_teachers(root, resources): + """One single-GPU deployment per uploaded teacher model, serving it as-is (no hot-load).""" + from fireworks.training.sdk.deployment import DeploymentConfig, DeploymentManager + manager = DeploymentManager(api_key=os.environ['FIREWORKS_API_KEY']) + for domain, model_id in resources['teacher_models'].items(): + deployment_id = resources['teacher_deployments'][domain] + manager.create_or_get(DeploymentConfig( + deployment_id=deployment_id, base_model=f'accounts/{resources["account"]}/models/{model_id}', + min_replica_count=1, max_replica_count=1, accelerator_type=TEACHER_GPU, enable_hot_load=False, + hot_load_bucket_type=None, disable_speculative_decoding=True, + deployment_shape=resources.get('teacher_shape'), # A validated shape; shapeless only without one. + accept_shapeless_risk=not resources.get('teacher_shape'), + description=f'OPD teacher: {domain}')) + log(root, 'teacher_requested', domain=domain, model=model_id, deployment_id=deployment_id) + + +def wait_ready(root, resources, start=None): + """Attach and provision eagerly: wait for the trainer (capacity, then healthy) and create the deployment. + + The SDK otherwise provisions on first use, which would count provisioning inside the first run's timings. + """ + from fireworks.training.sdk import FiretitanServiceClient + start = time.monotonic() if start is None else start + if not resources.get('teachers_only'): + service = FiretitanServiceClient.from_firetitan_config(**service_config(resources, 'opd-lease')) + try: + service._ensure_managed_handle() # Private in fireworks-ai 1.2.14: the eager form of first-use provisioning. + finally: + service.close() + if resources.get('teacher_deployments'): + from fireworks.training.sdk.deployment import DeploymentManager + manager = DeploymentManager(api_key=os.environ['FIREWORKS_API_KEY']) + paths = {} + for domain, deployment_id in resources['teacher_deployments'].items(): + manager.wait_for_ready(deployment_id, timeout_s=3600) + paths[domain] = f'{TEACHER_DEPLOYMENT_PREFIX}accounts/{resources["account"]}/deployments/{deployment_id}' + log(root, 'teacher_ready', domain=domain, deployment_id=deployment_id, seconds=time.monotonic() - start) + write(root/'teachers.json', paths) + write(root/'lease.json', {'ready': True, 'ready_utc': utc(), 'provisioning_seconds': time.monotonic() - start, + **{k: resources[k] for k in ('account', 'trainer_id', 'deployment_id')}, + 'teacher_deployments': resources.get('teacher_deployments', {})}) + log(root, 'ready', provisioning_seconds=time.monotonic() - start) + + +def drop_teacher(root, domain): + """Delete one teacher's deployment mid-lease (e.g. geometry, before OPD) and lower the spend-estimate rate.""" + import httpx + resources = read_json(root/'resources.json') + deployment_id = resources['teacher_deployments'][domain] + response = httpx.delete(f'https://api.fireworks.ai/v1/accounts/{resources["account"]}/deployments/{deployment_id}', + params={'ignoreChecks': 'true'}, headers={'Authorization': 'Bearer ' + os.environ['FIREWORKS_API_KEY']}, + timeout=30) + response.raise_for_status() + resources['conservative_usd_hour'] -= GPU_USD_HOUR # The watchdog re-reads resources.json. + resources.setdefault('dropped_teachers', {})[domain] = utc() + write(root/'resources.json', resources) + log(root, 'teacher_dropped', domain=domain, deployment_id=deployment_id, rate_usd_hour=resources['conservative_usd_hour']) + + +def down(root): + (root/'RELEASE').write_text(utc()) + for _ in range(360): + if (root/'cleanup.json').exists(): + print(json.dumps(read_json(root/'cleanup.json'), indent=2, default=str)[:2000]) + return + time.sleep(5) + raise SystemExit('Release not verified within 30 minutes; see lifecycle.jsonl') + + +def main(): + p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + p.add_argument('action', choices=('up', 'wait', 'down', 'billing', 'drop-teacher')) + p.add_argument('--domain', help='drop-teacher: the teacher to delete') + p.add_argument('--root', type=Path, required=True, help='The lease directory (new for up)') + p.add_argument('--account', help='Default: config/fireworks.local.json, then $FIREWORKS_ACCOUNT_ID') + p.add_argument('--budget-usd', type=float, help='Required for up: total spend estimate at which to stop') + p.add_argument('--teacher', action='append', default=[], metavar='DOMAIN=MODEL_ID', + help='An uploaded teacher model to deploy for OPD/MOPD (repeatable)') + p.add_argument('--sampler-replicas', type=int, default=1, help='Replicas of the student sampling deployment') + p.add_argument('--sampler-shape', help="Deployment shape version for the sampler (default: the training shape's paired 2-GPU shape)") + p.add_argument('--sampler-gpus', type=int, default=2, help='GPUs per sampler replica, for the spend estimate') + p.add_argument('--hot-load-transition', choices=('SYNC', 'ASYNC'), default='SYNC', + help='How the sampler swaps weights with requests in flight: SYNC lets them finish on the old ' + 'weights first; ASYNC (the Fireworks default) pauses and resumes them on the new weights') + p.add_argument('--teacher-shape', help='Validated deployment shape version for the teachers (default: shapeless)') + p.add_argument('--teachers-only', action='store_true', + help='Only the teacher deployments, for a serverless student (no trainer or sampling deployment)') + p.add_argument('--rate-usd-hour', type=float, help='Spend estimate rate; default: $13 per GPU of the lease') + args = p.parse_args() + teachers = dict(t.split('=', 1) for t in args.teacher) + # Trainer 4 GPUs (the training shape), the sampler's GPUs per replica, teachers 1 each. + trainer_and_sampler = 0 if args.teachers_only else 4 + args.sampler_gpus * args.sampler_replicas + rate = args.rate_usd_hour or GPU_USD_HOUR * (trainer_and_sampler + len(teachers)) + root = args.root.resolve() + if args.action == 'down': + return down(root) + settings = local_config('fireworks') + if args.action == 'drop-teacher': + os.environ['FIREWORKS_API_KEY'] = api_key('FIREWORKS_API_KEY', settings.get('api_key_profile_label')) + return drop_teacher(root, args.domain) + if args.action == 'billing': # Record billed usage again, e.g. hours after release once it has settled. + os.environ['FIREWORKS_API_KEY'] = api_key('FIREWORKS_API_KEY', settings.get('api_key_profile_label')) + rows = [json.loads(l) for l in (root/'billing.jsonl').read_text().splitlines()] + row = billing(root, read_json(root/'resources.json')['account'], 'later') + base = next(r for r in rows if r['label'] == 'baseline') + return print(json.dumps({'today_usd': row.get('total_usd'), 'lease_usd': round(row['total_usd'] - base['total_usd'], 2)})) + if args.action == 'wait': # Eagerly provision a lease whose `up` returned before its resources were ready. + os.environ['FIREWORKS_API_KEY'] = api_key('FIREWORKS_API_KEY', settings.get('api_key_profile_label')) + resources = read_json(root/'resources.json') + return wait_ready(root, resources, start=time.monotonic() - (time.time() - resources['budget_start_unix'])) + account = args.account or settings.get('account') or os.environ.get('FIREWORKS_ACCOUNT_ID') + if not account: p.error('Set --account, config/fireworks.local.json, or $FIREWORKS_ACCOUNT_ID') + if not args.budget_usd or args.budget_usd <= CLEANUP_RESERVE_USD: p.error(f'--budget-usd must exceed {CLEANUP_RESERVE_USD}') + os.environ['FIREWORKS_API_KEY'] = api_key('FIREWORKS_API_KEY', settings.get('api_key_profile_label')) + up(root, account, args.budget_usd, rate, teachers, args.sampler_replicas, args.sampler_shape, args.teacher_shape, + args.teachers_only, args.hot_load_transition) + + +if __name__ == '__main__': + main() diff --git a/hosted/opd/mopd.py b/hosted/opd/mopd.py new file mode 100644 index 0000000..7030cf2 --- /dev/null +++ b/hosted/opd/mopd.py @@ -0,0 +1,39 @@ +"""Multi-teacher on-policy distillation (MOPD): one fresh Qwen3.6-35B-A3B student learns several domains +at once, each from its own frozen teacher. + +Every update takes an equal share of `--prompts` from each domain; each sampled response is +scored by the teacher of its own domain, and the student takes one step on all of them +together (common.py describes the update). Thinking is on. + + python hosted/opd/mopd.py --teachers teachers.json --output /abs/mopd + +teachers.json maps each domain to its teacher's sampler path, for example the one +hosted/tinker/launch.py --teachers sft writes. Its domains are the ones trained. +""" +import argparse +import json +from pathlib import Path + +from common import Experiment, add_protocol_arguments, check_protocol_arguments, run +from shared import recipe + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument('--teachers', type=Path, required=True, help='JSON: {domain: teacher sampler path}') + add_protocol_arguments(parser) + args = parser.parse_args() + check_protocol_arguments(parser, args) + teachers = json.loads(args.teachers.read_text()) + if len(teachers) < 2 or not set(teachers) <= set(recipe.DOMAINS): + parser.error(f'--teachers needs two or more of {recipe.DOMAINS}; use opd.py for one') + if args.prompts % len(teachers): + parser.error('--prompts must split evenly over the domains') + share = args.prompts // len(teachers) + domains = [d for d in recipe.DOMAINS if d in teachers] # A fixed order, whatever the file's. + run(lambda: Experiment(args, 'mopd', {d: teachers[d] for d in domains}), + lambda experiment, update: [p for d in domains for p in experiment.train_prompts(d, update, share)]) + + +if __name__ == '__main__': + main() diff --git a/hosted/opd/opd.py b/hosted/opd/opd.py new file mode 100644 index 0000000..d42c52b --- /dev/null +++ b/hosted/opd/opd.py @@ -0,0 +1,30 @@ +"""On-policy distillation (OPD): a fresh Qwen3.6-35B-A3B student learns one domain from one frozen teacher. + +Every update the student answers `--prompts` training problems of the domain, the teacher +scores each sampled token, and the student takes one step towards the teacher on those +tokens (common.py describes the update). Thinking is on. + + python hosted/opd/opd.py --domain caesar_cipher --output /abs/opd-caesar \\ + --teacher tinker://.../sampler_weights/final + +Evaluation covers every recipe domain, so it also shows what happens to the domain not trained on. +""" +import argparse + +from common import Experiment, add_protocol_arguments, check_protocol_arguments, run +from shared import recipe + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument('--domain', choices=recipe.DOMAINS, required=True) + parser.add_argument('--teacher', required=True, help="The teacher's sampler path") + add_protocol_arguments(parser) + args = parser.parse_args() + check_protocol_arguments(parser, args) + run(lambda: Experiment(args, 'opd', {args.domain: args.teacher}), + lambda experiment, update: experiment.train_prompts(args.domain, update, args.prompts)) + + +if __name__ == '__main__': + main() diff --git a/hosted/opd/reference/teacher-check-rollouts.json b/hosted/opd/reference/teacher-check-rollouts.json new file mode 100644 index 0000000..a45ce76 --- /dev/null +++ b/hosted/opd/reference/teacher-check-rollouts.json @@ -0,0 +1 @@ +{"caesar_cipher": [{"prompt_tokens": [248045, 8678, 198, 21365, 264, 3377, 11, 678, 3274, 369, 310, 4087, 279, 3296, 539, 7047, 2923, 13909, 28253, 303, 264, 2708, 321, 3050, 11233, 13, 198, 12162, 488, 599, 3272, 883, 279, 31626, 1817, 11, 3300, 279, 4087, 303, 279, 2614, 3443, 25, 198, 27, 8944, 29, 8944, 1532, 510, 8944, 29, 198, 5236, 524, 10033, 678, 31626, 4613, 279, 4087, 9212, 11, 3300, 1132, 279, 1534, 4087, 13, 3095, 449, 3010, 369, 3766, 11, 488, 1220, 24660, 1732, 279, 3443, 314, 279, 2468, 14, 8944, 303, 421, 3010, 13, 248046, 198, 248045, 846, 198, 86577, 411, 51852, 30070, 1414, 25, 604, 38, 387, 12126, 30715, 53, 7227, 28089, 13481, 42928, 41814, 1977, 13, 38256, 1132, 279, 60839, 1414, 430, 678, 1534, 4087, 13, 248046, 198, 248045, 74455, 198, 248068, 198], "tokens": [760, 1156, 6587, 728, 310, 36865, 264, 51852, 30070, 1414, 25, 328, 41, 38, 387, 12126, 30715, 53, 7227, 28089, 13481, 42928, 41814, 1977, 3158, 271, 16, 13, 220, 2972, 2014, 53983, 279, 1879, 1414, 64700, 328, 41, 38, 387, 12126, 30715, 53, 7227, 28089, 13481, 42928, 41814, 1977, 3158, 198, 17, 13, 220, 2972, 27382, 1386, 279, 24656, 1654, 64700, 1049, 579, 264, 51852, 30070, 11, 864, 27700, 11592, 539, 264, 3529, 3202, 13, 198, 18, 13, 220, 2972, 35, 23544, 279, 6206, 64700, 198, 262, 348, 256, 8919, 506, 279, 1118, 3299, 328, 41, 38, 3158, 1049, 579, 2716, 13, 7474, 1330, 76641, 4105, 513, 328, 1580, 487, 328, 922, 487, 328, 669, 487, 328, 2685, 487, 328, 3024, 487, 328, 691, 3158, 198, 262, 348, 256, 6558, 579, 1380, 30787, 359, 41, 6, 1142, 13, 198, 285, 348, 256, 604, 1411, 353, 318, 13031, 471, 16, 8, 198, 285, 348, 256, 469, 1411, 426, 318, 13031, 471, 16, 8, 1411, 328, 2685, 3158, 1061, 5686, 24578, 13, 198, 262, 348, 256, 6558, 579, 1380, 359, 40, 6, 318, 13031, 471, 17, 553, 198, 285, 348, 256, 604, 1411, 353, 318, 13031, 471, 16, 553, 604, 1411, 462, 318, 13031, 471, 17, 553, 198, 285, 348, 256, 469, 1411, 426, 318, 13031, 471, 16, 553, 469, 1411, 458, 318, 13031, 471, 17, 553, 1411, 328, 1736, 3158, 198, 262, 348, 256, 6558, 579, 1380, 12910, 279, 2018, 3299, 328, 1977, 37, 3158, 198, 285, 348, 256, 1368, 6206, 369, 471, 16, 318, 41, 397, 40, 11, 469, 397, 37, 1590, 198, 309, 348, 256, 387, 1411, 496, 198, 309, 348, 256, 496, 1411, 443, 198, 309, 348, 256, 426, 1411, 458, 198, 309, 348, 256, 328, 5059, 3158, 328, 2685, 23287, 1, 3520, 5357, 13, 198, 285, 348, 256, 1368, 6206, 369, 471, 17, 318, 41, 397, 39, 11, 469, 397, 36, 1590, 198, 309, 348, 256, 387, 1411, 443, 198, 309, 348, 256, 496, 1411, 380, 198, 309, 348, 256, 426, 1411, 414, 198, 309, 348, 256, 328, 45, 5886, 3158, 47198, 914, 1353, 1040, 264, 3299, 13, 198, 262, 348, 256, 6558, 579, 9370, 279, 6206, 369, 471, 16, 13, 198, 285, 348, 256, 9045, 220, 16, 25, 604, 38, 1411, 11219, 198, 285, 348, 256, 9045, 220, 17, 25, 387, 12126, 1411, 23287, 198, 285, 348, 256, 9045, 220, 18, 25, 30715, 53, 7227, 1411, 907, 198, 309, 348, 256, 414, 1411, 351, 198, 309, 348, 256, 387, 1411, 496, 198, 309, 348, 256, 629, 1411, 533, 198, 309, 348, 256, 380, 1411, 436, 198, 309, 348, 256, 458, 1411, 414, 198, 309, 348, 256, 328, 8023, 34011, 3158, 328, 2685, 23287, 7057, 34011, 3158, 1061, 3520, 4574, 5357, 13, 198, 285, 348, 256, 9045, 220, 19, 25, 28089, 13481, 1411, 907, 198, 309, 348, 256, 533, 1411, 345, 198, 309, 348, 256, 629, 1411, 533, 198, 309, 348, 256, 326, 1411, 423, 198, 309, 348, 256, 496, 1411, 443, 198, 309, 348, 256, 328, 15834, 3158, 328, 2685, 23287, 7057, 34011, 79526, 3158, 198, 285, 348, 256, 9045, 220, 20, 25, 42928, 1411, 907, 198, 309, 348, 256, 533, 1411, 345, 198, 309, 348, 256, 353, 1411, 462, 198, 309, 348, 256, 426, 1411, 458, 198, 309, 348, 256, 328, 16721, 3158, 328, 2685, 23287, 7057, 34011, 79526, 3067, 3158, 198, 285, 348, 256, 9045, 220, 21, 25, 41814, 1977, 1411, 907, 198, 309, 348, 256, 443, 1411, 380, 198, 309, 348, 256, 387, 1411, 496, 198, 309, 348, 256, 387, 1411, 496, 198, 309, 348, 256, 496, 1411, 443, 198, 309, 348, 256, 328, 10218, 691, 3158, 328, 2685, 23287, 7057, 34011, 79526, 3067, 11090, 691, 3158, 271, 19, 13, 220, 2972, 60239, 64700, 198, 262, 348, 256, 16722, 25, 604, 38, 387, 12126, 30715, 53, 7227, 28089, 13481, 42928, 41814, 1977, 198, 262, 348, 256, 25662, 25, 471, 16, 318, 21871, 31177, 30070, 440, 6206, 283, 220, 16, 557, 1429, 310, 28658, 11, 748, 567, 31192, 220, 16, 310, 36865, 11, 466, 6206, 27484, 539, 220, 16, 553, 198, 262, 348, 256, 604, 12, 16, 28, 40, 11, 469, 12, 16, 79388, 1411, 11219, 198, 262, 348, 256, 387, 12, 16, 28, 46, 11, 496, 12, 16, 50247, 11, 426, 12, 16, 82295, 1411, 23287, 198, 262, 348, 256, 414, 12, 16, 39566, 11, 387, 12, 16, 28, 46, 11, 629, 12, 16, 28, 52, 11, 380, 12, 16, 66522, 11, 458, 12, 16, 47013, 1411, 7057, 34011, 198, 262, 348, 256, 533, 12, 16, 58423, 11, 629, 12, 16, 28, 52, 11, 326, 12, 16, 70206, 11, 496, 12, 16, 50247, 1411, 79526, 198, 262, 348, 256, 533, 12, 16, 58423, 11, 353, 12, 16, 28, 39, 11, 426, 12, 16, 82295, 1411, 3067, 198, 262, 348, 256, 443, 12, 16, 48010, 11, 387, 12, 16, 28, 46, 11, 387, 12, 16, 28, 46, 11, 496, 12, 16, 50247, 1411, 11090, 691, 271, 20, 13, 220, 2972, 18770, 8984, 3268, 2820, 64700, 198, 262, 348, 256, 561, 60839, 1414, 369, 328, 2685, 23287, 7057, 34011, 79526, 3067, 11090, 691, 3158, 198, 262, 348, 256, 561, 9640, 16561, 364, 348, 3138, 9, 279, 60839, 1414, 430, 279, 1534, 4087, 13, 198, 262, 348, 256, 14606, 25, 361, 8944, 29, 2685, 23287, 7057, 34011, 79526, 3067, 11090, 691, 510, 8944, 29, 271, 21, 13, 220, 2972, 7144, 1716, 16484, 64700, 328, 58387, 1132, 279, 60839, 1414, 430, 678, 1534, 4087, 1149, 3054, 524, 10033, 31626, 4613, 279, 9212, 13, 271, 262, 6558, 579, 1716, 413, 279, 1105, 12482, 13, 561, 1879, 369, 38175, 13, 561, 2468, 1220, 4222, 381, 38175, 466, 4472, 1105, 13, 31874, 11, 364, 1439, 44969, 11, 19720, 279, 1105, 314, 279, 1879, 369, 28488, 7014, 5024, 5752, 11, 466, 32328, 310, 5129, 11316, 70848, 13, 328, 2685, 23287, 7057, 34011, 79526, 3067, 11090, 691, 1, 72346, 279, 1879, 1647, 13, 271, 262, 6558, 579, 7440, 279, 6206, 4944, 13, 198, 262, 1368, 279, 30070, 1414, 369, 1510, 34, 63, 321, 13983, 369, 1510, 32, 7561, 6206, 369, 478, 17, 13, 198, 262, 5514, 11, 61152, 1510, 41, 63, 1411, 41774, 1510, 40, 27653, 1510, 40, 63, 369, 1518, 1510, 41, 27653, 1987, 279, 6206, 1429, 310, 28658, 557, 478, 16, 13, 198, 262, 1946, 36865, 11, 567, 6206, 471, 16, 13, 198, 262, 38643, 13, 271, 262, 5536, 25, 11219, 23287, 7057, 34011, 79526, 3067, 11090, 691, 13, 271, 262, 13428, 11, 369, 328, 41, 38, 1, 1411, 328, 2685, 1, 466, 328, 1580, 42418, 198, 262, 604, 1411, 353, 9987, 16, 681, 469, 1411, 326, 318, 13031, 478, 16, 21, 553, 2233, 11, 6206, 1902, 381, 6570, 13, 198, 262, 604, 397, 40, 11, 469, 397, 37, 13, 7065, 17589, 13, 198, 262, 387, 397, 46, 11, 496, 397, 45, 11, 426, 397, 36, 13, 7065, 17589, 13, 198, 262, 414, 397, 34, 11, 387, 397, 46, 11, 629, 397, 52, 11, 380, 397, 43, 11, 458, 397, 35, 13, 7065, 17589, 13, 198, 262, 533, 397, 51, 11, 629, 397, 52, 11, 326, 397, 49, 11, 496, 397, 45, 13, 7065, 17589, 13, 198, 262, 533, 397, 51, 11, 353, 397, 39, 11, 426, 397, 36, 13, 7065, 17589, 13, 198, 262, 443, 397, 44, 11, 387, 397, 46, 11, 387, 397, 46, 11, 496, 397, 45, 13, 7065, 17589, 13, 271, 262, 561, 16629, 369, 328, 2685, 23287, 7057, 34011, 79526, 3067, 11090, 691, 3158, 271, 262, 12650, 1716, 314, 279, 2483, 3443, 25, 198, 262, 361, 8944, 29, 8944, 1532, 510, 8944, 29, 198, 262, 328, 58387, 1132, 279, 60839, 1414, 430, 678, 1534, 4087, 1149, 271, 262, 1987, 353, 668, 2468, 1066, 279, 1414, 4613, 279, 9212, 13, 271, 262, 3648, 777, 1716, 25, 2091, 1017, 866, 975, 5129, 6206, 30, 198, 262, 56255, 16, 18, 30, 198, 262, 604, 1411, 457, 198, 262, 469, 1411, 345, 198, 262, 328, 17904, 20564, 2233, 13, 198, 262, 25662, 471, 16, 4816, 279, 1132, 19214, 799, 16876, 6163, 4105, 13, 271, 262, 12650, 21134, 7832, 13, 198, 262, 2824, 25, 11219, 23287, 7057, 34011, 79526, 3067, 11090, 691, 198, 262, 26786, 25, 361, 8944, 29, 2685, 23287, 7057, 34011, 79526, 3067, 11090, 691, 510, 8944, 29, 198, 248069, 271, 760, 1879, 1414, 369, 264, 51852, 30070, 1332, 1754, 6321, 682, 978, 28028, 4487, 539, 220, 16, 2234, 303, 279, 26887, 318, 32, 8772, 417, 11, 417, 8772, 351, 11, 4831, 34163, 1946, 36865, 424, 11, 567, 6206, 1754, 6321, 26661, 539, 220, 16, 318, 33, 8772, 357, 11, 351, 8772, 417, 11, 4831, 34163, 271, 9764, 579, 1387, 424, 1441, 25, 198, 16, 13, 220, 2972, 41, 38, 92217, 604, 12, 16, 28, 40, 11, 469, 12, 16, 79388, 56014, 18899, 3, 2972, 2685, 332, 198, 17, 13, 220, 2972, 1977, 37, 92217, 387, 12, 16, 28, 46, 11, 496, 12, 16, 50247, 11, 426, 12, 16, 82295, 56014, 18899, 3, 2972, 5059, 332, 198, 18, 13, 220, 2972, 9992, 53, 7227, 92217, 414, 12, 16, 39566, 11, 387, 12, 16, 28, 46, 11, 629, 12, 16, 28, 52, 11, 380, 12, 16, 66522, 11, 458, 12, 16, 47013, 56014, 18899, 3, 2972, 8023, 34011, 332, 198, 19, 13, 220, 2972, 21556, 13481, 92217, 533, 12, 16, 58423, 11, 629, 12, 16, 28, 52, 11, 326, 12, 16, 70206, 11, 496, 12, 16, 50247, 56014, 18899, 3, 2972, 15834, 332, 198, 20, 13, 220, 2972, 2202, 37, 92217, 533, 12, 16, 58423, 11, 353, 12, 16, 28, 39, 11, 426, 12, 16, 82295, 56014, 18899, 3, 2972, 16721, 332, 198, 21, 13, 220, 2972, 25391, 1977, 92217, 443, 12, 16, 48010, 11, 387, 12, 16, 28, 46, 11, 387, 12, 16, 28, 46, 11, 496, 12, 16, 50247, 56014, 18899, 3, 2972, 10218, 691, 332, 271, 34995, 5562, 1439, 4105, 2961, 303, 279, 16629, 25, 328, 2685, 23287, 7057, 34011, 79526, 3067, 11090, 691, 3158, 271, 27, 8944, 29, 2685, 23287, 7057, 34011, 79526, 3067, 11090, 691, 510, 8944, 29, 248046], "teacher_logprobs": [-2.695134401321411, -0.002069357316941023, -7.176141662057489e-05, -0.9742010831832886, -4.887569048150908e-06, -6.711257447022945e-05, -0.018152771517634392, -0.003302241675555706, -0.00026008085114881396, -0.08339299261569977, -0.054974041879177094, -0.012575967237353325, -6.16293036728166e-05, -1.811964830267243e-05, -7.10462118149735e-05, -1.2636104656849056e-05, -0.0003095386200584471, -3.71926071238704e-05, -6.437280717364047e-06, -4.672895011026412e-05, -1.7046782886609435e-05, -0.0003657863999251276, -4.23184028477408e-05, -0.00019941726350225508, -0.12744084000587463, -0.20143526792526245, -0.010266589000821114, -5.125986263010418e-06, -2.8609820219571702e-05, -1.3589766240329482e-05, -0.023388462141156197, 0.0, -1.4305104514278355e-06, -1.0860035419464111, -2.286139488220215, -0.31406170129776, -1.2305161952972412, -9.059865078597795e-06, -1.07287787614041e-05, -6.437280717364047e-06, -3.576278118089249e-07, -5.9126061387360096e-05, -9.298280929215252e-06, -9.536738616588991e-07, -4.8993817472364753e-05, -2.0265558760002023e-06, -0.0016916978638619184, -4.160317621426657e-05, -3.2186455882765586e-06, -0.3485034704208374, -0.03664553910493851, -0.1602807641029358, -2.9802276912960224e-06, -7.152531907195225e-06, -1.2874520507466514e-05, -0.14102381467819214, -5.960462772236497e-07, -2.7894584491150454e-05, -3.676780939102173, -0.012187534943223, -7.271740287251305e-06, -0.7270968556404114, -0.04673067480325699, -0.4614582061767578, -0.0005085367010906339, -0.003676086664199829, -0.5459561347961426, -0.008704221807420254, -1.489302158355713, -0.3641720116138458, -0.2885647714138031, -0.00016842853801790625, -0.7153467535972595, -1.5137970447540283, -0.053821809589862823, -0.02125847339630127, -3.194758028257638e-05, -8.344646857949556e-07, -4.529942543740617e-06, -1.168244216387393e-05, -0.1679980307817459, -4.076874756719917e-05, -6.103329360485077e-05, -0.012711570598185062, -0.196751669049263, -0.2559025287628174, -5.364403477869928e-06, -0.00015817821258679032, -1.8000440832111053e-05, -0.6553832292556763, -0.08894708752632141, -1.035435676574707, -0.3008849024772644, -0.019874481484293938, -0.25218039751052856, -0.0001938155182870105, -1.2755313036905136e-05, -0.007820463739335537, -1.436332106590271, -0.29618343710899353, -0.5591367483139038, -0.5566388964653015, -1.1658720970153809, -0.7263123989105225, -0.0019194527994841337, -0.05558044835925102, -0.3587319850921631, -0.009876668453216553, -0.722070574760437, -0.000614216027315706, -4.768360213347478e-06, -0.2418171912431717, -0.0015113847330212593, -5.483612312673358e-06, -0.6902490258216858, -0.029760021716356277, -0.00016735584358684719, -0.97257000207901, -0.2812219262123108, -0.03374500945210457, -3.5766167640686035, -0.12706317007541656, -0.18024839460849762, -1.2310724258422852, -1.137073278427124, -0.00042763148667290807, -0.005930565297603607, -9.536738616588991e-07, -8.4638240878121e-06, -0.7123671174049377, -0.00010084597306558862, -0.6255922317504883, -0.14150159060955048, -2.7870261669158936, -0.0030976191628724337, -0.048786260187625885, -0.755169153213501, -1.027414083480835, -0.01714995875954628, -0.2519434094429016, -1.9073468138230965e-06, -9.536697689327411e-06, -0.19585983455181122, -0.1121201142668724, -0.033773936331272125, -0.5179650783538818, -0.06477301567792892, -0.06575756520032883, -4.911301948595792e-05, -0.13265152275562286, -0.5664265751838684, -0.009890125133097172, -3.2186455882765586e-06, -4.172316494077677e-06, -0.987546443939209, -0.0002338612830499187, -0.021446462720632553, -0.15825879573822021, -0.00044050050200894475, -1.811964830267243e-05, -0.0004637358069885522, -0.0006529344827868044, -2.4679741859436035, -0.003841403406113386, -0.0008222059695981443, -0.2495078295469284, -0.816143810749054, -0.7785109281539917, -0.21628960967063904, -0.0005704921204596758, -0.011386999860405922, -0.012528995983302593, -4.172316494077677e-06, -7.152555099310121e-07, -0.000436925794929266, -6.6756979322235566e-06, -2.9296844005584717, -4.926053047180176, -2.521641254425049, -0.2652384340763092, -0.588250458240509, -0.029150601476430893, -0.017848940566182137, -0.06568589806556702, -0.32089629769325256, -0.017549939453601837, -0.0031829429790377617, -1.168244216387393e-05, -1.1920922133867862e-06, -0.0052674319595098495, -0.0007121411035768688, -1.9126296043395996, -1.3465666770935059, -0.6246362328529358, -0.009357050061225891, -0.1055857390165329, -3.2247297763824463, -3.4649295806884766, -0.2176172137260437, -1.0470874309539795, -0.08188904076814651, -0.0006654431927017868, -0.00012361239350866526, -0.0010794533882290125, -0.07418074458837509, -0.25612160563468933, -0.08897577226161957, -3.397406908334233e-05, -2.7418097943154862e-06, -0.017418036237359047, -0.00011300401820335537, -1.6053560972213745, -0.12106010317802429, -7.688703772146255e-05, -1.3351351299206726e-05, -0.0006211258587427437, -0.006345481611788273, -0.0009370227344334126, -1.8954096958623268e-05, -0.003363786730915308, -8.583032467868179e-06, -1.7523612768854946e-05, -9.953480184776708e-05, -0.00010263393050990999, -0.17976905405521393, -0.12161222100257874, -2.90866428258596e-05, -0.005972633603960276, -0.01737937517464161, -0.5600182414054871, -0.034600287675857544, -5.900685573578812e-05, -2.0265558760002023e-06, -0.00676934327930212, -1.156323378381785e-05, -0.4884929656982422, -4.808074951171875, -0.06110594421625137, -0.6351664066314697, -0.0012275787303224206, -0.002448182087391615, -5.221230458118953e-05, -1.0371154530730564e-05, -0.7576839327812195, -0.011557528749108315, -0.006717480253428221, -7.390948667307384e-06, -9.65590606938349e-06, -0.03694462776184082, -0.18425072729587555, -0.0314309224486351, -0.018282704055309296, -0.028092235326766968, -0.8036921620368958, -0.36372533440589905, -0.2710655927658081, -0.0024889700580388308, -0.1917843222618103, -0.0035053968895226717, -4.2676016164477915e-05, -0.0016104121459648013, -0.31521058082580566, -2.802450656890869, -0.0006417360855266452, -0.0019378946162760258, -4.291525328881107e-06, -0.0012694880133494735, -0.006549796089529991, -0.00019524575327523053, -0.012013227678835392, -3.2186455882765586e-06, -8.344646857949556e-07, -5.960462772236497e-07, -0.00048375347978435457, -2.9802276912960224e-06, -6.329813186312094e-05, -0.0001851148990681395, -1.0728830375228426e-06, -2.50339189733495e-06, -1.1920928244535389e-07, -6.675497570540756e-05, -4.0531076592742465e-06, -0.00010251473577227443, -0.029893429949879646, -5.209310256759636e-05, -2.3841574147809297e-05, -1.5258672647178173e-05, -0.8701550364494324, -0.01061625499278307, -0.008189194835722446, -0.9489398002624512, -0.004320216365158558, -0.00018094333063345402, -2.033046007156372, -0.3761446177959442, -0.024207493290305138, -0.05371945723891258, -0.001053255284205079, -0.2835819125175476, -5.364403477869928e-06, -6.9141146923357155e-06, -0.20151673257350922, -6.12716976320371e-05, -6.592056161025539e-05, -0.008135514333844185, -0.022261371836066246, -0.011396311223506927, -0.0037859445437788963, -1.3589766240329482e-05, -0.04332895949482918, -0.0025634069461375475, -8.201262971851975e-05, -2.3245540432981215e-05, -0.02982921525835991, -0.001023007556796074, -2.3841574147809297e-05, -7.152531907195225e-06, -1.4305104514278355e-06, -1.0728830375228426e-06, -2.2172682292875834e-05, -1.0371154530730564e-05, -0.0009585791267454624, -0.00034254882484674454, -8.344646857949556e-07, -1.7881377516459906e-06, -4.9470632802695036e-05, -9.059865078597795e-06, -1.1920928244535389e-07, -0.0007722970913164318, -0.00013886917440686375, -1.6689286894688848e-06, -2.0265558760002023e-06, -1.0728830375228426e-06, -2.6464111215318553e-05, -2.3841855067985307e-07, -0.0020300988107919693, -0.008713675662875175, -4.768360213347478e-06, -3.933898824470816e-06, -2.884823152271565e-05, -2.706014311115723e-05, -0.18803395330905914, -0.011607137508690357, -0.0019620470702648163, -0.5737950801849365, -1.5497195136049413e-06, -1.055634617805481, -0.02834777720272541, -0.5181466341018677, -0.08049831539392471, -0.0004481264913920313, -0.04312460124492645, -0.056291285902261734, -3.576278118089249e-07, -7.152531907195225e-06, -0.153799906373024, -2.0265373677830212e-05, -1.0633331537246704, -0.07699552178382874, -0.0016513533191755414, -0.000894027529284358, -0.07653337717056274, -0.006248936057090759, -4.532138824462891, -0.6627835035324097, -3.0603511333465576, -0.00012373158824630082, -8.284702198579907e-05, -1.9556081295013428, -0.010148354806005955, -0.14325755834579468, -9.250213042832911e-05, -0.22593744099140167, -0.033287692815065384, -0.00021801002731081098, -0.0013330630026757717, -0.0222991444170475, -0.0006310140597634017, -9.536738616588991e-07, -8.344646857949556e-07, -5.829164365422912e-05, -1.0728830375228426e-06, -2.6702524337451905e-05, -5.245195097813848e-06, -1.9788545614574105e-05, -1.2755313036905136e-05, -0.0001408954558428377, -0.00016926287207752466, -0.00644521601498127, -0.030278747901320457, -1.1920922133867862e-06, -1.6689286894688848e-06, -0.0008485292200930417, -3.576278118089249e-07, -3.099436753473128e-06, -1.597391747054644e-05, -0.00030524839530698955, -3.6954811548639555e-06, -5.364403477869928e-06, -1.2159587144851685, -2.631364107131958, -0.0004349001101218164, -0.0003404037852305919, -7.676783570786938e-05, -2.622600959512056e-06, -0.00018010901112575084, -0.004188218619674444, -0.0006043276516720653, -0.0061314101330935955, -8.344646857949556e-07, -1.6689286894688848e-06, -8.344646857949556e-07, -4.8636207793606445e-05, -1.1086402082582936e-05, -5.864924969500862e-05, -0.00040642108069732785, -5.960462772236497e-07, -3.576278118089249e-07, -2.3841855067985307e-07, -5.411955135059543e-05, -8.106198947643861e-06, -0.00022289653134066612, -0.0014335366431623697, -4.172316494077677e-06, -5.8412379075889476e-06, -1.1920928244535389e-07, -9.7508447652217e-05, -6.794906312279636e-06, -0.00012063252506777644, -0.0004640932602342218, -4.768370445162873e-07, -2.1815061700181104e-05, -4.768370445162873e-07, -1.9550132492440753e-05, -3.099436753473128e-06, -0.00010907054820563644, -0.003020369913429022, -3.2186455882765586e-06, -7.271740287251305e-06, -6.556489552167477e-06, -0.01888977363705635, -0.04964476823806763, -0.0010347970528528094, -0.23546282947063446, -1.07916259765625, -0.005647655110806227, -0.00032872517476789653, -0.00021860594279132783, -7.986703712958843e-05, -1.1434831619262695, -0.852293848991394, -0.827614426612854, -0.24193839728832245, -0.052355267107486725, -0.017410889267921448, -0.003656132845208049, -0.1298661231994629, 0.0, -5.960462772236497e-07, -0.0006412595394067466, 0.0, -6.139089964563027e-05, -5.960462772236497e-07, -2.5987286790041253e-05, -2.9802276912960224e-06, -9.893881360767409e-05, -0.00012385078298393637, -6.794906312279636e-06, -2.264974000354414e-06, -1.9311717551317997e-05, -2.3841855067985307e-07, -1.0132738680113107e-05, -5.602820692729438e-06, -6.878139538457617e-05, -0.0002090712368953973, -1.311301275563892e-06, -4.768370445162873e-07, -1.5497195136049413e-06, -2.539125671319198e-05, -2.145764938177308e-06, -2.074220174108632e-05, -0.00012373158824630082, -1.311301275563892e-06, -1.5497195136049413e-06, -9.536738616588991e-07, -2.3841574147809297e-05, -7.152555099310121e-07, -2.8967437174287625e-05, -0.0001072826053132303, -9.536738616588991e-07, -3.6954811548639555e-06, -1.1920922133867862e-06, -8.11782301752828e-05, -4.768370445162873e-07, -7.784063927829266e-05, -0.00014137222024146467, -2.145764938177308e-06, -3.6954811548639555e-06, -7.152531907195225e-06, -0.00025233422638848424, -0.00752128753811121, -0.0017910643946379423, -0.12854602932929993, -0.0022244728170335293, -0.0010101459920406342, -0.00013183678674977273, -2.622600959512056e-06, -6.83045873302035e-05, -0.0015126940561458468, -0.047768484801054, -0.001312348060309887, -1.1920928244535389e-07, -2.50339189733495e-06, -0.0002946419408544898, -3.576278118089249e-07, -2.4914430468925275e-05, -1.2159273865108844e-05, -0.001989886863157153, -3.6954811548639555e-06, -0.0001072826053132303, -4.8636207793606445e-05, -3.933898824470816e-06, -5.960462772236497e-07, -4.768370445162873e-07, -0.0010531361913308501, -4.529942543740617e-06, -1.728519782773219e-05, -0.0001370812824461609, -5.960462772236497e-07, -4.768370445162873e-07, -2.3841855067985307e-07, -0.0022891764529049397, -1.3708974620385561e-05, -0.00013720047718379647, -0.002303924411535263, -9.536738616588991e-07, -7.152555099310121e-07, -2.264974000354414e-06, -7.033323527139146e-06, -2.264974000354414e-06, -2.4318398573086597e-05, -0.0006157647585496306, -3.099436753473128e-06, -1.1920928244535389e-07, -2.50339189733495e-06, -0.00026353701832704246, -3.218599158572033e-05, -0.00015078838623594493, -0.16403570771217346, -0.007863635197281837, -6.353653589030728e-05, -4.0531076592742465e-06, -9.298280929215252e-06, -0.00012587709352374077, -1.4305012882687151e-05, -0.008711548522114754, -0.00044383687782101333, -0.001745130866765976, 0.0, -2.145764938177308e-06, -0.00010358751023886725, -2.3841855067985307e-07, -7.152531907195225e-06, -3.576272320060525e-06, -4.4225667807040736e-05, -1.0609570381348021e-05, -9.536738616588991e-07, -7.319182623177767e-05, -1.7165990357170813e-05, -2.50339189733495e-06, -2.622600959512056e-06, -8.344646857949556e-07, -1.7881233361549675e-05, -3.3378546504536644e-06, -0.00011872540198964998, -6.794698856538162e-05, -1.4305104514278355e-06, -1.07287787614041e-05, -2.3841855067985307e-07, -3.814689989667386e-06, -1.7881377516459906e-06, -8.22540732769994e-06, -3.242440288886428e-05, -2.0265558760002023e-06, -6.6756979322235566e-06, -1.6689286894688848e-06, -0.0001714082609396428, -2.145764938177308e-06, -2.407998726994265e-05, -2.312633478140924e-05, -1.6689286894688848e-06, -3.3378546504536644e-06, -2.264974000354414e-06, -9.011816291604191e-05, -2.3841855067985307e-07, -2.038458114839159e-05, -9.42901024245657e-05, -2.0265558760002023e-06, -7.152555099310121e-07, -2.5629668016335927e-05, -9.643566590966657e-05, -0.0009199915803037584, -0.00010358751023886725, -0.006445571314543486, -0.013534686528146267, -0.0007857572636567056, -2.8490614567999728e-05, -9.059865078597795e-06, -1.1920922133867862e-06, -2.610649426060263e-05, -0.00011991735664196312, -1.811964830267243e-05, -2.622600959512056e-06, -0.0013116337358951569, -0.15855060517787933, -0.026457538828253746, -7.152555099310121e-07, -4.5536911784438416e-05, -1.9311717551317997e-05, -0.8544261455535889, -0.0007557396893389523, -0.012788075022399426, -7.152531907195225e-06, -0.006861694622784853, -7.83174327807501e-05, -1.4261579513549805, -0.0017183552263304591, -0.004169462248682976, -0.34913307428359985, -0.004038631450384855, -4.172316494077677e-06, -2.0146166207268834e-05, -1.6689286894688848e-06, -3.576278118089249e-07, -5.8412379075889476e-06, -4.291525328881107e-06, -0.00018142008048016578, -1.4781842764932662e-05, -1.9073468138230965e-06, -5.3881147323409095e-05, -4.768370445162873e-07, -1.847726889536716e-05, -3.564294092939235e-05, -0.126693457365036, -0.1961856484413147, -0.03125970438122749, -3.528532761265524e-05, -0.0299428328871727, -3.8461289405822754, -0.0005289109540171921, -1.3959908485412598, -0.24556998908519745, -1.126644492149353, -6.048418998718262, -0.4778292179107666, -0.5241779088973999, -8.606789588928223, -0.06680827587842941, -0.060873329639434814, -0.017429867759346962, -0.08396268635988235, -0.06896335631608963, -0.6760100722312927, -2.025620937347412, -2.407998726994265e-05, -5.8412379075889476e-06, -0.08128948509693146, -7.70062324590981e-05, -2.416977882385254, -0.0022452878765761852, -0.42096444964408875, -3.5284364223480225, -0.06369242817163467, -0.000842692912556231, -0.0015813715290278196, -0.012813614681363106, -0.020080024376511574, -0.0010370597010478377, -3.576278118089249e-07, -2.8609820219571702e-05, -0.0966465100646019, -2.012660026550293, -2.4676019165781327e-05, -0.638085126876831, -0.0001805857609724626, -0.033662255853414536, -0.00014935807848814875, -5.364403477869928e-06, -4.0531076592742465e-06, -2.7656173188006505e-05, -0.21067997813224792, -0.00048411093303002417, -0.00017450717859901488, -2.622600959512056e-06, -7.152555099310121e-07, -1.5497195136049413e-06, -1.2278481335670222e-05, -2.50339189733495e-06, -1.9073468138230965e-06, -2.0265373677830212e-05, -2.264974000354414e-06, -8.344646857949556e-07, -7.748573807475623e-06, -2.0265558760002023e-06, -2.47952248173533e-05, -1.0609570381348021e-05, -9.536738616588991e-07, -6.556489552167477e-06, -4.172316494077677e-06, -3.2186455882765586e-06, -3.933898824470816e-06, -1.5497195136049413e-06, -1.3947389561508317e-05, -4.458328112377785e-05, -4.291525328881107e-06, -2.3841855067985307e-07, -2.622600959512056e-06, -7.629365427419543e-06, -3.576272320060525e-06, -5.960462772236497e-07, -6.556489552167477e-06, -1.311301275563892e-06, -8.821448318485636e-06, -7.390948667307384e-06, -1.0847986231965479e-05, -1.9311717551317997e-05, -2.264974000354414e-06, -1.311301275563892e-06, -1.6689286894688848e-06, -9.059865078597795e-06, -3.099436753473128e-06, -8.904537389753386e-05, -0.0003364712174516171, -4.410734163684538e-06, -3.85038583772257e-05, -8.702239938429557e-06, -1.3828182090946939e-05, -4.0531076592742465e-06, -1.7881377516459906e-06, -1.3828182090946939e-05, -4.291525328881107e-06, -1.0728830375228426e-06, -4.5298504119273275e-05, -5.960446742392378e-06, -8.594620157964528e-05, -7.152531907195225e-06, -0.0002115741081070155, -7.152555099310121e-07, -3.576278118089249e-07, -1.311301275563892e-06, -3.182837463100441e-05, -3.4570634852570947e-06, -1.6689286894688848e-06, -3.6954811548639555e-06, -1.7881377516459906e-06, -4.172316494077677e-06, -1.8000440832111053e-05, -0.00011181206355104223, -4.410646579344757e-05, -3.2186455882765586e-06, -2.7418097943154862e-06, -3.123234637314454e-05, -1.680836794548668e-05, -1.0490362910786644e-05, -2.7656173188006505e-05, -5.483612312673358e-06, -4.935142715112306e-05, -8.821448318485636e-06, -2.50339189733495e-06, -1.811964830267243e-05, -1.847726889536716e-05, -5.4834770708112046e-05, -9.154854342341423e-05, -8.22540732769994e-06, -1.4305104514278355e-06, -1.0251946150674485e-05, -0.00020108585886191577, -1.6689286894688848e-06, -1.7881377516459906e-06, -3.2186455882765586e-06, -1.1920922133867862e-06, -3.6954811548639555e-06, -1.8715683836489916e-05, -1.1324817933200393e-05, -2.5033637939486653e-05, -3.576272320060525e-06, -1.9073468138230965e-06, -2.002696055569686e-05, -1.6331539882230572e-05, -6.318072337307967e-06, -4.887569048150908e-06, -1.4185804502631072e-05, -1.2040065485052764e-05, -0.00012313561455812305, -1.311301275563892e-06, -1.1920922133867862e-06, -1.5497195136049413e-06, -1.9192511899746023e-05, -5.006777428206988e-06, -2.264974000354414e-06, -9.179073458653875e-06, -2.9802276912960224e-06, -4.410734163684538e-06, -3.504691630951129e-05, -1.0967194612021558e-05, -2.372236667724792e-05, -2.0265558760002023e-06, -3.4570634852570947e-06, -0.0003405229654163122, -7.080780778778717e-05, -1.9311717551317997e-05, -8.702239938429557e-06, -9.536738616588991e-07, -4.172316494077677e-06, -4.076874756719917e-05, -1.3708974620385561e-05, -5.722029527532868e-06, -0.0002329078270122409, -7.271740287251305e-06, -8.940656698541716e-06, -7.152555099310121e-07, -0.22588342428207397, -0.2812795042991638, -3.576278118089249e-07, -2.038458114839159e-05, -3.6954811548639555e-06, -0.23164422810077667, -1.507411003112793, -0.26476699113845825, -3.099393507000059e-05, -1.4305104514278355e-06, -0.14318916201591492, -2.3483953555114567e-05, -0.31335699558258057, -3.1709168979432434e-05, -0.09206124395132065, -0.132893905043602, -0.019036220386624336, -1.5735502529423684e-05, -0.0028084381483495235, -0.0001255195093108341, -4.279521817807108e-05, -1.1801649634435307e-05, -1.5497195136049413e-06, -2.4914430468925275e-05, -3.8742269680369645e-05, -3.4570634852570947e-06, -1.6689286894688848e-06, -3.909988299710676e-05, -0.22549834847450256, -3.5523738915799186e-05, -4.6491513785440475e-06, -4.160317621426657e-05, -0.0602082833647728, -1.4183986186981201, -0.0018587708473205566, -0.4290409982204437, -0.23600170016288757, -1.1920922133867862e-06, -4.291525328881107e-06, -4.768370445162873e-07, -0.00010156115604331717, -7.152555099310121e-07, -0.3799026310443878, -1.7523612768854946e-05, -0.00011586471373448148, -9.536738616588991e-07, -0.38870924711227417, -1.7023001909255981, -0.00017534149810671806, -5.8412379075889476e-06, -0.00018749863374978304, -0.22039252519607544, -0.02069522812962532, -0.3630238175392151, -2.753696753643453e-05, -0.0047140889801084995, -0.12288560718297958, -2.109982233378105e-05, -1.1920858014491387e-05, -2.622600959512056e-06, -1.168244216387393e-05, -3.2186455882765586e-06, -6.556489552167477e-06, -1.1086402082582936e-05, -3.8265450712060556e-05, -4.4225667807040736e-05, -0.005951303523033857, -0.00568297877907753, -0.6783034205436707, -3.933898824470816e-06, -0.000219321038457565, -0.005708107724785805, -0.050240352749824524, -0.20174621045589447, -0.9216251373291016, -0.0002208704245276749, -1.313612937927246, -0.21308749914169312, -7.319182623177767e-05, -6.556489552167477e-06, -1.3589766240329482e-05, -1.1920922133867862e-06, -0.0001472126314183697, -4.3748852476710454e-05, -1.0728830375228426e-06, -4.0531076592742465e-06, -0.0074920631013810635, -5.5272674560546875, -0.00042083943844772875, -0.061139024794101715, -0.272755891084671, -0.17360909283161163, -0.6109206080436707, -1.415257215499878, -0.004651084542274475, -0.05006643384695053, -0.04251744970679283, -1.4574202299118042, -0.0022081772331148386, -0.9470878839492798, -0.4755776524543762, -2.1873528957366943, -7.14790153503418, -0.2575872242450714, -0.0005722792120650411, -0.033556319773197174, -0.015044203959405422, -0.007610493805259466, -0.10322421044111252, -0.07195302844047546, -0.6923884749412537, -0.03902316093444824, -0.0518559068441391, -0.8489068746566772, -0.06230078637599945, -0.004400807432830334, -0.589992344379425, -5.951225280761719, -1.1137492656707764, -0.014644159935414791, -0.11669281870126724, -0.000993711524643004, -3.857012987136841, -0.7197709679603577, -0.5208179950714111, -4.994744449504651e-05, -2.3668062686920166, -0.17099449038505554, -0.27044805884361267, -1.3197367191314697, -0.0006700892699882388, -0.4571157693862915, -0.8556172847747803, -2.4303808212280273, -0.725773811340332, -0.1550305336713791, -0.029005281627178192, -0.575984001159668, -0.572054922580719, -0.8985821008682251, -0.00010847456724150106, -0.23551370203495026, -0.06510905176401138, -1.9110831022262573, -0.0054673426784574986, -2.296812057495117, -0.1144736111164093, -2.753696753643453e-05, -5.006664650863968e-05, -5.364403477869928e-06, -2.7894584491150454e-05, -2.9802276912960224e-06, -8.344646857949556e-07, -5.483612312673358e-06, -0.0013433012645691633, -2.013146162033081, -0.003018111689016223, -0.396295428276062, -0.3088489770889282, -0.023835543543100357, -0.3668735921382904, -4.434487345861271e-05, -0.18399359285831451, -0.0005709686665795743, -4.991566181182861, -0.0016004146309569478, -0.3075283467769623, -1.1009451150894165, -0.10277769714593887, -0.010948230512440205, -3.576214658096433e-05, -2.1984927654266357, -2.0488040447235107, -1.4911532402038574, -0.7749158143997192, -0.011980600655078888, -3.6409435272216797, -3.1459076404571533, -0.6754988431930542, -0.1500183641910553, -1.1125890016555786, -0.20286934077739716, -0.0001754606782924384, -0.21662065386772156, -0.010489096865057945, -0.1477874517440796, -0.01379703264683485, -0.07046941667795181, -0.0007993363542482257, -0.14908146858215332, -0.241497203707695, -3.635817120084539e-05, -0.43913397192955017, -1.3251533508300781, -0.8440845608711243, -0.03702768683433533, -0.3021217882633209, -0.014503883197903633, -0.1966504156589508, -6.0437283536884934e-05, -2.9802276912960224e-06, -0.0002623452164698392, -0.018398577347397804, -1.247617483139038, -0.8339338302612305, -0.001988935051485896, -0.2838938236236572, -0.585521936416626, -1.1920928244535389e-07, -4.172316494077677e-06, -0.12276352196931839, -0.04846365749835968, -1.758744239807129, -0.7949874401092529, -2.208528518676758, -0.5436936616897583, -0.35322141647338867, -0.003218353958800435, -0.19037465751171112, -0.0006411403883248568, -0.8386712074279785, -1.6908926963806152, -0.00011359999916749075, -0.7181062698364258, -0.00027223696815781295, -0.0016307639889419079, -0.02617816999554634, -0.1166810393333435, -0.06714919954538345, -1.7881377516459906e-06, -0.02978142723441124, -0.22629818320274353, -0.009843735955655575, -3.467127561569214, -1.5735502529423684e-05, -1.6689286894688848e-06, -1.6093124941107817e-05, -1.5447337627410889, -0.0015488782664760947, -0.018293472006917, -7.402622577501461e-05, -2.0265558760002023e-06, -1.4305104514278355e-06, -1.2159273865108844e-05, -2.622600959512056e-06, -7.152555099310121e-07, -1.1920928244535389e-07, -0.039445314556360245, -0.033652111887931824, -0.001157448161393404, -0.6197654604911804, -1.156323378381785e-05, -0.47492536902427673, -3.3166708946228027, -0.8608038425445557, -0.002651273040100932, -0.5767338871955872, -0.8023296594619751, -0.00016652150952722877, -0.3039718568325043, -0.3874300718307495, -1.1272581815719604, -0.023831702768802643, -1.522475242614746, -0.07002715021371841, -0.0003810394846368581, -1.549708758830093e-05, -0.29227304458618164, -0.4484717547893524, -0.01655637100338936, -1.0950260162353516, -2.074220174108632e-05, -1.6169121265411377, -0.23704761266708374, -0.0003407612966839224, -3.8220086097717285, -2.2159852981567383, -1.098550796508789, -0.17407116293907166, -0.05103418231010437, -1.7371137142181396, -0.3399946987628937, -0.04543015733361244, -0.6963681578636169, -2.9851274490356445, -0.16652706265449524, -3.504691630951129e-05, -0.13291926681995392, -0.039785269647836685, -0.0001323135511483997, -1.5258672647178173e-05, -0.068708635866642, -5.271708011627197, -0.14804428815841675, -0.44924452900886536, -0.00015758226800244302, -4.2676016164477915e-05, -0.02705177664756775, -0.8212394118309021, -4.080041885375977, -0.0014134189113974571, -0.12131618708372116, -0.8266716599464417, -0.03372852876782417, -1.1532913446426392, -0.009759804233908653, -0.0008864049450494349, -0.023482901975512505, -0.0003133521240670234, -1.7881377516459906e-06, -3.3854863431770355e-05, -0.0010452766437083483, -6.19869097135961e-05, -2.3841830625315197e-06, -1.0967194612021558e-05, -0.00026544384309090674, -0.0016230279579758644, -4.6491513785440475e-06, -0.016914160922169685, -0.07893821597099304, -0.005279408767819405, -0.034353479743003845, -1.4066597032069694e-05, -6.115249561844394e-05, -0.0035784514620900154, -0.00012218205665703863, -5.960462772236497e-07, -7.390948667307384e-06, -0.0021510813385248184, -5.602820692729438e-06, -4.291525328881107e-06, -0.00013469743134919554, -0.001209838199429214, -1.883488948806189e-05, -1.9073468138230965e-06, -3.397406908334233e-05, -4.0411134250462055e-05, -2.3603161025675945e-05, -8.4638240878121e-06, -0.00011252723925281316, -0.0004378790326882154, -0.008186238817870617, -8.344646857949556e-07, -0.0034985069651156664, -0.026571886613965034, -0.00027378625236451626, -0.0015855372184887528, -1.9073468138230965e-06, -7.152531907195225e-06, -3.85038583772257e-05, -4.088794958079234e-05, -2.7418097943154862e-06, -1.883488948806189e-05, -7.331102824537084e-05, -2.3603161025675945e-05, -9.536697689327411e-06, -7.986990567587782e-06, -8.797258487902582e-05, -3.540453326422721e-05, -5.8412379075889476e-06, -6.318072337307967e-06, -2.1219027985353023e-05, -5.4238757002167404e-05, -1.6689286894688848e-06, -2.396077979938127e-05, -0.00010394509445177391, -3.0874729418428615e-05, -0.008488408289849758, -2.9802276912960224e-06, -2.145764938177308e-06, -2.6940935640595853e-05, -1.1801649634435307e-05, -1.585470999998506e-05, -5.960446742392378e-06, -3.2066785934148356e-05, -2.4437606043647975e-05, -1.1801649634435307e-05, -2.1815061700181104e-05, -3.611976353568025e-05, -0.0001431601122021675, -2.3841855067985307e-07, -2.2411095415009186e-05, -0.0001255195093108341, -4.172316494077677e-06, -2.7417760065873154e-05, -1.1920922133867862e-06, -3.576272320060525e-06, -1.2516897186287679e-05, -3.3378546504536644e-06, -4.529942543740617e-06, -2.3841830625315197e-06, -5.8530047681415454e-05, -3.0636318115284666e-05, -1.597391747054644e-05, -1.1920928244535389e-07, -3.576214658096433e-05, -1.728519782773219e-05, -1.6689286894688848e-06, -7.510157047363464e-06, -1.5139465176616795e-05, -4.660974445869215e-05, -9.536738616588991e-07, -1.168244216387393e-05, -0.0001641377166379243, -5.3165931603871286e-05, -0.29387161135673523, -0.6578349471092224, -0.5248598456382751, -0.5140325427055359, -0.003732145531103015, -1.4543427823809907e-05, -8.22540732769994e-06, -6.318072337307967e-06, -1.3828182090946939e-05, -7.629365427419543e-06, -3.3378546504536644e-06, -1.1920922133867862e-06, -0.00013600854435935616, -0.10013486444950104, -8.356221951544285e-05, -0.49084293842315674, -0.3798672556877136, -0.07841029763221741, -0.002419879427179694, -1.551650881767273, -0.14287972450256348, -0.8259844183921814, -0.130514994263649, -4.482168878894299e-05, -0.031824078410863876, -1.645074735279195e-05, -0.0005839549703523517, -2.257272958755493, -0.00015567521040793508, -1.0728830375228426e-06, -5.495397272170521e-05, -1.5139465176616795e-05, -0.7576150894165039, -0.018519818782806396, -1.3244990110397339, -0.3240419626235962, -4.362964682513848e-05, -4.887569048150908e-06, -1.7046782886609435e-05, -7.152555099310121e-07, -0.005967774894088507, -3.7431014789035544e-05, -1.2278481335670222e-05, -2.622600959512056e-06, -0.0012985378270968795, -0.29731297492980957, -1.490105023549404e-05, -0.8042519688606262, -1.5790367126464844, -0.04408448934555054, -0.34214672446250916, -1.3982346057891846, -0.06271278113126755, -0.8779534697532654, -0.1951160877943039, -3.0397906812140718e-05, -0.00490509532392025, -0.0036634968128055334, -0.07034008949995041, -0.000276765669696033, -0.6495901942253113, -3.240640640258789, -0.01696830987930298, -1.4935123920440674, -0.5744197964668274, -1.0684806108474731, -0.09867598861455917, -0.17101258039474487, -4.232364654541016, -0.42098119854927063, -0.42941489815711975, -0.021857207641005516, -0.00038580605178140104, -0.21390144526958466, -0.14469650387763977, -0.03810510411858559, -0.45545947551727295, -0.02095336839556694, -7.033323527139146e-06, -0.0005726366653107107, -0.1920035481452942, -0.00022301571152638644, -0.0360269621014595, -1.2159273865108844e-05, -0.011394189670681953, -0.0002087136817863211, -0.0003923600015696138, -0.01120217889547348, -2.5152843591058627e-05, -0.5058790445327759, -0.0005143749876879156, -0.5938342809677124, -0.6733996272087097, -0.004155335016548634, -0.474145770072937, -0.06238064542412758, -0.24182094633579254, -0.43956702947616577, -0.17606393992900848, -1.4750289916992188, -0.12424539774656296, -0.42329835891723633, -0.06497020274400711, -0.5945934057235718, -2.292099714279175, -0.16776250302791595, -0.5433495044708252, -0.031870726495981216, -0.008243818767368793, -1.6927575416048057e-05, -0.4159987270832062, -0.15861865878105164, -1.3837871551513672, -0.31330904364585876, -0.002478862414136529, -0.007890719920396805, -0.7873705625534058, -1.311301275563892e-06, -0.00216166814789176, -4.0649541915627196e-05, -2.50339189733495e-06, -7.510157047363464e-06, -1.1086402082582936e-05, -1.311301275563892e-06, -3.576278118089249e-07, -6.794906312279636e-06, -0.20631371438503265, -0.2815049886703491, -0.49939069151878357, -0.006905026268213987, -0.006944567430764437, -0.0001212284987559542, -0.001692292862571776, -0.14975392818450928, -2.169585604860913e-05, -1.4781842764932662e-05, -5.9602869441732764e-05, -1.4662635294371285e-05, -1.9073468138230965e-06, -2.90866428258596e-05, -2.3841830625315197e-06, -5.638440416078083e-05, -0.00014137222024146467, -2.2053474822314456e-05, -0.5761516690254211, -0.5285900235176086, -5.722029527532868e-06, -1.7871897220611572, -1.9484553337097168, -0.021117720752954483, -0.22997450828552246, -1.397193431854248, -0.0024782679975032806, -0.000377583724912256, -1.4878572225570679, -0.00658781174570322, -2.312633478140924e-05, -0.430074542760849, -0.0002479245886206627, -5.221230458118953e-05, -0.026256553828716278, -1.764281842042692e-05, -0.0383785218000412, -3.075552376685664e-05, -0.4089033901691437, -0.3162408471107483, -2.0265558760002023e-06, -0.00013445904187392443, -0.4853963851928711, -1.647128939628601, -0.08331742882728577, -1.3828182090946939e-05, -0.04302788898348808, -0.14950983226299286, -1.5497195136049413e-06, -7.152555099310121e-07, -0.00010144196130568162, -0.00025090406415984035, -0.04173877835273743, -0.002423209138214588, -8.260862523457035e-05, -0.4349285066127777, -5.722029527532868e-06, -0.02071741409599781, -1.3315696716308594, -0.01472639013081789, -2.9682672902708873e-05, -0.0434451624751091, -7.152555099310121e-07, -5.1973900554003194e-05, -2.3841855067985307e-07, -5.01893949508667, -0.5691508650779724, -0.0005820487276650965, -6.318072337307967e-06, -0.012523110024631023, -0.05485657602548599, -1.4066597032069694e-05, -5.125986263010418e-06, -0.5762065052986145, -0.0024782679975032806, -0.0006690170848742127, -5.304672595229931e-05, -4.315191268920898, -6.580135959666222e-05, -1.2026498317718506, -1.2235239744186401, -0.00014041867689229548, -3.291382074356079, -0.01815195195376873, -1.0370776653289795, -4.291525328881107e-06, -0.01866491511464119, -0.007239537313580513, -0.0003277718205936253, -0.029869018122553825, -0.313355416059494, -0.3408677577972412, -2.4516360759735107, -0.00010525626566959545, -1.178320050239563, -0.4308532774448395, -1.4305104514278355e-06, -3.099436753473128e-06, -2.8490614567999728e-05, -7.152555099310121e-07, -0.00010168035078095272, -0.25247958302497864, -0.00016866691294126213, -2.264974000354414e-06, -0.010173489339649677, -1.7881377516459906e-06, -3.576278118089249e-07, -2.5033637939486653e-05, -1.2040065485052764e-05, -2.3841855067985307e-07, -3.576272320060525e-06, -7.152555099310121e-07, -4.172316494077677e-06, -1.0728830375228426e-06, -8.344646857949556e-07, -5.936446541454643e-05, -4.6491513785440475e-06, -1.1920928244535389e-07, -1.5139465176616795e-05, -9.536738616588991e-07, -1.1920928244535389e-07, -1.7881377516459906e-06, -2.264974000354414e-06, -3.576272320060525e-06, -3.3378546504536644e-06, -3.576278118089249e-07, -6.9141146923357155e-06, -3.933898824470816e-06, -2.3841855067985307e-07, -3.4570634852570947e-06, -1.1920922133867862e-06, -2.3841855067985307e-07, -2.3841855067985307e-07, -2.50339189733495e-06, -2.0265558760002023e-06, -2.3841855067985307e-07, -5.245071224635467e-05, -6.198863957251888e-06, -1.1920922133867862e-06, -4.410734163684538e-06, -1.5497195136049413e-06, -1.3708974620385561e-05, -5.960462772236497e-07, -4.768370445162873e-07, -1.5497195136049413e-06, -0.0010487301042303443, -8.22540732769994e-06, 0.0, -5.602820692729438e-06, -4.768370445162873e-07, -6.556489552167477e-06, -5.722029527532868e-06, -2.861018856492592e-06, -4.6491513785440475e-06, -7.152555099310121e-07, -2.3841855067985307e-07, -1.4305104514278355e-06, -6.9141146923357155e-06, -2.7418097943154862e-06, -1.0609570381348021e-05, -4.0649541915627196e-05, -2.7418097943154862e-06, -4.291525328881107e-06, -2.861018856492592e-06, -1.7881377516459906e-06, -4.529942543740617e-06, -1.1920928244535389e-07, -2.264974000354414e-06, -2.861018856492592e-06, -2.3841855067985307e-07, -8.940656698541716e-06, -2.0265558760002023e-06, -1.1920928244535389e-07, -2.3841855067985307e-07, -3.576278118089249e-07, -2.253030106658116e-05, -1.5497195136049413e-06, -1.1920928244535389e-07, -4.100715523236431e-05, -1.1801649634435307e-05, -3.6954811548639555e-06, -5.483612312673358e-06, -2.622600959512056e-06, -1.6689160474925302e-05, -8.344646857949556e-07, -1.311301275563892e-06, -1.3470558769768104e-05, -2.622600959512056e-06, -3.576278118089249e-07, -2.3841830625315197e-06, -5.960462772236497e-07, -4.529942543740617e-06, -7.033323527139146e-06, -1.7881377516459906e-06, -6.9141146923357155e-06, -1.3112935448589269e-05, -9.536738616588991e-07, -3.814689989667386e-06, -2.622600959512056e-06, -3.576272320060525e-06, -1.1920922133867862e-06, -2.7418097943154862e-06, -1.2874520507466514e-05, -2.0265558760002023e-06, -1.0728830375228426e-06, -8.22540732769994e-06, -1.311301275563892e-06, -3.576278118089249e-07, 0.0, -9.536738616588991e-07, -3.933898824470816e-06, -5.960462772236497e-07, -0.0001110968878492713, -3.814689989667386e-06, -1.5497195136049413e-06, -2.622600959512056e-06, -8.344646857949556e-07, -3.0397906812140718e-05, -1.6689286894688848e-06, -1.4305104514278355e-06, -1.680836794548668e-05, -7.152531907195225e-06, -8.344646857949556e-07, -1.7881377516459906e-06, -5.960462772236497e-07, -1.4305104514278355e-06, -1.1801649634435307e-05, -2.9802276912960224e-06, -1.0728830375228426e-06, -2.7418097943154862e-06, -1.1920922133867862e-06, -8.702239938429557e-06, -6.079655122448457e-06, -1.0728830375228426e-06, -6.079655122448457e-06, -1.5497195136049413e-06, -8.344646857949556e-07, -3.576272320060525e-06, -5.960462772236497e-07, -1.1920922133867862e-06, -2.264974000354414e-06, -9.881961887003854e-05, -4.410734163684538e-06, -3.6954811548639555e-06, -5.245195097813848e-06, -2.9802276912960224e-06, -0.0008088654140010476, -2.0265373677830212e-05, -8.344646857949556e-07, -4.0531076592742465e-06, -4.529942543740617e-06, -1.1920922133867862e-06, -5.483612312673358e-06, -7.152555099310121e-07, -1.9073468138230965e-06, -4.0531076592742465e-06, -1.4305104514278355e-06, -2.3841830625315197e-06, -2.3841855067985307e-07, -2.861018856492592e-06, -1.8596476365928538e-05, -8.34461570775602e-06, -3.933898824470816e-06, -2.861018856492592e-06, -2.145764938177308e-06, -3.099436753473128e-06, -9.298280929215252e-06, -3.099436753473128e-06, -1.5497195136049413e-06, -9.059865078597795e-06, -2.264974000354414e-06, -2.3841855067985307e-07, -1.311301275563892e-06, -1.1920928244535389e-07, -1.1920922133867862e-06, -2.3841855067985307e-07, -0.00013159839727450162, -4.31528314948082e-05, -0.8964225649833679, -7.629365427419543e-06, -0.431106299161911, -0.16624343395233154, -2.399109125137329, -9.536738616588991e-07, -0.06675820797681808, -1.806227207183838, -0.5250357389450073, -0.02394123002886772, -2.002696055569686e-05, -3.099436753473128e-06, -8.344646857949556e-07, -5.960462772236497e-07, -5.722029527532868e-06, -3.576278118089249e-07, -3.576278118089249e-07, -2.3841855067985307e-07, -0.04306042939424515, -0.0001225396408699453, -1.3828182090946939e-05, -1.311301275563892e-06, -6.9141146923357155e-06, -9.179073458653875e-06, -1.2993727978027891e-05, -2.145764938177308e-06, -1.0728830375228426e-06, -5.483612312673358e-06, -5.960462772236497e-07, -5.960462772236497e-07, -3.576278118089249e-07, -2.3841830625315197e-06, -2.95634672511369e-05, -3.576272320060525e-06, -2.9802276912960224e-06], "student_logprobs": [-0.6900197267532349, -0.010116256773471832, -0.00033742457162588835, -0.5233765840530396, -2.002696055569686e-05, -9.60780744208023e-05, -0.05490013584494591, -0.00303082843311131, -0.00023922442051116377, -0.23576639592647552, -0.2524796724319458, -0.008953659795224667, -0.00016044282529037446, -2.932505594799295e-05, -4.815939246327616e-05, -1.2636104656849056e-05, -0.0004374024283606559, -2.610649426060263e-05, -7.271740287251305e-06, -0.00011884459672728553, -1.4305012882687151e-05, -0.0004818470624741167, -5.2927523938706145e-05, -0.00015448330668732524, -2.020915985107422, -0.6328393816947937, -0.03395428881049156, -5.245195097813848e-06, -6.437094270950183e-05, -1.8715683836489916e-05, -0.01607787050306797, 0.0, -2.9802276912960224e-06, -0.9265004396438599, -1.6383144855499268, -0.2018834799528122, -0.9053189754486084, -1.7762025890988298e-05, -9.775113539944869e-06, -4.172316494077677e-06, -3.576278118089249e-07, -6.97350042173639e-05, -7.271740287251305e-06, -1.311301275563892e-06, -4.029192859889008e-05, -9.536738616588991e-07, -0.00100192881654948, -2.825220326485578e-05, -4.887569048150908e-06, -0.43112242221832275, -0.018115894868969917, -0.12696532905101776, -3.6954811548639555e-06, -4.529942543740617e-06, -1.2278481335670222e-05, -0.059538863599300385, -4.768370445162873e-07, -5.98412734689191e-05, -3.798675775527954, -0.021217042580246925, -1.1444026313256472e-05, -0.7883281707763672, -0.08142465353012085, -0.5756580233573914, -0.0008768765837885439, -0.002879285253584385, -1.0551764965057373, -0.01719401776790619, -1.299927830696106, -0.2944146990776062, -0.2599864900112152, -0.00019059749320149422, -0.7354309558868408, -1.0631425380706787, -0.02963828295469284, -0.029000069946050644, -4.6132929128361866e-05, -7.152555099310121e-07, -7.152531907195225e-06, -9.298280929215252e-06, -0.18076679110527039, -2.253030106658116e-05, -5.304672595229931e-05, -0.014316700398921967, -0.16832509636878967, -0.25544363260269165, -5.960446742392378e-06, -0.00023767507809679955, -5.364403477869928e-06, -0.4004979729652405, -0.11283161491155624, -1.4126150608062744, -0.2873202860355377, -0.018334439024329185, -0.3135789930820465, -0.0002203936892328784, -9.775113539944869e-06, -0.008854884654283524, -1.2677016258239746, -0.2217981517314911, -0.43686604499816895, -0.4568982720375061, -1.086632490158081, -0.6520951986312866, -0.0015393561916425824, -0.03865748271346092, -0.3885374963283539, -0.013120625168085098, -0.6821235418319702, -0.0004553949984256178, -4.410734163684538e-06, -0.23643402755260468, -0.0009205871028825641, -5.602820692729438e-06, -0.6446844935417175, -0.016040096059441566, -0.0001854724541772157, -0.8454846143722534, -0.2014615833759308, -0.04311055690050125, -3.5049633979797363, -0.08904088288545609, -0.22589483857154846, -1.1218920946121216, -1.2236428260803223, -0.0004745787591673434, -0.012511572800576687, -1.0728830375228426e-06, -3.3378546504536644e-06, -0.778614342212677, -0.00012814177898690104, -0.5947968363761902, -0.13835415244102478, -3.196530818939209, -0.0016052944120019674, -0.06219381093978882, -0.5947542786598206, -0.7095733880996704, -0.01426299475133419, -0.28116342425346375, -1.6689286894688848e-06, -2.9802276912960224e-06, -0.17545516788959503, -0.11091434210538864, -0.02953585796058178, -0.6623629927635193, -0.05930262804031372, -0.0764671191573143, -5.864924969500862e-05, -0.12909163534641266, -0.5101549625396729, -0.005112787708640099, -2.145764938177308e-06, -9.536738616588991e-07, -1.0620554685592651, -0.000311802898067981, -0.026974499225616455, -0.11772824823856354, -0.0006349454633891582, -1.0967194612021558e-05, -0.0001722425949992612, -0.0005766874528490007, -2.4691100120544434, -0.00493724225088954, -0.0008965287124738097, -0.2500179409980774, -0.8442312479019165, -0.853484034538269, -0.1720115691423416, -0.0011248220689594746, -0.016360541805624962, -0.02055404521524906, -4.0531076592742465e-06, -5.960462772236497e-07, -0.0005498804384842515, -6.9141146923357155e-06, -3.166419506072998, -5.164754390716553, -2.3113465309143066, -0.29767951369285583, -0.6324012875556946, -0.022887935861945152, -0.015462283045053482, -0.04247768968343735, -0.2783028185367584, -0.025678575038909912, -0.004083035048097372, -1.7165990357170813e-05, -3.576278118089249e-07, -0.006634707562625408, -0.0010855264263227582, -1.5044050216674805, -1.343536615371704, -0.6357471346855164, -0.00942731648683548, -0.10409247130155563, -3.4047975540161133, -3.4842562675476074, -0.18344004452228546, -0.8929510712623596, -0.08150563389062881, -0.00029345019720494747, -7.724463648628443e-05, -0.0006324436399154365, -0.07688966393470764, -0.2786765694618225, -0.06199551373720169, -0.00010859376925509423, -4.768370445162873e-07, -0.014422692358493805, -0.00012981049076188356, -2.0211615562438965, -0.0959654226899147, -3.421248038648628e-05, -2.002696055569686e-05, -0.0006820021662861109, -0.005385059397667646, -0.001043133088387549, -1.3351351299206726e-05, -0.0027661179192364216, -1.2516897186287679e-05, -8.821448318485636e-06, -4.446407547220588e-05, -7.235741941258311e-05, -0.17976824939250946, -0.0870257243514061, -2.276871418871451e-05, -0.0037934263236820698, -0.011140525341033936, -0.7008658051490784, -0.03460293635725975, -4.494089080253616e-05, -9.536738616588991e-07, -0.004289355129003525, -1.4185804502631072e-05, -0.3062437176704407, -5.381356239318848, -0.05299661308526993, -0.6011545658111572, -0.0011067223967984319, -0.002557342639192939, -5.8410845667822286e-05, -1.0132738680113107e-05, -0.7576942443847656, -0.010417843237519264, -0.004622369538992643, -5.483612312673358e-06, -2.264974000354414e-06, -0.036924753338098526, -0.21257227659225464, -0.03341267630457878, -0.012254423461854458, -0.028195271268486977, -0.8667022585868835, -0.2462826520204544, -0.1963893175125122, -0.0017143089789897203, -0.2144659161567688, -0.0029714026022702456, -5.781483559985645e-05, -0.0006095697171986103, -0.3024829626083374, -2.563579559326172, -0.0009307105210609734, -0.0024857595562934875, -4.529942543740617e-06, -0.00144091690890491, -0.006532978732138872, -0.00017379203927703202, -0.009798052720725536, -2.861018856492592e-06, -8.344646857949556e-07, 0.0, -0.00036483307485468686, -2.50339189733495e-06, -7.593343616463244e-05, -0.0001851148990681395, -1.0728830375228426e-06, -3.814689989667386e-06, 0.0, -5.8530047681415454e-05, -3.099436753473128e-06, -8.5588610090781e-05, -0.026400532573461533, -4.60137271147687e-05, -4.9828242481453344e-05, -5.602820692729438e-06, -0.8311730623245239, -0.0075484998524188995, -0.007712468039244413, -0.9293995499610901, -0.003365093609318137, -0.00015054999676067382, -2.5082175731658936, -0.40079888701438904, -0.01751643791794777, -0.057227931916713715, -0.0011510183103382587, -0.28315162658691406, -6.079655122448457e-06, -1.311301275563892e-06, -0.20153729617595673, -5.2927523938706145e-05, -6.663577369181439e-05, -0.008975517004728317, -0.0197649747133255, -0.007862688973546028, -0.002449014689773321, -1.2755313036905136e-05, -0.03004428930580616, -0.001972635742276907, -7.021180499577895e-05, -1.168244216387393e-05, -0.018298037350177765, -0.0006521005416288972, -1.2755313036905136e-05, -8.583032467868179e-06, -1.4305104514278355e-06, -3.576278118089249e-07, -2.074220174108632e-05, -6.9141146923357155e-06, -0.0007407305529341102, -0.00039319414645433426, -7.152555099310121e-07, -5.960462772236497e-07, -1.1920922133867862e-06, -4.51792984677013e-05, -1.1920928244535389e-07, -0.0006333967321552336, -0.00015484087634831667, -2.0265558760002023e-06, -2.861018856492592e-06, 0.0, -4.458328112377785e-05, -2.3841855067985307e-07, -0.0012759171659126878, -0.008718756958842278, -3.6954811548639555e-06, -2.264974000354414e-06, -8.344646857949556e-07, -2.098061486321967e-05, -0.07256075739860535, -0.0035438849590718746, -0.0016950300196185708, -0.41510260105133057, -2.145764938177308e-06, -0.9761484265327454, -0.031069045886397362, -0.6746014952659607, -0.05245029181241989, -0.0006313714548014104, -0.029786981642246246, -0.04906449094414711, -9.536738616588991e-07, -1.0609570381348021e-05, -0.24085815250873566, -2.5033637939486653e-05, -0.9750641584396362, -0.060481783002614975, -0.0020839895587414503, -0.000783732277341187, -0.08421026170253754, -0.01388767920434475, -4.284721851348877, -0.6876510381698608, -2.787400007247925, -0.00013267113536130637, -2.13382354559144e-05, -2.4478909969329834, -0.015071330592036247, -0.11337871104478836, -7.617183291586116e-05, -0.2524920701980591, -0.03867296501994133, -0.0002574589161667973, -0.0011553048389032483, -0.023773731663823128, -0.0004228651523590088, -9.536738616588991e-07, -4.768370445162873e-07, -7.176141662057489e-05, -2.50339189733495e-06, -2.3245540432981215e-05, -3.576272320060525e-06, -9.059865078597795e-06, -6.437280717364047e-06, -0.00015639036428183317, -0.00017557987303007394, -0.005198295693844557, -0.02371855266392231, -3.576272320060525e-06, -1.0728830375228426e-06, -0.0008039817912504077, -7.152555099310121e-07, -1.168244216387393e-05, -1.537788011773955e-05, -0.0003256267518736422, -1.7881377516459906e-06, -2.50339189733495e-06, -1.1299068927764893, -2.6405186653137207, -0.0004117832868359983, -0.0003815161471720785, -9.858122211880982e-05, -3.6954811548639555e-06, -0.00018737945356406271, -0.0042349896393716335, -0.0003670972364488989, -0.0046617635525763035, -9.536738616588991e-07, -1.6689286894688848e-06, -2.3841855067985307e-07, -5.61460001335945e-05, -1.1324817933200393e-05, -5.149708886165172e-05, -0.0004172646440565586, -5.960462772236497e-07, -2.3841855067985307e-07, 0.0, -7.068861305015162e-05, -9.65590606938349e-06, -0.00017927470616996288, -0.0012496052077040076, -3.576278118089249e-07, -3.4570634852570947e-06, -1.1920928244535389e-07, -8.11782301752828e-05, -9.894321920000948e-06, -0.00014149141497910023, -0.0006562701892107725, -2.3841855067985307e-07, -1.8358061424805783e-05, -2.3841855067985307e-07, -1.7404405298293568e-05, -3.3378546504536644e-06, -0.00016342257731594145, -0.002686465159058571, -3.099436753473128e-06, -7.033323527139146e-06, -2.7418097943154862e-06, -0.025036433711647987, -0.02746637351810932, -0.00038628268521279097, -0.2605762481689453, -1.295372724533081, -0.007310189306735992, -0.00047231485950760543, -0.0002383901592111215, -7.116541382856667e-05, -1.3201934099197388, -0.7158500552177429, -0.8143535852432251, -0.32830342650413513, -0.04858846589922905, -0.016510525718331337, -0.0025228597223758698, -0.10184789448976517, -2.3841855067985307e-07, -4.768370445162873e-07, -0.0007382289622910321, -1.1920928244535389e-07, -6.460934673668817e-05, -5.960462772236497e-07, -2.5748875486897305e-05, -2.3841830625315197e-06, -6.890059739816934e-05, -0.0003090619284193963, -7.986990567587782e-06, -2.50339189733495e-06, -6.437280717364047e-06, 0.0, -1.6569954823353328e-05, -6.437280717364047e-06, -6.258291978156194e-05, -0.00020990552729927003, -1.311301275563892e-06, -1.1920928244535389e-07, -1.1920928244535389e-07, -2.610649426060263e-05, -2.0265558760002023e-06, -2.1934269170742482e-05, -0.0001212284987559542, -5.960462772236497e-07, -1.4305104514278355e-06, -2.3841855067985307e-07, -2.1815061700181104e-05, -7.152555099310121e-07, -3.4927710657939315e-05, -0.0001720042055239901, -7.152555099310121e-07, -2.861018856492592e-06, 0.0, -2.9205850296420977e-05, -3.576278118089249e-07, -7.259582343976945e-05, -0.00010013079008786008, -1.5497195136049413e-06, -8.344646857949556e-07, -9.536738616588991e-07, -0.00011503035057103261, -0.006689061410725117, -0.0016583750257268548, -0.11428095400333405, -0.0019649022724479437, -0.0007198841194622219, -0.00014411364099942148, -2.264974000354414e-06, -8.201262971851975e-05, -0.0007605044520460069, -0.055493615567684174, -0.0012968709925189614, -1.1920928244535389e-07, -2.3841830625315197e-06, -0.0003200257197022438, -9.536738616588991e-07, -4.1483970562694594e-05, -2.861018856492592e-06, -0.0015564957866445184, -2.7418097943154862e-06, -0.00010966652916977182, -3.40932747349143e-05, -2.3841830625315197e-06, -3.4570634852570947e-06, -4.768370445162873e-07, -0.0003232433518860489, -6.556489552167477e-06, -1.7523612768854946e-05, -0.00022289653134066612, -4.768370445162873e-07, -5.364403477869928e-06, 0.0, -0.000321336614433676, -1.764281842042692e-05, -0.00012134769349358976, -0.0020452074240893126, -5.960462772236497e-07, -6.6756979322235566e-06, -1.1920928244535389e-07, -5.006777428206988e-06, -1.311301275563892e-06, -2.098061486321967e-05, -0.00039593485416844487, -1.5497195136049413e-06, -2.3841855067985307e-07, -4.768370445162873e-07, -0.00025567124248482287, -2.3483953555114567e-05, -0.00013517419574782252, -0.16451001167297363, -0.0032922611571848392, -6.222531374078244e-05, -3.6954811548639555e-06, -1.3351351299206726e-05, -7.343022298300639e-05, -9.65590606938349e-06, -0.005938741844147444, -0.00031263710116036236, -0.0013521108776330948, 0.0, -9.536738616588991e-07, -0.00019870213873218745, -3.576278118089249e-07, -4.768360213347478e-06, -2.145764938177308e-06, -1.1920858014491387e-05, -5.364403477869928e-06, -9.536738616588991e-07, -7.30726242181845e-05, -1.764281842042692e-05, -1.1920922133867862e-06, -1.0728830375228426e-06, -1.1920928244535389e-07, -1.8954096958623268e-05, -2.861018856492592e-06, -0.00011264643399044871, -6.687417771900073e-05, -9.536738616588991e-07, -7.986990567587782e-06, 0.0, -2.50339189733495e-06, -1.4305104514278355e-06, -7.748573807475623e-06, -4.005352093372494e-05, -1.0728830375228426e-06, -6.437280717364047e-06, -2.3841855067985307e-07, -9.083335316972807e-05, -1.7881377516459906e-06, -1.1801649634435307e-05, -3.6000557884108275e-05, -1.4305104514278355e-06, -1.4305104514278355e-06, -3.576278118089249e-07, -8.189342770492658e-05, -2.3841855067985307e-07, -1.4066597032069694e-05, -9.929640509653836e-05, -1.1920922133867862e-06, -9.536738616588991e-07, -1.5497195136049413e-06, -7.545663538621739e-05, -0.0006468587671406567, -1.9550132492440753e-05, -0.002450560452416539, -0.009951265528798103, -0.000502817565575242, -2.6702524337451905e-05, -1.3351351299206726e-05, -1.9073468138230965e-06, -3.302042750874534e-05, -0.0002224197960458696, -2.7179348762729205e-05, -1.9073468138230965e-06, -0.0036131355445832014, -0.14164024591445923, -0.016212882474064827, -9.536738616588991e-07, -6.365573790390044e-05, -1.5258672647178173e-05, -0.8649954199790955, -0.0010825494537129998, -0.011435674503445625, -5.722029527532868e-06, -0.005081832408905029, -1.549708758830093e-05, -1.3110147714614868, -0.002836611121892929, -0.005745088681578636, -0.38772645592689514, -0.00488362368196249, -4.410734163684538e-06, -1.8715683836489916e-05, -7.152555099310121e-07, -1.1920928244535389e-07, -7.152531907195225e-06, -1.6689286894688848e-06, -0.00025328766787424684, -1.0609570381348021e-05, -5.960446742392378e-06, -0.0002803409588523209, -3.576278118089249e-07, -4.994744449504651e-05, -1.4185804502631072e-05, -0.1077359989285469, -0.17153991758823395, -0.02710770070552826, -3.015949550899677e-05, -0.033982519060373306, -2.9490559101104736, -0.0003411188081372529, -1.0200231075286865, -0.17225533723831177, -0.8997697830200195, -6.036230087280273, -0.4315369129180908, -0.633786141872406, -9.240616798400879, -0.06626476347446442, -0.06297697871923447, -0.017230460420250893, -0.09049732238054276, -0.06705299764871597, -0.8399354219436646, -1.7331362962722778, -2.682172998902388e-05, -5.245195097813848e-06, -0.08343477547168732, -8.05822346592322e-05, -2.6462035179138184, -0.0020224847830832005, -0.5538468956947327, -2.9322872161865234, -0.0568055585026741, -0.0006762839620932937, -0.001427346607670188, -0.025555524975061417, -0.02080346643924713, -0.0007127366843633354, -4.768370445162873e-07, -1.3708974620385561e-05, -0.078593909740448, -1.8853504657745361, -2.2291887944447808e-05, -0.6986204385757446, -0.0001674750237725675, -0.03806092217564583, -0.00013159839727450162, -4.529942543740617e-06, -3.4570634852570947e-06, -4.8993817472364753e-05, -0.22591641545295715, -0.00041976699139922857, -0.0003535122668836266, -8.344646857949556e-07, -1.311301275563892e-06, -3.576278118089249e-07, -7.867782187531702e-06, -3.099436753473128e-06, -1.5497195136049413e-06, -2.706014311115723e-05, -2.3841830625315197e-06, -9.536738616588991e-07, -8.583032467868179e-06, -1.1920922133867862e-06, -2.4318398573086597e-05, -9.536697689327411e-06, -1.7881377516459906e-06, -7.748573807475623e-06, -5.483612312673358e-06, -2.622600959512056e-06, -4.172316494077677e-06, -2.3841830625315197e-06, -1.1920858014491387e-05, -7.331102824537084e-05, -4.768370445162873e-07, -3.576278118089249e-07, -1.311301275563892e-06, -9.894321920000948e-06, -3.4570634852570947e-06, -3.576278118089249e-07, -9.65590606938349e-06, -2.3841830625315197e-06, -5.8412379075889476e-06, -5.006777428206988e-06, -3.814689989667386e-06, -1.0967194612021558e-05, -2.0265558760002023e-06, -1.9073468138230965e-06, -2.145764938177308e-06, -7.390948667307384e-06, -4.768360213347478e-06, -3.504691630951129e-05, -1.1205610462639015e-05, -9.298280929215252e-06, -4.672895011026412e-05, -1.0967194612021558e-05, -1.0251946150674485e-05, -5.483612312673358e-06, -2.622600959512056e-06, -1.2755313036905136e-05, -4.529942543740617e-06, -4.768370445162873e-07, -4.4225667807040736e-05, -4.529942543740617e-06, -9.142934868577868e-05, -5.125986263010418e-06, -0.00014757021563127637, -1.1920928244535389e-07, -5.960462772236497e-07, -8.344646857949556e-07, -3.015949550899677e-05, -2.145764938177308e-06, -1.0728830375228426e-06, -4.172316494077677e-06, -2.3841830625315197e-06, -5.125986263010418e-06, -1.1205610462639015e-05, -3.0278701160568744e-05, -6.997340824455023e-05, -9.536738616588991e-07, -3.2186455882765586e-06, -2.3007127310847864e-05, -9.65590606938349e-06, -2.729855441430118e-05, -8.702239938429557e-06, -6.6756979322235566e-06, -3.40932747349143e-05, -7.033323527139146e-06, -2.7418097943154862e-06, -8.821448318485636e-06, -1.7523612768854946e-05, -4.386805812828243e-05, -4.732496745418757e-05, -9.536738616588991e-07, -3.576278118089249e-07, -3.2186455882765586e-06, -0.00017105070583056659, -1.1920922133867862e-06, -7.152555099310121e-07, -2.264974000354414e-06, -1.7881377516459906e-06, -3.4570634852570947e-06, -1.0847986231965479e-05, -3.969590397900902e-05, -1.168244216387393e-05, -1.4305104514278355e-06, -3.576272320060525e-06, -1.9907753085135482e-05, -1.9550132492440753e-05, -2.0265558760002023e-06, -2.861018856492592e-06, -9.417489309271332e-06, -1.1086402082582936e-05, -8.83301836438477e-05, -4.768370445162873e-07, -3.576278118089249e-07, -3.576278118089249e-07, -2.098061486321967e-05, -5.125986263010418e-06, -1.5497195136049413e-06, -7.867782187531702e-06, -4.291525328881107e-06, -3.6954811548639555e-06, -2.539125671319198e-05, -2.3841574147809297e-05, -1.6569954823353328e-05, -2.861018856492592e-06, -4.6491513785440475e-06, -0.0004920940846204758, -3.540453326422721e-05, -1.0251946150674485e-05, -7.271740287251305e-06, -9.536738616588991e-07, -5.602820692729438e-06, -1.6093124941107817e-05, -1.5139465176616795e-05, -9.775113539944869e-06, -5.07818695041351e-05, -7.629365427419543e-06, -1.4543427823809907e-05, -9.536738616588991e-07, -0.25252532958984375, -0.2520604133605957, -3.576278118089249e-07, -2.7179348762729205e-05, -3.3378546504536644e-06, -0.20488330721855164, -1.3966498374938965, -0.34775635600090027, -5.578839045483619e-05, -1.6689286894688848e-06, -0.11340031772851944, -2.253030106658116e-05, -0.20158027112483978, -1.7404405298293568e-05, -0.0827701985836029, -0.20455244183540344, -0.029438285157084465, -1.3589766240329482e-05, -0.008618067018687725, -0.00010549465514486656, -5.507317473529838e-05, -1.2040065485052764e-05, -8.344646857949556e-07, -2.8132995794294402e-05, -2.3841574147809297e-05, -4.887569048150908e-06, -1.0728830375228426e-06, -0.0001699779968475923, -0.11290966719388962, -2.3483953555114567e-05, -2.861018856492592e-06, -2.5629668016335927e-05, -0.0632612481713295, -1.422014594078064, -0.0019163592951372266, -0.34875810146331787, -0.29313158988952637, -2.3841830625315197e-06, -7.986990567587782e-06, -3.576278118089249e-07, -0.0001147919538198039, -1.1920922133867862e-06, -0.31049153208732605, -2.3007127310847864e-05, -0.00011419598013162613, -1.311301275563892e-06, -0.39465004205703735, -1.6011061668395996, -0.00016366096679121256, -5.960446742392378e-06, -5.304672595229931e-05, -0.14087285101413727, -0.013340851292014122, -0.13177502155303955, -2.169585604860913e-05, -0.005654885899275541, -0.09576601535081863, -2.2053474822314456e-05, -8.821448318485636e-06, -3.2186455882765586e-06, -9.894321920000948e-06, -5.006777428206988e-06, -1.0609570381348021e-05, -6.556489552167477e-06, -4.51792984677013e-05, -2.5033637939486653e-05, -0.005945615470409393, -0.0015038858400657773, -0.6463885307312012, -4.410734163684538e-06, -0.00023529145983047783, -0.004557341802865267, -0.06612953543663025, -0.226091206073761, -0.7799097299575806, -0.00017248096992261708, -1.3136399984359741, -0.1701374053955078, -4.3748852476710454e-05, -6.6756979322235566e-06, -1.3112935448589269e-05, -9.536738616588991e-07, -6.186770770000294e-05, -2.8490614567999728e-05, -8.344646857949556e-07, -3.4570634852570947e-06, -0.002952860901132226, -5.077699184417725, -0.00029345019720494747, -0.0876336470246315, -0.1566917896270752, -0.20347264409065247, -0.4538908004760742, -1.4227527379989624, -0.0009199915803037584, -0.042735178023576736, -0.05379650369286537, -1.5132114887237549, -0.002496223896741867, -1.059636116027832, -0.6170956492424011, -2.066518545150757, -7.182731628417969, -0.24476554989814758, -0.0004637358069885522, -0.03973028063774109, -0.01898382045328617, -0.013523630797863007, -0.15735892951488495, -0.05045187845826149, -0.6841297149658203, -0.05593232437968254, -0.05186733603477478, -0.9989488124847412, -0.07059306651353836, -0.0048412722535431385, -0.5384079813957214, -5.511801242828369, -1.1046257019042969, -0.012503214180469513, -0.0808330625295639, -0.001142564113251865, -3.5040502548217773, -0.7470360994338989, -0.4450824558734894, -4.7205765440594405e-05, -2.173161506652832, -0.15302757918834686, -0.2613004148006439, -1.2407145500183105, -0.000593962671700865, -0.35710737109184265, -0.7195988893508911, -2.5296571254730225, -0.6596041321754456, -0.1642100214958191, -0.03549475595355034, -0.575982391834259, -0.5574370622634888, -0.7647674679756165, -7.724463648628443e-05, -0.18959110975265503, -0.04545533284544945, -2.350782632827759, -0.005668873433023691, -2.3446311950683594, -0.09881530702114105, -3.0636318115284666e-05, -7.164221460698172e-05, -3.814689989667386e-06, -4.565611743601039e-05, -3.814689989667386e-06, -2.50339189733495e-06, -4.291525328881107e-06, -0.0024995533749461174, -2.1214089393615723, -0.0021716600749641657, -0.5084633827209473, -0.22212670743465424, -0.023020517081022263, -0.33638665080070496, -3.731181277544238e-05, -0.20126010477542877, -0.0011887634173035622, -4.612823009490967, -0.0016931259306147695, -0.29498592019081116, -1.141097068786621, -0.0902191624045372, -0.017887115478515625, -1.5735502529423684e-05, -2.089967727661133, -1.8465406894683838, -1.6784275770187378, -0.8657925724983215, -0.007784269750118256, -3.521324872970581, -2.999523639678955, -0.8462132215499878, -0.13015638291835785, -1.0682626962661743, -0.25323572754859924, -0.0001728385395836085, -0.22165614366531372, -0.013147097080945969, -0.11232970654964447, -0.010860028676688671, -0.04557618498802185, -0.001520311925560236, -0.1177147924900055, -0.30535754561424255, -1.6212332411669195e-05, -0.33942756056785583, -1.532120704650879, -0.9528497457504272, -0.04053272306919098, -0.29911696910858154, -0.013448006473481655, -0.15076708793640137, -5.221230458118953e-05, -3.4570634852570947e-06, -0.0002090712368953973, -0.02408507652580738, -1.4009437561035156, -0.9805890321731567, -0.0032557835802435875, -0.29108843207359314, -0.4920610189437866, -1.1920928244535389e-07, -4.172316494077677e-06, -0.13801921904087067, -0.04986979812383652, -1.757205843925476, -0.7565698623657227, -1.9273574352264404, -0.6509132385253906, -0.3663448393344879, -0.0032834685407578945, -0.10286621749401093, -0.00031668893643654883, -0.5887801051139832, -1.7092280387878418, -5.245071224635467e-05, -0.8134710788726807, -0.0003116837178822607, -0.002019629580900073, -0.04206835478544235, -0.10776040703058243, -0.05152399092912674, -1.7881377516459906e-06, -0.02057483419775963, -0.2825014293193817, -0.014239607378840446, -3.1232192516326904, -2.1815061700181104e-05, -1.7881377516459906e-06, -8.583032467868179e-06, -1.5372309684753418, -0.0012292455649003386, -0.026478318497538567, -6.496695277746767e-05, -2.264974000354414e-06, -3.3378546504536644e-06, -1.7523612768854946e-05, -2.622600959512056e-06, -2.7179348762729205e-05, -1.311301275563892e-06, -0.03641900420188904, -0.007611677050590515, -0.0006212450098246336, -0.5536268949508667, -1.0490362910786644e-05, -0.42490580677986145, -3.462890863418579, -0.8137584924697876, -0.003364143194630742, -0.47482919692993164, -0.8149405121803284, -0.00014625910262111574, -0.2880585193634033, -0.42923542857170105, -1.2479817867279053, -0.02975342608988285, -1.361396312713623, -0.062042683362960815, -0.0004397855664137751, -5.8412379075889476e-06, -0.2954549789428711, -0.4746701419353485, -0.011585102416574955, -1.1862140893936157, -1.585470999998506e-05, -1.6309500932693481, -0.18530087172985077, -0.0001445904199499637, -3.6827232837677, -2.2891783714294434, -1.1247543096542358, -0.16554540395736694, -0.04983407258987427, -1.7362942695617676, -0.35812172293663025, -0.03850482031702995, -0.5784719586372375, -3.3449978828430176, -0.14819109439849854, -3.0636318115284666e-05, -0.1355881541967392, -0.05677199363708496, -0.00014840454969089478, -8.940656698541716e-06, -0.0562172494828701, -5.887912750244141, -0.14499589800834656, -0.4934888780117035, -0.00015662873920518905, -2.8967437174287625e-05, -0.02250209078192711, -0.7667915225028992, -3.7108078002929688, -0.0010482537327334285, -0.12622535228729248, -0.8986120820045471, -0.04864932969212532, -0.920523464679718, -0.012873165309429169, -0.0005169962532818317, -0.03232118859887123, -0.00024136967840604484, -1.7881377516459906e-06, -2.753696753643453e-05, -0.0009241600637324154, -4.2676016164477915e-05, -2.7418097943154862e-06, -1.4424220353248529e-05, -0.00020108585886191577, -0.0015694693429395556, -5.8412379075889476e-06, -0.016906658187508583, -0.05493127927184105, -0.004648118279874325, -0.03754449635744095, -1.1205610462639015e-05, -4.589452510117553e-05, -0.002658287761732936, -6.05564855504781e-05, -8.344646857949556e-07, -5.006777428206988e-06, -0.0017220443114638329, -5.364403477869928e-06, -2.50339189733495e-06, -5.113947918289341e-05, -0.0005502378917299211, -2.3007127310847864e-05, -2.622600959512056e-06, -1.4424220353248529e-05, -4.2676016164477915e-05, -1.7523612768854946e-05, -1.2159273865108844e-05, -0.00010442188795423135, -0.00035661060246638954, -0.007085079327225685, -4.768370445162873e-07, -0.0026391458231955767, -0.012895642779767513, -0.00018606838420964777, -0.0013717536348849535, -1.6689286894688848e-06, -7.986990567587782e-06, -2.658331868587993e-05, -4.2199197196168825e-05, -3.4570634852570947e-06, -2.109982233378105e-05, -4.792098479811102e-05, -1.823885577323381e-05, -7.152531907195225e-06, -7.748573807475623e-06, -6.985420623095706e-05, -3.075552376685664e-05, -5.364403477869928e-06, -2.145764938177308e-06, -1.9907753085135482e-05, -3.790783375734463e-05, -8.344646857949556e-07, -1.7523612768854946e-05, -8.082063141046092e-05, -2.264974000354414e-06, -0.003480806713923812, -2.7418097943154862e-06, -2.622600959512056e-06, -1.8596476365928538e-05, -1.6569954823353328e-05, -4.172316494077677e-06, -6.318072337307967e-06, -2.4199192921514623e-05, -3.361645576660521e-05, -1.7046782886609435e-05, -3.755022044060752e-05, -2.2411095415009186e-05, -8.439661905867979e-05, -5.960462772236497e-07, -2.253030106658116e-05, -0.00012540031457319856, -2.145764938177308e-06, -2.0503786799963564e-05, -9.536738616588991e-07, -4.0531076592742465e-06, -1.2159273865108844e-05, -2.145764938177308e-06, -3.933898824470816e-06, -1.7881377516459906e-06, -6.460934673668817e-05, -2.6464111215318553e-05, -1.9430925021879375e-05, -9.536738616588991e-07, -3.540453326422721e-05, -1.1086402082582936e-05, -1.6689286894688848e-06, -7.748573807475623e-06, -1.0371154530730564e-05, -2.753696753643453e-05, -1.1920928244535389e-07, -1.3351351299206726e-05, -0.00016234986833296716, -3.6954195820726454e-05, -0.2947358787059784, -0.6770281791687012, -0.5252234935760498, -0.5357192754745483, -0.002016298472881317, -1.6331539882230572e-05, -9.059865078597795e-06, -9.179073458653875e-06, -2.3483953555114567e-05, -7.152531907195225e-06, -2.622600959512056e-06, -1.1920922133867862e-06, -0.0001559135998832062, -0.09596802294254303, -4.637133679352701e-05, -0.44352948665618896, -0.369498074054718, -0.0782528966665268, -0.0015732781030237675, -1.6515066623687744, -0.2255975902080536, -0.7576354742050171, -0.1538732945919037, -1.9192511899746023e-05, -0.038892798125743866, -1.0251946150674485e-05, -0.0008437649230472744, -2.2574334144592285, -0.0003777029050979763, -3.576278118089249e-07, -4.660974445869215e-05, -1.4662635294371285e-05, -0.6931657791137695, -0.012713807635009289, -1.5487983226776123, -0.37589195370674133, -3.373566141817719e-05, -3.2186455882765586e-06, -1.7165990357170813e-05, -7.152555099310121e-07, -0.0019860798493027687, -2.407998726994265e-05, -6.318072337307967e-06, -3.099436753473128e-06, -0.00023600654094479978, -0.20560137927532196, -9.894321920000948e-06, -0.7569898366928101, -1.5496381521224976, -0.026324482634663582, -0.27854448556900024, -1.3447072505950928, -0.04353760927915573, -0.8816882371902466, -0.15949980914592743, -3.397406908334233e-05, -0.005021816119551659, -0.0026324877981096506, -0.04358109459280968, -0.0002936885575763881, -0.6033709049224854, -3.429521083831787, -0.013619006611406803, -1.4926865100860596, -0.6595169305801392, -1.063952922821045, -0.09968254715204239, -0.1642419844865799, -4.728325366973877, -0.4771721661090851, -0.47478559613227844, -0.022003335878252983, -7.176141662057489e-05, -0.22787164151668549, -0.11436856538057327, -0.04945485666394234, -0.43119701743125916, -0.020961539819836617, -4.410734163684538e-06, -0.0005787128466181457, -0.1748238056898117, -0.0002455409849062562, -0.043801672756671906, -6.437280717364047e-06, -0.010104455053806305, -0.0002699726028367877, -0.000427274004323408, -0.006810310296714306, -1.0609570381348021e-05, -0.46642956137657166, -0.0006594866863451898, -0.4689430892467499, -0.5169046521186829, -0.005309646483510733, -0.38693109154701233, -0.04881986230611801, -0.20658202469348907, -0.3940649926662445, -0.13722732663154602, -1.3682271242141724, -0.10628539323806763, -0.39681652188301086, -0.07344839721918106, -0.5143271684646606, -2.2314043045043945, -0.21001936495304108, -0.5133510828018188, -0.024786092340946198, -0.008137405849993229, -1.0371154530730564e-05, -0.3414222002029419, -0.13789969682693481, -1.4728658199310303, -0.2812159061431885, -0.0021891456563025713, -0.0025512785650789738, -1.095101237297058, -2.7418097943154862e-06, -0.0024779110681265593, -4.410646579344757e-05, -2.145764938177308e-06, -8.821448318485636e-06, -1.2636104656849056e-05, -1.5497195136049413e-06, -4.172316494077677e-06, -7.033323527139146e-06, -0.24834543466567993, -0.11303327977657318, -0.5899413824081421, -0.007182846311479807, -0.004003605805337429, -9.345571743324399e-05, -0.0014065144350752234, -0.1326701045036316, -2.288792165927589e-05, -1.3232143828645349e-05, -0.00021717573690693825, -2.038458114839159e-05, -2.3841830625315197e-06, -4.649054244509898e-05, -9.536738616588991e-07, -0.0002101439022226259, -0.00011955977242905647, -2.8729025871143676e-05, -0.8261289596557617, -0.7441821694374084, -9.417489309271332e-06, -1.8717968463897705, -1.8497986793518066, -0.017378907650709152, -0.206337571144104, -1.3973325490951538, -0.002516557462513447, -0.0003510097449179739, -1.2421557903289795, -0.005814898293465376, -2.253030106658116e-05, -0.47484543919563293, -0.00013839241000823677, -5.2569914259947836e-05, -0.015986833721399307, -5.483612312673358e-06, -0.02074018307030201, -1.7881233361549675e-05, -0.2737317383289337, -0.3863644599914551, -1.7881377516459906e-06, -9.917721035890281e-05, -0.4890812039375305, -1.5908139944076538, -0.08564179390668869, -2.2053474822314456e-05, -0.043036337941884995, -0.11508803069591522, -1.7881377516459906e-06, -5.960462772236497e-07, -4.8993817472364753e-05, -0.0005778788472525775, -0.05135210603475571, -0.0012996093137189746, -8.308542601298541e-05, -0.3504977524280548, -4.529942543740617e-06, -0.021000295877456665, -1.5146887302398682, -0.005421103443950415, -1.7523612768854946e-05, -0.07958795875310898, -7.152555099310121e-07, -5.8530047681415454e-05, -1.1920928244535389e-07, -5.391632556915283, -0.597592294216156, -0.0005998004344291985, -5.483612312673358e-06, -0.0076158177107572556, -0.04264973849058151, -1.4185804502631072e-05, -4.6491513785440475e-06, -0.42896905541419983, -0.0014485353603959084, -0.000328367663314566, -4.51792984677013e-05, -3.8414793014526367, -5.328513361746445e-05, -1.2260267734527588, -1.2235023975372314, -7.557583012385294e-05, -3.4127018451690674, -0.008615939877927303, -1.0959992408752441, -4.887569048150908e-06, -0.016411438584327698, -0.005904730875045061, -0.0008031480247154832, -0.03413690999150276, -0.2812303900718689, -0.274009108543396, -2.8658225536346436, -0.00013028726971242577, -1.0259060859680176, -0.4755607843399048, -1.4305104514278355e-06, -2.622600959512056e-06, -2.1457441107486375e-05, -7.152555099310121e-07, -8.546940807718784e-05, -0.22737595438957214, -0.00015793983766343445, -1.7881377516459906e-06, -0.008831252343952656, -2.145764938177308e-06, -2.3841855067985307e-07, -1.728519782773219e-05, -9.298280929215252e-06, -1.1920928244535389e-07, -2.264974000354414e-06, -1.1920928244535389e-07, -3.6954811548639555e-06, -7.152555099310121e-07, -9.536738616588991e-07, -4.660974445869215e-05, -3.3378546504536644e-06, -1.1920928244535389e-07, -1.7523612768854946e-05, -3.576278118089249e-07, -1.1920928244535389e-07, -2.3841830625315197e-06, -1.6689286894688848e-06, -1.5497195136049413e-06, -2.622600959512056e-06, -3.576278118089249e-07, -1.9073468138230965e-06, -3.099436753473128e-06, -1.1920928244535389e-07, -1.5497195136049413e-06, -1.311301275563892e-06, -3.576278118089249e-07, -1.1920928244535389e-07, -1.311301275563892e-06, -1.4305104514278355e-06, -1.1920928244535389e-07, -3.2066785934148356e-05, -3.6954811548639555e-06, -7.152555099310121e-07, -1.0728830375228426e-06, -1.5497195136049413e-06, -9.536697689327411e-06, -1.1920928244535389e-07, -1.1920928244535389e-07, -1.0728830375228426e-06, -0.0005640584276989102, -5.364403477869928e-06, 0.0, -3.099436753473128e-06, -2.3841855067985307e-07, -2.9802276912960224e-06, -2.7418097943154862e-06, -1.7881377516459906e-06, -1.7881377516459906e-06, -9.536738616588991e-07, -2.3841855067985307e-07, -1.9073468138230965e-06, -4.291525328881107e-06, -2.145764938177308e-06, -2.622600959512056e-06, -9.60780744208023e-05, -1.4305104514278355e-06, -3.6954811548639555e-06, -1.5497195136049413e-06, -1.5497195136049413e-06, -4.172316494077677e-06, -1.1920928244535389e-07, -1.7881377516459906e-06, -1.5497195136049413e-06, 0.0, -6.6756979322235566e-06, -2.3841830625315197e-06, -1.1920928244535389e-07, 0.0, -2.3841855067985307e-07, -2.8967437174287625e-05, -9.536738616588991e-07, -1.1920928244535389e-07, -2.2172682292875834e-05, -1.3351351299206726e-05, -1.5497195136049413e-06, -4.887569048150908e-06, -1.5497195136049413e-06, -7.033323527139146e-06, -5.960462772236497e-07, -1.0728830375228426e-06, -1.1086402082582936e-05, -1.5497195136049413e-06, -2.3841855067985307e-07, -2.50339189733495e-06, -4.768370445162873e-07, -2.861018856492592e-06, -3.576272320060525e-06, -1.7881377516459906e-06, -1.9073468138230965e-06, -8.34461570775602e-06, -1.0728830375228426e-06, -2.861018856492592e-06, -2.50339189733495e-06, -5.8412379075889476e-06, -8.344646857949556e-07, -1.4305104514278355e-06, -8.22540732769994e-06, -1.1920922133867862e-06, 0.0, -4.0531076592742465e-06, -1.6689286894688848e-06, -2.3841855067985307e-07, 0.0, -3.576278118089249e-07, -3.099436753473128e-06, -3.576278118089249e-07, -7.629103492945433e-05, -2.861018856492592e-06, -1.311301275563892e-06, -1.9073468138230965e-06, -2.3841855067985307e-07, -1.4066597032069694e-05, -7.152555099310121e-07, -9.536738616588991e-07, -1.3112935448589269e-05, -3.2186455882765586e-06, -2.3841855067985307e-07, -1.311301275563892e-06, -4.768370445162873e-07, -1.311301275563892e-06, -8.22540732769994e-06, -4.768360213347478e-06, -4.768370445162873e-07, -5.960462772236497e-07, -1.0728830375228426e-06, -4.6491513785440475e-06, -2.50339189733495e-06, -1.1920922133867862e-06, -3.099436753473128e-06, -1.0728830375228426e-06, -3.576278118089249e-07, -1.1920928244535389e-07, -3.576278118089249e-07, -7.152555099310121e-07, -1.311301275563892e-06, -6.782778655178845e-05, -4.887569048150908e-06, -2.145764938177308e-06, -1.5497195136049413e-06, -1.5497195136049413e-06, -0.00018142008048016578, -5.125986263010418e-06, -5.960462772236497e-07, -4.768360213347478e-06, -3.576272320060525e-06, -4.768370445162873e-07, -3.4570634852570947e-06, -5.960462772236497e-07, -1.7881377516459906e-06, -3.6954811548639555e-06, -1.0728830375228426e-06, -2.264974000354414e-06, -4.768370445162873e-07, -1.7881377516459906e-06, -1.0490362910786644e-05, -5.245195097813848e-06, -5.722029527532868e-06, -2.7418097943154862e-06, -1.6689286894688848e-06, -2.50339189733495e-06, -7.748573807475623e-06, -4.172316494077677e-06, -1.5497195136049413e-06, -7.748573807475623e-06, -1.0728830375228426e-06, -2.3841855067985307e-07, -1.1920928244535389e-07, -1.1920928244535389e-07, -1.1920922133867862e-06, -1.1920928244535389e-07, -8.189342770492658e-05, -3.0397906812140718e-05, -0.946895956993103, -4.0531076592742465e-06, -0.35138821601867676, -0.18474477529525757, -2.578446388244629, -7.152555099310121e-07, -0.056811414659023285, -2.3512563705444336, -0.4748242497444153, -0.02100052870810032, -9.775113539944869e-06, -3.933898824470816e-06, -7.152555099310121e-07, -4.768370445162873e-07, -5.960446742392378e-06, -2.3841855067985307e-07, -4.768370445162873e-07, -1.1920928244535389e-07, -0.026363378390669823, -7.366862701019272e-05, -7.152531907195225e-06, -8.344646857949556e-07, -8.583032467868179e-06, -6.6756979322235566e-06, -2.169585604860913e-05, -1.0728830375228426e-06, -1.311301275563892e-06, -5.8412379075889476e-06, -5.960462772236497e-07, -4.768370445162873e-07, -2.3841855067985307e-07, -2.264974000354414e-06, -3.0874729418428615e-05, -1.0728830375228426e-06, -3.576272320060525e-06], "index": 5113, "score": 1.0}, {"prompt_tokens": [248045, 8678, 198, 21365, 264, 3377, 11, 678, 3274, 369, 310, 4087, 279, 3296, 539, 7047, 2923, 13909, 28253, 303, 264, 2708, 321, 3050, 11233, 13, 198, 12162, 488, 599, 3272, 883, 279, 31626, 1817, 11, 3300, 279, 4087, 303, 279, 2614, 3443, 25, 198, 27, 8944, 29, 8944, 1532, 510, 8944, 29, 198, 5236, 524, 10033, 678, 31626, 4613, 279, 4087, 9212, 11, 3300, 1132, 279, 1534, 4087, 13, 3095, 449, 3010, 369, 3766, 11, 488, 1220, 24660, 1732, 279, 3443, 314, 279, 2468, 14, 8944, 303, 421, 3010, 13, 248046, 198, 248045, 846, 198, 86577, 411, 51852, 30070, 1414, 25, 47815, 53, 4823, 3580, 24505, 41, 23683, 2990, 457, 69516, 24505, 47815, 762, 30264, 53, 9690, 53, 46427, 47815, 12117, 13, 38256, 1132, 279, 60839, 1414, 430, 678, 1534, 4087, 13, 248046, 198, 248045, 74455, 198, 248068, 198], "tokens": [760, 1156, 6587, 728, 310, 36865, 264, 51852, 30070, 1414, 13, 198, 760, 1414, 369, 25, 328, 73966, 53, 4823, 3580, 24505, 41, 23683, 2990, 457, 69516, 24505, 47815, 762, 30264, 53, 9690, 53, 46427, 47815, 12117, 1149, 198, 40, 1144, 310, 1423, 279, 6206, 1328, 318, 3946, 314, 11592, 28028, 1142, 553, 198, 9764, 579, 1380, 310, 36865, 424, 539, 30787, 279, 11592, 1142, 13, 198, 10522, 4105, 303, 51852, 272, 79243, 25, 328, 73966, 53, 1, 1362, 381, 328, 16721, 487, 328, 3843, 487, 328, 54495, 487, 4831, 13, 198, 9764, 579, 1228, 413, 328, 73966, 53, 1, 1411, 328, 16721, 763, 198, 42, 1411, 345, 25, 710, 369, 220, 16, 16, 337, 6321, 11, 345, 369, 220, 17, 15, 337, 13, 25662, 283, 478, 24, 13, 198, 56, 1411, 462, 25, 783, 369, 220, 17, 20, 337, 11, 462, 369, 220, 23, 337, 13, 220, 17, 20, 478, 220, 24, 283, 220, 18, 19, 1411, 220, 18, 19, 471, 220, 17, 21, 283, 220, 23, 318, 39, 553, 38643, 13, 198, 53, 1411, 458, 25, 629, 369, 220, 17, 17, 302, 11, 458, 369, 220, 20, 337, 13, 220, 17, 17, 478, 220, 24, 283, 220, 18, 16, 1411, 220, 18, 16, 471, 220, 17, 21, 283, 220, 20, 318, 36, 553, 38643, 13, 198, 4272, 279, 6206, 364, 86298, 369, 478, 24, 318, 269, 6206, 2047, 539, 471, 24, 11, 466, 1245, 539, 220, 16, 22, 553, 13428, 11, 5129, 51852, 6206, 3254, 24656, 557, 393, 36, 2007, 8, 283, 318, 87, 478, 580, 8, 1088, 2508, 220, 17, 21, 12576, 198, 2592, 710, 7, 16, 15, 8, 1411, 345, 7, 16, 24, 681, 1179, 393, 16, 15, 478, 580, 283, 220, 16, 24, 1088, 25406, 580, 283, 220, 24, 12576, 198, 4272, 310, 36865, 11, 567, 6206, 478, 24, 13, 6558, 579, 36865, 279, 4220, 886, 440, 478, 24, 13, 198, 42, 1411, 345, 198, 56, 1411, 462, 198, 53, 1411, 458, 198, 9620, 198, 51, 1411, 351, 318, 16, 24, 10, 24, 28, 17, 23, 1411, 220, 17, 8, 198, 49, 1411, 357, 318, 16, 22, 10, 24, 28, 17, 21, 1411, 220, 15, 8, 1411, 32645, 423, 369, 220, 16, 22, 318, 15, 20834, 290, 25, 357, 28, 15, 11, 417, 28, 16, 1076, 423, 28, 16, 22, 553, 220, 16, 22, 10, 24, 28, 17, 21, 1411, 220, 15, 1411, 357, 13, 38643, 13, 198, 34, 1411, 436, 318, 17, 10, 24, 28, 16, 16, 1411, 436, 8, 198, 34, 1411, 436, 198, 53, 1411, 458, 198, 40, 1411, 423, 318, 23, 10, 24, 28, 16, 22, 1411, 423, 8, 198, 41, 1411, 326, 318, 24, 10, 24, 28, 16, 23, 1411, 326, 8, 198, 4272, 328, 2301, 3580, 24505, 41, 1, 1411, 328, 24622, 4180, 3158, 59187, 328, 2301, 3580, 24505, 41, 1, 1411, 351, 357, 436, 436, 458, 423, 326, 13, 198, 5666, 25, 328, 35125, 2990, 1, 198, 45, 1411, 457, 318, 16, 18, 10, 24, 28, 17, 17, 1411, 457, 8, 198, 53, 1411, 458, 198, 40, 1411, 423, 198, 53, 1411, 458, 198, 1, 35125, 2990, 1, 1411, 328, 12100, 762, 3158, 59187, 13, 198, 5666, 25, 328, 54, 69516, 24505, 1, 198, 54, 1411, 426, 318, 17, 17, 10, 24, 28, 18, 16, 1411, 220, 20, 1411, 426, 8, 198, 53, 1411, 458, 198, 45, 1411, 457, 198, 53, 1411, 458, 198, 40, 1411, 423, 198, 57943, 69516, 24505, 1, 1411, 328, 11091, 38050, 3158, 59187, 13, 198, 5666, 25, 328, 73966, 762, 1, 198, 42, 1411, 345, 198, 56, 1411, 462, 198, 49, 1411, 357, 198, 36, 1411, 443, 318, 19, 10, 24, 28, 16, 18, 1411, 443, 8, 198, 1, 73966, 762, 1, 1411, 328, 3496, 1058, 3158, 59187, 13, 198, 5666, 25, 328, 16651, 53, 9690, 53, 1, 198, 42, 1411, 345, 198, 45, 1411, 457, 198, 53, 1411, 458, 198, 34, 1411, 436, 198, 44, 1411, 629, 318, 16, 17, 10, 24, 28, 17, 16, 1411, 629, 8, 198, 53, 1411, 458, 198, 1, 16651, 53, 9690, 53, 1, 1411, 328, 45498, 2662, 4343, 3158, 59187, 13, 198, 5666, 25, 328, 17680, 1, 198, 37, 1411, 496, 318, 20, 10, 24, 28, 16, 19, 1411, 496, 8, 198, 54, 1411, 426, 198, 1, 17680, 1, 1411, 328, 12126, 3158, 59187, 13, 198, 5666, 25, 328, 73966, 12117, 1149, 198, 42, 1411, 345, 198, 56, 1411, 462, 198, 53, 1411, 458, 198, 35, 1411, 380, 318, 18, 10, 24, 28, 16, 17, 1411, 380, 8, 198, 1, 73966, 12117, 1149, 1411, 328, 16721, 44, 1149, 59187, 13, 271, 9153, 60839, 1414, 25, 328, 16721, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 1149, 198, 9764, 579, 1923, 1716, 13, 198, 17848, 25, 47815, 53, 4823, 3580, 24505, 41, 23683, 2990, 457, 69516, 24505, 47815, 762, 30264, 53, 9690, 53, 46427, 47815, 12117, 13, 198, 4741, 13623, 25, 3067, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 13, 198, 33807, 8798, 13542, 13, 198, 760, 6206, 369, 478, 24, 364, 86298, 13, 198, 760, 1622, 16561, 310, 328, 58387, 1132, 279, 60839, 1414, 430, 678, 1534, 4087, 1149, 198, 3925, 25, 361, 8944, 29, 16721, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 3786, 8944, 29, 198, 3840, 59429, 25, 561, 3889, 682, 264, 4031, 506, 279, 809, 13, 353, 668, 2830, 424, 13, 198, 18770, 4087, 8798, 29208, 13, 198, 40, 668, 7936, 279, 2468, 13, 198, 2324, 7123, 23185, 13, 198, 1138, 25, 3067, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 13, 198, 16936, 13, 695, 40, 668, 1066, 3443, 424, 12083, 13, 198, 27, 8944, 29, 16721, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 3786, 8944, 29, 198, 13784, 11, 12910, 328, 2301, 3580, 24505, 41, 1, 1411, 24932, 4180, 13, 351, 7, 17, 7030, 24, 66522, 7, 16, 16, 553, 436, 7, 16, 16, 7030, 24, 28, 52, 7, 17, 15, 11504, 13428, 13, 198, 9764, 579, 1347, 43924, 15060, 13, 198, 51, 7, 16, 24, 8, 478, 220, 24, 283, 220, 17, 23, 1411, 220, 17, 318, 34, 553, 38643, 13, 198, 49, 7, 16, 22, 8, 478, 220, 24, 283, 220, 17, 21, 1411, 220, 15, 318, 32, 553, 38643, 13, 198, 34, 7, 17, 8, 478, 220, 24, 283, 220, 16, 16, 318, 43, 553, 38643, 13, 198, 34, 7, 17, 8, 478, 220, 24, 283, 220, 16, 16, 318, 43, 553, 38643, 13, 198, 53, 7, 17, 16, 8, 478, 220, 24, 283, 220, 18, 15, 1411, 220, 19, 318, 36, 553, 38643, 13, 198, 40, 7, 23, 8, 478, 220, 24, 283, 220, 16, 22, 318, 49, 553, 38643, 13, 198, 41, 7, 24, 8, 478, 220, 24, 283, 220, 16, 23, 318, 50, 553, 38643, 13, 198, 4272, 4823, 3580, 24505, 41, 1411, 24932, 4180, 13, 7179, 13, 271, 3840, 457, 69516, 24505, 1411, 27025, 38050, 13, 198, 54, 7, 17, 17, 7030, 24, 28, 18, 16, 397, 20, 7584, 553, 198, 53, 7, 17, 16, 7030, 24, 28, 18, 15, 397, 19, 10406, 553, 198, 45, 7, 16, 18, 7030, 24, 28, 17, 17, 13628, 553, 198, 53, 7, 17, 16, 7030, 24, 28, 18, 15, 397, 19, 10406, 553, 198, 40, 7, 23, 7030, 24, 28, 16, 22, 2697, 553, 198, 11091, 38050, 13, 7179, 13, 271, 3840, 47815, 762, 1411, 86909, 13, 198, 42, 7, 16, 15, 7030, 24, 28, 16, 24, 4004, 553, 198, 56, 7, 17, 19, 7030, 24, 28, 18, 18, 397, 22, 10578, 553, 198, 49, 7, 16, 22, 7030, 24, 28, 17, 21, 397, 15, 4205, 553, 198, 36, 7, 19, 7030, 24, 28, 16, 18, 7947, 553, 198, 3496, 1058, 13, 7179, 13, 271, 3840, 30264, 53, 9690, 53, 1411, 34379, 2662, 4343, 13, 198, 42, 397, 51, 198, 45, 397, 54, 198, 53, 397, 36, 198, 34, 397, 43, 198, 44, 7, 16, 17, 7030, 24, 28, 17, 16, 12054, 553, 198, 53, 397, 36, 13, 198, 45498, 2662, 4343, 13, 7179, 13, 271, 3840, 46427, 1411, 2913, 13, 198, 37, 7, 20, 7030, 24, 28, 16, 19, 18657, 553, 198, 54, 7, 17, 17, 7030, 24, 28, 18, 16, 397, 20, 7584, 553, 198, 12126, 13, 7179, 13, 271, 3840, 47815, 12117, 1411, 71198, 13, 198, 42, 397, 51, 198, 56, 397, 39, 198, 53, 397, 36, 198, 35, 7, 18, 7030, 24, 28, 16, 17, 3088, 553, 198, 16721, 44, 13, 7179, 13, 271, 33807, 369, 6235, 13, 198, 4935, 8798, 6681, 13, 198, 81427, 286, 13, 695, 3925, 25, 361, 8944, 29, 16721, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 3786, 8944, 29, 198, 16936, 13, 695, 3833, 8652, 1406, 25, 6807, 51852, 272, 79243, 513, 2574, 440, 6206, 471, 24, 364, 24656, 11, 748, 86298, 369, 478, 24, 13, 353, 1429, 478, 24, 13, 1049, 4138, 13, 198, 18770, 21134, 369, 5354, 13, 695, 2665, 4799, 1414, 13, 4450, 279, 2483, 3443, 13, 198, 40, 668, 6707, 279, 1965, 13, 695, 11748, 7415, 269, 29377, 14, 60239, 2261, 3272, 25, 695, 3742, 1017, 866, 69330, 30, 2233, 13, 695, 760, 16629, 369, 264, 3750, 12278, 466, 11316, 13, 328, 760, 79077, 998, 16253, 1056, 28279, 314, 1070, 1149, 35649, 5357, 13, 198, 18622, 13, 695, 4935, 8798, 1622, 13, 198, 81427, 82, 13, 695, 27, 8944, 29, 16721, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 3786, 8944, 29, 198, 16936, 13, 695, 4935, 22738, 13, 695, 58, 18770, 4109, 60, 198, 2425, 25, 47815, 53, 4823, 3580, 24505, 41, 23683, 2990, 457, 69516, 24505, 47815, 762, 30264, 53, 9690, 53, 46427, 47815, 12117, 13, 198, 4741, 13623, 25, 3067, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 13, 198, 41058, 13542, 13, 14606, 4252, 13, 198, 16936, 13, 695, 81427, 82, 13, 695, 58, 4935, 22738, 60, 1411, 361, 8944, 29, 16721, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 3786, 8944, 29, 198, 2324, 1603, 13, 695, 69570, 7415, 269, 29377, 14, 8839, 2261, 22188, 25, 353, 1220, 5790, 6511, 1954, 8798, 13, 561, 9640, 5533, 660, 18009, 11, 748, 279, 4087, 1220, 381, 660, 18009, 430, 60839, 13, 353, 599, 424, 303, 660, 18009, 13, 7441, 5946, 198, 18770, 21134, 22738, 13, 695, 27, 8944, 29, 16721, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 3786, 8944, 29, 198, 16936, 13, 695, 58, 81427, 82, 60, 198, 248069, 271, 27, 8944, 29, 16721, 24932, 4180, 457, 40775, 27025, 38050, 86909, 34379, 2662, 4343, 2913, 71198, 3786, 8944, 29, 248046], "teacher_logprobs": [-2.5791499614715576, -0.002402041107416153, -5.447716102935374e-05, -0.757690966129303, -4.291525328881107e-06, -5.376194530981593e-05, -0.0031794970855116844, -0.0029870914295315742, -0.0002964295563288033, -0.0924551859498024, -1.8048436641693115, -0.252271443605423, -0.07216344773769379, -0.5286787748336792, -0.006503488402813673, -0.00044109628652222455, -0.3588781952857971, -4.0411134250462055e-05, -1.6689286894688848e-06, -2.276871418871451e-05, -1.5497195136049413e-06, -5.960446742392378e-06, -1.2397689715726301e-05, -0.00024959311122074723, -8.451581379631534e-05, -2.13382354559144e-05, -1.1920858014491387e-05, -3.4689302992774174e-05, -1.764281842042692e-05, -6.592056161025539e-05, -2.992108420585282e-05, -1.3947389561508317e-05, -2.7656173188006505e-05, -9.059865078597795e-06, -0.00012182447244413197, -1.9550132492440753e-05, -2.8490614567999728e-05, -1.3529539108276367, -2.9298923015594482, -0.5277044773101807, -0.0012306743301451206, -2.7418097943154862e-06, -0.6567514538764954, -0.0001267114421352744, -0.02779686078429222, -1.63224458694458, -1.7343966960906982, -1.729701042175293, -0.00044741155579686165, -1.9048876762390137, -0.826718270778656, -2.8655309677124023, -2.6216323375701904, -0.9771146774291992, -0.3983566462993622, -0.00029297350556589663, -1.0015573501586914, -0.8776799440383911, -2.675971508026123, -0.2825886011123657, -0.271281361579895, -0.09588278084993362, -1.3962286710739136, -0.03867778182029724, -0.8617801070213318, -1.135056734085083, -0.6516289114952087, -1.1162855625152588, -1.4274810552597046, -0.4360247254371643, -1.2187049388885498, -0.04391827806830406, -0.00036161558819003403, -0.9580151438713074, -0.48256978392601013, -1.0917458534240723, -0.018382074311375618, -0.08993929624557495, -1.726210355758667, -0.0010880271438509226, -0.0155399851500988, -0.022843310609459877, -0.9670813083648682, -0.001328896265476942, -0.6335386633872986, -0.00012587709352374077, -0.12930849194526672, -0.09783868491649628, -0.039718251675367355, -0.17985370755195618, -0.00011550712952157483, -0.003290478838607669, -1.5144124031066895, -1.490105023549404e-05, -0.5986308455467224, -4.026268005371094, -0.05311135947704315, -0.004830239340662956, -2.145764938177308e-06, -1.5020257706055418e-05, -0.44824016094207764, -8.583032467868179e-06, -4.625213477993384e-05, -1.9137834310531616, -0.0008300673216581345, -0.0013622299302369356, -0.16613903641700745, -0.008496446534991264, -0.6429111361503601, -0.8584531545639038, -2.2203614711761475, -0.026551224291324615, -0.00035279724397696555, -0.31391608715057373, -0.5240828990936279, -0.07012517750263214, -0.04936159402132034, -4.482168878894299e-05, -2.3841855067985307e-07, -2.50339189733495e-06, -1.680836794548668e-05, -0.00038509105797857046, -0.0009183242218568921, -0.2257736623287201, -0.12016374617815018, -0.7688398361206055, -0.029843216761946678, -0.0007487116381525993, -0.34015190601348877, -0.6834851503372192, -0.012094496749341488, -0.0006199345225468278, -0.0008348317351192236, -2.3007127310847864e-05, -0.00022766382608097047, -0.0016232660273090005, -6.6756979322235566e-06, -1.3470558769768104e-05, -0.0010143141262233257, -0.023335358127951622, -0.11466442793607712, -1.6927575416048057e-05, -3.576272320060525e-06, -3.099436753473128e-06, -0.0009291622554883361, -0.0699644610285759, -0.0003743662964552641, -0.05839008092880249, -0.0029122107662260532, -0.00021324267436284572, -0.3131852149963379, -0.0006082591717131436, -0.007884452119469643, -4.672895011026412e-05, 0.0, -7.510157047363464e-06, -1.9073468138230965e-06, -0.036846283823251724, -9.083335316972807e-05, -0.2255440205335617, -4.23184028477408e-05, -0.229362353682518, 0.0, -1.9907753085135482e-05, -1.2159273865108844e-05, -2.3841855067985307e-07, -1.3947389561508317e-05, -6.139089964563027e-05, -0.2170025259256363, -4.5298504119273275e-05, -3.707340147229843e-05, -0.3467654883861542, -0.0001879753835964948, -1.9550132492440753e-05, -9.011816291604191e-05, -2.861018856492592e-06, -1.7762025890988298e-05, -4.768370445162873e-07, -5.245195097813848e-06, -2.3841855067985307e-07, -4.768370445162873e-07, -7.867782187531702e-06, -0.0003195490571670234, -0.003017992712557316, -0.0019433676498010755, -4.756337511935271e-05, -1.1920922133867862e-06, -2.3841855067985307e-07, -8.284702198579907e-05, -8.344646857949556e-07, -2.7418097943154862e-06, -5.829164365422912e-05, -1.7881377516459906e-06, -7.152531907195225e-06, -1.9311717551317997e-05, -7.152555099310121e-07, -0.00038342276820912957, -1.7881377516459906e-06, -2.3841855067985307e-07, -1.7046782886609435e-05, -3.015949550899677e-05, -2.1934269170742482e-05, -3.576278118089249e-07, -0.0007790867821313441, -5.483612312673358e-06, -1.7881377516459906e-06, -1.537788011773955e-05, -3.814689989667386e-06, -1.5497195136049413e-06, -1.1920928244535389e-07, -2.50339189733495e-06, -4.851700214203447e-05, -5.2569914259947836e-05, -1.0013530300057027e-05, -1.6689286894688848e-06, -1.6689286894688848e-06, -2.264974000354414e-06, -0.000447530735982582, -0.0020400919020175934, -0.00582840945571661, -0.018562881276011467, -6.385342597961426, -0.4358554482460022, -0.030347790569067, -1.2462419271469116, -0.0022452878765761852, -0.027466604486107826, -0.0011200590524822474, -2.4689314365386963, -1.5592904090881348, -0.0008070787298493087, -2.0182082653045654, -0.14388395845890045, -0.31592556834220886, -0.38634192943573, -0.6144494414329529, -0.00015507926582358778, -0.42917224764823914, -0.25563666224479675, -0.0013584204716607928, -0.02907012775540352, -1.2229485511779785, -0.00028200942324474454, -4.703715801239014, -0.007122719660401344, -2.808636426925659, -1.7436171770095825, -1.3311290740966797, -1.996919870376587, -2.490222930908203, -1.1587121486663818, -0.013991953805088997, -6.198863957251888e-06, -2.622600959512056e-06, -0.17985349893569946, -1.9073468138230965e-06, -0.02710932493209839, -0.20635929703712463, -8.940656698541716e-06, -0.0010457528987899423, -0.10028757154941559, -0.0003833036171272397, -2.9802276912960224e-06, -7.152555099310121e-07, -0.012526053003966808, -1.801326870918274, -0.2548603415489197, -0.5214228630065918, -1.5905752182006836, -0.00016223068814724684, -0.42873069643974304, -0.0033205393701791763, -0.09125614166259766, -0.00288380216807127, -6.770858453819528e-05, -1.680836794548668e-05, -5.364403477869928e-06, -0.14313627779483795, -0.5960735082626343, -0.023632168769836426, -0.07060417532920837, -0.018161432817578316, -0.0043462105095386505, -0.004669594578444958, -0.1269359290599823, -1.9073468138230965e-06, -6.985420623095706e-05, -3.313963316031732e-05, -9.07141511561349e-05, -0.17585228383541107, -1.4066597032069694e-05, -0.038054727017879486, -0.0002650863316375762, -0.00026603974401950836, -0.0019610952585935593, -0.24270331859588623, -0.7465909719467163, -2.0275566577911377, -0.0182794276624918, -0.00029488030122593045, -0.0385371670126915, -0.1173480674624443, -2.916712760925293, -0.05098637938499451, -1.391657829284668, -1.777395248413086, -2.3841830625315197e-06, -2.8038365840911865, -0.005613515619188547, -0.09073425084352493, -1.5075381994247437, -0.16947731375694275, -0.7130817174911499, -8.11782301752828e-05, -0.5168745517730713, -0.5235897302627563, -0.13762402534484863, -0.5250243544578552, -0.0027523275930434465, -0.011296008713543415, -0.00015615197480656207, -9.178694017464295e-05, -0.000691770575940609, -0.00011538793478393927, -5.6622808187967166e-05, -2.682172998902388e-05, -5.411955135059543e-05, -0.004503465257585049, -1.1194957494735718, -0.07557597011327744, -0.005328618921339512, -0.10554336756467819, -0.0023538758978247643, -0.4779352843761444, -0.5781927704811096, -7.045020902296528e-05, -0.07625296711921692, -0.00022194306075107306, -0.0040803044103085995, -1.2278481335670222e-05, -4.303362584323622e-05, -1.140059471130371, -0.01948723942041397, -0.0002549561613705009, -0.2481199949979782, -0.09254854172468185, -0.00019929806876461953, -1.1444026313256472e-05, -0.0047102924436330795, -0.01816798932850361, -0.0002489972102921456, -8.248942322097719e-05, -1.5020257706055418e-05, -5.411955135059543e-05, -2.7418097943154862e-06, -3.576272320060525e-06, -1.07287787614041e-05, -0.002131572924554348, -0.010031994432210922, -0.09629210829734802, -2.016974925994873, -1.8880319595336914, -3.6698813438415527, -0.5145117044448853, -0.18422316014766693, -0.00015496007108595222, -1.0847986231965479e-05, -0.20142395794391632, -2.4023702144622803, -0.16048933565616608, -0.00010954733443213627, -3.6000557884108275e-05, -2.4409704208374023, -0.14114618301391602, -0.011050808243453503, -6.437280717364047e-06, -0.06911499798297882, -0.6792758703231812, -3.2186455882765586e-06, -6.9141146923357155e-06, -0.15656977891921997, -0.10185704380273819, -5.936446541454643e-05, -8.344646857949556e-07, -7.510157047363464e-06, -0.02302517741918564, -0.07978114485740662, -4.0649541915627196e-05, -1.4543427823809907e-05, -4.410734163684538e-06, -0.0005401106318458915, -0.005929261911660433, -6.437280717364047e-06, -3.814689989667386e-06, -0.05333233252167702, -0.05837433785200119, -0.007643263787031174, -0.9722810387611389, -0.0007468056865036488, -0.0013769917422905564, -0.6999127268791199, -1.680836794548668e-05, -0.002209128811955452, -0.0007613382767885923, -0.00016437610611319542, -0.0026183391455560923, -0.02641725167632103, -0.00039664984797127545, -7.533743337262422e-05, -1.2993727978027891e-05, -2.861018856492592e-06, -6.556489552167477e-06, -1.6331539882230572e-05, -0.4791807532310486, -0.0050953528843820095, -0.14364711940288544, -0.004554850049316883, -0.00047672350774519145, -3.814689989667386e-06, -4.9232225137529895e-05, -0.0014317509485408664, -0.0001287377526750788, -3.71926071238704e-05, -0.0003381395654287189, -0.004843763541430235, -0.0004972175229340792, -1.1920858014491387e-05, -0.0002632986579556018, -0.001508170971646905, -7.068861305015162e-05, -2.7418097943154862e-06, -7.748303323751315e-05, -4.768370445162873e-07, -9.536738616588991e-07, -8.702239938429557e-06, -0.0017472729086875916, -0.00014935807848814875, -8.272782724816352e-05, -0.000760385300964117, -9.16677454370074e-05, -1.6569954823353328e-05, -0.0008594871615059674, -2.2172682292875834e-05, -3.0278701160568744e-05, -7.152555099310121e-07, -2.1457441107486375e-05, -4.768370445162873e-07, -2.264974000354414e-06, -2.264974000354414e-06, -4.172316494077677e-06, -0.0010570659069344401, -2.4437606043647975e-05, -0.002586117247119546, -0.44773226976394653, -0.2897240221500397, -0.055335380136966705, -1.9073468138230965e-06, -2.1219027985353023e-05, -4.1483970562694594e-05, -9.179073458653875e-06, -0.012962954118847847, -0.00024768622824922204, -0.13118216395378113, -0.012705803848803043, -0.7871478199958801, -2.624401092529297, -0.9915730953216553, -3.323557138442993, -0.005637934897094965, -0.0005018643569201231, -0.00011598391574807465, -0.3646616041660309, -0.22663210332393646, -0.9244511127471924, -0.10342159122228622, -6.48477507638745e-05, -8.4638240878121e-06, -0.0003000047872774303, -3.123234637314454e-05, -2.264974000354414e-06, -0.0381123311817646, -0.8116331696510315, -0.520330011844635, -1.2248482704162598, -0.16095587611198425, -6.592056161025539e-05, -5.030505417380482e-05, -0.0016920547932386398, -0.3488501012325287, -9.083335316972807e-05, -0.07585424929857254, -0.0019490785198286176, -0.011099264025688171, -2.95634672511369e-05, -8.821098163025454e-05, -4.6491513785440475e-06, -0.0001454247540095821, -1.0847986231965479e-05, -2.0265558760002023e-06, -1.4424220353248529e-05, -0.3874960243701935, -0.001134824356995523, -6.496695277746767e-05, -0.0008492438937537372, -3.862306402879767e-05, -7.748573807475623e-06, -1.0132738680113107e-05, -0.1431533247232437, -0.00014220656885299832, -1.6927575416048057e-05, -2.586808113846928e-05, -0.0006617502076551318, -0.001259844284504652, -4.768370445162873e-07, -0.0031890033278614283, -0.0028276958037167788, -0.7188410758972168, -0.06990154832601547, -9.655486064730212e-05, -2.622600959512056e-06, -4.768370445162873e-07, -0.001522930571809411, -2.372799873352051, -0.722657322883606, -0.08150574564933777, -0.802006185054779, -0.05810340866446495, -0.00176214799284935, -0.007056789472699165, -1.9073468138230965e-06, -2.3603161025675945e-05, -1.6689160474925302e-05, -2.3841574147809297e-05, -6.16293036728166e-05, -6.97350042173639e-05, -0.0015335239004343748, -8.940656698541716e-06, -0.0001746263587847352, -9.97731985989958e-05, -0.026310084387660027, -3.111314072157256e-05, -0.00014399446081370115, -7.891343557275832e-05, -6.723177648382261e-05, -2.169585604860913e-05, -6.0794889577664435e-05, -4.410734163684538e-06, -0.07225372642278671, -4.291525328881107e-06, -0.013671688735485077, -0.0069285850040614605, -0.0002329078270122409, -0.00012516192509792745, -0.0005658455775119364, -1.764281842042692e-05, -1.5497195136049413e-06, -4.768360213347478e-06, -5.8410845667822286e-05, -0.0002474478678777814, -6.079655122448457e-06, -4.8874615458771586e-05, -0.000959412835072726, -0.024645136669278145, -5.125868119648658e-05, -4.339123915997334e-05, -0.0010213402565568686, -0.0002451834443490952, -4.291525328881107e-06, -9.595887240720913e-05, -0.00016389934171456844, -0.0017744047800078988, -0.00010334911348763853, -7.533743337262422e-05, -1.311301275563892e-06, -1.4305104514278355e-06, -2.3245540432981215e-05, -0.017606280744075775, -0.006757739465683699, -0.008264036849141121, -0.024127202108502388, -0.0006792622152715921, -0.0007715824176557362, -0.00345918582752347, -7.211902266135439e-05, -4.124556289752945e-05, -0.0001823735801735893, -6.5205356804654e-05, -3.015949550899677e-05, -0.100333072245121, -0.080612912774086, -0.0001370812824461609, -5.245195097813848e-06, -0.00015937011630740017, -1.4543427823809907e-05, -6.198863957251888e-06, -1.9073468138230965e-06, -4.0649541915627196e-05, -0.0008609164506196976, -6.556489552167477e-06, -2.2291887944447808e-05, -0.0030544791370630264, -0.00021371940965764225, -1.7881377516459906e-06, -0.0005446380819194019, -1.9114257097244263, -0.00038068200228735805, -2.0265558760002023e-06, -3.933898824470816e-06, -1.0728830375228426e-06, -2.264974000354414e-06, -8.4638240878121e-06, -0.0011235122801735997, -0.0001461399078834802, -0.00012909532233607024, -0.0009912106906995177, -0.0014975772937759757, -0.0007239343249239028, -1.2755313036905136e-05, -1.0728830375228426e-06, -1.0728830375228426e-06, -2.992108420585282e-05, -0.0009352362249046564, -0.0005079409456811845, -0.00010346830822527409, -0.0021729685831815004, -0.00023767507809679955, -0.0004909025738015771, -0.0005273620481602848, -2.6464111215318553e-05, -9.65590606938349e-06, -0.00043752157944254577, -7.629365427419543e-06, -4.410734163684538e-06, -1.168244216387393e-05, -0.00035315475543029606, -0.00034397884155623615, -5.6265202147187665e-05, -1.1801649634435307e-05, -5.483612312673358e-06, -5.709961988031864e-05, -0.0031655936036258936, -1.4305104514278355e-06, -7.390948667307384e-06, -0.00024863966973498464, -5.602679812000133e-05, -4.768370445162873e-07, -2.1576648578047752e-05, -1.2278481335670222e-05, -7.86750388215296e-05, -1.2993727978027891e-05, -0.0001299296854995191, -0.0008256602450273931, -0.004779343493282795, -7.152555099310121e-07, -0.0005046047735959291, -0.008647141046822071, -5.149708886165172e-05, -1.2993727978027891e-05, -1.7881377516459906e-06, -1.5497195136049413e-06, -4.768370445162873e-07, -1.7881377516459906e-06, -8.583032467868179e-06, -0.00013350549852475524, -2.6464111215318553e-05, -4.768360213347478e-06, -0.0005067494930699468, -0.00030655929003842175, -1.0728830375228426e-06, -1.0371154530730564e-05, -2.074220174108632e-05, -4.327203714638017e-05, -2.7418097943154862e-06, -7.629365427419543e-06, -3.2186455882765586e-06, -3.576278118089249e-07, -4.768370445162873e-07, -2.5033637939486653e-05, -1.597391747054644e-05, -0.033873047679662704, -0.10858431458473206, -0.000393432448618114, -0.0003094194398727268, -0.0033422820270061493, -5.9126061387360096e-05, -0.00024196557933464646, -0.00017152745567727834, -3.302042750874534e-05, -3.611976353568025e-05, -9.440929716220126e-05, -2.5033637939486653e-05, -0.000604565953835845, -7.676783570786938e-05, -3.576272320060525e-06, -0.00040797016117721796, -0.020553812384605408, -5.2927523938706145e-05, -1.6689286894688848e-06, -5.960446742392378e-06, -7.152555099310121e-07, -1.9073468138230965e-06, -3.2543604902457446e-05, -9.595887240720913e-05, -9.154854342341423e-05, -8.22540732769994e-06, -0.0003084660565946251, -8.928377064876258e-05, -7.152555099310121e-07, -2.372236667724792e-05, -0.08161550760269165, -0.0003685271949507296, -5.447716102935374e-05, -3.6954811548639555e-06, -1.311301275563892e-06, -1.5616295058862306e-05, -1.6689160474925302e-05, -4.911301948595792e-05, -0.0008542464347556233, -1.5020257706055418e-05, -8.439661905867979e-05, -0.0012224590172991157, -9.536738616588991e-07, -9.179073458653875e-06, -4.029192859889008e-05, -1.9788545614574105e-05, -2.4640676975250244, -0.0005628670332953334, -6.758938252460212e-05, -1.537788011773955e-05, -2.3841830625315197e-06, -3.576272320060525e-06, -7.83174327807501e-05, -3.504691630951129e-05, -2.3841830625315197e-06, -5.483612312673358e-06, -0.00012396997772157192, -2.3841830625315197e-06, -3.242440288886428e-05, -3.504691630951129e-05, -0.0002351722796447575, -3.540453326422721e-05, -0.00130425242241472, -0.0009288049186579883, -7.903263758635148e-05, -2.622600959512056e-06, -3.3378546504536644e-06, -1.5497195136049413e-06, -2.0265558760002023e-06, -2.3245540432981215e-05, -1.9192511899746023e-05, -4.529942543740617e-06, -0.0007631250191479921, -0.00014780859055463225, -0.0008025524439290166, -0.0001311216183239594, -1.9907753085135482e-05, -0.11287686973810196, -7.152555099310121e-07, -1.311301275563892e-06, -0.002945372834801674, -0.011185086332261562, -0.04441557452082634, -0.212965726852417, -0.001081834896467626, -0.006758449599146843, -0.7205831408500671, -0.028115415945649147, -0.008934756740927696, -2.145764938177308e-06, -0.003967866767197847, -9.142934868577868e-05, -0.00047851080307736993, -3.4450891689630225e-05, -6.425174069590867e-05, -0.0006302992696873844, -6.770858453819528e-05, -0.00017796363681554794, -0.00015698630886618048, -0.011347398161888123, -1.811964830267243e-05, -9.059865078597795e-06, -0.00010525626566959545, -0.0018047487828880548, -0.00020704510097857565, -0.22674299776554108, -0.5641514658927917, -0.02058476023375988, -0.0315408930182457, -0.00010430268594063818, -0.4652327001094818, -0.0002618685248307884, -1.8268307447433472, -1.2874520507466514e-05, -0.0038468660786747932, -3.576278118089249e-07, -9.65590606938349e-06, -7.152555099310121e-07, -1.1920922133867862e-06, -7.152555099310121e-07, -1.4781842764932662e-05, -5.125868119648658e-05, -1.4543427823809907e-05, -3.4570634852570947e-06, -3.0397906812140718e-05, -1.6212332411669195e-05, -7.390948667307384e-06, -2.3245540432981215e-05, -8.22540732769994e-06, -1.0728830375228426e-06, 0.0, -4.494089080253616e-05, -5.8412379075889476e-06, -7.390948667307384e-06, -0.0015095992712303996, -7.319182623177767e-05, -0.3276565968990326, -0.027615511789917946, -0.0016824151389300823, -0.0004189328756183386, -0.00021717573690693825, -3.2186455882765586e-06, -2.0265558760002023e-06, -1.5497195136049413e-06, -4.529942543740617e-06, -2.9802276912960224e-06, -1.1324817933200393e-05, -5.006777428206988e-06, -2.7418097943154862e-06, -6.079655122448457e-06, -6.6756979322235566e-06, -7.164221460698172e-05, -9.119095193454996e-05, -0.1427622139453888, -1.5863138437271118, -1.0196928977966309, -0.013880272395908833, -0.0003960540343541652, -0.19132672250270844, -0.11223135888576508, -1.2728166580200195, -0.6103836297988892, -0.5427530407905579, -0.0017683359328657389, -1.5488100051879883, -0.029938898980617523, -0.5864420533180237, -0.046327631920576096, -1.2932238578796387, -2.796316623687744, -0.14132969081401825, -0.47532421350479126, -1.910573124885559, -0.0035188202746212482, -2.3841855067985307e-07, -4.768370445162873e-07, -6.6756979322235566e-06, -1.1920922133867862e-06, -2.3841855067985307e-07, -8.725739462533966e-05, -9.65590606938349e-06, -5.125986263010418e-06, -0.0001532914029667154, -0.12219452112913132, -0.5929673910140991, -0.002460430609062314, -0.0957324355840683, -3.8980677345534787e-05, -0.007256579585373402, -0.07037687301635742, -5.543078441405669e-05, -9.179073458653875e-06, -2.264974000354414e-06, -5.960462772236497e-07, -1.1920922133867862e-06, -3.4570634852570947e-06, -1.883488948806189e-05, -3.4570634852570947e-06, -2.0265558760002023e-06, -5.364403477869928e-06, -7.152531907195225e-06, -1.9788545614574105e-05, -0.07991354912519455, -5.507317473529838e-05, -5.483612312673358e-06, -0.018453113734722137, -0.7292721271514893, -0.8890070915222168, -0.03048771247267723, -0.5180079936981201, -0.010436010546982288, -0.04939086735248566, -0.00023362292267847806, -0.00020930961181875318, -0.004205906298011541, -3.576278118089249e-07, -2.3841855067985307e-07, -0.010529793798923492, -0.10702258348464966, -0.04792996495962143, -0.062345027923583984, -6.198863957251888e-06, -7.033323527139146e-06, -0.000740968796890229, -0.8366580009460449, -1.5282890796661377, -0.10004599392414093, -3.891618251800537, -4.768370445162873e-07, -0.0009085579658858478, -1.1027021408081055, -0.014659079723060131, -0.6467176079750061, -0.04083935171365738, -0.05595599487423897, -1.9220900535583496, -0.25513434410095215, -1.7736879587173462, -3.158996332786046e-05, -0.016911230981349945, -3.659658250398934e-05, -0.029409462586045265, -3.299083709716797, -4.625213477993384e-05, -0.46646374464035034, -2.3245540432981215e-05, -2.50339189733495e-06, -2.622600959512056e-06, -2.3841855067985307e-07, -3.576278118089249e-07, -3.6954811548639555e-06, -1.07287787614041e-05, -2.3841830625315197e-06, -1.9073468138230965e-06, -3.814689989667386e-06, -2.50339189733495e-06, -1.823885577323381e-05, -5.411955135059543e-05, -1.2397689715726301e-05, -0.32353833317756653, -9.536738616588991e-07, -0.000567275274079293, -2.1039202213287354, -0.2519761919975281, -3.0512561798095703, -0.5387212038040161, -0.00019441144831944257, -0.9332094192504883, -0.0008467426523566246, -0.00031609306461177766, -0.7732438445091248, -2.5152843591058627e-05, -9.727005090098828e-05, -2.634490556374658e-05, -6.198863957251888e-06, -9.65590606938349e-06, -1.311301275563892e-06, -7.152555099310121e-07, -1.311301275563892e-06, -2.0265558760002023e-06, -7.271740287251305e-06, -3.933898824470816e-06, -4.768360213347478e-06, -3.814689989667386e-06, -2.7418097943154862e-06, -4.6491513785440475e-06, -0.0038091023452579975, -3.659658250398934e-05, -1.4305104514278355e-06, -3.361645576660521e-05, -0.48262518644332886, -3.576278118089249e-07, -2.4579036235809326, -2.8642477989196777, -0.4462651312351227, -1.7881377516459906e-06, -9.536738616588991e-07, -3.576278118089249e-07, -0.013781982474029064, -0.044201288372278214, -0.5581978559494019, -0.0002848696312867105, -0.20516511797904968, -0.51607346534729, -1.0211918354034424, -0.0031251898035407066, -0.9259043335914612, -0.05409931764006615, -1.1404032707214355, -0.040117207914590836, -0.00011991735664196312, -0.000560245884116739, -1.2950862646102905, -1.4946566820144653, -0.03728199005126953, -9.095255518332124e-05, -9.285972191719338e-05, -0.01726561039686203, -0.002985308412462473, -0.12004820257425308, -0.06866355985403061, -0.04913939908146858, -0.0002649671514518559, -1.6331539882230572e-05, -0.06382406502962112, -0.2520279884338379, -0.0704503133893013, -0.018384765833616257, -0.02291112206876278, -1.0728830375228426e-06, -0.5175305008888245, -0.0009557208395563066, -0.23353883624076843, -0.07547836750745773, -0.023878494277596474, -1.8412061929702759, -0.446209192276001, -0.0006980843027122319, -7.986990567587782e-06, -0.008341945707798004, -1.4257054328918457, -0.019251639023423195, -0.0068863206543028355, -0.00016139635408762842, -0.0003466005437076092, -0.0013816345017403364, -0.014178258366882801, -0.0022955990862101316, -0.02136303298175335, -5.376194530981593e-05, -0.9897555112838745, -0.004667577799409628, -0.3493070900440216, -0.008304115384817123, -3.2543604902457446e-05, -0.0012051946250721812, -0.0007383481133729219, -1.0728830375228426e-06, -1.811964830267243e-05, -0.0005380851216614246, -1.5497195136049413e-06, -5.8412379075889476e-06, -2.861018856492592e-06, -0.00011169286881340668, -2.3841855067985307e-07, 0.0, -2.861018856492592e-06, -2.3841830625315197e-06, -1.1920922133867862e-06, -5.054346183896996e-05, -0.00036197309964336455, -8.868777513271198e-05, -7.724463648628443e-05, -4.994744449504651e-05, -2.2411095415009186e-05, -1.5497195136049413e-06, -0.00011967896716669202, -1.3112935448589269e-05, -6.794906312279636e-06, -4.5060096454108134e-05, -4.410734163684538e-06, -2.634490556374658e-05, -6.437280717364047e-06, -5.876845170860179e-05, -2.3841830625315197e-06, -1.4305104514278355e-06, -2.3841830625315197e-06, -5.483612312673358e-06, -0.01603798381984234, -5.447716102935374e-05, -1.4305104514278355e-06, -3.015949550899677e-05, -9.536738616588991e-07, -7.438383181579411e-05, -0.0006429274217225611, -0.0010562323732301593, -0.0014485353603959084, -5.006777428206988e-06, -0.0010368215152993798, -5.006777428206988e-06, -3.671578815556131e-05, -2.3841855067985307e-07, -1.490105023549404e-05, -3.9934315282152966e-05, -2.634490556374658e-05, -3.933898824470816e-06, -9.417489309271332e-06, -2.264974000354414e-06, -4.3987260141875595e-05, -3.576278118089249e-07, -9.417489309271332e-06, -8.34461570775602e-06, -2.0265558760002023e-06, -4.6967357775429264e-05, -0.00033623288618400693, -0.005560644436627626, -0.008957440964877605, -5.125986263010418e-06, -6.639736966462806e-05, -7.986990567587782e-06, -3.576278118089249e-07, -1.0967194612021558e-05, -8.344646857949556e-07, -3.576278118089249e-07, -2.3841855067985307e-07, -0.0002814135223161429, -3.981510963058099e-05, -3.862306402879767e-05, -0.0042847259901463985, -0.052113596349954605, -0.0002165798214264214, -0.3229384124279022, -2.9682672902708873e-05, -2.0861407392658293e-05, -1.1324817933200393e-05, -4.768370445162873e-07, -1.2993727978027891e-05, -1.9073468138230965e-06, -1.1444026313256472e-05, -3.576278118089249e-07, -1.5497195136049413e-06, -9.536738616588991e-07, -3.2186455882765586e-06, -1.6689286894688848e-06, -5.483612312673358e-06, -2.0265558760002023e-06, -6.913899414939806e-05, -9.536738616588991e-07, -0.001213052892126143, -7.986990567587782e-06, -5.960462772236497e-07, -2.3483953555114567e-05, -5.960462772236497e-07, -7.510157047363464e-06, -1.7881377516459906e-06, -3.361645576660521e-05, -2.3841855067985307e-07, -1.9073468138230965e-06, -8.583032467868179e-06, -1.9073468138230965e-06, -2.0265558760002023e-06, -8.344646857949556e-07, -7.152555099310121e-07, -3.576272320060525e-06, -1.1920922133867862e-06, -0.0009583409409970045, -0.5611003637313843, -0.1278763711452484, -2.9802276912960224e-06, -2.50339189733495e-06, -5.960462772236497e-07, -0.015897786244750023, -0.007696734741330147, -0.00017081231635529548, -0.0013390155509114265, -2.0231683254241943, -0.0019301610300317407, -1.0538363456726074, -1.173594355583191, -1.1519395112991333, -0.00019739109848160297, -0.0007139279623515904, -0.225557342171669, -8.666139910928905e-05, -3.0874729418428615e-05, -0.3143039345741272, -4.0411134250462055e-05, -1.3232143828645349e-05, -5.125986263010418e-06, -2.50339189733495e-06, -8.511180931236595e-05, -0.8259544372558594, -1.549708758830093e-05, -0.10021714866161346, -0.07899748533964157, -5.483612312673358e-06, -0.007494193036109209, -0.00041952868923544884, -0.002915657591074705, -0.029771246016025543, -2.859273672103882, -1.168244216387393e-05, -5.304672595229931e-05, -2.622600959512056e-06, -1.764281842042692e-05, -2.2053474822314456e-05, -2.9802276912960224e-06, -4.6132929128361866e-05, -0.03815146163105965, -1.4543427823809907e-05, -7.152555099310121e-07, -6.508615479106084e-05, -1.7881377516459906e-06, -2.169585604860913e-05, -2.729855441430118e-05, -2.6464111215318553e-05, -1.9073468138230965e-06, -1.3708974620385561e-05, -9.894321920000948e-06, -5.960462772236497e-07, -1.0728830375228426e-06, -3.576278118089249e-07, -4.768370445162873e-07, -3.0874729418428615e-05, -0.011087710037827492, -0.0022818022407591343, -0.06515104323625565, -9.238292841473594e-05, -0.004787529818713665, -3.0397906812140718e-05, -1.8000440832111053e-05, -0.000341476290486753, -2.7418097943154862e-06, -0.00045694399159401655, -0.020754894241690636, -1.537788011773955e-05, -4.768370445162873e-07, -6.6756979322235566e-06, -1.0728830375228426e-06, -4.6491513785440475e-06, -2.5987286790041253e-05, -2.145764938177308e-06, -1.1920928244535389e-07, -9.65590606938349e-06, -2.0265558760002023e-06, -4.0531076592742465e-06, -4.768370445162873e-07, -3.814689989667386e-06, -3.194758028257638e-05, -3.0278701160568744e-05, -7.510157047363464e-06, -0.011164574883878231, -0.05278121680021286, -0.0004323977918829769, -0.00014447122521232814, -0.5760621428489685, -7.152555099310121e-07, -1.1801649634435307e-05, -0.011522175744175911, -0.9768891334533691, -0.00672647962346673, -2.372236667724792e-05, -0.0004188137245364487, -0.00038378025055862963, -1.3589766240329482e-05, -0.0011208925861865282, -0.053070321679115295, -2.0146166207268834e-05, -1.0013530300057027e-05, -0.026311825960874557, -1.811964830267243e-05, -4.95898348162882e-05, -6.556489552167477e-06, -3.814689989667386e-06, -8.821448318485636e-06, -0.029754118993878365, -0.00013398226292338222, -2.145764938177308e-06, -8.4638240878121e-06, -1.9073468138230965e-06, -2.145764938177308e-06, -5.471556869451888e-05, -6.05564855504781e-05, -7.510157047363464e-06, -2.861018856492592e-06, -1.0013530300057027e-05, -1.5497195136049413e-06, -0.00032217081752605736, -1.6927575416048057e-05, -0.00013314791431184858, -0.04364500194787979, -5.006777428206988e-06, -2.3841855067985307e-07, -1.6689286894688848e-06, -2.7418097943154862e-06, -7.152555099310121e-07, -1.6689286894688848e-06, -2.622600959512056e-06, -0.01252240315079689, -0.00010215714428341016, -5.364274329622276e-05, -0.0001515035255579278, -1.8954096958623268e-05, -2.658331868587993e-05, -0.0015287628630176187, -1.6331539882230572e-05, -2.0265558760002023e-06, -3.325883881188929e-05, -2.50339189733495e-06, -7.033323527139146e-06, -2.3841855067985307e-07, -3.576278118089249e-07, -7.033323527139146e-06, -7.748573807475623e-06, -7.748573807475623e-06, -3.5523738915799186e-05, -0.00021324267436284572, -0.00365922087803483, -1.1920928244535389e-07, -1.7046782886609435e-05, -2.3841855067985307e-07, -3.4570634852570947e-06, -8.821448318485636e-06, -0.001576372655108571, -4.529942543740617e-06, -1.1920928244535389e-07, 0.0, -1.0728830375228426e-06, -6.9141146923357155e-06, -1.1920922133867862e-06, -4.0531076592742465e-06, -2.8729025871143676e-05, -8.344646857949556e-07, -0.00031549722189083695, -0.5251435041427612, -3.6954811548639555e-06, -0.06737244129180908, -8.177422569133341e-05, -0.016146961599588394, -3.45700973412022e-05, -0.0006705658161081374, -4.768360213347478e-06, -1.5497195136049413e-06, -2.7418097943154862e-06, -1.7404405298293568e-05, -2.145764938177308e-06, -0.014282620511949062, -0.00010907054820563644, -0.00488706398755312, -3.099393507000059e-05, -0.0036188371013849974, -1.1801649634435307e-05, -9.417489309271332e-06, -0.0003381395654287189, -3.576272320060525e-06, -1.0728830375228426e-06, -8.344646857949556e-07, -3.099436753473128e-06, -0.005963390693068504, -1.0537735223770142, -0.012212500907480717, -0.00025650550378486514, -3.731181277544238e-05, -7.152555099310121e-07, -0.14268645644187927, -2.145764938177308e-06, -0.00020525732543319464, -9.905801562126726e-05, -1.3112935448589269e-05, -4.529942543740617e-06, -2.4676019165781327e-05, -1.1920922133867862e-06, -3.325883881188929e-05, -0.0033557077404111624, -0.00565867917612195, -3.0278701160568744e-05, -3.2186455882765586e-06, -5.221230458118953e-05, -2.777537883957848e-05, -4.0531076592742465e-06, -0.0016470688860863447, -5.364274329622276e-05, -2.169585604860913e-05, -7.152531907195225e-06, -1.1920922133867862e-06, -3.576278118089249e-07, -5.125986263010418e-06, -6.4490144723095e-05, -1.2516897186287679e-05, -3.099393507000059e-05, -3.218599158572033e-05, -0.06201859563589096, -1.7881377516459906e-06, -0.0002643712505232543, -0.003657320514321327, -2.7418097943154862e-06, -0.00011693747364915907, -0.2520673871040344, -1.7404405298293568e-05, -1.0967194612021558e-05, -8.821098163025454e-05, -2.6702524337451905e-05, -0.00035577642847783864, -0.00011562632425921038, -1.9788545614574105e-05, -1.0728830375228426e-06, -9.417489309271332e-06, -8.344646857949556e-07, -8.583032467868179e-06, -0.0001941730733960867, -2.0503786799963564e-05, -3.2543604902457446e-05, -0.004130998160690069, -0.0025678062811493874, -9.65590606938349e-06, -1.1086402082582936e-05, -0.013102740980684757, -2.6702524337451905e-05, -9.65590606938349e-06, -0.0007420408655889332, -2.50339189733495e-06, -1.6689286894688848e-06, -4.768370445162873e-07, -5.960462772236497e-07, -2.50339189733495e-06, -1.0728830375228426e-06, -8.344646857949556e-07, -6.6756979322235566e-06, -9.894321920000948e-06, -4.529942543740617e-06, -7.92710343375802e-05, -1.4305104514278355e-06, -2.145764938177308e-06, -7.152555099310121e-07, -1.0728830375228426e-06, -6.437280717364047e-06, -9.179073458653875e-06, -4.625213477993384e-05, -8.940656698541716e-06, -0.0001774868869688362, -0.00015567521040793508, -1.5497195136049413e-06, -4.124556289752945e-05, -8.344646857949556e-07, -3.7788631743751466e-05, -0.4914965331554413, -0.0004049911512993276, -1.3490195274353027, -0.07889261096715927, -0.005487616173923016, -0.5491496920585632, -0.004193560685962439, -0.9375216960906982, -5.8412379075889476e-06, -1.6331539882230572e-05, -1.6239190101623535, -1.1581493616104126, -3.576272320060525e-06, -0.0011644733604043722, -1.4229685068130493, -0.2574547231197357, -0.08035806566476822, -2.098061486321967e-05, -0.00045217780279926956, -0.0005752577562816441, -7.867782187531702e-06, -2.3841574147809297e-05, -2.264974000354414e-06, -1.1920928244535389e-07, -1.0728830375228426e-06, -1.5497195136049413e-06, -9.894321920000948e-06, -3.099436753473128e-06, -3.576278118089249e-07, -1.9073468138230965e-06, -8.344646857949556e-07, -7.390948667307384e-06, -9.226373367710039e-05, -3.242440288886428e-05, -5.960462772236497e-07, -2.9205850296420977e-05, -0.1456049382686615, -2.50339189733495e-06, -1.3828182090946939e-05, -1.3118103742599487, -0.0005092516075819731, -0.44027218222618103, -4.410734163684538e-06, -0.7500842809677124, -0.492337703704834, -1.7098311185836792, -0.0003810394846368581, -0.48416846990585327, -3.958786725997925, -0.14508973062038422, -2.1342344284057617, -0.40257853269577026, -0.16544538736343384, -2.360034942626953, -0.8989232778549194, -0.10368535667657852, -0.37283453345298767, -0.5815814733505249, -0.0007570500019937754, -1.4305104514278355e-06, -0.0031985098030418158, -0.00016962042718660086, -1.9461791515350342, -0.7015296816825867, -0.005183590110391378, -2.7656173188006505e-05, -0.4350399374961853, -0.2911001741886139, -0.07063505053520203, -0.14448481798171997, -0.0006837890832684934, -0.2754652500152588, -2.638627052307129, -2.1547696590423584, -0.0016719423001632094, -7.152555099310121e-07, -0.05202680081129074, -2.3807735443115234, -0.0027523275930434465, -0.008980006910860538, -0.009707624092698097, -0.6750497817993164, -0.0004120216181036085, -2.823215961456299, -0.05019262433052063, -7.152555099310121e-07, -0.0021891456563025713, -2.0410499572753906, -0.13799595832824707, -0.6322396397590637, -2.018585681915283, -0.02146163024008274, -1.1368807554244995, -2.610649426060263e-05, -0.9721508622169495, -1.8596476365928538e-05, -2.145764938177308e-06, -1.4305104514278355e-06, -2.7418097943154862e-06, -0.00019572250312194228, -0.0004368066438473761, -8.701899787411094e-05, -1.5139465176616795e-05, -0.1437259018421173, -0.8824120163917542, -0.025703206658363342, -5.960446742392378e-06, -0.8954668045043945, -0.04232172295451164, -2.129551649093628, -4.565611743601039e-05, -0.4496771991252899, -0.04917287826538086, -2.574681282043457, -0.5139200091362, -0.4774366021156311, -0.25643306970596313, -0.12461728602647781, -0.2245289534330368, -2.1361050605773926, -1.3573956489562988, -0.06544843316078186, -0.013143920339643955, -0.008224193006753922, -6.198863957251888e-06, -0.00027164106722921133, -9.179073458653875e-06, -0.0003833036171272397, -2.861018856492592e-06, -5.602820692729438e-06, -0.001178999780677259, -0.8494711518287659, -0.05570605769753456, -0.07672583311796188, -0.34856051206588745, -1.0117923021316528, -2.50339189733495e-06, -5.876845170860179e-05, -0.0002683041093405336, -0.03993968665599823, -0.11780804395675659, -3.302042750874534e-05, -0.22869053483009338, -0.9307283163070679, -0.004716106224805117, -1.0728830375228426e-06, -2.8132995794294402e-05, -0.2831563651561737, -0.00018416139937471598, -1.1444026313256472e-05, -6.9141146923357155e-06, -6.794906312279636e-06, -1.311301275563892e-06, -8.344646857949556e-07, -2.3841855067985307e-07, -5.960462772236497e-07, -2.0265558760002023e-06, -1.2040065485052764e-05, -1.9073468138230965e-06, -1.0728830375228426e-06, -1.7881377516459906e-06, -5.960462772236497e-07, -2.50339189733495e-06, -2.109982233378105e-05, -2.9682672902708873e-05, -5.960462772236497e-07, -0.027863292023539543, -0.022939782589673996, -1.4305104514278355e-06, -5.125986263010418e-06, -2.7246649265289307, -0.0009261847590096295, 0.0, -2.6702524337451905e-05, -0.19026733934879303, -0.5976230502128601, -7.629365427419543e-06, -0.04859357327222824, -8.797258487902582e-05, -1.268641710281372, -2.3841855067985307e-07, -6.294052582234144e-05, 0.0, -4.887569048150908e-06, -4.768370445162873e-07, -3.576278118089249e-07, 0.0, -3.099436753473128e-06, -1.4424220353248529e-05, -1.7881377516459906e-06, -9.536738616588991e-07, -1.3947389561508317e-05, -4.410734163684538e-06, -1.1920928244535389e-07, -6.6756979322235566e-06, 0.0, -6.079655122448457e-06, 0.0, -2.4437606043647975e-05, -2.7418097943154862e-06, -1.1205610462639015e-05, -0.0004316828562878072, -4.410734163684538e-06, -1.453895926475525, -0.7050499320030212, -0.00023278864682652056, -5.125986263010418e-06, -1.168244216387393e-05, -5.960462772236497e-07, -2.3841855067985307e-07, -3.576278118089249e-07, -2.3841855067985307e-07, -1.8715683836489916e-05, -9.179073458653875e-06, -1.7881377516459906e-06, -7.152555099310121e-07, -5.960462772236497e-07, -9.536738616588991e-07, -7.092700980138034e-05, -2.7418097943154862e-06, -4.291525328881107e-06, -0.1696820706129074, -0.7741565108299255, -1.5497195136049413e-06, -1.3813109397888184, -0.30463284254074097, -5.960462772236497e-07, -0.4390721023082733, -1.102571964263916, -6.437280717364047e-06, -1.585470999998506e-05, -0.05903790891170502, -0.014364292845129967, -0.0019481267081573606, -0.0003046525234822184, -0.15328603982925415, -0.0021168221719563007, -5.400034933700226e-05, -2.622600959512056e-06, -0.020068924874067307, -0.4356759488582611, -7.986990567587782e-06, -7.510157047363464e-06, -8.821448318485636e-06, -7.152531907195225e-06, -5.602820692729438e-06, -1.311301275563892e-06, -1.9073468138230965e-06, -1.0728830375228426e-06, -2.5152843591058627e-05, -1.1801649634435307e-05, -4.291525328881107e-06, -1.311301275563892e-06, -2.3841830625315197e-06, -5.960462772236497e-07, -1.3828182090946939e-05, -8.702239938429557e-06, -6.341733387671411e-05, -4.768370445162873e-07, -0.00721787940710783, -0.7024062275886536, -0.014202469028532505, -1.6689286894688848e-06, -8.106198947643861e-06, -0.0706760436296463, -5.006777428206988e-06, -3.576278118089249e-07, -3.576278118089249e-07, -2.1576648578047752e-05, -0.06406807154417038, -0.004086121916770935, -1.6648235321044922, -0.021351829171180725, -0.05507603660225868, -0.19008991122245789, -0.9923024773597717, -0.5055489540100098, -0.0012872274965047836, -0.0069880131632089615, -1.9713894128799438, -0.22075441479682922, -2.0981624126434326, -1.9777084589004517, -0.13309867680072784, -5.8410845667822286e-05, -0.5621964335441589, -0.22161459922790527, -1.14786958694458, -0.8674592971801758, -0.14337964355945587, -0.2424980252981186, -0.0037430718075484037, -4.529942543740617e-06, -3.146562099456787, -0.46933430433273315, -0.6096848845481873, -0.8396257758140564, -2.989189863204956, -0.6242204308509827, -0.17590267956256866, -0.0010495636379346251, -4.291525328881107e-06, -1.406319260597229, -0.5864269137382507, -0.0002321927313460037, -0.05488253012299538, -0.07772847265005112, -0.2683890461921692, -0.052152518182992935, -3.290122185717337e-05, -0.0007727735792286694, -0.12601466476917267, -3.7431014789035544e-05, -1.311301275563892e-06, -3.933898824470816e-06, -8.940656698541716e-06, -3.099436753473128e-06, -1.4305104514278355e-06, -2.3841855067985307e-07, -2.3841855067985307e-07, -1.6689286894688848e-06, -9.536697689327411e-06, -3.814689989667386e-06, -8.344646857949556e-07, -7.152555099310121e-07, -2.3841855067985307e-07, -1.7881233361549675e-05, -9.179073458653875e-06, -3.8265450712060556e-05, -2.3841855067985307e-07, -0.030207039788365364, -0.004163763951510191, -2.264974000354414e-06, -1.168244216387393e-05, -1.5069836378097534, -1.5718555450439453, -0.00022790218645241112, -4.100715523236431e-05, -0.04769779369235039, -0.0007308434578590095, -2.50339189733495e-06, -0.00025245340657420456, -2.7418097943154862e-06, -1.6689286894688848e-06, -7.152555099310121e-07, -7.033323527139146e-06, -5.960462772236497e-07, -9.536738616588991e-07, -7.152555099310121e-07, -2.3841855067985307e-07, -5.960462772236497e-07, -4.6491513785440475e-06, -2.264974000354414e-06, -3.576278118089249e-07, -7.152555099310121e-07, -3.576278118089249e-07, -9.536697689327411e-06, -9.417489309271332e-06, -1.3351351299206726e-05, -4.768370445162873e-07, -1.4305104514278355e-06], "student_logprobs": [-0.4076656699180603, -0.015165501274168491, -0.00027021096320822835, -0.6327109932899475, -1.0609570381348021e-05, -6.818538531661034e-05, -0.020555097609758377, -0.004718597512692213, -0.00019572250312194228, -0.23772822320461273, -0.6937184929847717, -0.127329483628273, -0.08186454325914383, -1.1893420219421387, -0.008449992164969444, -0.0041287424974143505, -0.1347087025642395, -0.00021371940965764225, -4.410734163684538e-06, -3.9934315282152966e-05, -2.0265558760002023e-06, -4.6491513785440475e-06, -6.437280717364047e-06, -0.00022599527437705547, -9.500529267825186e-05, -4.637133679352701e-05, -1.9311717551317997e-05, -1.2636104656849056e-05, -2.5629668016335927e-05, -0.00011062010162277147, -2.8013790142722428e-05, -3.4570634852570947e-06, -1.6569954823353328e-05, -8.22540732769994e-06, -0.00013350549852475524, -1.7523612768854946e-05, -1.823885577323381e-05, -0.008943617343902588, -1.2234723567962646, -0.24408972263336182, -0.01089269295334816, -4.291525328881107e-06, -0.7343441843986511, -9.405170567333698e-05, -0.028911709785461426, -1.3219380378723145, -2.556230068206787, -1.8679471015930176, -0.0002090712368953973, -1.8067829608917236, -0.7656939625740051, -3.0132853984832764, -2.7863104343414307, -0.35662105679512024, -0.1869954913854599, -0.00016950124700088054, -0.6842201948165894, -1.1315603256225586, -3.0474092960357666, -0.30236729979515076, -0.1883385330438614, -0.07598772644996643, -1.7015633583068848, -0.029315927997231483, -0.8471308350563049, -1.14090096950531, -0.31939274072647095, -1.201120138168335, -1.7839677333831787, -0.36551886796951294, -1.1324222087860107, -0.023795152083039284, -0.0004389514506328851, -0.6049924492835999, -0.3108336925506592, -0.8340004086494446, -0.00835435837507248, -0.08947198837995529, -1.8699760437011719, -0.000780635280534625, -0.007239773869514465, -0.02327258326113224, -1.2525241374969482, -0.0009716795175336301, -0.4526904225349426, -9.285972191719338e-05, -0.04437417909502983, -0.10316872596740723, -0.06323964893817902, -0.28123030066490173, -5.6265202147187665e-05, -0.0018081996822729707, -1.5102839469909668, -1.3470558769768104e-05, -0.4463960528373718, -3.7555840015411377, -0.04234194755554199, -0.0030095544643700123, -3.2186455882765586e-06, -1.1086402082582936e-05, -0.34052595496177673, -6.794906312279636e-06, -4.053033626405522e-05, -1.7037886381149292, -0.0007310817018151283, -0.0010010951664298773, -0.14847248792648315, -0.005291385110467672, -0.4553490877151489, -0.7575640678405762, -2.137849807739258, -0.01832577772438526, -0.00013386306818574667, -0.28148186206817627, -0.7585539817810059, -0.03817257657647133, -0.036829277873039246, -4.017272294731811e-05, -1.1920928244535389e-07, -4.529942543740617e-06, -8.583032467868179e-06, -0.00018249277491122484, -0.0007163104019127786, -0.3135770857334137, -0.10662554949522018, -0.7280974984169006, -0.0153661435469985, -0.000557267339900136, -0.2996799647808075, -0.6554251909255981, -0.013045204803347588, -0.0005165196489542723, -0.00026651646476238966, -1.2874520507466514e-05, -0.00016926287207752466, -0.0010849310783669353, -4.887569048150908e-06, -1.811964830267243e-05, -0.0003409996279515326, -0.020587561652064323, -0.07993479073047638, -1.1086402082582936e-05, -1.7881377516459906e-06, -3.933898824470816e-06, -0.0007693191873840988, -0.04302023723721504, -0.0002924968139268458, -0.042707424610853195, -0.0022400543093681335, -0.00014137222024146467, -0.2574351727962494, -0.0005596501869149506, -0.004467269405722618, -3.683499380713329e-05, -1.1920928244535389e-07, -6.198863957251888e-06, -1.1920922133867862e-06, -0.0183913204818964, -9.345571743324399e-05, -0.2015160620212555, -3.111314072157256e-05, -0.3894684612751007, 0.0, -1.6212332411669195e-05, -5.722029527532868e-06, -1.1920928244535389e-07, -6.198863957251888e-06, -4.124556289752945e-05, -0.15530246496200562, -1.6093124941107817e-05, -3.71926071238704e-05, -0.31809160113334656, -4.8397800128441304e-05, -4.887569048150908e-06, -0.00011383838864276186, -1.311301275563892e-06, -9.417489309271332e-06, -4.768370445162873e-07, -3.2186455882765586e-06, -2.3841855067985307e-07, -3.576278118089249e-07, -5.006777428206988e-06, -0.0001722425949992612, -0.0019444384379312396, -0.0009216589969582856, -4.362964682513848e-05, -3.576278118089249e-07, -1.1920928244535389e-07, -6.282132380874828e-05, -1.5497195136049413e-06, -1.5497195136049413e-06, -4.446407547220588e-05, -7.152555099310121e-07, -1.3232143828645349e-05, -1.3232143828645349e-05, -3.576278118089249e-07, -0.00011717586312443018, -1.1920922133867862e-06, -3.576278118089249e-07, -1.2040065485052764e-05, -1.7881233361549675e-05, -1.5020257706055418e-05, -1.1920928244535389e-07, -0.0010682597057893872, -1.7881377516459906e-06, -1.311301275563892e-06, -2.47952248173533e-05, -4.410734163684538e-06, -1.4305104514278355e-06, 0.0, -2.50339189733495e-06, -5.495397272170521e-05, -2.5033637939486653e-05, -5.125986263010418e-06, -9.536738616588991e-07, -2.0265558760002023e-06, -8.344646857949556e-07, -0.00043752157944254577, -0.0010132422903552651, -0.0029577340465039015, -0.016566572710871696, -7.134505271911621, -0.2179587334394455, -0.020874682813882828, -0.7696530818939209, -0.0011826909612864256, -0.023438768461346626, -0.0007110689766705036, -2.349302291870117, -1.4762922525405884, -0.000814582861494273, -1.8050320148468018, -0.22635656595230103, -0.31168878078460693, -0.38847815990448, -0.6061562895774841, -9.285972191719338e-05, -0.3136627972126007, -0.16301769018173218, -0.0008543655276298523, -0.03373014181852341, -1.0026931762695312, -4.768258077092469e-05, -4.887014389038086, -0.004614300560206175, -2.681187868118286, -2.0625815391540527, -1.152420997619629, -1.9352736473083496, -2.2013304233551025, -1.4253499507904053, -0.00633458374068141, -2.7418097943154862e-06, -1.7881377516459906e-06, -0.14269669353961945, -4.768370445162873e-07, -0.07053496688604355, -0.14879664778709412, -7.867782187531702e-06, -0.0009891861118376255, -0.07893205434083939, -0.0003828269545920193, -5.960462772236497e-07, -8.344646857949556e-07, -0.004626759793609381, -1.709872841835022, -0.23458479344844818, -0.48517125844955444, -2.3556602001190186, -0.00011848701251437888, -0.3868909478187561, -0.001209242851473391, -0.08361790329217911, -0.0023517352528870106, -6.90197994117625e-05, -1.1205610462639015e-05, -4.410734163684538e-06, -0.18001960217952728, -0.6419010162353516, -0.027080899104475975, -0.113536536693573, -0.02055206149816513, -0.00420008972287178, -0.005267550237476826, -0.14267860352993011, -1.4305104514278355e-06, -2.3007127310847864e-05, -2.3007127310847864e-05, -3.671578815556131e-05, -0.07095224410295486, -8.22540732769994e-06, -0.03804933279752731, -0.00022539935889653862, -0.0001358893496217206, -0.001043133088387549, -0.24163362383842468, -0.8059572577476501, -2.2719314098358154, -0.012607398442924023, -0.00013934595335740596, -0.03870243951678276, -0.10455933213233948, -2.641674041748047, -0.04965554550290108, -1.3869314193725586, -1.8778619766235352, -5.960462772236497e-07, -3.5665998458862305, -0.005080290138721466, -0.09021110087633133, -1.4583775997161865, -0.17395055294036865, -0.6242830157279968, -4.768258077092469e-05, -0.3695756793022156, -0.9742996692657471, -0.18065963685512543, -0.4369969367980957, -0.0022916740272194147, -0.011981424875557423, -0.00014065706636756659, -8.535020606359467e-05, -0.00069165148306638, -0.00010072677832795307, -4.3748852476710454e-05, -1.7762025890988298e-05, -3.838465272565372e-05, -0.0056958990171551704, -0.8077143430709839, -0.08357777446508408, -0.005997992120683193, -0.10056445002555847, -0.0016750366194173694, -0.2840803861618042, -0.6816835403442383, -6.5205356804654e-05, -0.07110858708620071, -0.0001879753835964948, -0.0028062984347343445, -6.556489552167477e-06, -1.597391747054644e-05, -1.1391435861587524, -0.02441621758043766, -0.00019262365822214633, -0.3568198084831238, -0.09235725551843643, -0.00013541258522309363, -6.6756979322235566e-06, -0.00660652294754982, -0.01251816563308239, -0.0001839230244513601, -0.00010048838157672435, -1.0132738680113107e-05, -3.218599158572033e-05, -1.9073468138230965e-06, -2.861018856492592e-06, -4.6491513785440475e-06, -0.0010399178136140108, -0.005358262453228235, -0.044839903712272644, -1.362795352935791, -2.0953996181488037, -3.539144277572632, -0.4680044651031494, -0.19334018230438232, -0.00014911970356479287, -8.4638240878121e-06, -0.1426796317100525, -2.39836049079895, -0.12280938774347305, -0.00013648532330989838, -4.2676016164477915e-05, -1.860852599143982, -0.13444465398788452, -0.011049393564462662, -3.099436753473128e-06, -0.06613544374704361, -0.4843566417694092, -2.145764938177308e-06, -3.814689989667386e-06, -0.12918978929519653, -0.11075268685817719, -5.936446541454643e-05, -4.768370445162873e-07, -2.861018856492592e-06, -0.03201955556869507, -0.04527624696493149, -2.372236667724792e-05, -8.22540732769994e-06, -3.099436753473128e-06, -0.0006255338666960597, -0.0031780709978193045, -5.006777428206988e-06, -1.7881377516459906e-06, -0.050865042954683304, -0.04533366858959198, -0.00673500495031476, -1.3356053829193115, -0.00039748396375216544, -0.00162743148393929, -0.9354971051216125, -6.437280717364047e-06, -0.0015256681945174932, -0.0006895071710459888, -0.00011789103882620111, -0.001813197392039001, -0.007667633704841137, -0.000262106885202229, -8.880697714630514e-05, -1.0132738680113107e-05, -1.9073468138230965e-06, -4.529942543740617e-06, -1.4662635294371285e-05, -0.4784844219684601, -0.004282470792531967, -0.12766717374324799, -0.004038156475871801, -0.00030751267331652343, -2.3841830625315197e-06, -5.5549986427649856e-05, -0.001822478836402297, -7.807903602952138e-05, -1.2040065485052764e-05, -0.00037520044133998454, -0.012639416381716728, -0.0002498314715921879, -5.8412379075889476e-06, -0.00024577934527769685, -0.001505790394730866, -6.0794889577664435e-05, -1.9073468138230965e-06, -2.9205850296420977e-05, -3.576278118089249e-07, -7.152555099310121e-07, -5.483612312673358e-06, -0.0013491347199305892, -0.00011407678539399058, -7.533743337262422e-05, -0.0006273209000937641, -8.642300235806033e-05, -1.2040065485052764e-05, -0.0007082099909894168, -1.0013530300057027e-05, -1.7165990357170813e-05, -8.344646857949556e-07, -4.768360213347478e-06, -3.576278118089249e-07, -2.264974000354414e-06, -2.3841855067985307e-07, -3.6954811548639555e-06, -0.0017190692014992237, -1.168244216387393e-05, -0.0029078128281980753, -0.5085974931716919, -0.31757596135139465, -0.04331560060381889, -8.344646857949556e-07, -1.4781842764932662e-05, -3.909988299710676e-05, -6.6756979322235566e-06, -0.009097079746425152, -0.00010871296399272978, -0.10103210061788559, -0.004123400431126356, -0.8836885094642639, -3.251901149749756, -0.9945614337921143, -3.559727668762207, -0.0032088477164506912, -0.00022682955022901297, -4.2437604861333966e-05, -0.3583873212337494, -0.16136276721954346, -0.7602478861808777, -0.10978637635707855, -5.769562994828448e-05, -5.722029527532868e-06, -0.00018606838420964777, -2.7656173188006505e-05, -2.7418097943154862e-06, -0.03370098024606705, -0.6391346454620361, -0.4305588901042938, -1.4074759483337402, -0.1275842934846878, -7.509902934543788e-05, -9.870042413240299e-05, -0.001459724735468626, -0.31351765990257263, -5.686121585313231e-05, -0.05360082909464836, -0.001741322805173695, -0.014203761704266071, -2.2172682292875834e-05, -2.682172998902388e-05, -3.933898824470816e-06, -0.00020811776630580425, -7.152531907195225e-06, -1.7881377516459906e-06, -1.2397689715726301e-05, -0.3138510286808014, -0.0009392855572514236, -4.1483970562694594e-05, -0.000302030734019354, -5.507317473529838e-05, -5.722029527532868e-06, -1.07287787614041e-05, -0.12726420164108276, -0.0001528146385680884, -9.894321920000948e-06, -2.6702524337451905e-05, -0.0006597249885089695, -0.0006119524477981031, -4.768370445162873e-07, -0.0017089537577703595, -0.0022033003624528646, -0.5036742091178894, -0.03862927109003067, -5.090107151772827e-05, -1.4305104514278355e-06, -2.3841855067985307e-07, -0.001912789884954691, -2.386784791946411, -0.42584481835365295, -0.0628390684723854, -0.732629656791687, -0.07627329230308533, -0.0009151084814220667, -0.015037980861961842, -1.6689286894688848e-06, -2.372236667724792e-05, -1.2755313036905136e-05, -1.4185804502631072e-05, -3.7788631743751466e-05, -0.00012909532233607024, -0.0008186326595023274, -8.940656698541716e-06, -0.00010859376925509423, -5.1377883210079744e-05, -0.018155112862586975, -2.169585604860913e-05, -0.00011252723925281316, -0.00018308870494365692, -5.2689116273541003e-05, -2.1934269170742482e-05, -2.8490614567999728e-05, -2.0265558760002023e-06, -0.07374963909387589, -3.099436753473128e-06, -0.009400981478393078, -0.009995171800255775, -0.00011562632425921038, -9.560128091834486e-05, -0.0002627027570270002, -1.7404405298293568e-05, -1.0728830375228426e-06, -3.3378546504536644e-06, -5.745722592109814e-05, -0.00020346954988781363, -4.6491513785440475e-06, -4.207999518257566e-05, -0.0010582567192614079, -0.01073030661791563, -5.8887653722194955e-05, -0.00011050090688513592, -0.0010383697226643562, -0.00020001317898277193, -5.960446742392378e-06, -6.997340824455023e-05, -0.0002586507180240005, -0.003365925280377269, -7.319182623177767e-05, -3.564294092939235e-05, -1.1920922133867862e-06, -5.960462772236497e-07, -1.8715683836489916e-05, -0.015933571383357048, -0.002388126915320754, -0.007648824248462915, -0.023238342255353928, -0.0007258402765728533, -0.0006884350441396236, -0.005731695331633091, -0.00014900050882715732, -2.396077979938127e-05, -0.00013410145766101778, -5.090107151772827e-05, -4.2437604861333966e-05, -0.11289092153310776, -0.1022079735994339, -0.00019274283840786666, -7.986990567587782e-06, -9.858122211880982e-05, -1.6569954823353328e-05, -6.437280717364047e-06, -2.0265558760002023e-06, -4.9232225137529895e-05, -0.00027700403006747365, -5.8412379075889476e-06, -2.5748875486897305e-05, -0.002378731733188033, -0.0001250427303602919, -7.152555099310121e-07, -0.00043132537393830717, -2.1274917125701904, -0.00023112009512260556, -1.6689286894688848e-06, -2.264974000354414e-06, -8.344646857949556e-07, -1.311301275563892e-06, -3.814689989667386e-06, -0.001985484967008233, -6.544376083184034e-05, -8.284702198579907e-05, -0.001257820287719369, -0.0006606780225411057, -0.00015221867943182588, -7.629365427419543e-06, -1.0728830375228426e-06, -7.152555099310121e-07, -3.266281055402942e-05, -0.0008499585674144328, -0.0002498314715921879, -5.054346183896996e-05, -0.0014334175502881408, -0.00011014331539627165, -0.0003010773507412523, -0.0005544078885577619, -2.3007127310847864e-05, -1.2397689715726301e-05, -0.00051771110156551, -2.861018856492592e-06, -2.264974000354414e-06, -3.814689989667386e-06, -0.0002162222663173452, -0.0002714027068577707, -1.7762025890988298e-05, -7.867782187531702e-06, -3.099436753473128e-06, -3.194758028257638e-05, -0.0036860634572803974, -1.0728830375228426e-06, -4.291525328881107e-06, -0.0001517419150331989, -4.255681051290594e-05, -4.768370445162873e-07, -9.65590606938349e-06, -6.6756979322235566e-06, -8.332382276421413e-05, -6.6756979322235566e-06, -0.00012194366718176752, -0.000726197671610862, -0.0029401429928839207, -4.768370445162873e-07, -0.00043084874050691724, -0.008639931678771973, -6.05564855504781e-05, -1.9788545614574105e-05, -1.7881377516459906e-06, -2.861018856492592e-06, -3.576278118089249e-07, -1.4305104514278355e-06, -5.245195097813848e-06, -8.05822346592322e-05, -2.038458114839159e-05, -5.125986263010418e-06, -0.0005032941699028015, -0.0003488647344056517, -3.576278118089249e-07, -8.106198947643861e-06, -1.1444026313256472e-05, -2.47952248173533e-05, -1.9073468138230965e-06, -2.861018856492592e-06, -1.0728830375228426e-06, -3.576278118089249e-07, -4.768370445162873e-07, -1.0847986231965479e-05, -7.986990567587782e-06, -0.03980085253715515, -0.13278159499168396, -0.00014935807848814875, -0.0001776060671545565, -0.0032928551081568003, -4.2676016164477915e-05, -0.0002090712368953973, -7.4741430580616e-05, -2.658331868587993e-05, -5.769562994828448e-05, -0.00024303820100612938, -1.764281842042692e-05, -0.00042417587246745825, -2.038458114839159e-05, -2.622600959512056e-06, -0.00020621081057470292, -0.01251863595098257, -6.09140915912576e-05, -1.311301275563892e-06, -5.125986263010418e-06, -8.344646857949556e-07, -2.3841830625315197e-06, -1.0371154530730564e-05, -5.9602869441732764e-05, -5.090107151772827e-05, -5.245195097813848e-06, -0.00038771264371462166, -5.8530047681415454e-05, -5.960462772236497e-07, -4.95898348162882e-05, -0.09061023592948914, -0.00018070495571009815, -3.361645576660521e-05, -1.9073468138230965e-06, -8.344646857949556e-07, -1.9550132492440753e-05, -8.34461570775602e-06, -1.549708758830093e-05, -0.0004318020073696971, -9.298280929215252e-06, -1.7762025890988298e-05, -0.0019857229199260473, -7.152555099310121e-07, -3.933898824470816e-06, -2.729855441430118e-05, -3.504691630951129e-05, -0.5760931968688965, -0.0003178806509822607, -3.707340147229843e-05, -9.417489309271332e-06, -7.152555099310121e-07, -1.6689286894688848e-06, -6.90197994117625e-05, -3.099393507000059e-05, -1.311301275563892e-06, -4.0531076592742465e-06, -0.00011491115583339706, -1.4305104514278355e-06, -2.4914430468925275e-05, -2.610649426060263e-05, -0.0001501924270996824, -1.7046782886609435e-05, -0.0006292270263656974, -0.0009240409708581865, -6.711257447022945e-05, -1.9073468138230965e-06, -5.483612312673358e-06, -1.4305104514278355e-06, -2.9802276912960224e-06, -6.9141146923357155e-06, -1.4424220353248529e-05, -3.3378546504536644e-06, -0.0006656814948655665, -0.00010048838157672435, -0.0008886678842827678, -0.00012110930401831865, -2.5987286790041253e-05, -0.07891982048749924, -2.3841855067985307e-07, -7.152555099310121e-07, -0.0021771318279206753, -0.005504569970071316, -0.030081884935498238, -0.1613071709871292, -0.00047398300375789404, -0.016080567613244057, -0.499638170003891, -0.03653842583298683, -0.013539272360503674, -2.3841830625315197e-06, -0.003965373151004314, -0.00034564718953333795, -0.00037579628406092525, -2.4914430468925275e-05, -9.48860906646587e-05, -0.0002978596603497863, -0.00010907054820563644, -0.00017975145601667464, -7.164221460698172e-05, -0.003395151812583208, -9.536697689327411e-06, -1.0609570381348021e-05, -4.577531944960356e-05, -0.004167919047176838, -1.2755313036905136e-05, -0.16123726963996887, -0.6162506341934204, -0.012532292865216732, -0.04021844267845154, -0.00014757021563127637, -0.35124456882476807, -0.000285584683297202, -1.7177313566207886, -1.4781842764932662e-05, -0.00483095133677125, -1.1920928244535389e-07, -7.033323527139146e-06, -4.768370445162873e-07, -7.152555099310121e-07, -3.576278118089249e-07, -1.4066597032069694e-05, -7.30726242181845e-05, -7.152531907195225e-06, -3.933898824470816e-06, -2.2172682292875834e-05, -1.4662635294371285e-05, -4.291525328881107e-06, -2.3841574147809297e-05, -2.0265558760002023e-06, -8.344646857949556e-07, 0.0, -5.2569914259947836e-05, -7.033323527139146e-06, -7.748573807475623e-06, -6.067568756407127e-05, -4.0649541915627196e-05, -0.29690706729888916, -0.022230826318264008, -0.001629216712899506, -0.00048780461656861007, -0.00019822540343739092, -2.264974000354414e-06, -1.3470558769768104e-05, -7.152555099310121e-07, -2.3841830625315197e-06, -1.7881377516459906e-06, -1.6331539882230572e-05, -3.933898824470816e-06, -1.1920922133867862e-06, -4.0531076592742465e-06, -6.437280717364047e-06, -9.572047565598041e-05, -2.4437606043647975e-05, -0.17980441451072693, -2.559265613555908, -0.8356835842132568, -0.020783383399248123, -0.00030393750057555735, -0.16857056319713593, -0.11138923466205597, -1.5042897462844849, -0.5127093195915222, -0.49736666679382324, -0.0013727060286328197, -1.5140365362167358, -0.03314908593893051, -0.5841472744941711, -0.04491262137889862, -1.3032004833221436, -2.4907073974609375, -0.21324411034584045, -0.47584041953086853, -2.01812481880188, -0.005467698443681002, -3.576278118089249e-07, -7.152555099310121e-07, -7.390948667307384e-06, -1.4305104514278355e-06, -2.3841855067985307e-07, -6.0437283536884934e-05, -1.1086402082582936e-05, -4.887569048150908e-06, -9.691245941212401e-05, -0.11766011267900467, -0.6916072964668274, -0.0011648305226117373, -0.05299197882413864, -3.2066785934148356e-05, -0.01259456668049097, -0.04440508410334587, -3.766942609217949e-05, -4.529942543740617e-06, -1.7881377516459906e-06, -4.768370445162873e-07, -9.536738616588991e-07, -2.861018856492592e-06, -1.9311717551317997e-05, -6.079655122448457e-06, -2.0265558760002023e-06, -3.933898824470816e-06, -5.006777428206988e-06, -3.3378044463461265e-05, -0.0023335388395935297, -3.58813522325363e-05, -2.9802276912960224e-06, -0.010759790427982807, -0.5550222396850586, -0.836743175983429, -0.017358053475618362, -0.692503809928894, -0.020046841353178024, -0.08972691744565964, -8.725739462533966e-05, -0.00013016807497479022, -0.0015441172290593386, -4.768370445162873e-07, -1.1920928244535389e-07, -0.005742599721997976, -0.11853107064962387, -0.06717249751091003, -0.16086362302303314, -6.198863957251888e-06, -2.145764938177308e-06, -0.00043525759247131646, -0.960599958896637, -1.5567290782928467, -0.06759374588727951, -4.132913589477539, -4.768370445162873e-07, -0.0007701530121266842, -1.150797724723816, -0.010557160712778568, -0.7019291520118713, -0.07273080945014954, -0.07731787115335464, -2.15720534324646, -0.2855638563632965, -1.517369270324707, -3.528532761265524e-05, -0.011151253245770931, -1.728519782773219e-05, -0.04235440492630005, -3.4828968048095703, -4.756337511935271e-05, -0.5255099534988403, -1.7046782886609435e-05, -2.622600959512056e-06, -1.7881377516459906e-06, -2.3841855067985307e-07, -3.576278118089249e-07, -1.311301275563892e-06, -1.1205610462639015e-05, -3.3378546504536644e-06, -1.6689286894688848e-06, -1.311301275563892e-06, -1.9073468138230965e-06, -4.410646579344757e-05, -5.006777428206988e-06, -1.3470558769768104e-05, -0.21731480956077576, -2.145764938177308e-06, -8.999896090244874e-05, -2.297534704208374, -0.3132956326007843, -2.7054736614227295, -0.6966296434402466, -0.00011276562872808427, -0.6276267170906067, -0.0005247407825663686, -0.00036638224264606833, -0.6246585249900818, -1.6212332411669195e-05, -0.0001599660754436627, -2.6225699912174605e-05, -6.437280717364047e-06, -4.768360213347478e-06, -1.1920922133867862e-06, -1.1920928244535389e-07, -1.0728830375228426e-06, -3.814689989667386e-06, -1.5139465176616795e-05, -5.245195097813848e-06, -5.245195097813848e-06, -2.264974000354414e-06, -1.1920922133867862e-06, -5.722029527532868e-06, -6.031808152329177e-05, -5.411955135059543e-05, -1.1920922133867862e-06, -2.90866428258596e-05, -0.6817994117736816, -7.152555099310121e-07, -2.0793702602386475, -2.7921199798583984, -0.4405415654182434, -1.1920922133867862e-06, -5.960462772236497e-07, -3.576278118089249e-07, -0.007424027193337679, -0.030193043872714043, -0.37441709637641907, -0.00013743886665906757, -0.25463664531707764, -0.47130417823791504, -1.0063079595565796, -0.0020761380437761545, -0.9750957489013672, -0.0667235255241394, -1.056922435760498, -0.020752208307385445, -0.00010585224663373083, -0.0004372832481749356, -1.156671404838562, -1.2023917436599731, -0.05400366336107254, -6.83045873302035e-05, -4.935142715112306e-05, -0.020414479076862335, -0.002954049501568079, -0.18007825314998627, -0.04488150775432587, -0.06244448199868202, -0.0001463782973587513, -1.9907753085135482e-05, -0.05325344204902649, -0.20151887834072113, -0.07940112054347992, -0.01272734347730875, -0.01569349505007267, -9.536738616588991e-07, -0.48100072145462036, -0.0010889797704294324, -0.20918820798397064, -0.093769371509552, -0.028996944427490234, -1.8348150253295898, -0.3435589075088501, -0.0006157647585496306, -4.529942543740617e-06, -0.006772184744477272, -1.5184434652328491, -0.020389484241604805, -0.009181066416203976, -0.00016973962192423642, -0.00034588552080094814, -0.0009584600338712335, -0.013533039949834347, -0.002400376135483384, -0.026750406250357628, -3.266281055402942e-05, -0.9178503751754761, -0.0019564551766961813, -0.34910279512405396, -0.006592311896383762, -2.5152843591058627e-05, -0.0009291622554883361, -0.00048720886115916073, -8.344646857949556e-07, -1.6569954823353328e-05, -0.0002648479712661356, -8.344646857949556e-07, -1.2636104656849056e-05, -2.622600959512056e-06, -9.202533692587167e-05, -1.1920928244535389e-07, 0.0, -1.7881377516459906e-06, -1.9073468138230965e-06, -7.152555099310121e-07, -1.6927575416048057e-05, -0.0002348147245356813, -2.8729025871143676e-05, -2.3483953555114567e-05, -1.8954096958623268e-05, -1.0847986231965479e-05, -1.0728830375228426e-06, -7.021180499577895e-05, -7.390948667307384e-06, -5.483612312673358e-06, -6.5205356804654e-05, -3.576272320060525e-06, -3.766942609217949e-05, -7.986990567587782e-06, -6.747018051100895e-05, -3.099436753473128e-06, -1.9073468138230965e-06, -1.9073468138230965e-06, -7.152531907195225e-06, -0.018152771517634392, -8.129743218887597e-05, -7.152555099310121e-07, -9.059865078597795e-06, -7.152555099310121e-07, -4.172238186583854e-05, -0.0003610197745729238, -0.0004957877099514008, -0.00047100416850298643, -2.861018856492592e-06, -0.001328539103269577, -5.602820692729438e-06, -2.5748875486897305e-05, -5.960462772236497e-07, -1.0490362910786644e-05, -4.649054244509898e-05, -2.5629668016335927e-05, -2.861018856492592e-06, -1.2874520507466514e-05, -1.311301275563892e-06, -3.814624506048858e-05, -3.576278118089249e-07, -9.536697689327411e-06, -4.768360213347478e-06, -1.6689286894688848e-06, -1.4543427823809907e-05, -0.00015877417172305286, -0.002230895683169365, -0.009709158912301064, -4.529942543740617e-06, -0.00024375328212045133, -5.125986263010418e-06, -4.768370445162873e-07, -6.318072337307967e-06, -4.768370445162873e-07, -3.576278118089249e-07, -2.3841855067985307e-07, -0.00021562635083682835, -3.433168603805825e-05, -2.932505594799295e-05, -0.004313213750720024, -0.05006915330886841, -0.00015472168161068112, -0.29080548882484436, -2.4437606043647975e-05, -2.5033637939486653e-05, -1.0251946150674485e-05, -2.3841855067985307e-07, -3.1709168979432434e-05, -2.3841830625315197e-06, -1.156323378381785e-05, -3.576278118089249e-07, -2.145764938177308e-06, -2.145764938177308e-06, -3.2186455882765586e-06, -2.0265558760002023e-06, -3.814689989667386e-06, -1.5497195136049413e-06, -3.71926071238704e-05, -3.576278118089249e-07, -0.0005458295345306396, -1.2278481335670222e-05, -7.152555099310121e-07, -1.764281842042692e-05, -2.3841855067985307e-07, -9.059865078597795e-06, -3.099436753473128e-06, -1.9073304429184645e-05, -2.3841855067985307e-07, -2.861018856492592e-06, -3.814689989667386e-06, -1.5497195136049413e-06, -1.311301275563892e-06, -9.536738616588991e-07, -5.960462772236497e-07, -3.576272320060525e-06, -7.152555099310121e-07, -0.0007333449902944267, -0.5795182585716248, -0.09491413086652756, -2.145764938177308e-06, -2.861018856492592e-06, -8.344646857949556e-07, -0.015435636974871159, -0.005247628316283226, -4.410646579344757e-05, -0.0017118098912760615, -1.888314962387085, -0.0011713793501257896, -1.1369444131851196, -1.1917259693145752, -1.0640137195587158, -0.00014423283573705703, -0.00036483307485468686, -0.1798936426639557, -8.880697714630514e-05, -1.8954096958623268e-05, -0.30095335841178894, -4.029192859889008e-05, -5.722029527532868e-06, -4.291525328881107e-06, -2.0265558760002023e-06, -3.5523738915799186e-05, -0.5759504437446594, -1.2993727978027891e-05, -0.1128273531794548, -0.05497020483016968, -4.410734163684538e-06, -0.004804139491170645, -0.00027509720530360937, -0.0014524635626003146, -0.02631867676973343, -2.5073740482330322, -4.529942543740617e-06, -2.074220174108632e-05, -1.311301275563892e-06, -5.245195097813848e-06, -1.6927575416048057e-05, -1.7881377516459906e-06, -4.577531944960356e-05, -0.03811497241258621, -9.65590606938349e-06, -5.960462772236497e-07, -5.6622808187967166e-05, -1.1920922133867862e-06, -1.537788011773955e-05, -1.5616295058862306e-05, -1.597391747054644e-05, -1.4305104514278355e-06, -1.5735502529423684e-05, -2.9802276912960224e-06, -7.152555099310121e-07, -1.0728830375228426e-06, -3.576278118089249e-07, -3.576278118089249e-07, -2.4199192921514623e-05, -0.00463993102312088, -0.0019497923785820603, -0.06355897337198257, -8.5588610090781e-05, -0.0053708311170339584, -2.2053474822314456e-05, -7.271740287251305e-06, -0.0004365683125797659, -1.1920922133867862e-06, -0.0003922408213838935, -0.014274746179580688, -2.0146166207268834e-05, -4.768370445162873e-07, -4.172316494077677e-06, -9.536738616588991e-07, -4.291525328881107e-06, -1.9907753085135482e-05, -2.3841855067985307e-07, -1.1920928244535389e-07, -4.887569048150908e-06, -2.264974000354414e-06, -3.6954811548639555e-06, -3.576278118089249e-07, -3.814689989667386e-06, -2.407998726994265e-05, -2.109982233378105e-05, -3.6954811548639555e-06, -0.006774197798222303, -0.07101420313119888, -0.0002786724944598973, -4.970903682988137e-05, -0.6932425498962402, -3.576278118089249e-07, -7.748573807475623e-06, -0.012773598544299603, -0.7599620819091797, -0.004022602923214436, -1.6212332411669195e-05, -0.0005984899471513927, -0.0002980979916173965, -9.298280929215252e-06, -0.0009286858257837594, -0.06056594103574753, -1.597391747054644e-05, -1.0371154530730564e-05, -0.020551009103655815, -7.033323527139146e-06, -2.7656173188006505e-05, -7.867782187531702e-06, -2.264974000354414e-06, -5.245195097813848e-06, -0.02324754372239113, -8.296622399939224e-05, -2.0265558760002023e-06, -4.0531076592742465e-06, -1.5497195136049413e-06, -1.1920922133867862e-06, -1.07287787614041e-05, -4.2676016164477915e-05, -6.198863957251888e-06, -1.7881377516459906e-06, -6.079655122448457e-06, -1.5497195136049413e-06, -0.00024256148026324809, -1.1086402082582936e-05, -4.768258077092469e-05, -0.02157771773636341, -3.6954811548639555e-06, -2.3841855067985307e-07, -1.5497195136049413e-06, -1.6689286894688848e-06, -5.960462772236497e-07, -1.1920922133867862e-06, -1.1920922133867862e-06, -0.01604936271905899, -0.00010740180005086586, -2.062299427052494e-05, -0.00011097769311163574, -1.8596476365928538e-05, -2.706014311115723e-05, -0.001057423185557127, -2.253030106658116e-05, -1.311301275563892e-06, -2.4914430468925275e-05, -1.4305104514278355e-06, -2.3841830625315197e-06, -2.3841855067985307e-07, -4.768370445162873e-07, -7.390948667307384e-06, -7.510157047363464e-06, -4.768360213347478e-06, -2.253030106658116e-05, -0.00029380773776210845, -0.000491021724883467, 0.0, -1.0490362910786644e-05, -1.1920928244535389e-07, -1.5497195136049413e-06, -1.1801649634435307e-05, -0.0011002921964973211, -1.4305104514278355e-06, -1.1920928244535389e-07, 0.0, -5.960462772236497e-07, -5.125986263010418e-06, -7.152555099310121e-07, -1.4305104514278355e-06, -1.2040065485052764e-05, -4.768370445162873e-07, -7.748303323751315e-05, -0.5778350830078125, -1.5497195136049413e-06, -0.06798062473535538, -7.986703712958843e-05, -0.01826760545372963, -1.6331539882230572e-05, -0.0003567297535482794, -5.245195097813848e-06, -9.536738616588991e-07, -1.311301275563892e-06, -7.629365427419543e-06, -2.145764938177308e-06, -0.012591623701155186, -6.05564855504781e-05, -0.00435594329610467, -2.109982233378105e-05, -0.0022021110635250807, -9.894321920000948e-06, -8.22540732769994e-06, -0.00014137222024146467, -2.622600959512056e-06, -9.536738616588991e-07, -5.960462772236497e-07, -2.145764938177308e-06, -0.005266008898615837, -0.8259924054145813, -0.0073204850777983665, -0.0001255195093108341, -6.794906312279636e-06, -3.576278118089249e-07, -0.11282799392938614, -1.4305104514278355e-06, -7.879423355916515e-05, -4.7205765440594405e-05, -8.22540732769994e-06, -1.311301275563892e-06, -1.9907753085135482e-05, -8.344646857949556e-07, -1.9311717551317997e-05, -0.004217896144837141, -0.006248343735933304, -2.169585604860913e-05, -3.099436753473128e-06, -4.577531944960356e-05, -2.4318398573086597e-05, -2.9802276912960224e-06, -0.0010240792762488127, -4.160317621426657e-05, -1.9430925021879375e-05, -5.8412379075889476e-06, -8.344646857949556e-07, -3.576278118089249e-07, -4.6491513785440475e-06, -3.9457496313843876e-05, -1.2159273865108844e-05, -3.075552376685664e-05, -2.884823152271565e-05, -0.07892169803380966, -8.344646857949556e-07, -3.313963316031732e-05, -0.0032038569916039705, -2.50339189733495e-06, -0.00010013079008786008, -0.251986026763916, -1.0251946150674485e-05, -6.198863957251888e-06, -6.508615479106084e-05, -1.2159273865108844e-05, -9.881961887003854e-05, -4.9232225137529895e-05, -9.775113539944869e-06, -2.3841855067985307e-07, -2.50339189733495e-06, -3.576278118089249e-07, -1.7881377516459906e-06, -0.00023815179883968085, -1.4424220353248529e-05, -3.659658250398934e-05, -0.00465927179902792, -0.0044574192725121975, -2.7418097943154862e-06, -7.271740287251305e-06, -0.011370735242962837, -1.9192511899746023e-05, -8.34461570775602e-06, -0.0013490156270563602, -2.50339189733495e-06, -1.5497195136049413e-06, -1.1920928244535389e-07, -3.576278118089249e-07, -6.556489552167477e-06, -1.4305104514278355e-06, -4.768370445162873e-07, -6.079655122448457e-06, -7.629365427419543e-06, -3.2186455882765586e-06, -4.362964682513848e-05, -1.311301275563892e-06, -1.6689286894688848e-06, -5.960462772236497e-07, -1.1920922133867862e-06, -7.510157047363464e-06, -5.364403477869928e-06, -1.8715683836489916e-05, -8.940656698541716e-06, -0.0001102625101339072, -0.00011002412065863609, -8.344646857949556e-07, -4.2914423829643056e-05, -5.960462772236497e-07, -2.8490614567999728e-05, -0.5391895174980164, -0.0006287504802457988, -1.1707826852798462, -0.054884787648916245, -0.003688676515594125, -0.5429688096046448, -0.002921600826084614, -0.666246771812439, -2.9802276912960224e-06, -1.0967194612021558e-05, -1.412889003753662, -1.206166386604309, -5.125986263010418e-06, -0.0003746046277228743, -1.4669103622436523, -0.13526801764965057, -0.0696256086230278, -1.3708974620385561e-05, -0.00047231485950760543, -0.00037543877260759473, -6.9141146923357155e-06, -1.8715683836489916e-05, -1.4305104514278355e-06, -8.344646857949556e-07, -8.344646857949556e-07, -5.960462772236497e-07, -1.1920858014491387e-05, -2.7418097943154862e-06, -3.576278118089249e-07, -2.3841830625315197e-06, -4.768370445162873e-07, -2.109982233378105e-05, -1.8596476365928538e-05, -3.123234637314454e-05, -4.768370445162873e-07, -1.4781842764932662e-05, -0.09967434406280518, -3.099436753473128e-06, -3.6954811548639555e-06, -1.996826410293579, -0.000552263343706727, -0.5420316457748413, -4.291525328881107e-06, -0.7771167755126953, -0.49916738271713257, -1.5136542320251465, -0.0006217215559445322, -0.5167911648750305, -3.9145359992980957, -0.1663176268339157, -1.7662025690078735, -0.2644553482532501, -0.19355808198451996, -1.9961669445037842, -0.6936944127082825, -0.09643208980560303, -0.45559537410736084, -0.5275871753692627, -0.0006652049487456679, -8.344646857949556e-07, -0.0008174415561370552, -5.376194530981593e-05, -2.06821346282959, -0.7506166696548462, -0.004906400106847286, -1.9788545614574105e-05, -0.43316349387168884, -0.24640575051307678, -0.0848030298948288, -0.14425936341285706, -0.0005404680268839002, -0.20197056233882904, -2.7018818855285645, -1.9571725130081177, -0.00129413278773427, -1.0728830375228426e-06, -0.045822177082300186, -2.2296817302703857, -0.002532372483983636, -0.0049335649237036705, -0.004554256331175566, -0.5087721943855286, -0.00027259447961114347, -3.1722195148468018, -0.04461533576250076, -7.152555099310121e-07, -0.0013299677520990372, -2.258091449737549, -0.13670289516448975, -0.5500624179840088, -1.9110677242279053, -0.02387360669672489, -1.1368792057037354, -1.2278481335670222e-05, -0.7980622053146362, -8.583032467868179e-06, -8.106198947643861e-06, -2.50339189733495e-06, -1.5497195136049413e-06, -0.00014578233822248876, -0.00023862851958256215, -1.7165990357170813e-05, -3.6954811548639555e-06, -0.11348801106214523, -0.719049870967865, -0.02549254149198532, -6.556489552167477e-06, -0.9644983410835266, -0.029117833822965622, -2.130664825439453, -4.0411134250462055e-05, -0.3538638949394226, -0.05001971498131752, -2.2011184692382812, -0.5560036897659302, -0.3690510392189026, -0.24614864587783813, -0.15894320607185364, -0.19907061755657196, -1.6383475065231323, -1.6688286066055298, -0.05540756508708, -0.005178609397262335, -0.009396021254360676, -5.364403477869928e-06, -0.0002809368306770921, -6.556489552167477e-06, -0.00029774048016406596, -1.5497195136049413e-06, -4.0531076592742465e-06, -0.0003849719068966806, -0.6166912317276001, -0.0718468576669693, -0.060178425163030624, -0.31335464119911194, -0.8491682410240173, -2.622600959512056e-06, -2.4676019165781327e-05, -0.0002559096028562635, -0.017074374482035637, -0.08547446876764297, -2.5629668016335927e-05, -0.26169395446777344, -0.6022234559059143, -0.00319446949288249, -8.344646857949556e-07, -8.821448318485636e-06, -0.3875230550765991, -7.915183232398704e-05, -7.629365427419543e-06, -3.4570634852570947e-06, -6.6756979322235566e-06, -1.6689286894688848e-06, -1.6689286894688848e-06, -1.1920928244535389e-07, -1.1920928244535389e-07, -7.152555099310121e-07, -8.4638240878121e-06, -3.3378546504536644e-06, -3.814689989667386e-06, -7.152555099310121e-07, -3.576278118089249e-07, -1.168244216387393e-05, -5.722029527532868e-06, -2.8132995794294402e-05, -3.576278118089249e-07, -0.025450240820646286, -0.019083237275481224, -2.3841830625315197e-06, -2.145764938177308e-06, -2.8210952281951904, -0.0007889734115451574, 0.0, -1.2636104656849056e-05, -0.1063857153058052, -0.7275447845458984, -3.6954811548639555e-06, -0.023248009383678436, -2.253030106658116e-05, -0.7836130261421204, -2.3841855067985307e-07, -5.090107151772827e-05, 0.0, -4.291525328881107e-06, -3.576278118089249e-07, -2.3841855067985307e-07, 0.0, -2.50339189733495e-06, -4.6491513785440475e-06, -1.1920922133867862e-06, -5.722029527532868e-06, -1.680836794548668e-05, -3.933898824470816e-06, -2.3841855067985307e-07, -6.437280717364047e-06, 0.0, -2.145764938177308e-06, 0.0, -1.9192511899746023e-05, -1.9073468138230965e-06, -1.0251946150674485e-05, -2.145764938177308e-06, -1.7881377516459906e-06, -1.5615299940109253, -0.6804471015930176, -0.00023147765023168176, -3.933898824470816e-06, -1.6331539882230572e-05, -4.768370445162873e-07, -4.768370445162873e-07, -4.768370445162873e-07, -3.576278118089249e-07, -1.311301275563892e-06, -1.0132738680113107e-05, -1.9073468138230965e-06, -1.0728830375228426e-06, -2.3841855067985307e-07, -7.152555099310121e-07, -9.893881360767409e-05, -9.536738616588991e-07, -2.622600959512056e-06, -0.11606346070766449, -0.641929030418396, -1.5497195136049413e-06, -1.2921065092086792, -0.2267092764377594, -4.768370445162873e-07, -0.4261739253997803, -0.9478947520256042, -7.510157047363464e-06, -1.9430925021879375e-05, -0.04699696600437164, -0.004694867879152298, -0.00018559163436293602, -7.903263758635148e-05, -0.11924596130847931, -0.0010119322687387466, -2.062299427052494e-05, -3.2186455882765586e-06, -0.0054848892614245415, -0.32436564564704895, -7.152531907195225e-06, -6.318072337307967e-06, -7.033323527139146e-06, -7.748573807475623e-06, -2.861018856492592e-06, -7.152555099310121e-07, -5.960462772236497e-07, -9.536738616588991e-07, -2.7418097943154862e-06, -8.940656698541716e-06, -5.722029527532868e-06, -2.50339189733495e-06, -1.5497195136049413e-06, -3.576278118089249e-07, -4.851700214203447e-05, -2.50339189733495e-06, -4.637133679352701e-05, -2.3841855067985307e-07, -0.0033528562635183334, -0.6507970094680786, -0.026352116838097572, -1.6689286894688848e-06, -2.264974000354414e-06, -0.05509307235479355, -2.145764938177308e-06, -1.1920928244535389e-07, -3.576278118089249e-07, -1.07287787614041e-05, -0.09035531431436539, -0.0011747133685275912, -1.3224520683288574, -0.01460198499262333, -0.07419967651367188, -0.21408575773239136, -0.7813470363616943, -0.3447580933570862, -0.0012646066024899483, -0.0036896266974508762, -1.8575185537338257, -0.2271762192249298, -2.2355873584747314, -2.069409132003784, -0.07441648095846176, -6.353653589030728e-05, -0.6057047843933105, -0.11436229944229126, -0.9515644907951355, -1.4275494813919067, -0.12727878987789154, -0.13965141773223877, -0.003276220755651593, -5.245195097813848e-06, -3.0412983894348145, -0.3943798840045929, -0.6296138763427734, -0.7824786305427551, -3.10367751121521, -0.9076679944992065, -0.1552063226699829, -0.0013456823071464896, -5.483612312673358e-06, -1.4062148332595825, -0.611797571182251, -8.594620157964528e-05, -0.029750533401966095, -0.08429189771413803, -0.26024895906448364, -0.03963002935051918, -1.2755313036905136e-05, -0.00031394799589179456, -0.10452250391244888, -3.9457496313843876e-05, -5.960462772236497e-07, -3.2186455882765586e-06, -6.6756979322235566e-06, -2.9802276912960224e-06, -1.9073468138230965e-06, -5.960462772236497e-07, -4.768370445162873e-07, -1.1920922133867862e-06, -5.8412379075889476e-06, -6.198863957251888e-06, -2.3841855067985307e-07, -4.768370445162873e-07, -3.576278118089249e-07, -2.6940935640595853e-05, -2.861018856492592e-06, -4.0649541915627196e-05, -5.960462772236497e-07, -0.03446047753095627, -0.0025254758074879646, -2.0265558760002023e-06, -2.3841830625315197e-06, -1.4100860357284546, -1.8130009174346924, -0.00016389934171456844, -3.111314072157256e-05, -0.04077228903770447, -0.00041166413575410843, -2.50339189733495e-06, -1.6689286894688848e-06, -1.311301275563892e-06, -9.536738616588991e-07, -8.344646857949556e-07, -8.702239938429557e-06, -4.768370445162873e-07, -8.344646857949556e-07, -3.576278118089249e-07, -1.1920928244535389e-07, -5.960462772236497e-07, -5.125986263010418e-06, -2.0265558760002023e-06, -1.1920928244535389e-07, -2.3841855067985307e-07, -2.3841855067985307e-07, -1.2755313036905136e-05, -2.861018856492592e-06, -1.4066597032069694e-05, -4.768370445162873e-07, -3.576278118089249e-07], "index": 4480, "score": 0.0}], "simple_geometry": [{"prompt_tokens": [248045, 8678, 198, 21365, 264, 3377, 11, 678, 3274, 369, 310, 4087, 279, 3296, 539, 7047, 2923, 13909, 28253, 303, 264, 2708, 321, 3050, 11233, 13, 198, 12162, 488, 599, 3272, 883, 279, 31626, 1817, 11, 3300, 279, 4087, 303, 279, 2614, 3443, 25, 198, 27, 8944, 29, 8944, 1532, 510, 8944, 29, 198, 5236, 524, 10033, 678, 31626, 4613, 279, 4087, 9212, 11, 3300, 1132, 279, 1534, 4087, 13, 3095, 449, 3010, 369, 3766, 11, 488, 1220, 24660, 1732, 279, 3443, 314, 279, 2468, 14, 8944, 303, 421, 3010, 13, 248046, 198, 248045, 846, 198, 21365, 264, 63704, 28412, 440, 220, 18, 10744, 11, 1141, 1118, 220, 17, 14362, 25111, 513, 25, 220, 17, 23, 13, 15, 214679, 220, 18, 21, 13, 15, 224024, 3437, 369, 279, 6420, 314, 279, 9376, 14362, 8937, 318, 258, 11995, 11504, 5423, 1132, 279, 8937, 430, 678, 4087, 32484, 524, 2873, 279, 7896, 303, 678, 4087, 13, 248046, 198, 248045, 74455, 198, 248068, 198], "tokens": [760, 1156, 6587, 310, 1423, 279, 6420, 314, 279, 4688, 14362, 8937, 314, 264, 63704, 28412, 440, 220, 18, 10744, 318, 64, 20837, 553, 271, 16, 13, 220, 2972, 27382, 1386, 279, 5893, 64700, 357, 28412, 440, 220, 18, 10744, 369, 264, 20837, 13, 198, 17, 13, 220, 2972, 3692, 528, 279, 3234, 314, 14362, 25111, 314, 264, 20837, 64700, 561, 2542, 314, 279, 14362, 25111, 314, 264, 20837, 369, 2590, 220, 16, 23, 15, 11995, 13, 198, 18, 13, 220, 2972, 27382, 1386, 279, 3750, 25111, 64700, 561, 1118, 8937, 369, 393, 17, 23, 13, 15, 24094, 41868, 3, 321, 279, 2018, 8937, 369, 393, 18, 21, 13, 15, 24094, 41868, 12576, 198, 19, 13, 220, 2972, 1591, 685, 279, 22868, 64700, 6558, 393, 87, 3, 381, 279, 6420, 314, 279, 9376, 8937, 13, 198, 262, 25273, 17, 23, 13, 15, 478, 220, 18, 21, 13, 15, 478, 830, 283, 220, 16, 23, 15, 13682, 198, 20, 13, 220, 2972, 46270, 279, 2542, 314, 279, 3750, 25111, 64700, 198, 262, 25273, 17, 23, 13, 15, 478, 220, 18, 21, 13, 15, 283, 220, 21, 19, 13, 15, 13682, 198, 21, 13, 220, 2972, 50, 3815, 364, 393, 87, 60439, 332, 198, 262, 25273, 87, 283, 220, 16, 23, 15, 471, 220, 21, 19, 13, 15, 13682, 198, 262, 25273, 87, 283, 220, 16, 16, 21, 13, 15, 13682, 198, 22, 13, 220, 2972, 3840, 279, 3443, 64700, 561, 1156, 4439, 364, 328, 3138, 279, 8937, 1, 321, 328, 5236, 524, 2873, 279, 7896, 3158, 561, 3010, 3766, 303, 279, 9640, 4774, 279, 3443, 5802, 5686, 1040, 1066, 279, 1324, 13, 13428, 11, 3221, 506, 279, 7359, 328, 4350, 449, 3010, 369, 3766, 11, 488, 1220, 24660, 1732, 279, 3443, 314, 279, 2468, 14, 8944, 303, 421, 3010, 1149, 198, 262, 348, 256, 561, 9640, 3070, 914, 3404, 6435, 264, 3050, 3010, 314, 2468, 3443, 4613, 279, 1414, 3766, 11, 694, 424, 2640, 25, 328, 5423, 1132, 279, 8937, 430, 678, 4087, 13, 3054, 524, 2873, 279, 7896, 303, 678, 4087, 1149, 198, 262, 348, 256, 561, 5129, 3443, 10897, 369, 29587, 8944, 29, 8944, 1532, 510, 8944, 29, 27653, 271, 23, 13, 220, 2972, 18770, 21261, 64700, 198, 262, 393, 16, 23, 15, 471, 220, 21, 19, 283, 220, 16, 16, 21, 12576, 271, 262, 6558, 579, 1923, 1716, 13, 198, 262, 393, 17, 23, 478, 220, 18, 21, 283, 220, 21, 19, 12576, 198, 262, 393, 16, 23, 15, 471, 220, 21, 19, 283, 220, 16, 16, 21, 12576, 198, 262, 2091, 424, 63704, 30, 7179, 11, 393, 16, 16, 21, 198, 248069, 271, 760, 2542, 314, 279, 14362, 25111, 314, 264, 63704, 28412, 440, 393, 77, 3, 10744, 369, 2574, 539, 279, 14377, 4771, 77, 12, 17, 8, 1088, 14695, 220, 16, 23, 15, 24094, 41868, 12576, 198, 2381, 264, 28412, 440, 220, 18, 10744, 318, 64, 20837, 681, 393, 77, 28, 18, 12576, 198, 8917, 393, 28, 318, 18, 12, 17, 8, 1088, 14695, 220, 16, 23, 15, 24094, 41868, 283, 220, 16, 1088, 14695, 220, 16, 23, 15, 24094, 41868, 283, 220, 16, 23, 15, 24094, 41868, 12576, 271, 21365, 1330, 25111, 25, 393, 17, 23, 13, 15, 24094, 41868, 3, 321, 393, 18, 21, 13, 15, 24094, 41868, 12576, 198, 9764, 279, 4688, 8937, 381, 393, 87, 12576, 198, 3, 17, 23, 13, 15, 478, 220, 18, 21, 13, 15, 478, 830, 283, 220, 16, 23, 15, 3, 198, 3, 21, 19, 13, 15, 478, 830, 283, 220, 16, 23, 15, 3, 198, 60859, 283, 220, 16, 23, 15, 471, 220, 21, 19, 13, 15, 3, 198, 60859, 283, 220, 16, 16, 21, 13, 15, 3, 271, 27, 8944, 29, 16, 16, 21, 13, 15, 510, 8944, 29, 248046], "teacher_logprobs": [-0.2818501889705658, -0.033768635243177414, -0.000537727726623416, -0.28120049834251404, -5.483612312673358e-06, -2.3841855067985307e-07, -0.02212379314005375, 0.0, -2.3841830625315197e-06, -0.03465855494141579, -0.11290466040372849, -1.5497195136049413e-06, -5.960462772236497e-07, -1.0728830375228426e-06, -0.1359478384256363, -0.0013865152141079307, -0.0010212211636826396, -2.622600959512056e-06, -3.814689989667386e-06, -8.583032467868179e-06, -0.1893223375082016, -7.462222856702283e-05, -7.510157047363464e-06, -0.25207579135894775, -0.14268015325069427, -0.00021336186910048127, -2.3841830625315197e-06, -1.6331539882230572e-05, -3.576214658096433e-05, -0.00117649941239506, 0.0, -2.2053474822314456e-05, -1.066859245300293, -0.7595238089561462, -0.07992114126682281, -0.4742719531059265, -1.1920928244535389e-07, -1.1920922133867862e-06, -7.152555099310121e-07, -1.2397689715726301e-05, -5.960462772236497e-07, -0.00038068200228735805, -4.529942543740617e-06, -2.0265558760002023e-06, -1.0609570381348021e-05, -5.245195097813848e-06, -2.3841855067985307e-07, -1.9073468138230965e-06, -2.8013790142722428e-05, -0.004164832178503275, -1.9073468138230965e-06, -0.00344089069403708, -0.0350906141102314, -0.0125422989949584, -0.04421280696988106, -4.0531076592742465e-06, -0.6372246146202087, -4.172316494077677e-06, -0.0015510206576436758, -1.2874520507466514e-05, -5.602820692729438e-06, -2.7418097943154862e-06, -1.1920928244535389e-07, -0.00014065706636756659, -0.0077801295556128025, -7.390948667307384e-06, -2.455681169521995e-05, -0.10022750496864319, -0.005303243175148964, -0.0007656264933757484, -0.015140256844460964, -0.5760250091552734, -1.1920928244535389e-07, 0.0, -2.3841855067985307e-07, -0.03370017558336258, -0.0003761537664104253, -0.0013223483692854643, -0.0086353225633502, -7.152555099310121e-07, -5.8412379075889476e-06, -8.344646857949556e-07, -0.22655056416988373, -1.1920928244535389e-07, -0.007984624244272709, -3.7732794284820557, -0.9314572811126709, -1.5497195136049413e-06, -1.0552691221237183, -0.6697893738746643, -1.50248122215271, -0.0029867347329854965, -0.22562545537948608, -0.001091004116460681, -4.0531076592742465e-06, -0.0008201810996979475, -7.152555099310121e-07, -0.029768235981464386, -3.576278118089249e-07, -0.09906182438135147, -2.90866428258596e-05, -1.1920928244535389e-07, -1.5497195136049413e-06, -0.001505433232523501, -8.344646857949556e-07, -2.6702524337451905e-05, -2.861018856492592e-06, -5.960462772236497e-07, -4.172316494077677e-06, -2.264974000354414e-06, -2.622600959512056e-06, -4.768370445162873e-07, -1.1444026313256472e-05, -0.0036899829283356667, -0.00023398046323563904, -3.576278118089249e-07, -9.536738616588991e-07, -1.0013530300057027e-05, -0.033416252583265305, 0.0, -0.0003355178632773459, -7.152555099310121e-07, -2.47952248173533e-05, -0.0005554801900871098, -0.19493639469146729, -0.00040737437666393816, -1.6689286894688848e-06, 0.0, -2.3841855067985307e-07, -0.03093729540705681, 0.0, 0.0, -0.4742822051048279, -1.9171180725097656, -3.3378546504536644e-06, -0.0052381413988769054, -0.0007195267826318741, -0.4289077818393707, -0.0431283675134182, -3.576278118089249e-07, -0.03808318451046944, -4.768370445162873e-07, -0.00017987063620239496, -5.722029527532868e-06, 0.0, -1.0251946150674485e-05, -1.5258672647178173e-05, -4.768370445162873e-07, -1.4305104514278355e-06, -7.152555099310121e-07, -5.960462772236497e-07, -7.152555099310121e-07, -5.364403477869928e-06, -8.344646857949556e-07, -3.576278118089249e-07, -2.682172998902388e-05, -9.536738616588991e-07, -5.006664650863968e-05, -2.3841855067985307e-07, -5.960462772236497e-07, 0.0, -0.7644615173339844, -2.634490556374658e-05, -4.351044481154531e-05, -1.1920922133867862e-06, -0.14271312952041626, -0.00046075694262981415, -7.152555099310121e-07, -1.1920922133867862e-06, -0.004092295654118061, -4.768370445162873e-07, -9.786603914108127e-05, -7.033323527139146e-06, -1.9073468138230965e-06, -0.20141635835170746, -3.576278118089249e-07, -1.1920928244535389e-07, -1.7881377516459906e-06, -1.311301275563892e-06, -2.3841855067985307e-07, -7.676783570786938e-05, -5.006777428206988e-06, -1.6689286894688848e-06, -1.5497195136049413e-06, -1.4424220353248529e-05, -7.271740287251305e-06, -3.182837463100441e-05, -2.3841855067985307e-07, -2.407998726994265e-05, -5.960462772236497e-07, -0.0009268993744626641, 0.0, -1.1920928244535389e-07, -2.3841855067985307e-07, -5.447716102935374e-05, -3.4570634852570947e-06, 0.0, -0.11339638382196426, -2.3841855067985307e-07, -3.2186455882765586e-06, -1.4305104514278355e-06, 0.0, -8.344646857949556e-07, -1.4305104514278355e-06, -0.12697918713092804, -0.0002037079248111695, -2.3841855067985307e-07, -2.50339189733495e-06, -1.311301275563892e-06, -3.576278118089249e-07, -4.291525328881107e-06, -3.576278118089249e-07, -8.344646857949556e-07, -1.9073468138230965e-06, -0.002478862414136529, -7.152555099310121e-07, -3.576278118089249e-07, -3.576278118089249e-07, -0.000558220490347594, -3.2543604902457446e-05, -0.008630831725895405, -2.861018856492592e-06, -2.0265558760002023e-06, -4.529942543740617e-06, -7.617183291586116e-05, -7.986990567587782e-06, -0.9742062091827393, -2.3841855067985307e-07, -2.7418097943154862e-06, -0.014282267540693283, -0.0028461208567023277, -2.3841855067985307e-07, -7.390948667307384e-06, -4.172316494077677e-06, -0.4579479694366455, -0.8778383135795593, -1.4113974571228027, -0.30611875653266907, -0.018927205353975296, -0.03075963631272316, -2.029618263244629, -0.6328969597816467, -1.8409630060195923, -0.1927807778120041, -5.125986263010418e-06, -0.003008722560480237, -0.891208291053772, -0.006331622134894133, -0.0015705404803156853, -0.0070914714597165585, -9.536738616588991e-07, -1.8358061424805783e-05, -2.7418097943154862e-06, -2.169585604860913e-05, -0.06997168809175491, -1.2374993562698364, -0.5863826870918274, -3.18587589263916, -0.9235706329345703, -0.09311021119356155, -0.13756263256072998, -7.41780424118042, -1.8065468072891235, -0.63592129945755, -6.066205978393555, -4.856618404388428, -0.0030690969433635473, -1.2046600580215454, -0.014739193953573704, -0.00021252757869660854, -0.6860020160675049, -1.0941998958587646, -1.1920928244535389e-07, -1.3594790697097778, -0.0001045410826918669, -0.006959720514714718, -2.6116127967834473, -0.22899018228054047, -1.9625890254974365, -2.002696055569686e-05, -1.1920922133867862e-06, -1.6689286894688848e-06, -6.318072337307967e-06, -0.40864649415016174, -2.3007127310847864e-05, -3.93382906622719e-05, -3.6954811548639555e-06, -8.821448318485636e-06, -1.1205610462639015e-05, -0.0003551805857568979, -0.09044570475816727, -3.6954811548639555e-06, -9.512448741588742e-05, -7.152555099310121e-07, -3.0517112463712692e-05, -3.099436753473128e-06, -0.0003190723655279726, -2.0265558760002023e-06, -0.013640644028782845, -1.290639042854309, -0.00879757571965456, -0.6215551495552063, -0.00710863433778286, -1.1323798894882202, -0.015378469601273537, -1.9952421188354492, -2.145764938177308e-06, -0.38266873359680176, -4.165553569793701, -1.3133714199066162, -0.16852927207946777, -0.2462300956249237, -1.1550663709640503, -1.6661226749420166, -0.2693195939064026, -4.277515411376953, -0.021144097670912743, -0.15085668861865997, -0.7076634168624878, -0.520551323890686, -0.6833359599113464, -0.9295293688774109, -0.20617684721946716, -4.512855052947998, -0.18954043090343475, -0.174388125538826, -4.2437604861333966e-05, -7.033323527139146e-06, -0.0004175029753241688, -9.893881360767409e-05, -2.2172682292875834e-05, -9.536697689327411e-06, -0.2539406418800354, -0.0023321115877479315, -5.960462772236497e-07, -2.3841574147809297e-05, -7.152555099310121e-07, -1.1801649634435307e-05, -6.460934673668817e-05, -6.675497570540756e-05, -1.6569954823353328e-05, -0.0011848341673612595, -0.2033270299434662, -0.00038092033355496824, -2.109982233378105e-05, -3.015949550899677e-05, -1.660201907157898, -1.4946410655975342, -0.35728269815444946, -0.10533513873815536, -0.14194458723068237, -0.02638961933553219, -1.6569954823353328e-05, -0.0008182753226719797, -0.17239591479301453, -0.0007465674425475299, -1.311301275563892e-06, -3.2782016205601394e-05, -7.676783570786938e-05, -1.811964830267243e-05, -0.8260185122489929, -1.9836232662200928, -7.152555099310121e-07, -2.7418097943154862e-06, -5.245195097813848e-06, -0.33585724234580994, -1.4970927238464355, -0.5135365724563599, -0.38719069957733154, -4.529942543740617e-06, -0.13085663318634033, -0.02633458375930786, -0.00015341058315243572, -1.3232143828645349e-05, -6.198863957251888e-06, -0.4740975499153137, -0.42893335223197937, -5.304672595229931e-05, -1.07287787614041e-05, -5.960462772236497e-07, -1.3470558769768104e-05, -9.894321920000948e-06, -6.19869097135961e-05, -0.1426967978477478, -0.31333214044570923, -0.7050686478614807, -0.3464478850364685, -4.8397800128441304e-05, -0.0016811060486361384, -0.2811744213104248, -0.24791023135185242, -0.021023524925112724, -5.519237674889155e-05, -1.0928688049316406, -0.014525619335472584, -0.04314526543021202, -1.811964830267243e-05, -4.768370445162873e-07, -0.00016485285595990717, -8.070142939686775e-05, -0.004205550067126751, -0.00011419598013162613, -0.0016030330443754792, -7.462222856702283e-05, -0.11282607167959213, -1.3708974620385561e-05, -1.1920928244535389e-07, -3.4570634852570947e-06, -0.029834190383553505, -0.0007444233051501215, -1.1920922133867862e-06, -3.576272320060525e-06, -7.152555099310121e-07, -4.95898348162882e-05, -0.061983637511730194, -0.013519514352083206, -1.168244216387393e-05, -9.97731985989958e-05, -7.521823135903105e-05, -2.038458114839159e-05, -1.9073468138230965e-06, -0.38687846064567566, -0.0009223736124113202, -0.7004821300506592, -0.15436530113220215, -1.0713022947311401, -1.4305104514278355e-06, -0.33854013681411743, -3.433168603805825e-05, -0.4474891424179077, -0.0008737798780202866, -1.5735502529423684e-05, -6.09140915912576e-05, -2.7130842208862305, -4.1483970562694594e-05, -1.0371154530730564e-05, -0.6532222628593445, -0.19861488044261932, -5.960462772236497e-07, -0.001328896265476942, -6.05564855504781e-05, -3.576272320060525e-06, -5.400034933700226e-05, -0.02054797299206257, -1.353020429611206, -6.603976362384856e-05, -0.0003216941258870065, -0.02987804263830185, -2.7179348762729205e-05, -3.707340147229843e-05, -1.883488948806189e-05, -6.031808152329177e-05, -0.0007397775771096349, -1.1920928244535389e-07, -0.02327980473637581, -1.9073468138230965e-06, -0.0017952292691916227, -4.768370445162873e-07, -0.08893989026546478, -1.7881377516459906e-06, -1.6689286894688848e-06, -8.4638240878121e-06, -4.768370445162873e-07, -8.344646857949556e-07, -2.0265558760002023e-06, -5.960462772236497e-07, 0.0, -0.09243670850992203, -2.3841855067985307e-07, -1.0013530300057027e-05, -0.14395281672477722, -0.002569114323705435, -8.201262971851975e-05, -0.04958260431885719, 0.0, -0.0002960720448754728, -3.576278118089249e-07, -1.9073468138230965e-06, -0.6327539086341858, -0.00021145492792129517, -8.106198947643861e-06, -0.048689864575862885, -0.8989375829696655, -1.1920928244535389e-07, -0.04859175905585289, -3.576278118089249e-07, -0.07981108129024506, -1.1801649634435307e-05, -0.049142349511384964, -1.9470317363739014, -1.4424220353248529e-05, -1.1920928244535389e-07, 0.0, -0.00010895135346800089, -7.152555099310121e-07, 0.0, -7.152555099310121e-07, 0.0, -3.576278118089249e-07, -4.410734163684538e-06, -9.536738616588991e-07, 0.0, -0.0036019703838974237, -5.960462772236497e-07, -9.536738616588991e-07, -1.1920928244535389e-07, -5.960462772236497e-07, -0.0015310243470594287, -1.1920928244535389e-07, -2.145764938177308e-06, -2.3841855067985307e-07, -3.2186455882765586e-06, -2.0265558760002023e-06, -3.576278118089249e-07, -1.6689286894688848e-06, 0.0, -4.768370445162873e-07, -5.960462772236497e-07, -3.576278118089249e-07, -4.410734163684538e-06, -5.125986263010418e-06, -1.1920928244535389e-07, -1.6689286894688848e-06, -0.018151482567191124, -0.34201860427856445, -0.9205870032310486, -0.014305888675153255, -1.4124295711517334, -0.10044089704751968, -0.00018880968855228275, -1.0728830375228426e-06, -0.004622013308107853, -7.152555099310121e-07, -9.65590606938349e-06, -1.1920928244535389e-07, -3.576278118089249e-07, 0.0, -3.576272320060525e-06, -1.4305104514278355e-06, -4.768370445162873e-07, -5.006777428206988e-06, -3.576278118089249e-07, -2.7418097943154862e-06, -2.3841855067985307e-07, -2.0265558760002023e-06, -1.5020257706055418e-05, -0.021034030243754387, -0.20144881308078766, -0.09451215714216232, -5.364403477869928e-06, -1.1920928244535389e-07, -0.0002037079248111695, -3.576272320060525e-06, -1.4305104514278355e-06, -8.49926145747304e-05, -0.19351142644882202, -1.2278481335670222e-05, -3.3378546504536644e-06, -0.0003361137059982866, -2.145764938177308e-06, -0.001929566147737205, -7.152555099310121e-07, -2.3841855067985307e-07, -1.1920922133867862e-06, -6.198863957251888e-06, -1.4305104514278355e-06, -9.536738616588991e-07, -1.311301275563892e-06, -2.3841855067985307e-07, -1.311301275563892e-06, -1.311301275563892e-06, -1.1920928244535389e-07, -3.576278118089249e-07, -0.0006515049026347697, -2.145764938177308e-06, -7.152531907195225e-06, -4.410734163684538e-06, -2.0265558760002023e-06, -0.12695398926734924, -8.344646857949556e-07, -4.768370445162873e-07, -3.6954811548639555e-06, -8.344646857949556e-07, -2.6940935640595853e-05, -8.344646857949556e-07, -1.1920928244535389e-07, -2.3841855067985307e-07, -7.152531907195225e-06, -1.1920928244535389e-07, -1.2159273865108844e-05, -1.1920928244535389e-07, -2.3841855067985307e-07, 0.0, -1.5497195136049413e-06, -2.3841855067985307e-07, -1.1920928244535389e-07, -1.1920928244535389e-07, -1.4305104514278355e-06, -3.3378546504536644e-06, -0.020551243796944618, 0.0, -1.0728830375228426e-06, 0.0, -5.8412379075889476e-06, -1.1920928244535389e-07, -9.536738616588991e-07, -1.0728830375228426e-06, -1.6689286894688848e-06, -4.0531076592742465e-06, -2.2380640506744385, -1.5497195136049413e-06, -1.0967194612021558e-05, -0.0010873125866055489, -0.5919036865234375, -9.536738616588991e-07, -2.2411095415009186e-05, -0.000176652567461133, -2.109982233378105e-05, -2.253030106658116e-05, -2.018218994140625, -1.1086402082582936e-05, -0.00027307120035402477, -1.9907753085135482e-05, -1.0371154530730564e-05, -5.924526340095326e-05], "student_logprobs": [-0.1210956946015358, -0.06318839639425278, -0.0032732500694692135, -0.2254779040813446, -4.362964682513848e-05, 0.0, -0.01675345189869404, 0.0, -7.152555099310121e-07, -0.08053680509328842, -0.07000470161437988, -1.5497195136049413e-06, -1.7881377516459906e-06, -1.311301275563892e-06, -0.04963977634906769, -0.0007626485894434154, -0.0022521864157170057, -5.8412379075889476e-06, -3.099436753473128e-06, -8.940656698541716e-06, -0.17835749685764313, -0.00036399890086613595, -8.4638240878121e-06, -0.3695593476295471, -0.5231347680091858, -0.0025122768711298704, -5.483612312673358e-06, -0.00014065706636756659, -0.00012659224739763886, -0.0011547094909474254, 0.0, -7.748573807475623e-06, -0.950526237487793, -0.5789155960083008, -0.14725011587142944, -0.7579206228256226, -4.768370445162873e-07, -9.179073458653875e-06, -2.622600959512056e-06, -1.4185804502631072e-05, -1.311301275563892e-06, -0.0002320735511602834, -5.006777428206988e-06, -1.6689160474925302e-05, -5.94836674281396e-05, -1.0013530300057027e-05, -7.152555099310121e-07, -5.722029527532868e-06, -4.7205765440594405e-05, -0.03544677793979645, -2.264974000354414e-06, -0.009622382931411266, -0.05294483155012131, -0.014245718717575073, -0.12694348394870758, -6.6756979322235566e-06, -0.5271027088165283, -1.1086402082582936e-05, -0.0025754161179065704, -1.6927575416048057e-05, -2.2411095415009186e-05, -5.006777428206988e-06, -1.1920928244535389e-07, -0.0002335037279408425, -0.02362273819744587, -1.0251946150674485e-05, -5.185469490243122e-05, -0.08895930647850037, -0.0051231058314442635, -0.0007539528887718916, -0.005074241664260626, -0.6933821439743042, -2.3841855067985307e-07, 0.0, -1.1920928244535389e-07, -0.0700901672244072, -0.0020527022425085306, -0.0008605591137893498, -0.004095857031643391, -7.152555099310121e-07, -8.940656698541716e-06, -1.311301275563892e-06, -0.3168574869632721, -3.576278118089249e-07, -0.039171844720840454, -3.288072109222412, -0.8710388541221619, -3.4570634852570947e-06, -1.2346827983856201, -0.42100512981414795, -1.226946473121643, -0.005992304068058729, -0.20266474783420563, -0.0023076115176081657, -2.861018856492592e-06, -6.961580220377073e-05, -3.576278118089249e-07, -0.054909273982048035, -4.768370445162873e-07, -0.14294326305389404, -2.5748875486897305e-05, -1.1920928244535389e-07, -1.5497195136049413e-06, -0.0013291343348100781, -3.576278118089249e-07, -3.814624506048858e-05, -1.2636104656849056e-05, -1.1920922133867862e-06, -3.3378546504536644e-06, -2.3841855067985307e-07, -8.344646857949556e-07, -9.536738616588991e-07, -1.4066597032069694e-05, -0.005365969613194466, -9.83428253675811e-05, -3.576278118089249e-07, -2.50339189733495e-06, -8.583032467868179e-06, -0.03424370288848877, 0.0, -0.0002960720448754728, -4.768360213347478e-06, -4.51792984677013e-05, -0.001328777172602713, -0.3789931535720825, -0.0012752028414979577, -1.7881377516459906e-06, 0.0, -2.3841855067985307e-07, -0.047230325639247894, 0.0, 0.0, -0.5761807560920715, -1.3199623823165894, -7.986990567587782e-06, -0.006725650746375322, -0.0019394413102418184, -0.4289853572845459, -0.02999408356845379, -2.3841855067985307e-07, -5.8887653722194955e-05, -2.3841855067985307e-07, -0.002805704018101096, -3.3378546504536644e-06, 0.0, -9.059865078597795e-06, -7.390948667307384e-06, -2.3841855067985307e-07, -7.152555099310121e-07, -4.768370445162873e-07, -4.768370445162873e-07, -5.960462772236497e-07, -4.410734163684538e-06, -1.5497195136049413e-06, -5.960462772236497e-07, -0.001511027687229216, -1.311301275563892e-06, -5.447716102935374e-05, -1.1920928244535389e-07, -3.576278118089249e-07, -2.3841855067985307e-07, -0.9857484102249146, -6.997340824455023e-05, -4.4225667807040736e-05, -1.311301275563892e-06, -0.14270775020122528, -0.0003046525234822184, -5.960462772236497e-07, -1.1920922133867862e-06, -0.009781171567738056, -2.3841855067985307e-07, -0.0001599660754436627, -5.364403477869928e-06, -2.9802276912960224e-06, -0.018150780349969864, -4.768370445162873e-07, -1.1920928244535389e-07, -1.0728830375228426e-06, -2.7418097943154862e-06, 0.0, -1.2755313036905136e-05, -4.768370445162873e-07, -2.7418097943154862e-06, -1.1920922133867862e-06, -1.1205610462639015e-05, -3.6954811548639555e-06, -1.311301275563892e-06, -1.1920928244535389e-07, -3.075552376685664e-05, -7.152555099310121e-07, -0.0009278521756641567, 0.0, -1.1920928244535389e-07, -2.3841855067985307e-07, -6.16293036728166e-05, -3.576272320060525e-06, 0.0, -0.16331003606319427, -2.3841855067985307e-07, -6.198863957251888e-06, -2.50339189733495e-06, 0.0, -4.768370445162873e-07, -1.1920922133867862e-06, -0.08897577226161957, -0.00017963226127903908, -1.1920928244535389e-07, -1.7881377516459906e-06, -9.536738616588991e-07, -2.3841855067985307e-07, -8.49926145747304e-05, -1.1920928244535389e-07, -2.3841855067985307e-07, -8.344646857949556e-07, -2.777537883957848e-05, -1.0728830375228426e-06, -3.576278118089249e-07, -5.960462772236497e-07, -0.0010401559993624687, -6.508615479106084e-05, -0.008627521805465221, -1.9073468138230965e-06, -1.1920922133867862e-06, -4.0531076592742465e-06, -1.9550132492440753e-05, -2.9802276912960224e-06, -0.0890125259757042, 0.0, -2.0265558760002023e-06, -0.05497652664780617, -0.0019738255068659782, 0.0, -1.490105023549404e-05, -2.264974000354414e-06, -0.6671292185783386, -0.7231975793838501, -1.2707586288452148, -0.15641818940639496, -0.016812652349472046, -0.024046091362833977, -2.1497626304626465, -0.42888420820236206, -1.6701180934906006, -0.22660425305366516, -3.2186455882765586e-06, -0.003731789067387581, -1.1128575801849365, -0.009317603893578053, -0.0015191215788945556, -0.0090414397418499, -1.6689286894688848e-06, -2.2411095415009186e-05, -2.3841830625315197e-06, -1.537788011773955e-05, -0.06201635301113129, -1.0740509033203125, -0.47543850541114807, -2.882425308227539, -0.9054298996925354, -0.1067727655172348, -0.17047813534736633, -6.661328315734863, -1.8323012590408325, -0.6135590076446533, -6.206151008605957, -4.857578277587891, -0.0024877809919416904, -1.2978012561798096, -0.013237920589745045, -0.000364713923772797, -0.6851308345794678, -0.974402904510498, -1.1920928244535389e-07, -1.1393519639968872, -0.00013159839727450162, -0.006695455405861139, -2.6091573238372803, -0.2057073563337326, -1.9910273551940918, -1.6927575416048057e-05, -2.145764938177308e-06, -7.152555099310121e-07, -5.006777428206988e-06, -0.3106670081615448, -8.22540732769994e-06, -3.58813522325363e-05, -5.125986263010418e-06, -8.4638240878121e-06, -2.50339189733495e-06, -0.0002299282787134871, -0.055287670344114304, -1.1920922133867862e-06, -0.0008985534077510238, -3.576278118089249e-07, -1.168244216387393e-05, -1.7881377516459906e-06, -0.0008780676289461553, -1.9073468138230965e-06, -0.010257267393171787, -1.1018428802490234, -0.007890601642429829, -0.6550048589706421, -0.011845133267343044, -1.1879024505615234, -0.02068938873708248, -1.9919646978378296, -1.9073468138230965e-06, -0.42985957860946655, -3.6678757667541504, -1.3134151697158813, -0.1557859182357788, -0.2338748723268509, -0.8484010696411133, -1.5187757015228271, -0.15120743215084076, -4.164862632751465, -0.021506084129214287, -0.20356357097625732, -0.7101168036460876, -0.5670925378799438, -0.7159036993980408, -0.961832582950592, -0.21071425080299377, -4.142359256744385, -0.19757619500160217, -0.14990651607513428, -3.862306402879767e-05, -2.7418097943154862e-06, -0.00041631137719377875, -6.747018051100895e-05, -1.3589766240329482e-05, -4.529942543740617e-06, -0.24840858578681946, -0.0016563518438488245, -8.344646857949556e-07, -2.5748875486897305e-05, -1.7881377516459906e-06, -1.7881233361549675e-05, -5.900685573578812e-05, -2.3245540432981215e-05, -1.0132738680113107e-05, -0.0009277330245822668, -0.1822039783000946, -0.00038115866482257843, -1.1444026313256472e-05, -2.884823152271565e-05, -2.0729098320007324, -1.6286684274673462, -0.33503830432891846, -0.1375350058078766, -0.14549404382705688, -0.04396311193704605, -2.9801878554280847e-05, -0.0012731788447126746, -0.22272223234176636, -0.0016853903653100133, -7.152555099310121e-07, -2.825220326485578e-05, -7.4741430580616e-05, -1.7404405298293568e-05, -0.8260054588317871, -1.68686842918396, -1.1920922133867862e-06, -4.768360213347478e-06, -5.245195097813848e-06, -0.42130813002586365, -1.546069622039795, -0.7332382798194885, -0.34886306524276733, -7.867782187531702e-06, -0.14084552228450775, -0.016052061691880226, -0.0002134810492862016, -6.9141146923357155e-06, -1.1444026313256472e-05, -0.4287237823009491, -0.42888522148132324, -4.076874756719917e-05, -8.737658936297521e-05, -7.152555099310121e-07, -6.318072337307967e-06, -1.2397689715726301e-05, -4.255681051290594e-05, -0.16026026010513306, -0.7578203678131104, -0.538929283618927, -0.2435109168291092, -3.6954195820726454e-05, -0.007229122798889875, -0.2811971604824066, -0.4443559944629669, -0.026047401130199432, -3.45700973412022e-05, -0.8885443210601807, -0.014341965317726135, -0.026397978886961937, -5.900685573578812e-05, -2.3841855067985307e-07, -0.00036125810584053397, -4.7205765440594405e-05, -0.00336307380348444, -4.911301948595792e-05, -0.0009990707039833069, -3.7788631743751466e-05, -0.11282596737146378, -2.2053474822314456e-05, -1.1920928244535389e-07, -2.264974000354414e-06, -0.016077635809779167, -0.0012256737099960446, -1.311301275563892e-06, -7.629365427419543e-06, -5.960462772236497e-07, -8.797258487902582e-05, -0.029764071106910706, -0.014013935811817646, -1.6927575416048057e-05, -7.188061863416806e-05, -5.030505417380482e-05, -3.755022044060752e-05, -2.3841830625315197e-06, -0.5759481191635132, -0.0017133570509031415, -0.9812970757484436, -0.22703459858894348, -0.801493227481842, -2.0265558760002023e-06, -0.39676210284233093, -2.0265373677830212e-05, -0.327604204416275, -0.000685576000250876, -1.1920858014491387e-05, -4.9828242481453344e-05, -2.0446255207061768, -2.1815061700181104e-05, -3.6000557884108275e-05, -0.6324049234390259, -0.368645042181015, -5.960462772236497e-07, -0.0024808840826153755, -5.090107151772827e-05, -3.576272320060525e-06, -9.929640509653836e-05, -0.02325255051255226, -1.3868604898452759, -6.8662193370983e-05, -0.0008370947907678783, -0.033761028200387955, -5.817244164063595e-05, -7.56950321374461e-05, -9.775113539944869e-06, -5.006664650863968e-05, -0.0016686100279912353, -1.1920928244535389e-07, -0.07894769310951233, -2.264974000354414e-06, -0.0004912600270472467, -4.768370445162873e-07, -0.06993623077869415, -2.0265558760002023e-06, -1.1920922133867862e-06, -5.483612312673358e-06, -2.3841855067985307e-07, -3.576278118089249e-07, -2.3841855067985307e-07, 0.0, 0.0, -0.049639664590358734, -2.3841855067985307e-07, -1.0013530300057027e-05, -0.10792125761508942, -0.004324727226048708, -0.0005609607324004173, -0.044113460928201675, 0.0, -0.0011703077470883727, -2.3841855067985307e-07, -3.814689989667386e-06, -0.632794976234436, -0.00033408781746402383, -8.940656698541716e-06, -0.05500258877873421, -0.9745840430259705, -1.1920928244535389e-07, -0.029754003509879112, -2.3841855067985307e-07, -0.09778549522161484, -9.894321920000948e-06, -0.02988012507557869, -2.603245973587036, -8.821448318485636e-06, -1.1920928244535389e-07, 0.0, -7.486063259420916e-05, -7.152555099310121e-07, 0.0, -5.960462772236497e-07, 0.0, -1.1920928244535389e-07, 0.0, -1.9073468138230965e-06, 0.0, -0.001704074558801949, -5.960462772236497e-07, -4.768370445162873e-07, -1.1920928244535389e-07, -4.768370445162873e-07, -0.001946580014191568, -1.1920928244535389e-07, -3.099436753473128e-06, -2.3841830625315197e-06, 0.0, -3.576278118089249e-07, -7.152555099310121e-07, -2.0265558760002023e-06, 0.0, -4.768370445162873e-07, -1.1920922133867862e-06, -3.576278118089249e-07, -1.0728830375228426e-06, -4.529942543740617e-06, -1.1920928244535389e-07, -2.0265558760002023e-06, -0.00975708942860365, -0.6832998991012573, -0.9979574680328369, -0.03012017346918583, -0.9120010733604431, -0.14300154149532318, -0.00040236959466710687, -4.768370445162873e-07, -0.00020394629973452538, -1.1920928244535389e-07, -1.6689286894688848e-06, -2.3841855067985307e-07, -2.3841855067985307e-07, 0.0, -3.3378546504536644e-06, -2.145764938177308e-06, -9.536738616588991e-07, -1.2636104656849056e-05, -3.576278118089249e-07, -1.311301275563892e-06, -2.3841855067985307e-07, -3.4570634852570947e-06, -2.372236667724792e-05, -0.004250658676028252, -0.25200650095939636, -0.093827985227108, -4.887569048150908e-06, -1.1920928244535389e-07, -0.00014029949670657516, -5.722029527532868e-06, -9.536738616588991e-07, -0.0002961912250611931, -0.2422790229320526, -1.2874520507466514e-05, -2.7418097943154862e-06, -1.1801649634435307e-05, -7.152555099310121e-07, -0.005929972976446152, -4.768370445162873e-07, 0.0, -1.0728830375228426e-06, -2.861018856492592e-06, -1.5497195136049413e-06, -5.960462772236497e-07, -9.536738616588991e-07, -2.3841855067985307e-07, -1.4305104514278355e-06, -1.0728830375228426e-06, -4.768370445162873e-07, -3.576278118089249e-07, -0.007822119630873203, -2.264974000354414e-06, -6.318072337307967e-06, -1.311301275563892e-06, -9.536738616588991e-07, -0.009758152067661285, -3.576278118089249e-07, -3.576278118089249e-07, -2.3841830625315197e-06, -7.152555099310121e-07, -1.9430925021879375e-05, -8.344646857949556e-07, -2.50339189733495e-06, 0.0, -3.4570634852570947e-06, 0.0, -9.775113539944869e-06, -1.1920928244535389e-07, -1.1920928244535389e-07, -2.3841855067985307e-07, -8.344646857949556e-07, -1.1920928244535389e-07, -2.3841855067985307e-07, -1.1920928244535389e-07, -7.152555099310121e-07, -1.9073468138230965e-06, -0.0021871235221624374, 0.0, -7.152555099310121e-07, 0.0, -5.483612312673358e-06, -1.1920928244535389e-07, -8.344646857949556e-07, -1.1920922133867862e-06, -1.7881377516459906e-06, -2.264974000354414e-06, -0.429097056388855, -1.1920928244535389e-07, -1.1324817933200393e-05, -0.00023052419419400394, -0.3162836730480194, -9.536738616588991e-07, -2.8609820219571702e-05, -0.00014232576359063387, -2.288792165927589e-05, -1.3112935448589269e-05, -0.3489367365837097, -3.099436753473128e-06, -0.00013696208770852536, -2.4199192921514623e-05, -1.3232143828645349e-05, -1.9788545614574105e-05], "index": 1720, "score": 0.0}, {"prompt_tokens": [248045, 8678, 198, 21365, 264, 3377, 11, 678, 3274, 369, 310, 4087, 279, 3296, 539, 7047, 2923, 13909, 28253, 303, 264, 2708, 321, 3050, 11233, 13, 198, 12162, 488, 599, 3272, 883, 279, 31626, 1817, 11, 3300, 279, 4087, 303, 279, 2614, 3443, 25, 198, 27, 8944, 29, 8944, 1532, 510, 8944, 29, 198, 5236, 524, 10033, 678, 31626, 4613, 279, 4087, 9212, 11, 3300, 1132, 279, 1534, 4087, 13, 3095, 449, 3010, 369, 3766, 11, 488, 1220, 24660, 1732, 279, 3443, 314, 279, 2468, 14, 8944, 303, 421, 3010, 13, 248046, 198, 248045, 846, 198, 21365, 264, 63704, 28412, 440, 220, 19, 10744, 11, 1141, 1118, 220, 18, 14362, 25111, 513, 25, 220, 20, 15, 13, 15, 214679, 220, 16, 18, 16, 13, 15, 214679, 220, 20, 20, 13, 15, 224024, 3437, 369, 279, 6420, 314, 279, 9376, 14362, 8937, 318, 258, 11995, 11504, 5423, 1132, 279, 8937, 430, 678, 4087, 32484, 524, 2873, 279, 7896, 303, 678, 4087, 13, 248046, 198, 248045, 74455, 198, 248068, 198], "tokens": [760, 1156, 6587, 310, 1423, 279, 6420, 314, 279, 220, 19, 337, 14362, 8937, 314, 264, 63704, 28412, 440, 220, 19, 10744, 13, 271, 16, 13, 220, 2972, 27382, 1386, 279, 28412, 913, 64700, 357, 28412, 440, 220, 19, 10744, 369, 264, 28183, 41520, 13, 198, 17, 13, 220, 2972, 3692, 528, 279, 2542, 314, 14362, 25111, 14377, 64700, 561, 2542, 314, 279, 14362, 25111, 314, 264, 63704, 28412, 440, 393, 77, 3, 10744, 369, 2574, 539, 4771, 77, 471, 220, 17, 8, 1088, 14695, 220, 16, 23, 15, 61, 34521, 41868, 92, 12576, 198, 18, 13, 220, 2972, 46270, 279, 2542, 364, 264, 28183, 41520, 64700, 5514, 11, 393, 77, 283, 220, 19, 12576, 198, 262, 7860, 393, 28, 318, 19, 471, 220, 17, 8, 1088, 14695, 220, 16, 23, 15, 61, 34521, 41868, 92, 283, 220, 17, 1088, 14695, 220, 16, 23, 15, 61, 34521, 41868, 92, 283, 220, 18, 21, 15, 61, 34521, 41868, 92, 12576, 198, 19, 13, 220, 2972, 21365, 2663, 64700, 561, 2250, 3750, 14362, 25111, 513, 393, 20, 15, 13, 15, 61, 34521, 41868, 30675, 11, 393, 16, 18, 16, 13, 15, 61, 34521, 41868, 30675, 11, 321, 393, 20, 20, 13, 15, 61, 34521, 41868, 92, 12576, 198, 20, 13, 220, 2972, 1591, 685, 279, 22868, 64700, 6558, 393, 87, 3, 381, 279, 6420, 314, 279, 220, 19, 337, 8937, 13, 198, 262, 393, 20, 15, 13, 15, 478, 220, 16, 18, 16, 13, 15, 478, 220, 20, 20, 13, 15, 478, 830, 283, 220, 18, 21, 15, 12576, 198, 21, 13, 220, 2972, 50, 3815, 364, 393, 87, 60439, 332, 198, 262, 7860, 314, 2574, 25111, 393, 28, 220, 20, 15, 13, 15, 478, 220, 16, 18, 16, 13, 15, 478, 220, 20, 20, 13, 15, 3, 198, 262, 393, 20, 15, 478, 220, 16, 18, 16, 283, 220, 16, 23, 16, 3, 198, 262, 393, 16, 23, 16, 478, 220, 20, 20, 283, 220, 17, 18, 21, 3, 198, 262, 1987, 11, 393, 17, 18, 21, 478, 830, 283, 220, 18, 21, 15, 3, 198, 262, 393, 87, 283, 220, 18, 21, 15, 471, 220, 17, 18, 21, 3, 198, 262, 393, 87, 283, 220, 16, 17, 19, 3, 198, 22, 13, 220, 2972, 3840, 16484, 64700, 561, 28412, 369, 63704, 13, 357, 63704, 28412, 1902, 599, 660, 14362, 25111, 2599, 1056, 393, 16, 23, 15, 61, 34521, 41868, 92, 12576, 561, 16100, 8937, 369, 393, 16, 17, 19, 61, 34521, 41868, 30675, 11, 864, 369, 2599, 1056, 393, 16, 23, 15, 61, 34521, 41868, 30675, 11, 748, 279, 2876, 369, 18933, 13, 198, 23, 13, 220, 2972, 3925, 279, 4087, 64700, 561, 1156, 10897, 328, 5423, 1132, 279, 8937, 430, 678, 4087, 13, 3054, 524, 2873, 279, 7896, 303, 678, 4087, 1149, 198, 262, 561, 869, 369, 220, 16, 17, 19, 13, 271, 9764, 579, 1923, 1716, 279, 33633, 13, 198, 3, 20, 15, 478, 220, 20, 20, 283, 220, 16, 15, 20, 3, 198, 3, 16, 15, 20, 478, 220, 16, 18, 16, 283, 220, 17, 18, 21, 3, 198, 3, 18, 21, 15, 471, 220, 17, 18, 21, 283, 220, 16, 17, 19, 3, 271, 760, 21261, 369, 4252, 13, 271, 760, 3296, 16561, 364, 279, 6420, 303, 11995, 11, 694, 2640, 328, 5236, 524, 2873, 279, 7896, 3158, 1987, 1066, 279, 1324, 13, 271, 18770, 21134, 4816, 310, 381, 220, 16, 17, 19, 13, 198, 248069, 271, 760, 2542, 314, 279, 14362, 25111, 314, 264, 63704, 28412, 440, 393, 77, 3, 10744, 369, 16100, 1608, 279, 14377, 4771, 77, 471, 220, 17, 8, 1088, 14695, 220, 16, 23, 15, 61, 34521, 41868, 92, 12576, 271, 2381, 264, 28412, 440, 220, 19, 10744, 318, 64, 28183, 41520, 1590, 198, 13682, 59, 1272, 90, 8917, 92, 283, 318, 19, 471, 220, 17, 8, 1088, 14695, 220, 16, 23, 15, 61, 34521, 41868, 92, 283, 220, 17, 1088, 14695, 220, 16, 23, 15, 61, 34521, 41868, 92, 283, 220, 18, 21, 15, 61, 34521, 41868, 92, 13682, 271, 1596, 513, 2574, 2250, 314, 279, 14362, 25111, 25, 393, 20, 15, 13, 15, 61, 34521, 41868, 30675, 11, 393, 16, 18, 16, 13, 15, 61, 34521, 41868, 30675, 11, 321, 393, 20, 20, 13, 15, 61, 34521, 41868, 92, 12576, 6558, 393, 87, 3, 381, 279, 6420, 314, 279, 9376, 8937, 13, 1165, 716, 685, 279, 22868, 25, 198, 13682, 20, 15, 13, 15, 478, 220, 16, 18, 16, 13, 15, 478, 220, 20, 20, 13, 15, 478, 830, 283, 220, 18, 21, 15, 13682, 271, 5170, 11, 2542, 279, 3750, 25111, 25, 198, 13682, 20, 15, 13, 15, 478, 220, 16, 18, 16, 13, 15, 478, 220, 20, 20, 13, 15, 283, 220, 17, 18, 21, 13, 15, 13682, 271, 6820, 11, 11290, 364, 393, 87, 60439, 198, 13682, 17, 18, 21, 13, 15, 478, 830, 283, 220, 18, 21, 15, 13682, 198, 13682, 87, 283, 220, 18, 21, 15, 471, 220, 17, 18, 21, 13682, 198, 13682, 87, 283, 220, 16, 17, 19, 13682, 271, 760, 6420, 314, 279, 9376, 14362, 8937, 369, 220, 16, 17, 19, 11995, 13, 271, 27, 8944, 29, 16, 17, 19, 510, 8944, 29, 248046], "teacher_logprobs": [-0.20307038724422455, -0.055076491087675095, -0.0001793938863556832, -0.1797688603401184, -1.7881377516459906e-06, -2.3841855067985307e-07, -0.02862086333334446, 0.0, -1.7881377516459906e-06, -3.5578315258026123, -5.745722592109814e-05, -8.344646857949556e-07, -0.043062373995780945, -1.7881377516459906e-06, -4.768370445162873e-07, -9.536738616588991e-07, -0.009713409468531609, -0.25345614552497864, -0.01742330752313137, -3.4570634852570947e-06, -7.986990567587782e-06, -5.8412379075889476e-06, -1.0465655326843262, -0.03804910182952881, -0.00038556772051379085, -2.7418097943154862e-06, -3.373566141817719e-05, -2.455681169521995e-05, -0.0017441788222640753, 0.0, -1.3828182090946939e-05, -0.5745489001274109, -0.5463950634002686, -1.1370922327041626, -0.2690209448337555, -0.5762876868247986, -3.576278118089249e-07, -9.536738616588991e-07, -9.536738616588991e-07, -2.264974000354414e-06, -1.1920928244535389e-07, -0.043004363775253296, -3.802703940891661e-05, -1.8358061424805783e-05, -0.0021877181716263294, -0.0028430302627384663, -7.390948667307384e-06, -3.576278118089249e-07, -2.145764938177308e-06, -1.7523612768854946e-05, -0.05519314110279083, -5.483612312673358e-06, -0.00013600854435935616, -0.5441504120826721, -4.768360213347478e-06, -9.65590606938349e-06, -3.933898824470816e-06, -0.3245396316051483, -0.00038485272671096027, -0.6946748495101929, -5.125986263010418e-06, -8.523101132595912e-05, -0.3869664669036865, -0.004159965086728334, -1.1444026313256472e-05, -0.0006721144891344011, -0.3868754804134369, -1.2239466905593872, -0.0009346407605335116, -3.3378546504536644e-06, -2.0265558760002023e-06, -1.4305104514278355e-06, -1.0728830375228426e-06, -6.294052582234144e-05, -4.768360213347478e-06, -0.009763346053659916, -2.3841855067985307e-07, -0.06823853403329849, -3.576278118089249e-07, -0.5759400725364685, -2.3841855067985307e-07, -1.1920928244535389e-07, -4.768370445162873e-07, -8.344646857949556e-07, -7.152555099310121e-07, -7.152555099310121e-07, -2.9802276912960224e-06, -1.1920928244535389e-07, 0.0, -1.2261861562728882, -3.6954811548639555e-06, -9.536738616588991e-07, -1.6569954823353328e-05, -4.768370445162873e-07, -1.6689286894688848e-06, -0.006716888397932053, -9.536738616588991e-07, -4.887569048150908e-06, -2.7418097943154862e-06, -0.6327606439590454, -1.8596476365928538e-05, -0.02343958429992199, -0.03010687045753002, -0.3525923788547516, -8.999896090244874e-05, -7.152531907195225e-06, -0.2254287153482437, -1.805789828300476, -0.5759400725364685, -1.1920928244535389e-07, -1.1920928244535389e-07, -0.000553335587028414, -2.3841855067985307e-07, -6.6756979322235566e-06, -3.3378546504536644e-06, -0.023262567818164825, -5.578839045483619e-05, -0.07709628343582153, -0.38703811168670654, -1.2874520507466514e-05, -1.1920928244535389e-07, -1.1920928244535389e-07, 0.0, -1.7881377516459906e-06, -9.775113539944869e-06, -1.1920928244535389e-07, -2.3841855067985307e-07, 0.0, -7.152555099310121e-07, -1.4185804502631072e-05, 0.0, -4.768370445162873e-07, -0.000262106885202229, -1.1920928244535389e-07, -2.3841855067985307e-07, -0.026303697377443314, -8.49926145747304e-05, 0.0, 0.0, -3.576278118089249e-07, -2.3841855067985307e-07, -2.47952248173533e-05, -2.9802276912960224e-06, -2.3841855067985307e-07, -2.0265558760002023e-06, -4.768360213347478e-06, -1.1920928244535389e-07, -1.5497195136049413e-06, -7.390948667307384e-06, -3.933898824470816e-06, -4.768370445162873e-07, -1.0728830375228426e-06, -3.576278118089249e-07, -9.536738616588991e-07, -1.7881377516459906e-06, -3.576278118089249e-07, -3.576278118089249e-07, -2.861018856492592e-06, -5.960462772236497e-07, -3.3378546504536644e-06, -0.0013319915160536766, -2.3841830625315197e-06, -5.960446742392378e-06, -5.006777428206988e-06, -5.407790660858154, -0.7286739349365234, -4.8040190449682996e-05, -0.1442369669675827, -4.0843281745910645, -0.03987038508057594, -0.428767591714859, -1.4305104514278355e-06, -2.3841855067985307e-07, -0.016721097752451897, -0.007704069372266531, -2.145764938177308e-06, -0.030237920582294464, -7.152555099310121e-07, -0.0015262633096426725, -2.3841855067985307e-07, -7.152555099310121e-07, -0.0002636561985127628, -1.1920928244535389e-07, 0.0, -5.960446742392378e-06, -3.099436753473128e-06, -5.8412379075889476e-06, -4.410734163684538e-06, -1.4305104514278355e-06, -5.722029527532868e-06, -4.768370445162873e-07, -2.3841855067985307e-07, -2.622600959512056e-06, -1.1920928244535389e-07, -4.768370445162873e-07, -3.814689989667386e-06, -4.291525328881107e-06, -4.768370445162873e-07, -2.169585604860913e-05, -1.0728830375228426e-06, -1.4305104514278355e-06, -7.152555099310121e-07, -1.1920922133867862e-06, -1.3470558769768104e-05, -7.152555099310121e-07, -0.004159846343100071, -0.0015240018256008625, -9.536738616588991e-07, -4.410734163684538e-06, -9.417489309271332e-06, -0.16348962485790253, 0.0, -0.0003864018653985113, -7.152555099310121e-07, -0.00015186110977083445, -4.827859811484814e-05, -0.0497099906206131, -9.65590606938349e-06, -9.536738616588991e-07, 0.0, -3.576278118089249e-07, -0.048268526792526245, 0.0, -1.1920928244535389e-07, -2.305232286453247, -2.264974000354414e-06, -4.768370445162873e-07, -0.02332499250769615, -3.576272320060525e-06, -2.3007127310847864e-05, -2.622600959512056e-06, -0.00043108707177452743, -0.02057553455233574, -1.4305104514278355e-06, -0.2519915997982025, -5.960462772236497e-07, -7.652943895664066e-05, -2.264974000354414e-06, -3.4570634852570947e-06, -1.1920922133867862e-06, -4.6491513785440475e-06, -4.6491513785440475e-06, -1.1920922133867862e-06, -3.2186455882765586e-06, -2.038458114839159e-05, -4.768370445162873e-07, -1.585470999998506e-05, -1.549708758830093e-05, -1.6689286894688848e-06, -2.3841830625315197e-06, -2.861018856492592e-06, -1.4305104514278355e-06, -2.50339189733495e-06, -8.344646857949556e-07, -1.311301275563892e-06, -4.768370445162873e-07, -0.79652339220047, -2.0146166207268834e-05, -0.00017951308109331876, -1.1920928244535389e-07, -3.2186455882765586e-06, -8.344646857949556e-07, -2.473557472229004, -0.000964533886872232, -0.0011738798348233104, -0.6524118781089783, -2.145764938177308e-06, -0.0001805857609724626, -3.611976353568025e-05, -6.794906312279636e-06, -1.3708974620385561e-05, -0.409090131521225, -0.00020418466010596603, -4.38768196105957, -1.6689286894688848e-06, -0.011491064913570881, -7.986990567587782e-06, -8.583032467868179e-06, -1.823885577323381e-05, -4.410734163684538e-06, -1.2234543561935425, -5.960462772236497e-07, -5.960462772236497e-07, -1.1920922133867862e-06, -2.586808113846928e-05, -2.145764938177308e-06, -1.549708758830093e-05, -1.9430925021879375e-05, -4.768370445162873e-07, -8.22540732769994e-06, -1.5616295058862306e-05, -3.576272320060525e-06, -3.576272320060525e-06, -1.4662635294371285e-05, -1.1920922133867862e-06, -1.2446088790893555, -2.3841830625315197e-06, -8.702239938429557e-06, -0.0003303935518488288, -0.01112897228449583, -5.364403477869928e-06, -0.0380532331764698, -1.1920928244535389e-07, -0.20141732692718506, -0.0003943857445847243, -0.0009136793087236583, -4.017272294731811e-05, -1.1920922133867862e-06, -2.50339189733495e-06, -3.099436753473128e-06, -4.410734163684538e-06, -2.5510462364763953e-05, -2.3841855067985307e-07, -1.0728830375228426e-06, -4.768370445162873e-07, -3.4570634852570947e-06, -9.775113539944869e-06, -5.125986263010418e-06, -1.6212332411669195e-05, -3.4570634852570947e-06, -2.0265558760002023e-06, -1.1086402082582936e-05, -2.5987286790041253e-05, -1.9073468138230965e-06, -1.7523612768854946e-05, -3.576272320060525e-06, -7.510157047363464e-06, -0.00031120702624320984, -6.198863957251888e-06, -0.0001472126314183697, -0.07397119700908661, -1.0728830375228426e-06, -0.0007070187130011618, -0.0004468158003874123, -5.602820692729438e-06, -1.5139465176616795e-05, -0.04326845332980156, -6.949660019017756e-05, -1.7881377516459906e-06, -6.198863957251888e-06, -3.3378546504536644e-06, -2.3841855067985307e-07, -1.311301275563892e-06, -1.7014415264129639, -4.8636207793606445e-05, -2.7418097943154862e-06, -1.1920922133867862e-06, -1.0728830375228426e-06, -1.0728830375228426e-06, -1.9073468138230965e-06, -2.0265558760002023e-06, -5.960462772236497e-07, -1.0728830375228426e-06, -9.536738616588991e-07, -5.960462772236497e-07, -7.152555099310121e-07, -4.768370445162873e-07, -3.6954811548639555e-06, -2.7179348762729205e-05, -4.887569048150908e-06, -0.0026601902209222317, -0.04309331625699997, -0.1806889921426773, -1.0132738680113107e-05, -2.0503786799963564e-05, -6.723177648382261e-05, -1.7762025890988298e-05, -8.583032467868179e-06, -0.0013255628291517496, -0.17975012958049774, -0.0009010545909404755, -5.960462772236497e-07, -4.255681051290594e-05, -1.156323378381785e-05, -0.19075889885425568, -1.219909906387329, -0.0010996968485414982, -0.2300490438938141, -1.6212518215179443, -0.006725887302309275, -0.00044919890933670104, -0.4741060137748718, -1.987952470779419, -0.11876325309276581, -0.02992941252887249, -0.7976672053337097, -1.0013530300057027e-05, -0.07013306766748428, -3.40932747349143e-05, -4.768360213347478e-06, -0.039826855063438416, -1.1920928244535389e-07, -0.038050249218940735, -4.768370445162873e-07, -4.768370445162873e-07, -5.960462772236497e-07, -0.0001677133986959234, -1.9073468138230965e-06, -4.768370445162873e-07, -0.0006299418746493757, -2.861018856492592e-06, -2.1207621097564697, -0.01562331523746252, -0.00037579628406092525, -0.0019318267004564404, -0.002185458317399025, -2.0265558760002023e-06, -2.145764938177308e-06, -7.664863369427621e-05, -0.009378897957503796, -1.1920922133867862e-06, -1.7881377516459906e-06, -0.00026556302327662706, -1.0251946150674485e-05, -2.8609820219571702e-05, -0.0002992897352669388, -0.00829524826258421, -2.3841855067985307e-07, -0.00010966652916977182, -1.3947389561508317e-05, -7.152555099310121e-07, -7.152555099310121e-07, -0.0002029928145930171, -2.3841830625315197e-06, -7.152555099310121e-07, -2.5789191722869873, -0.004339801147580147, -0.008715211413800716, -0.5093414187431335, -2.1090331077575684, -0.8267994523048401, -0.0040849344804883, -7.176141662057489e-05, -0.02632215991616249, -0.002564001362770796, -5.722029527532868e-06, -1.8954096958623268e-05, -8.4638240878121e-06, -0.04548027738928795, -0.00036090059438720345, -0.31326574087142944, -1.0967194612021558e-05, -0.0028904585633426905, -0.07404425740242004, -0.01538527850061655, -0.48251408338546753, -0.05039248242974281, -3.3378546504536644e-06, -1.1920922133867862e-06, -0.0006456674309447408, -0.0002696150622796267, -3.9457496313843876e-05, -1.5497195136049413e-06, -0.06396687775850296, -0.00014494798961095512, -7.152555099310121e-07, -0.00010966652916977182, -3.099436753473128e-06, -1.0251946150674485e-05, -0.002286083996295929, -3.564294092939235e-05, -3.6954811548639555e-06, -0.0008952185744419694, -1.1728646755218506, -0.00016675988445058465, -0.6048350930213928, -1.4548338651657104, -0.00012420836719684303, -0.012528878636658192, -1.8596476365928538e-05, -5.125986263010418e-06, -9.536697689327411e-06, -0.018408408388495445, -0.12129602581262589, -0.5180586576461792, -0.011111052706837654, -7.629365427419543e-06, -0.28141888976097107, -0.04448649659752846, -1.3929436206817627, -0.00012516192509792745, -2.3841830625315197e-06, -0.09080729633569717, -0.0025378421414643526, -8.964136941358447e-05, -0.0006820021662861109, -1.1920928244535389e-07, -1.5019605159759521, -0.0013402060139924288, -0.0002983363519888371, -5.960462772236497e-07, -1.1920922133867862e-06, -2.622600959512056e-06, -6.437280717364047e-06, -0.023253483697772026, -2.622600959512056e-06, -2.9802276912960224e-06, -5.245195097813848e-06, -0.2254209965467453, -1.1920922133867862e-06, -1.9073468138230965e-06, -3.814689989667386e-06, -1.0251946150674485e-05, -1.1324817933200393e-05, -6.425174069590867e-05, -9.298280929215252e-06, -5.960446742392378e-06, -3.7431014789035544e-05, -7.271740287251305e-06, -7.033323527139146e-06, -6.723177648382261e-05, -5.471556869451888e-05, -1.728519782773219e-05, -0.00035613393993116915, -9.536738616588991e-07, -1.0728830375228426e-06, -2.3841830625315197e-06, -3.576278118089249e-07, -1.7046782886609435e-05, -0.0007232195930555463, -0.0005607224884442985, -0.6163802146911621, -0.0012517482973635197, -3.349725011503324e-05, -2.0146166207268834e-05, -3.123234637314454e-05, -0.05489460378885269, -1.701542615890503, -0.46076899766921997, -0.2386483997106552, -0.020609984174370766, -0.00011753345461329445, -1.4543427823809907e-05, -0.4758332371711731, -4.689886569976807, -1.636433720588684, -0.005506585352122784, -0.003846034873276949, -0.024121267721056938, -2.5923569202423096, -1.5137505531311035, -0.0004923323867842555, -0.5600412487983704, -0.10355498641729355, -2.7013258934020996, -0.01831945776939392, -0.0042190831154584885, -1.0728830375228426e-06, -1.823885577323381e-05, -1.3589766240329482e-05, -3.659658250398934e-05, -0.1428202986717224, -0.9866935610771179, -0.061242859810590744, -0.00615712022408843, -1.1920922133867862e-06, -0.020822497084736824, -0.3140263557434082, -0.1098388209939003, -0.03440277650952339, -0.2935640215873718, -2.3841855067985307e-07, -3.576278118089249e-07, -2.50339189733495e-06, -5.960446742392378e-06, -4.6491513785440475e-06, -2.098061486321967e-05, -0.00012861855793744326, -0.9746541380882263, -0.5500696897506714, -1.4305104514278355e-06, -1.1585915088653564, -0.022273262962698936, -7.033323527139146e-06, -0.0011742371134459972, -0.0001358893496217206, -5.960446742392378e-06, -9.536697689327411e-06, -0.001255200942978263, -0.1035015657544136, -0.018465636298060417, -0.00038652101648040116, -0.002262177411466837, -9.298280929215252e-06, -1.07287787614041e-05, -4.184158387943171e-05, -1.764281842042692e-05, -1.4266210794448853, -0.00038818930624984205, -2.3841855067985307e-07, -2.145764938177308e-06, -0.04935557767748833, -1.1920922133867862e-06, -0.2519375681877136, -1.1920928244535389e-07, -2.3841855067985307e-07, -7.152555099310121e-07, -6.556489552167477e-06, -1.490105023549404e-05, -9.536738616588991e-07, -1.6689286894688848e-06, -1.1920928244535389e-07, -1.1920928244535389e-07, -0.09805680066347122, -3.6954811548639555e-06, -7.152555099310121e-07, -3.957670196541585e-05, -2.145764938177308e-06, -0.9174057245254517, -0.007971970364451408, -1.6212332411669195e-05, -0.3568454384803772, -4.768370445162873e-07, -3.242440288886428e-05, -2.145764938177308e-06, -3.099436753473128e-06, -0.033707551658153534, -0.00016282663273159415, -4.446407547220588e-05, -1.6689286894688848e-06, -1.7052539587020874, -9.536738616588991e-07, -1.647078275680542, -2.6549246311187744, -7.152555099310121e-07, -1.311301275563892e-06, -1.4305012882687151e-05, -0.06995601207017899, -3.099436753473128e-06, -1.1920928244535389e-07, -2.3841855067985307e-07, -1.1920928244535389e-07, -7.152555099310121e-07, -9.536738616588991e-07, 0.0, -5.960462772236497e-07, 0.0, -2.3841855067985307e-07, -1.0728830375228426e-06, -2.50339189733495e-06, -1.7881377516459906e-06, -0.00033682872890494764, -5.960462772236497e-07, -4.0531076592742465e-06, -2.3841855067985307e-07, -4.756337511935271e-05, -2.3841855067985307e-07, -4.637133679352701e-05, -2.0265558760002023e-06, -2.3841855067985307e-07, -1.0013530300057027e-05, -4.887569048150908e-06, -1.1920928244535389e-07, -1.0728830375228426e-06, -2.7418097943154862e-06, -1.7881377516459906e-06, -6.794906312279636e-06, -3.302042750874534e-05, -1.7881377516459906e-06, -4.768370445162873e-07, -1.1920928244535389e-07, -7.152555099310121e-07, -1.5497195136049413e-06, -5.960462772236497e-07, -5.960462772236497e-07, -2.3841855067985307e-07, -2.3841855067985307e-07, -7.271740287251305e-06, -1.7881377516459906e-06, -0.994235634803772, -7.390948667307384e-06, -2.7418097943154862e-06, -0.00862598605453968, -0.07159409672021866, -2.586808113846928e-05, -0.16239899396896362, -1.9430925021879375e-05, -3.814689989667386e-06, -0.005332650151103735, -1.4305104514278355e-06, -4.768370445162873e-07, -0.011059768497943878, -7.152555099310121e-07, -6.544376083184034e-05, 0.0, -5.960462772236497e-07, -0.00030012393835932016, 0.0, 0.0, -4.887569048150908e-06, -1.9073468138230965e-06, -2.7418097943154862e-06, -1.9073468138230965e-06, -1.1920928244535389e-07, -5.960462772236497e-07, -7.152555099310121e-07, -5.960462772236497e-07, -7.033323527139146e-06, 0.0, 0.0, -1.0728830375228426e-06, -1.0728830375228426e-06, -3.576278118089249e-07, -9.775113539944869e-06, -5.960462772236497e-07, -3.576278118089249e-07, -2.3841855067985307e-07, -7.152555099310121e-07, -2.062299427052494e-05, 0.0, -0.11905379593372345, -0.08951427787542343, -1.7881377516459906e-06, -3.576278118089249e-07, -0.000336590368533507, 0.0, -0.0012040039291605353, 0.0, 0.0, -0.578680694103241, -0.514168381690979, -2.50339189733495e-06, -1.2821941375732422, -4.6352009773254395, -8.22540732769994e-06, -4.768360213347478e-06, -0.06199909746646881, -0.00036113892565481365, -0.005933290813118219, -1.847726889536716e-05, -0.00036352223833091557, -3.576278118089249e-07, -0.008632722310721874, -3.576278118089249e-07, -0.006722453515976667, -4.768370445162873e-07, -1.1920922133867862e-06, -8.344646857949556e-07, -1.1920928244535389e-07, -4.529942543740617e-06, -3.576278118089249e-07, -1.311301275563892e-06, -3.933898824470816e-06, -8.344646857949556e-07, -5.8412379075889476e-06, -1.9073468138230965e-06, -2.264974000354414e-06, -9.536738616588991e-07, -8.344646857949556e-07, -2.3841855067985307e-07, -7.152555099310121e-07, -1.1920928244535389e-07, -5.960462772236497e-07, 0.0, -0.004433089401572943, -0.004622369538992643, -0.006695337127894163, -7.152555099310121e-07, -0.13082829117774963, -1.311301275563892e-06, -0.04869190976023674, -2.3841830625315197e-06, -2.3841855067985307e-07, -5.960462772236497e-07, -4.6132929128361866e-05, -3.397406908334233e-05, -2.145764938177308e-06, -0.14268088340759277, -5.960462772236497e-07, -3.576278118089249e-07, -1.1920928244535389e-07, -0.0017034795600920916, -3.576278118089249e-07, -6.556489552167477e-06, -3.2186455882765586e-06, -1.1920928244535389e-07, -0.11282437294721603, -3.4570634852570947e-06, -1.1920922133867862e-06, -1.0728830375228426e-06, -6.079655122448457e-06, -5.960462772236497e-07, -1.0728830375228426e-06, -8.344646857949556e-07, -0.003259229240939021, -8.4638240878121e-06, -3.3378546504536644e-06, -0.050993289798498154, -1.1920928244535389e-07, -0.06997035443782806, -0.00043215948971919715, -0.01481002289801836, -0.0002184867626056075, -0.256600022315979, -1.1920928244535389e-07, -1.9311717551317997e-05, -1.1920928244535389e-07, -0.0003846143954433501, 0.0, -1.585470999998506e-05, -0.252032071352005, -2.3841855067985307e-07, -1.7881377516459906e-06, -0.088961161673069, -3.576278118089249e-07, -8.344646857949556e-07, -3.3378546504536644e-06, -8.344646857949556e-07, -4.768370445162873e-07, -9.536738616588991e-07, -4.768370445162873e-07, -7.152555099310121e-07, -8.034383063204587e-05, -1.0728830375228426e-06, -1.311301275563892e-06, -1.1920922133867862e-06, -3.576278118089249e-07, -4.768370445162873e-07, -8.344646857949556e-07, -2.3841855067985307e-07, -2.3841855067985307e-07, -7.152555099310121e-07, -1.1920928244535389e-07, -9.536738616588991e-07, -1.1920928244535389e-07, -1.1920922133867862e-06, -0.47420957684516907, -5.960462772236497e-07, -1.6689286894688848e-06, -2.50339189733495e-06, -3.576278118089249e-07, -7.152555099310121e-07, -8.4638240878121e-06, -1.0728830375228426e-06, -7.390948667307384e-06, -3.099436753473128e-06, -1.0251946150674485e-05, -0.5807924866676331, -0.12628419697284698, -9.643566590966657e-05, -5.960462772236497e-07, -0.0011094611836597323, -0.11287602037191391, -1.4305104514278355e-06, -1.4305104514278355e-06, -0.7576113939285278, -8.344646857949556e-07, -9.536738616588991e-07, -8.4638240878121e-06, -0.24048283696174622, -0.00013541258522309363, -0.5385320782661438, -4.529942543740617e-06, -1.0728830375228426e-06, -5.8412379075889476e-06, -8.225102646974847e-05, -2.3841855067985307e-07, -3.6954811548639555e-06, -2.264974000354414e-06, -1.537788011773955e-05, -8.344646857949556e-07, -5.960462772236497e-07], "student_logprobs": [-0.05672266334295273, -0.11415717005729675, -0.0017792837461456656, -0.17976227402687073, -1.168244216387393e-05, 0.0, -0.007602094206959009, 0.0, -3.576278118089249e-07, -3.8293259143829346, -6.6756979322235566e-06, -7.152555099310121e-07, -0.014207875356078148, -1.7881377516459906e-06, -2.145764938177308e-06, -1.0728830375228426e-06, -0.0024267768021672964, -0.2281215339899063, -0.01911235973238945, -6.556489552167477e-06, -6.318072337307967e-06, -7.390948667307384e-06, -1.3338613510131836, -0.11284290254116058, -0.00182819040492177, -1.0132738680113107e-05, -0.0004717191040981561, -6.854299135738984e-05, -0.0020987405441701412, 0.0, -9.298280929215252e-06, -0.5003356337547302, -0.7117588520050049, -0.829456090927124, -0.34372442960739136, -0.6342941522598267, -2.145764938177308e-06, -1.9788545614574105e-05, -4.6491513785440475e-06, -4.0531076592742465e-06, -1.9073468138230965e-06, -0.11288389563560486, -7.60526381782256e-05, -3.302042750874534e-05, -0.005953436251729727, -0.011189447715878487, -2.169585604860913e-05, -8.344646857949556e-07, -6.198863957251888e-06, -3.45700973412022e-05, -0.05071151629090309, -6.556489552167477e-06, -0.001302228425629437, -0.696714460849762, -1.6331539882230572e-05, -1.6331539882230572e-05, -7.390948667307384e-06, -0.5082643628120422, -0.0031967272516340017, -0.39548325538635254, -1.0013530300057027e-05, -0.0005535738891921937, -0.25201529264450073, -0.004194391425698996, -1.7165990357170813e-05, -0.0027774113696068525, -0.31327593326568604, -0.8272053003311157, -0.004658797290176153, -6.437280717364047e-06, -6.556489552167477e-06, -5.602820692729438e-06, -2.264974000354414e-06, -7.152301259338856e-05, -5.245195097813848e-06, -0.0021928329952061176, 0.0, -0.16981233656406403, -5.960462772236497e-07, -0.5759400725364685, -1.1920928244535389e-07, -1.1920928244535389e-07, -4.768370445162873e-07, -9.536738616588991e-07, -9.536738616588991e-07, -4.768370445162873e-07, -4.887569048150908e-06, -1.1920928244535389e-07, 0.0, -1.6040120124816895, -5.245195097813848e-06, -1.1920922133867862e-06, -2.098061486321967e-05, -5.960462772236497e-07, -2.9802276912960224e-06, -0.014166505075991154, -8.344646857949556e-07, -9.775113539944869e-06, -2.622600959512056e-06, -0.6932379007339478, -3.671578815556131e-05, -0.016301317140460014, -0.07425279915332794, -0.4801090955734253, -0.00026639728457666934, -6.794906312279636e-06, -0.25199130177497864, -1.162824273109436, -0.523124098777771, -2.3841855067985307e-07, -1.1920928244535389e-07, -0.0013262771535664797, -2.3841855067985307e-07, -8.344646857949556e-07, -1.0013530300057027e-05, -0.054975397884845734, -7.986703712958843e-05, -0.017554858699440956, -0.5233331322669983, -3.0874729418428615e-05, -1.1920928244535389e-07, 0.0, 0.0, -1.311301275563892e-06, -3.576278118089249e-07, 0.0, -2.3841855067985307e-07, -1.1920928244535389e-07, -3.576278118089249e-07, -4.0531076592742465e-06, 0.0, -3.576278118089249e-07, -0.00010978573118336499, -1.1920928244535389e-07, -3.576278118089249e-07, -0.018154526129364967, -5.638440416078083e-05, 0.0, 0.0, -1.4305104514278355e-06, -1.1920928244535389e-07, -8.22540732769994e-06, -5.125986263010418e-06, -8.702239938429557e-06, -2.622600959512056e-06, -7.390948667307384e-06, -1.6689286894688848e-06, -3.3378546504536644e-06, -1.6927575416048057e-05, -3.2186455882765586e-06, -2.3841855067985307e-07, -4.768370445162873e-07, -1.1920928244535389e-07, -2.3841855067985307e-07, -2.50339189733495e-06, -5.960462772236497e-07, -9.536738616588991e-07, -3.099436753473128e-06, -4.768370445162873e-07, -8.702239938429557e-06, -0.00248659192584455, -1.1920922133867862e-06, -1.823885577323381e-05, -7.271740287251305e-06, -3.780691146850586, -1.6535077095031738, -9.83428253675811e-05, -0.24294354021549225, -3.943044900894165, -0.07998012751340866, -0.5231817960739136, -1.9073468138230965e-06, -3.576278118089249e-07, -0.03464945778250694, -0.009858490899205208, -1.311301275563892e-06, -0.003237841185182333, -8.344646857949556e-07, -0.0022021110635250807, -2.3841855067985307e-07, -7.152555099310121e-07, -0.0004323977918829769, -1.1920928244535389e-07, 0.0, -6.079655122448457e-06, -2.145764938177308e-06, -4.887569048150908e-06, -1.6689286894688848e-06, -7.152555099310121e-07, -2.9802276912960224e-06, -5.960462772236497e-07, -8.344646857949556e-07, -1.1920922133867862e-06, -7.152555099310121e-07, -1.7881377516459906e-06, -5.006777428206988e-06, -7.986990567587782e-06, -3.576278118089249e-07, -4.172316494077677e-06, -1.1920922133867862e-06, -2.3841855067985307e-07, -1.6689286894688848e-06, -7.152555099310121e-07, -1.5258672647178173e-05, -4.768370445162873e-07, -0.006095983553677797, -0.0019594295881688595, -8.344646857949556e-07, -5.960446742392378e-06, -2.288792165927589e-05, -0.2433280199766159, 0.0, -0.00021371940965764225, -9.536738616588991e-07, -0.00045503751607611775, -0.00012063252506777644, -0.04463904723525047, -6.079655122448457e-06, -7.152555099310121e-07, 0.0, -2.3841855067985307e-07, -0.0747019425034523, 0.0, 0.0, -1.833946943283081, -8.344646857949556e-07, -3.576278118089249e-07, -0.043113868683576584, -3.6954811548639555e-06, -3.755022044060752e-05, -2.145764938177308e-06, -0.0008592489175498486, -0.018180865794420242, -9.536738616588991e-07, -0.003603514516726136, -1.1920928244535389e-07, -0.0011739989276975393, -2.145764938177308e-06, -2.3841830625315197e-06, -1.311301275563892e-06, -2.3841855067985307e-07, -2.264974000354414e-06, -4.768370445162873e-07, -2.7418097943154862e-06, -2.6702524337451905e-05, -2.3841855067985307e-07, -2.50339189733495e-06, -6.437280717364047e-06, -2.7418097943154862e-06, -1.5497195136049413e-06, -1.6689286894688848e-06, -1.0728830375228426e-06, -1.7881377516459906e-06, -1.1920928244535389e-07, -7.152555099310121e-07, -1.1920928244535389e-07, -1.7315330505371094, -2.2411095415009186e-05, -0.00022206225548870862, -1.1920928244535389e-07, -5.483612312673358e-06, -7.152555099310121e-07, -2.8080644607543945, -0.0015748253790661693, -0.00016008525562938303, -0.4690549373626709, -2.622600959512056e-06, -0.00018082413589581847, -4.160317621426657e-05, -7.986990567587782e-06, -1.2516897186287679e-05, -0.4350610077381134, -0.00020215852418914437, -4.758737564086914, -1.6689286894688848e-06, -0.010151894763112068, -1.0251946150674485e-05, -3.6954811548639555e-06, -1.1920858014491387e-05, -1.311301275563892e-06, -0.2811555862426758, -2.3841855067985307e-07, -1.1920928244535389e-07, -7.152555099310121e-07, -1.966933996300213e-05, -1.311301275563892e-06, -7.152555099310121e-07, -6.6756979322235566e-06, -7.152555099310121e-07, -1.6689286894688848e-06, -1.1801649634435307e-05, -2.622600959512056e-06, -3.4570634852570947e-06, -3.933898824470816e-06, -3.576278118089249e-07, -1.2798550128936768, -2.622600959512056e-06, -5.8412379075889476e-06, -0.00044860312482342124, -0.01259068213403225, -3.4570634852570947e-06, -0.1602344661951065, 0.0, -0.22541537880897522, -0.00047386385267600417, -0.0006288696313276887, -5.1616290875244886e-05, -8.344646857949556e-07, -1.311301275563892e-06, -1.9073468138230965e-06, -2.145764938177308e-06, -3.0874729418428615e-05, -2.3841855067985307e-07, -7.152555099310121e-07, -3.576278118089249e-07, -5.722029527532868e-06, -5.98412734689191e-05, -3.099436753473128e-06, -2.5033637939486653e-05, -2.9802276912960224e-06, -2.264974000354414e-06, -1.3947389561508317e-05, -3.0517112463712692e-05, -1.5497195136049413e-06, -3.1709168979432434e-05, -1.6689286894688848e-06, -1.4305104514278355e-06, -0.0029024637769907713, -5.364403477869928e-06, -7.319182623177767e-05, -0.06364801526069641, -1.311301275563892e-06, -0.0007843278581276536, -0.0004191712068859488, -3.6954811548639555e-06, -7.390948667307384e-06, -0.6937108635902405, -9.178694017464295e-05, -1.311301275563892e-06, -3.814689989667386e-06, -2.3841830625315197e-06, -1.311301275563892e-06, -4.768370445162873e-07, -2.3502235412597656, -0.0001245659514097497, -1.4305104514278355e-06, -8.344646857949556e-07, -7.152555099310121e-07, -5.960462772236497e-07, -8.344646857949556e-07, -8.344646857949556e-07, -5.960462772236497e-07, -5.960462772236497e-07, -8.344646857949556e-07, -4.768370445162873e-07, -7.152555099310121e-07, -2.3841855067985307e-07, -1.6689286894688848e-06, -1.811964830267243e-05, -4.410734163684538e-06, -0.0013543728273361921, -0.026355832815170288, -0.16049574315547943, -6.437280717364047e-06, -1.0013530300057027e-05, -4.029192859889008e-05, -1.2397689715726301e-05, -2.3841830625315197e-06, -0.0029372903518378735, -0.1602286696434021, -0.001111366436816752, -1.1920928244535389e-07, -4.672895011026412e-05, -7.629365427419543e-06, -0.19909699261188507, -1.4286926984786987, -0.00022623363474849612, -0.1843934953212738, -1.163993000984192, -0.01815686747431755, -0.0004976941272616386, -0.5759613513946533, -2.02487850189209, -0.1499001532793045, -0.023345256224274635, -0.6637111902236938, -5.125986263010418e-06, -0.054972801357507706, -2.3364747903542593e-05, -4.887569048150908e-06, -0.035238735377788544, -2.3841855067985307e-07, -0.020557083189487457, -2.3841855067985307e-07, -2.3841855067985307e-07, -3.576278118089249e-07, -7.92710343375802e-05, -1.7881377516459906e-06, -5.960462772236497e-07, -0.00071190285962075, -2.0265558760002023e-06, -1.8454673290252686, -0.01901738904416561, -0.00046397410915233195, -0.0024776733480393887, -0.000380324519937858, -1.1920922133867862e-06, -1.9073468138230965e-06, -4.660974445869215e-05, -0.016428792849183083, -1.5497195136049413e-06, -2.9802276912960224e-06, -0.00023338454775512218, -1.3112935448589269e-05, -3.218599158572033e-05, -0.00014995403762441128, -0.005210984963923693, -2.3841855067985307e-07, -6.651657167822123e-05, -1.490105023549404e-05, -8.344646857949556e-07, -2.3841855067985307e-07, -4.184158387943171e-05, -2.7418097943154862e-06, -1.9073468138230965e-06, -2.237839937210083, -0.003889378160238266, -0.008758462965488434, -0.40527451038360596, -2.203514814376831, -0.7589221000671387, -0.003607671707868576, -4.0649541915627196e-05, -0.026146583259105682, -0.0028780964203178883, -5.006777428206988e-06, -4.184158387943171e-05, -6.318072337307967e-06, -0.051919739693403244, -0.0004353767435532063, -0.2014203518629074, -1.0490362910786644e-05, -0.002621905878186226, -0.11866670101881027, -0.014830577187240124, -0.6287705302238464, -0.07361874729394913, -3.099436753473128e-06, -5.960462772236497e-07, -0.000786591088399291, -0.0008241117466241121, -2.2411095415009186e-05, -1.6689286894688848e-06, -0.0451340489089489, -0.00013422065239865333, -7.152555099310121e-07, -6.854299135738984e-05, -1.4305104514278355e-06, -5.960446742392378e-06, -0.0026228572241961956, -3.075552376685664e-05, -2.3841830625315197e-06, -0.0008198237628675997, -1.208079218864441, -0.0001394651480950415, -0.5928919315338135, -1.2507108449935913, -0.0002109781780745834, -0.03806884214282036, -1.6927575416048057e-05, -6.198863957251888e-06, -5.602820692729438e-06, -0.07165080308914185, -0.6246991157531738, -0.5584753155708313, -0.023308223113417625, -2.109982233378105e-05, -0.2814493775367737, -0.09794040024280548, -1.7614960670471191, -0.000715833914000541, -5.602820692729438e-06, -0.07416480779647827, -0.006061030086129904, -3.9934315282152966e-05, -0.0047214450314641, 0.0, -1.3132762908935547, -0.0010417040903121233, -0.00016056202002801, -4.768370445162873e-07, -8.344646857949556e-07, -2.622600959512056e-06, -6.6756979322235566e-06, -0.0380508229136467, -3.933898824470816e-06, -1.9073468138230965e-06, -3.576272320060525e-06, -0.20141781866550446, -9.536738616588991e-07, -1.0728830375228426e-06, -2.264974000354414e-06, -6.6756979322235566e-06, -7.867782187531702e-06, -4.207999518257566e-05, -4.887569048150908e-06, -4.887569048150908e-06, -4.076874756719917e-05, -7.867782187531702e-06, -4.529942543740617e-06, -0.00020454221521504223, -3.93382906622719e-05, -2.52720492426306e-05, -0.00013207517622504383, -2.145764938177308e-06, -7.152555099310121e-07, -2.9802276912960224e-06, -3.576278118089249e-07, -1.4781842764932662e-05, -0.00018594920402392745, -8.964136941358447e-05, -0.5564578771591187, -0.0007591941393911839, -1.5258672647178173e-05, -1.4662635294371285e-05, -2.47952248173533e-05, -0.048599932342767715, -1.9103267192840576, -0.3317382037639618, -0.3522474467754364, -0.016089601442217827, -9.119095193454996e-05, -3.886147169396281e-05, -0.8297318816184998, -3.8625171184539795, -1.703676700592041, -0.010656239464879036, -0.0035257101990282536, -0.021381234750151634, -2.6980810165405273, -1.4183117151260376, -0.0006082591717131436, -0.4801313281059265, -0.08169900625944138, -2.1982035636901855, -0.01367392297834158, -0.004742445424199104, -1.6689286894688848e-06, -1.7762025890988298e-05, -1.0251946150674485e-05, -2.6940935640595853e-05, -0.1130274310708046, -0.9941242337226868, -0.06691921502351761, -0.006099064368754625, -1.0728830375228426e-06, -0.028955718502402306, -0.5249372124671936, -0.14634419977664948, -0.03656486049294472, -0.3639310300350189, -2.3841855067985307e-07, -2.3841855067985307e-07, -3.576272320060525e-06, -5.483612312673358e-06, -2.9802276912960224e-06, -1.156323378381785e-05, -0.0009689403814263642, -0.7541006803512573, -0.8292323350906372, -1.1801649634435307e-05, -0.849418044090271, -0.055338311940431595, -1.0609570381348021e-05, -0.0010367024224251509, -0.00016449528629891574, -6.6756979322235566e-06, -1.680836794548668e-05, -0.001293775625526905, -0.08695480227470398, -0.030198942869901657, -0.00038652101648040116, -0.0036819064989686012, -7.867782187531702e-06, -1.8000440832111053e-05, -3.6954195820726454e-05, -2.396077979938127e-05, -1.330472707748413, -0.0006649667047895491, -2.3841855067985307e-07, -1.7881377516459906e-06, -0.07972324639558792, -8.344646857949556e-07, -0.38687652349472046, 0.0, -1.1920928244535389e-07, -4.768370445162873e-07, -9.059865078597795e-06, -1.0251946150674485e-05, -4.768370445162873e-07, -2.50339189733495e-06, 0.0, 0.0, -0.11539207398891449, -3.3378546504536644e-06, -8.344646857949556e-07, -3.504691630951129e-05, -1.9073468138230965e-06, -0.9594733715057373, -0.002969857305288315, -1.549708758830093e-05, -0.45560920238494873, -3.576278118089249e-07, -3.635817120084539e-05, -2.3841830625315197e-06, -5.245195097813848e-06, -0.07011206448078156, -0.00025996167096309364, -5.8053239627042785e-05, -2.264974000354414e-06, -1.316757321357727, -8.344646857949556e-07, -1.183588981628418, -3.238340139389038, -4.768370445162873e-07, -1.1920922133867862e-06, -8.22540732769994e-06, -0.11284194141626358, -5.602820692729438e-06, -1.1920928244535389e-07, -1.1920928244535389e-07, -1.1920928244535389e-07, -4.768370445162873e-07, -2.3841855067985307e-07, -2.3841855067985307e-07, -5.960462772236497e-07, 0.0, -1.1920928244535389e-07, -4.768370445162873e-07, -1.1920928244535389e-07, -3.576278118089249e-07, -0.00023195437097456306, -7.152555099310121e-07, -1.0728830375228426e-06, -4.768370445162873e-07, -6.770858453819528e-05, -1.1920928244535389e-07, -1.311301275563892e-06, -1.7881377516459906e-06, -1.1920928244535389e-07, -1.0490362910786644e-05, -4.410734163684538e-06, 0.0, -5.960462772236497e-07, -1.6689286894688848e-06, -9.894321920000948e-06, -1.9073468138230965e-06, -3.790783375734463e-05, -1.4305104514278355e-06, -2.3841855067985307e-07, -1.1920928244535389e-07, -1.6689286894688848e-06, -9.536738616588991e-07, -1.4305104514278355e-06, -4.768370445162873e-07, -2.3841855067985307e-07, -1.1920928244535389e-07, -8.34461570775602e-06, -1.6689286894688848e-06, -0.8241457343101501, -7.867782187531702e-06, -2.3841830625315197e-06, -0.014178375713527203, -0.1137320026755333, -1.6212332411669195e-05, -0.16421183943748474, -2.3841574147809297e-05, -5.8412379075889476e-06, -0.008409328758716583, -1.0728830375228426e-06, -2.3841855067985307e-07, -0.0007111880695447326, -2.3841855067985307e-07, -2.825220326485578e-05, 0.0, -1.4305104514278355e-06, -0.00023326536756940186, 0.0, 0.0, -4.0531076592742465e-06, -7.152555099310121e-07, -1.6689286894688848e-06, -8.344646857949556e-07, -1.1920928244535389e-07, -2.3841855067985307e-07, -1.311301275563892e-06, -8.344646857949556e-07, -6.6756979322235566e-06, 0.0, 0.0, -9.536738616588991e-07, -4.768370445162873e-07, -1.1920928244535389e-07, -2.7418097943154862e-06, -1.1920928244535389e-07, 0.0, -2.861018856492592e-06, -4.768370445162873e-07, -1.2993727978027891e-05, 0.0, -0.10489857941865921, -0.08935131132602692, -9.536738616588991e-07, -3.576278118089249e-07, -0.00015889335190877318, 0.0, -0.0018773326883092523, 0.0, 0.0, -0.47669291496276855, -0.4418356120586395, -1.9073468138230965e-06, -1.5024585723876953, -5.006956577301025, -7.986990567587782e-06, -2.0265558760002023e-06, -0.03366709500551224, -0.00040415700641460717, -0.008617476560175419, -1.3828182090946939e-05, -0.0002004899288294837, 0.0, -0.0001641377166379243, -4.768370445162873e-07, -0.054889753460884094, -2.3841855067985307e-07, -5.960462772236497e-07, -5.960462772236497e-07, -2.50339189733495e-06, -4.6491513785440475e-06, 0.0, -9.536738616588991e-07, -2.7418097943154862e-06, -2.3841855067985307e-07, -2.50339189733495e-06, -1.1920922133867862e-06, -1.0728830375228426e-06, -4.768370445162873e-07, -8.344646857949556e-07, -1.1920928244535389e-07, -3.576278118089249e-07, 0.0, -3.576278118089249e-07, 0.0, -0.012261842377483845, -0.003601732663810253, -0.008038313128054142, -3.576278118089249e-07, -0.09923606365919113, -1.1920922133867862e-06, -0.04871370643377304, -1.311301275563892e-06, -1.1920928244535389e-07, -2.3841855067985307e-07, -2.52720492426306e-05, -2.098061486321967e-05, -1.5497195136049413e-06, -0.03364703804254532, -4.768370445162873e-07, -1.1920928244535389e-07, -1.1920928244535389e-07, -0.005929854232817888, -2.3841855067985307e-07, -3.099436753473128e-06, -1.311301275563892e-06, 0.0, -0.10020766407251358, -2.50339189733495e-06, -8.344646857949556e-07, -8.344646857949556e-07, -1.311301275563892e-06, -3.576278118089249e-07, -5.960462772236497e-07, -1.0728830375228426e-06, -0.005373439285904169, -9.298280929215252e-06, -3.4570634852570947e-06, -0.002914112526923418, -1.1920928244535389e-07, -0.07892059534788132, -0.0004318020073696971, -0.011547512374818325, -9.07141511561349e-05, -0.2287055253982544, 0.0, -1.9073304429184645e-05, -1.1920928244535389e-07, -0.0002989322238136083, 0.0, -1.3589766240329482e-05, -0.25199630856513977, -1.1920928244535389e-07, -4.768370445162873e-07, -0.008617003448307514, -1.1920928244535389e-07, -4.768370445162873e-07, -2.145764938177308e-06, -9.536738616588991e-07, -2.3841855067985307e-07, -5.960462772236497e-07, -2.3841855067985307e-07, -2.3841855067985307e-07, -0.00023874769976828247, -8.344646857949556e-07, -1.311301275563892e-06, -3.576278118089249e-07, -2.3841855067985307e-07, -2.3841855067985307e-07, -2.3841855067985307e-07, 0.0, -1.1920928244535389e-07, -1.1920922133867862e-06, -1.1920928244535389e-07, -7.152555099310121e-07, 0.0, -3.576278118089249e-07, -1.053807020187378, -4.768370445162873e-07, -1.1920922133867862e-06, -2.0265558760002023e-06, -3.576278118089249e-07, -5.960462772236497e-07, -7.390948667307384e-06, -9.536738616588991e-07, -5.602820692729438e-06, -7.199982064776123e-05, -1.1801649634435307e-05, -0.5871596336364746, -0.1359272301197052, -0.00010918975021922961, -4.768370445162873e-07, -0.0009346407605335116, -0.11285588890314102, -9.536738616588991e-07, -1.311301275563892e-06, -1.4061613082885742, -9.536738616588991e-07, -1.1920922133867862e-06, -8.34461570775602e-06, -0.25678473711013794, -0.00017677174764685333, -0.6528276205062866, -7.510157047363464e-06, -1.7881377516459906e-06, -8.583032467868179e-06, -6.151010165922344e-05, -1.0728830375228426e-06, -1.5020257706055418e-05, -0.0011804286623373628, -2.038458114839159e-05, -7.152555099310121e-07, -4.768370445162873e-07], "index": 4291, "score": 1.0}]} \ No newline at end of file diff --git a/hosted/opd/report.py b/hosted/opd/report.py new file mode 100644 index 0000000..423efa6 --- /dev/null +++ b/hosted/opd/report.py @@ -0,0 +1,302 @@ +"""Summarize OPD and MOPD runs: quality, throughput, time breakdown, and a cost estimate. + + python hosted/opd/report.py RUN_DIR [RUN_DIR ...] --output REPORT_DIR \\ + [--teachers-campaign SFT campaign dir from hosted/tinker/launch.py] + +Writes REPORT.md (every chart's numbers are also a table there), summary.json, and PNG/SVG +figures. Only the retained run records are read; nothing is re-sampled. With +--teachers-campaign, the base model's and teachers' scores on the same benchmark are +shown alongside. +""" +import argparse +import json +from pathlib import Path +import statistics + +# Published Tinker list prices for Qwen3.6-35B-A3B, USD per million tokens (observed 2026-09-27). +# Illustrative only: not an invoice, and prefill caching is ignored. +PRICE = {'sampled': 1.335, 'prefill': 0.54, 'trained': 1.177} +DOMAINS = ('caesar_cipher', 'simple_geometry') +# Chart colors for a light background; the three series colors are distinguishable from each other. +SURFACE, INK, MUTED, GRID = '#fcfcfb', '#0b0b0b', '#52514e', '#e4e3df' +SERIES = ['#2a78d6', '#eb6834', '#1baf7a'] +NEUTRAL = '#a8a7a2' # "Other"-style remainder, not a series hue. + + +def read_jsonl(path): + return [json.loads(line) for line in Path(path).read_text().splitlines() if line.strip()] if Path(path).exists() else [] + + +def load(run): + run = Path(run) + config = json.loads((run/'config.json').read_text()) + events = read_jsonl(run/'events.jsonl') + return {'dir': run, 'name': run.name, 'kind': config['kind'], 'domains': config['domains'], 'config': config, + 'metrics': read_jsonl(run/'metrics.jsonl'), 'evaluations': read_jsonl(run/'evaluations.jsonl'), + 'events': events, 'complete': bool(events) and events[-1]['event'] == 'complete', + 'label': 'OPD (' + config['domains'][0] + ')' if config['kind'] == 'opd' else 'MOPD (' + ' + '.join(config['domains']) + ')', + 'short': config['kind'].upper()} + + +def references(campaign): + """Base and SFT-teacher scores on the same 100 x 3 benchmark, from an SFT campaign of hosted/tinker/launch.py.""" + if not campaign: + return {} + campaign = Path(campaign) + base = json.loads((campaign/'baseline/benchmark.json').read_text()) + refs = {'base': {d: base[d]['accuracy'] for d in DOMAINS}, 'teacher': {}} + for d in DOMAINS: + teacher = json.loads((campaign/('teacher-' + d.replace('_', '-'))/'teacher.json').read_text()) + refs['teacher'][d] = teacher['benchmark']['accuracy'] + return refs + + +def client_seconds(metric): + """Update time outside the service calls: saving rollouts, building the batch, computing metrics.""" + t = metric['timing'] + return t['update'] - t['rollout'] - t['train'] - t['sync'] + + +def summarize(r): + m, a = r['metrics'], r['config'] + timing = lambda key: [x['timing'][key] for x in m] + tokens = {'sampled': sum(x['response_tokens'] for x in m), 'prompt': sum(x['prompt_tokens'] for x in m)} + # Evaluation records keep mean response lengths; their prompt prefill (a few hundred tokens each) is ignored. + eval_tokens = {'sampled': sum(e[d]['mean_tokens'] * e[d]['samples'] for e in r['evaluations'] for d in DOMAINS)} + # Prefill: sampling prompts, teacher scoring of prompt + response. Training: prompt + response. + prefill = tokens['prompt'] + (tokens['prompt'] + tokens['sampled']) + trained = tokens['prompt'] + tokens['sampled'] + cost = {'sampling': (tokens['sampled'] + eval_tokens['sampled']) * PRICE['sampled'] / 1e6, + 'prefill': prefill * PRICE['prefill'] / 1e6, 'training': trained * PRICE['trained'] / 1e6} + start, end = r['events'][0]['utc'], r['events'][-1]['utc'] + median = lambda values: statistics.median(values) if values else None + return { + 'run': r['name'], 'kind': r['kind'], 'domains': r['domains'], 'complete': r['complete'], + 'updates': len(m), 'responses': sum(x['responses'] for x in m), 'started_utc': start, 'last_event_utc': end, + 'wall_seconds': r['events'][-1]['elapsed_seconds'], + 'update_seconds_total': sum(timing('update')), 'eval_seconds_total': sum(e['seconds'] for e in r['evaluations']), + 'median_seconds': {k: median(timing(k)) for k in ('update', 'rollout', 'train', 'sync', 'median_response', + 'p90_response', 'last_response', 'teacher_tail')}, + 'share_of_update_time': {**{k: sum(timing(k)) / sum(timing('update')) for k in ('rollout', 'train', 'sync')}, + 'client': sum(client_seconds(x) for x in m) / sum(timing('update'))} if m else {}, + 'median_throughput': {k: median([x['throughput'][k] for x in m]) for k in (m[0]['throughput'] if m else [])}, + 'tokens': {'sampled_train': tokens['sampled'], 'prompt_train': tokens['prompt'], + 'sampled_eval_estimate': round(eval_tokens['sampled']), 'trained': trained, 'prefill': prefill}, + 'truncated_train_responses': sum(x['truncated'] for x in m), + 'train_sampler_logprob_delta_abs_mean': median([x['train_sampler_logprob_delta_abs_mean'] for x in m + if x['train_sampler_logprob_delta_abs_mean'] is not None]), + 'cost_estimate_usd': {**cost, 'total': sum(cost.values())}, + 'evaluations': r['evaluations'], 'config': {k: a[k] for k in ('updates', 'prompts', 'learning_rate', 'rank', + 'max_tokens', 'eval_examples', 'eval_samples')}} + + +# Figures. + +def style(ax, title, ylabel): + ax.set_facecolor(SURFACE) + ax.set_title(title, loc='left', color=INK, fontsize=11) + ax.set_ylabel(ylabel, color=MUTED, fontsize=9) + ax.tick_params(colors=MUTED, labelsize=8, length=0) + ax.grid(axis='y', color=GRID, linewidth=0.8) + ax.set_axisbelow(True) + for side in ('top', 'right', 'left'): + ax.spines[side].set_visible(False) + ax.spines['bottom'].set_color(GRID) + + +def figure_legend(fig, ax): + """One legend in a row above the panels, clear of the data.""" + handles, labels = ax.get_legend_handles_labels() + fig.legend(handles, labels, frameon=False, fontsize=8, labelcolor=INK, ncol=len(labels), loc='upper center', + bbox_to_anchor=(0.5, 1.0)) + fig.subplots_adjust(top=0.8) + + +def save(fig, out, name): + fig.patch.set_facecolor(SURFACE) + fig.tight_layout(rect=(0, 0, 1, 0.9) if fig.legends else (0, 0, 1, 1)) + for ext in ('png', 'svg'): + fig.savefig(out/f'{name}.{ext}', dpi=200, facecolor=SURFACE) + + +def plot_accuracy(runs, refs, out, plt): + fig, axes = plt.subplots(1, len(DOMAINS), figsize=(10, 3.8), sharey=True) + for ax, d in zip(axes, DOMAINS): + style(ax, f'{d}: avg@3 on 100 dev problems', 'accuracy' if d == DOMAINS[0] else '') + for label, value in (('SFT teacher', refs.get('teacher', {}).get(d)), ('base', refs.get('base', {}).get(d))): + if value is not None: + ax.axhline(value, color=MUTED, linewidth=1, linestyle=(0, (4, 3))) + ax.annotate(f'{label} {value:.1%}', (1, value), xycoords=('axes fraction', 'data'), xytext=(0, 3), + textcoords='offset points', ha='right', color=MUTED, fontsize=8) + for color, r in zip(SERIES, runs): + points = [(e['update'], e[d]['accuracy']) for e in r['evaluations']] + if points: + xs, ys = zip(*points) + trained = d in r['domains'] + ax.plot(xs, ys, color=color, linewidth=2, marker='o', markersize=5, label=r['label'], + linestyle='-' if trained else (0, (2, 2))) + ax.annotate(f'{ys[-1]:.1%}', (xs[-1], ys[-1]), xytext=(6, -3), textcoords='offset points', + color=INK, fontsize=8) + ax.set_xlabel('update', color=MUTED, fontsize=9) + ax.set_ylim(0, 1.05) + ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda v, _: f'{v:.0%}')) + axes[0].legend(frameon=False, fontsize=8, loc='lower right', labelcolor=INK) + fig.text(0.01, 0.005, 'Dotted line: a domain the run does not train on.', color=MUTED, fontsize=7) + save(fig, out, 'accuracy') + + +def plot_time(runs, out, plt): + fig, axes = plt.subplots(1, len(runs), figsize=(5 * len(runs), 3.9), sharey=True, squeeze=False) + for ax, r in zip(axes[0], runs): + style(ax, f'{r["short"]}: seconds per update', 'seconds' if ax is axes[0][0] else '') + xs = [x['update'] for x in r['metrics']] + bottom = [0.0] * len(xs) + parts = [(SERIES[0], 'sample + teacher score', [x['timing']['rollout'] for x in r['metrics']]), + (SERIES[1], 'train step', [x['timing']['train'] for x in r['metrics']]), + (SERIES[2], 'weight sync', [x['timing']['sync'] for x in r['metrics']]), + (NEUTRAL, 'client-side', [client_seconds(x) for x in r['metrics']])] + for color, name, values in parts: + ax.bar(xs, values, bottom=bottom, color=color, width=0.8, label=name, edgecolor=SURFACE, linewidth=1) + bottom = [b + v for b, v in zip(bottom, values)] + ax.set_xlabel('update', color=MUTED, fontsize=9) + figure_legend(fig, axes[0][0]) + save(fig, out, 'time-per-update') + + +def plot_stragglers(runs, out, plt): + fig, axes = plt.subplots(1, len(runs), figsize=(5 * len(runs), 3.9), sharey=True, squeeze=False) + for ax, r in zip(axes[0], runs): + style(ax, f'{r["short"]}: when responses finish', 'seconds after update start' if ax is axes[0][0] else '') + xs = [x['update'] for x in r['metrics']] + for color, (key, name) in zip(SERIES, (('median_response', 'median response'), ('p90_response', '90th percentile'), + ('last_response', 'last response'))): + ax.plot(xs, [x['timing'][key] for x in r['metrics']], color=color, linewidth=2, marker='o', markersize=4, + label=name) + ax.set_xlabel('update', color=MUTED, fontsize=9) + figure_legend(fig, axes[0][0]) + save(fig, out, 'response-finish-times') + + +def plot_kl(runs, out, plt): + fig, axes = plt.subplots(1, len(DOMAINS), figsize=(10, 3.6), squeeze=False) + for ax, d in zip(axes[0], DOMAINS): + style(ax, f'{d}: sampled reverse KL to its teacher', 'nats per token' if d == DOMAINS[0] else '') + for color, r in zip(SERIES, runs): + if d in r['domains'] and r['metrics']: + ax.plot([x['update'] for x in r['metrics']], [x['by_domain'][d]['reverse_kl'] for x in r['metrics']], + color=color, linewidth=2, marker='o', markersize=4, label=r['label']) + ax.set_xlabel('update', color=MUTED, fontsize=9) + if ax.lines: + ax.legend(frameon=False, fontsize=8, labelcolor=INK) + save(fig, out, 'reverse-kl') + + +# Markdown. + +def table(header, rows): + lines = ['| ' + ' | '.join(header) + ' |', '|' + '---|' * len(header)] + return '\n'.join(lines + ['| ' + ' | '.join(str(c) for c in row) + ' |' for row in rows]) + + +def pct(value): + return '—' if value is None else f'{value:.1%}' + + +def seconds(value): + return '—' if value is None else f'{value:,.0f} s' + + +def markdown(runs, summaries, refs): + lags = sorted({r['config'].get('policy_lag', 0) for r in runs}) + mode = 'synchronous (policy lag 0)' if lags == [0] else f'asynchronous lookahead (maximum policy lag {max(lags)})' + parts = ['# OPD and MOPD on Tinker', '', + 'Fresh rank-64 LoRA students of Qwen3.6-35B-A3B, thinking on, distilled from frozen SFT teachers ' + f'(hosted/opd/sft.py). Sampled-token reverse KL, importance-sampling loss, no task reward, {mode}. ' + 'Evaluation: the first 100 dev problems of each domain, 3 samples at temperature 1.', ''] + parts += ['## Quality (avg@3)', ''] + rows = [] + if refs: + rows.append(['base model'] + [pct(refs['base'][d]) for d in DOMAINS]) + rows.append(['SFT teacher (own domain)'] + [pct(refs['teacher'][d]) for d in DOMAINS]) + for r in runs: + for e in r['evaluations']: + rows.append([f'{r["label"]}, update {e["update"]}'] + [pct(e[d]['accuracy']) for d in DOMAINS]) + parts += [table(['Model', *DOMAINS], rows), '', '![accuracy](accuracy.png)', ''] + parts += ['## Time and throughput', '', + 'Client wall time on a shared hosted service; not GPU utilization. Rollout is sampling with teacher ' + 'scoring overlapped; the teacher tail is the scoring left after the last response finishes.', ''] + header = ['', *[s['run'] for s in summaries]] + row = lambda name, f: [name, *[f(s) for s in summaries]] + parts += [table(header, [ + row('complete', lambda s: 'yes' if s['complete'] else f'no ({s["updates"]} updates)'), + row('updates × responses', lambda s: f'{s["updates"]} × {s["responses"] // max(1, s["updates"])}'), + row('wall time', lambda s: f'{s["wall_seconds"] / 60:,.1f} min'), + row('of which evaluation', lambda s: f'{s["eval_seconds_total"] / 60:,.1f} min'), + row('median update', lambda s: seconds(s['median_seconds']['update'])), + row('median rollout', lambda s: seconds(s['median_seconds']['rollout'])), + row('median train step', lambda s: seconds(s['median_seconds']['train'])), + row('median weight sync', lambda s: seconds(s['median_seconds']['sync'])), + row('median response finishes', lambda s: seconds(s['median_seconds']['median_response'])), + row('p90 response finishes', lambda s: seconds(s['median_seconds']['p90_response'])), + row('last response finishes', lambda s: seconds(s['median_seconds']['last_response'])), + row('teacher tail', lambda s: seconds(s['median_seconds']['teacher_tail'])), + row('share: rollout / train / sync / client-side', lambda s: ' / '.join(f'{v:.0%}' for v in s['share_of_update_time'].values())), + row('sampled tokens/s (median update)', lambda s: f'{s["median_throughput"].get("sampled_tokens_per_second", 0):,.0f}'), + row('trained tokens/s (median update)', lambda s: f'{s["median_throughput"].get("trained_tokens_per_second", 0):,.0f}'), + row('end-to-end response tokens/s', lambda s: f'{s["median_throughput"].get("update_response_tokens_per_second", 0):,.0f}'), + row('training responses truncated', lambda s: s['truncated_train_responses']), + row('train−sampler logprob gap (abs, median)', lambda s: f'{s["train_sampler_logprob_delta_abs_mean"]:.4f}' + if s['train_sampler_logprob_delta_abs_mean'] is not None else '—'), + ]), '', '![time per update](time-per-update.png)', '', '![response finish times](response-finish-times.png)', ''] + parts += ['## Tokens and estimated cost', '', + f'List prices: sampling ${PRICE["sampled"]}/M, prefill ${PRICE["prefill"]}/M, training ${PRICE["trained"]}/M. ' + 'Prefill counts the sampling prompts and the teachers scoring prompt + response; caching is ignored. ' + 'An estimate, not an invoice.', ''] + parts += [table(header, [ + row('sampled tokens (training)', lambda s: f'{s["tokens"]["sampled_train"] / 1e6:,.1f} M'), + row('sampled tokens (evaluation, est.)', lambda s: f'{s["tokens"]["sampled_eval_estimate"] / 1e6:,.1f} M'), + row('prefill tokens', lambda s: f'{s["tokens"]["prefill"] / 1e6:,.1f} M'), + row('trained tokens', lambda s: f'{s["tokens"]["trained"] / 1e6:,.1f} M'), + row('estimated cost', lambda s: f'${s["cost_estimate_usd"]["total"]:,.0f}'), + ]), ''] + parts += ['## Distillation signal', '', '![reverse KL](reverse-kl.png)', ''] + kl_rows = [] + for r in runs: + for x in r['metrics']: + kl_rows.append([r['label'], x['update'], *[f'{x["by_domain"][d]["reverse_kl"]:.4f}' if d in x['by_domain'] else '—' + for d in DOMAINS], + *[f'{x["by_domain"][d]["score"]:.0%}' if d in x['by_domain'] else '—' for d in DOMAINS]]) + parts += [table(['Run', 'update', *[f'{d} reverse KL' for d in DOMAINS], *[f'{d} train score' for d in DOMAINS]], + kl_rows), ''] + return '\n'.join(parts) + + +def main(): + p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + p.add_argument('runs', nargs='+', type=Path) + p.add_argument('--output', type=Path, required=True) + p.add_argument('--teachers-campaign', type=Path) + args = p.parse_args() + runs = [load(r) for r in args.runs] + refs = references(args.teachers_campaign) + summaries = [summarize(r) for r in runs] + args.output.mkdir(parents=True, exist_ok=True) + (args.output/'summary.json').write_text(json.dumps({'references': refs, 'runs': summaries}, indent=2)) + (args.output/'REPORT.md').write_text(markdown(runs, summaries, refs)) + try: + import matplotlib + matplotlib.use('Agg') + import matplotlib.pyplot as plt + except ImportError: + print('matplotlib not installed: tables only') + return + plt.rcParams['font.family'] = 'sans-serif' + plot_accuracy(runs, refs, args.output, plt) + plot_time(runs, args.output, plt) + plot_stragglers(runs, args.output, plt) + plot_kl(runs, args.output, plt) + print(f'wrote {args.output}/REPORT.md') + + +if __name__ == '__main__': + main() diff --git a/hosted/opd/run_fireworks.sh b/hosted/opd/run_fireworks.sh new file mode 100755 index 0000000..1f13a44 --- /dev/null +++ b/hosted/opd/run_fireworks.sh @@ -0,0 +1,65 @@ +#!/bin/bash +# The Fireworks OPD/MOPD campaign: lease with both uploaded teachers, teacher check, MOPD, then OPD +# (after dropping the geometry teacher), then release. Stops at the first failure; the watchdog +# releases everything at the budget or if this script dies. +# +# [SAMPLER_REPLICAS=5] [SAMPLER_SHAPE=... SAMPLER_GPUS=1] [TEACHER_SHAPE=...] [HOT_LOAD_TRANSITION=ASYNC] +# [RUN_ARGS=--skip-initial-eval] +# [MOPD_ONLY=1] hosted/opd/run_fireworks.sh /abs/new-lease-dir BUDGET_USD +# +# Rerunning with the same directory reuses a lease that is already up and a teacher check that passed. +set -u +LEASE=$1; BUDGET=$2 +HERE=$(cd "$(dirname "$0")" && pwd); cd "$HERE/../.." +PY=${PYTHON:-python} +CAESAR=qwen3p6-35b-a3b-caesar-cipher-sft-tinker +GEOMETRY=qwen3p6-35b-a3b-simple-geometry-sft-tinker +export FIREWORKS_LEASE=$LEASE TOKENIZERS_PARALLELISM=false PYTHONDONTWRITEBYTECODE=1 +say() { echo "{\"utc\": \"$(date -u +%FT%TZ)\", \"stage\": \"$1\"}"; } +fail() { say "failed: $1"; "$PY" hosted/opd/fireworks_lease.py down --root "$LEASE" > /dev/null; exit 1; } + +if [ -f "$LEASE/lease.json" ] && [ ! -f "$LEASE/TRAINING_FINISHED" ]; then + say lease_reused +else + say lease_up + "$PY" -u hosted/opd/fireworks_lease.py up --root "$LEASE" --budget-usd "$BUDGET" --sampler-replicas "${SAMPLER_REPLICAS:-1}" \ + ${SAMPLER_SHAPE:+--sampler-shape "$SAMPLER_SHAPE"} --sampler-gpus "${SAMPLER_GPUS:-2}" \ + ${TEACHER_SHAPE:+--teacher-shape "$TEACHER_SHAPE"} --hot-load-transition "${HOT_LOAD_TRANSITION:-SYNC}" \ + --teacher caesar_cipher=$CAESAR --teacher simple_geometry=$GEOMETRY > "$LEASE.lease.log" 2>&1 & + until [ -f "$LEASE/teachers.json" ] && grep -q '"ready"' "$LEASE/lease-events.jsonl" 2>/dev/null; do + kill -0 $! 2>/dev/null || fail lease_up; sleep 15 + done +fi +RUNS=$LEASE-runs; mkdir -p "$RUNS" + +if grep -q '"passed": true' "$LEASE/teacher-check.json" 2>/dev/null; then + say teacher_check_reused +else + say teacher_check + "$PY" -u hosted/opd/check_teachers.py --backend fireworks --teachers "$LEASE/teachers.json" \ + --rollouts hosted/opd/reference/teacher-check-rollouts.json --output "$LEASE/teacher-check.json" \ + > "$RUNS/teacher-check.log" 2>&1 || fail teacher_check +fi + +say mopd +"$PY" -u hosted/opd/mopd.py --backend fireworks --teachers "$LEASE/teachers.json" --output "$RUNS/mopd" ${RUN_ARGS:-} \ + > "$RUNS/mopd.log" 2>&1 || fail mopd + +if [ -n "${MOPD_ONLY:-}" ]; then + say release + "$PY" hosted/opd/fireworks_lease.py down --root "$LEASE" > /dev/null + say complete + exit 0 +fi + +say drop_geometry_teacher +"$PY" hosted/opd/fireworks_lease.py drop-teacher --root "$LEASE" --domain simple_geometry || fail drop_teacher + +say opd +CAESAR_TEACHER=$("$PY" -c "import json,sys; print(json.load(open(sys.argv[1]))['caesar_cipher'])" "$LEASE/teachers.json") +"$PY" -u hosted/opd/opd.py --backend fireworks --domain caesar_cipher --teacher "$CAESAR_TEACHER" --output "$RUNS/opd-caesar" ${RUN_ARGS:-} \ + > "$RUNS/opd-caesar.log" 2>&1 || fail opd + +say release +"$PY" hosted/opd/fireworks_lease.py down --root "$LEASE" > /dev/null +say complete diff --git a/hosted/opd/run_serverless.sh b/hosted/opd/run_serverless.sh new file mode 100755 index 0000000..3c1e445 --- /dev/null +++ b/hosted/opd/run_serverless.sh @@ -0,0 +1,32 @@ +#!/bin/bash +# MOPD with a serverless Qwen3.8-27B student and the uploaded Qwen3.6 SFT teachers: a teacher-only lease +# (one validated 1xB200 deployment per teacher, under the budget watchdog), the MOPD run on the serverless +# Training API, then release. Stops at the first failure; the watchdog releases the teachers at the budget +# or if this script dies. Serverless token spend is per token and is not covered by the watchdog. +# +# [RUN_ARGS="--max-tokens 16000 --eval-every 20"] hosted/opd/run_serverless.sh /abs/new-lease-dir TEACHER_BUDGET_USD +set -u +LEASE=$1; BUDGET=$2 +HERE=$(cd "$(dirname "$0")" && pwd); cd "$HERE/../.." +PY=${PYTHON:-python} +SHAPE=accounts/fireworks/deploymentShapes/rft-qwen3p6-35b-a3b-rl-b200-bf16-w1-p1/versions/a35zwd97 +export FIREWORKS_LEASE=$LEASE TOKENIZERS_PARALLELISM=false PYTHONDONTWRITEBYTECODE=1 +say() { echo "{\"utc\": \"$(date -u +%FT%TZ)\", \"stage\": \"$1\"}"; } +fail() { say "failed: $1"; "$PY" hosted/opd/fireworks_lease.py down --root "$LEASE" > /dev/null; exit 1; } + +say teachers_up +"$PY" -u hosted/opd/fireworks_lease.py up --root "$LEASE" --budget-usd "$BUDGET" --teachers-only --teacher-shape $SHAPE \ + --teacher caesar_cipher=qwen3p6-35b-a3b-caesar-cipher-sft-tinker \ + --teacher simple_geometry=qwen3p6-35b-a3b-simple-geometry-sft-tinker > "$LEASE.lease.log" 2>&1 & +until [ -f "$LEASE/teachers.json" ] && grep -q '"ready"' "$LEASE/lease-events.jsonl" 2>/dev/null; do + kill -0 $! 2>/dev/null || fail teachers_up; sleep 15 +done +RUNS=$LEASE-runs; mkdir -p "$RUNS" + +say mopd +"$PY" -u hosted/opd/mopd.py --backend fireworks-serverless --student-model qwen3.8-27b --teacher-model qwen3.6-35b-a3b \ + --teachers "$LEASE/teachers.json" --output "$RUNS/mopd" ${RUN_ARGS:-} > "$RUNS/mopd.log" 2>&1 || fail mopd + +say release +"$PY" hosted/opd/fireworks_lease.py down --root "$LEASE" > /dev/null +say complete diff --git a/hosted/opd/sft.py b/hosted/opd/sft.py new file mode 100644 index 0000000..534b7f5 --- /dev/null +++ b/hosted/opd/sft.py @@ -0,0 +1,444 @@ +"""SFT teachers from teacher traces, and the teacher benchmark, on a Tinker-compatible API. Thinking on. + +`train` fits one LoRA teacher for one domain on that domain's traces, then benchmarks it. +`bench` benchmarks the base model, or any sampler checkpoint, the same way: the first +`--bench-examples` problems of each dev split, `--bench-samples` responses each. + +The defaults reproduce the released teachers: every verified trace of +semianalysisai/Qwen3.6-35B-A3B-teacher-sft-caesar-geometry (sampled from the frozen +recipe.TEACHERS), one epoch, rank 64, and a 100-problem x 3-sample benchmark at temperature 1. +Every setting is a flag and is recorded in the run's config.json, so other teachers +(fewer or easier traces, another LR, rank, or epoch count, or another trace dataset +with the same layout) are one command away: + + python sft.py train --domain caesar_cipher --output /abs/teacher-caesar-cipher + python sft.py train --domain caesar_cipher --output /abs/t --min-correct 3 --traces-per-prompt 1 + python sft.py bench --output /abs/baseline [--checkpoint tinker://.../sampler_weights/final] + python sft.py train --domain caesar_cipher --output /abs/t --backend fireworks # see backend.py + +A trace dataset needs data//train-*.parquet files with columns id, task, question and +messages (system, user, assistant; the assistant turn is `\n......`), and the +system prompt and scorer labels of this repository's data/. The trace filters also need +correct_of_3, source_index and sample_index. Adapted from the cookbook's recipes/sl_loop.py. +No model weights are downloaded. +""" +import argparse +from concurrent.futures import ThreadPoolExecutor, as_completed +from datetime import datetime, timezone +import hashlib +import importlib.metadata +import json +import math +from pathlib import Path +import random +import statistics +import sys +import threading +import time +import traceback + +import tinker +from transformers import AutoTokenizer +from tinker_cookbook import renderers +from tinker_cookbook.hyperparam_utils import get_lr + +from common import MODEL, RENDERER, REVISION # common.py puts the repository on sys.path. +from backend import BACKENDS +from shared import recipe, scoring +from shared.puzzles import REPO_DATA, load_split + +DOMAINS = recipe.DOMAINS +# Defaults of every setting; each is also a command-line flag (underscores become dashes). +DEFAULTS = { + # Traces: the released dataset, pinned. By default every trace is used; the filters keep prompts the + # teacher solved at least `min_correct` of 3 times, at most `traces_per_prompt` traces each (the lowest + # sample indices; 0 keeps all), and then at most `max_rows` seeded-random rows (0 keeps all). + 'dataset': 'semianalysisai/Qwen3.6-35B-A3B-teacher-sft-caesar-geometry', + 'revision': 'b21bb3e75b1ca903f4b69bd609adc89bd99543f1', + 'min_correct': 1, 'traces_per_prompt': 0, 'max_rows': 0, + # Training, cookbook SFT defaults: sum-over-token cross entropy on the completion, the cookbook's + # recommended LoRA LR for this model (about 5e-4), linear decay to zero. Each epoch is a fresh seeded + # shuffle whose final partial batch is dropped, so every update sees the same number of sequences. + 'learning_rate': get_lr(MODEL), 'lr_schedule': 'linear', 'batch_size': 128, 'epochs': 1, 'rank': 64, + 'save_every': 25, + # The benchmark: the first 100 problems of each dev split, 3 samples each at temperature 1, the + # sampling used to generate the dataset and to report the GRPO teachers' held-out avg@8. + 'bench_examples': 100, 'bench_samples': 3, 'bench_temperature': 1.0, +} +ADAM = {'beta1': 0.9, 'beta2': 0.95, 'eps': 1e-8, 'weight_decay': 0.0, 'grad_clip_norm': 0.0} + + +def lr_at(step, steps, peak, schedule='linear'): + """The learning rate of update `step` (0-based) of `steps`: linear or cosine decay to zero, or constant.""" + if schedule == 'constant': + return peak + if schedule == 'cosine': + return peak * 0.5 * (1 + math.cos(math.pi * step / steps)) + assert schedule == 'linear' + return peak * (1 - step / steps) + + +def select_traces(rows, min_correct=1, traces_per_prompt=0, max_rows=0, seed=0): + """Apply the trace filters, keeping the dataset's row order.""" + if min_correct > 1: + rows = [r for r in rows if r['correct_of_3'] >= min_correct] + if traces_per_prompt: + kept = {} + for r in sorted(rows, key=lambda r: (r['source_index'], r['sample_index'])): + kept.setdefault(r['source_index'], []) + if len(kept[r['source_index']]) < traces_per_prompt: + kept[r['source_index']].append(r['id']) + keep = {i for ids in kept.values() for i in ids} + rows = [r for r in rows if r['id'] in keep] + if max_rows and len(rows) > max_rows: + keep = set(random.Random(seed).sample(range(len(rows)), max_rows)) + rows = [r for i, r in enumerate(rows) if i in keep] + return rows + + +def epoch_order(n, epochs, seed): + """Row indices for every update: one seeded shuffle per epoch, the final partial batch dropped later.""" + orders = [] + for epoch in range(epochs): + order = list(range(n)) + random.Random(seed + 1000 * epoch).shuffle(order) + orders.append(order) + return orders + + +def sft_datum(prompt_tokens, completion_tokens): + """Next-token datum trained on the completion only (thinking, answer, and end-of-turn).""" + assert prompt_tokens and completion_tokens + sequence = prompt_tokens + completion_tokens + weights = [0.0] * (len(prompt_tokens) - 1) + [1.0] * len(completion_tokens) + shape = [len(sequence) - 1] + return tinker.Datum(model_input=tinker.ModelInput.from_ints(sequence[:-1]), loss_fn_inputs={ + 'target_tokens': tinker.TensorData(data=sequence[1:], dtype='int64', shape=shape), + 'weights': tinker.TensorData(data=weights, dtype='float32', shape=shape)}) + + +def sequence_nll(output, prompt_length, completion_length): + """One sequence's summed completion NLL from a cross_entropy forward_backward output. + + Tinker returns per-token logprobs; Fireworks returns the weighted sum directly as `loss`. + """ + if 'logprobs' in output: + logprobs = output['logprobs'].data[prompt_length - 1:] + assert len(logprobs) == completion_length + return -sum(logprobs) + return float(output['loss'].data[0]) + + +def summarize(records, domain, k): + """avg@k, pass@k, and length statistics of one domain's benchmark records.""" + rows = [r for r in records if r['domain'] == domain] + problems = {} + for r in rows: + problems.setdefault(r['index'], []).append(r['score']) + return {'problems': len(problems), 'samples': len(rows), 'correct': sum(r['score'] for r in rows), + 'accuracy': statistics.mean(r['score'] for r in rows), + f'pass_at_{k}': statistics.mean(max(s) for s in problems.values()), + f'all_{k}_correct': statistics.mean(min(s) for s in problems.values()), + 'mean_tokens': statistics.mean(len(r['tokens']) for r in rows), + 'max_tokens': max(len(r['tokens']) for r in rows), + 'truncated': sum(r['stop_reason'] == 'length' for r in rows), + 'unfinished_thinking': sum('' not in r['response'] for r in rows)} + + +class Run: + def __init__(self, args): + self.args = args + self.out = args.output.resolve() + self.out.mkdir(parents=True, exist_ok=False) + self.started = time.monotonic() + self.write_lock = threading.Lock() + self.pool = ThreadPoolExecutor(max_workers=args.concurrency) + config = {k: str(v) if isinstance(v, Path) else v for k, v in vars(args).items()} + config.update(adam=ADAM, model=MODEL, tokenizer_revision=REVISION, + changed_from_defaults=sorted(k for k, v in DEFAULTS.items() if getattr(args, k) != v)) + self.event('start', config=config) + (self.out/'config.json').write_text(json.dumps(config, indent=2)) + packages = ('tinker', 'tinker-cookbook', 'transformers', 'torch', 'numpy', 'pyarrow', 'fireworks-ai') + versions = {} + for name in packages: + try: + versions[name] = importlib.metadata.version(name) + except importlib.metadata.PackageNotFoundError: + versions[name] = None + (self.out/'versions.json').write_text(json.dumps(versions, indent=2)) + (self.out/'source.py').write_text(Path(__file__).read_text()) + (self.out/'backend.py').write_text(Path(__file__).with_name('backend.py').read_text()) + (self.out/'scoring.py').write_text(Path(scoring.__file__).read_text()) + (self.out/'data-manifest.json').write_text((REPO_DATA/'manifest.json').read_text()) + self.tokenizer = AutoTokenizer.from_pretrained(MODEL, revision=REVISION) + self.renderer = renderers.get_renderer(RENDERER, self.tokenizer, model_name=MODEL) + self.stop = self.renderer.get_stop_sequences() + self.backend = BACKENDS[args.backend](experiment=f'sft-{args.mode}-{self.out.name}') + (self.out/'account.json').write_text(json.dumps(self.backend.account, indent=2)) + (self.out/'capabilities.json').write_text(json.dumps(self.backend.capabilities(MODEL), indent=2, default=str)) + self.dev = {d: load_split(d, 'dev') for d in DOMAINS} + self.event('ready', stop=self.stop, tokenizer_revision=REVISION) + + def write(self, filename, row): + with self.write_lock, (self.out/filename).open('a') as f: + f.write(json.dumps(row, allow_nan=False) + '\n') + + def event(self, name, **data): + row = {'event': name, 'utc': datetime.now(timezone.utc).isoformat(), + 'elapsed_seconds': time.monotonic() - self.started, **data} + self.write('events.jsonl', row) + print(json.dumps(row), flush=True) + + def prompt(self, messages): + # The pinned HF template, which the traces were generated and verified with; fail if the cookbook differs. + ids = self.tokenizer.apply_chat_template(messages, tokenize=True, return_dict=False, + add_generation_prompt=True, enable_thinking=recipe.ENABLE_THINKING) + assert ids == self.renderer.build_generation_prompt(messages).to_ints(), 'Pinned HF and cookbook prompt tokens differ' + return ids + + # Training data. + + def load_traces(self, domain): + """The domain's selected traces, checked before any training is paid for.""" + import pyarrow.parquet as pq + from huggingface_hub import snapshot_download + a = self.args + root = Path(snapshot_download(a.dataset, repo_type='dataset', revision=a.revision)) + files = sorted((root/'data'/domain).glob('train-*.parquet')) + assert files, f'No parquet files for {domain}' + hashes = {f.name: hashlib.sha256(f.read_bytes()).hexdigest() for f in files} + columns = ['id', 'task', 'messages', 'question'] + if a.min_correct > 1 or a.traces_per_prompt: + columns += ['correct_of_3', 'source_index', 'sample_index'] + released = [r for f in files for r in pq.read_table(f, columns=columns).to_pylist()] + rows = select_traces(released, a.min_correct, a.traces_per_prompt, a.max_rows, a.seed) + assert rows, 'The trace filters kept no rows' + + system = self.dev[domain][0]['prompt'][0]['content'] + dev_questions = {r['prompt'][-1]['content'] for d in DOMAINS for r in self.dev[d]} + for r in rows: + m = r['messages'] + assert r['task'] == domain and [x['role'] for x in m] == ['system', 'user', 'assistant'], r['id'] + assert m[0]['content'] == system and m[1]['content'] == r['question'], r['id'] + assert r['question'] not in dev_questions, f'Dev problem in SFT data: {r["id"]}' + text = m[2]['content'] + assert text.startswith('\n') and text.count('') == 1 and text.count('') == 1, r['id'] + record = {'domain': domain, 'repo': a.dataset, 'revision': a.revision, 'files': hashes, + 'released_rows': len(released), 'rows': len(rows), 'unique_prompts': len({r['question'] for r in rows}), + 'filters': {k: getattr(a, k) for k in ('min_correct', 'traces_per_prompt', 'max_rows')}} + (self.out/'sft-data.json').write_text(json.dumps(record, indent=2)) + self.event('traces_loaded', **record) + return rows + + def tokenize(self, row): + """Prompt and completion tokens; the completion runs through the end-of-turn token.""" + m = row['messages'] + prompt = self.prompt(m[:2]) + full = self.tokenizer.apply_chat_template(m, tokenize=True, return_dict=False, + enable_thinking=recipe.ENABLE_THINKING) + end = self.tokenizer.convert_tokens_to_ids('<|im_end|>') + assert full[:len(prompt)] == prompt, f'Prompt is not a prefix of the rendered trace: {row["id"]}' + assert full[-2:] == [end, *self.tokenizer.encode('\n')], row['id'] + completion = full[len(prompt):-1] # Drop the template's newline after <|im_end|>. + assert len(prompt) + len(completion) <= recipe.CONTEXT_LENGTH, row['id'] + return prompt, completion + + def build_batch(self, rows, orders, step): + """One update's rows, tokens and datums, and how long rendering them took (seconds).""" + start = time.monotonic() + size = self.args.batch_size + epoch, b = divmod(step, len(rows) // size) + batch = [rows[i] for i in orders[epoch][b*size:(b+1)*size]] + tokens = [self.tokenize(r) for r in batch] + return batch, tokens, [sft_datum(p, c) for p, c in tokens], time.monotonic() - start + + def train(self): + a = self.args + rows = self.load_traces(a.domain) + orders = epoch_order(len(rows), a.epochs, a.seed + DOMAINS.index(a.domain)) + per_epoch = len(rows) // a.batch_size + steps = per_epoch * a.epochs + assert per_epoch > 0, f'Fewer rows ({len(rows)}) than one batch ({a.batch_size})' + assert 0 <= a.start_step < steps + for i in orders[0][:a.batch_size]: # Render one batch up front: fail before creating a client. + self.tokenize(rows[i]) + self.event('plan', rows=len(rows), steps=steps, dropped_per_epoch=len(rows) - per_epoch*a.batch_size, + learning_rate=a.learning_rate, lr_schedule=a.lr_schedule) + if a.checkpoint: + train = self.backend.resume_student(a.checkpoint) + else: + train = self.backend.new_student(MODEL, a.rank, a.seed, {'domain': a.domain, 'run': self.out.name, 'method': 'sft'}) + (self.out/'model-info.json').write_text(train.get_info().model_dump_json(indent=2)) + self.event('training_client_created') + prefetch = ThreadPoolExecutor(max_workers=1) + try: + # One update stays in flight while the next batch is rendered and submitted. + next_batch = prefetch.submit(self.build_batch, rows, orders, a.start_step) + pending = None + self.last_finish = time.monotonic() + for step in range(a.start_step, steps): + wait = time.monotonic() + batch, tokens, datums, build_seconds = next_batch.result() + timing = {'build_seconds': build_seconds, 'batch_wait_seconds': time.monotonic() - wait} + if step + 1 < steps: + next_batch = prefetch.submit(self.build_batch, rows, orders, step + 1) + lr = lr_at(step, steps, a.learning_rate, a.lr_schedule) + timing['submitted'] = time.monotonic() + fb = train.forward_backward(datums, loss_fn='cross_entropy') + opt = train.optim_step(tinker.AdamParams(learning_rate=lr, **ADAM)) + timing['submit_seconds'] = time.monotonic() - timing['submitted'] + if pending: + self.finish(train, *pending, steps) + pending = (step, batch, tokens, fb, opt, lr, timing) + self.finish(train, *pending, steps) + finally: + prefetch.shutdown(wait=True, cancel_futures=True) + sync_start = time.monotonic() + client, sampler = self.backend.sync(train, 'final') + self.event('sampler_saved', path=sampler, sync_seconds=time.monotonic() - sync_start) + results = self.benchmark(client, (a.domain,), sampler) + final = [json.loads(line) for line in (self.out/'checkpoints.jsonl').read_text().splitlines()][-1] + assert final['step'] == steps + teacher = {'domain': a.domain, 'method': 'sft', 'steps': steps, 'sampler_path': sampler, + 'state_path': final['state_path'], 'benchmark': results[a.domain]} + with (self.out/'teacher.json').open('x') as f: + json.dump(teacher, f, indent=2) + self.event('complete', optimizer_updates=steps - a.start_step, total_optimizer_updates=steps) + + def finish(self, train, step, batch, tokens, fb, opt, lr, timing, steps): + """Wait for one update, record it, and save optimizer state when scheduled. + + Updates are pipelined (the next is submitted before this one is awaited), so step_seconds + (submit to result) overlaps its neighbours; interval_seconds, the time between consecutive + completions, is the throughput measure. It includes any checkpoint save before it, during + which the trainer keeps working on the next update, so the run-level rate (total tokens over + total time) is the figure to report. + """ + fb, opt = fb.result(), opt.result() + done = time.monotonic() + interval, self.last_finish = done - self.last_finish, done + nll, weight = 0.0, 0 + for (prompt, completion), output in zip(tokens, fb.loss_fn_outputs, strict=True): + nll += sequence_nll(output, len(prompt), len(completion)) + weight += len(completion) + prompt_tokens = sum(len(p) for p, _ in tokens) + row = {'step': step + 1, 'utc': datetime.now(timezone.utc).isoformat(), 'sequences': len(batch), + 'learning_rate': lr, 'prompt_tokens': prompt_tokens, 'completion_tokens': weight, + 'train_nll': nll / weight, 'step_seconds': done - timing['submitted'], 'interval_seconds': interval, + 'tokens_per_second': (prompt_tokens + weight) / interval, + 'build_seconds': timing['build_seconds'], 'batch_wait_seconds': timing['batch_wait_seconds'], + 'submit_seconds': timing['submit_seconds'], 'ids': [r['id'] for r in batch], + 'forward_backward_metrics': fb.metrics, 'optimizer_metrics': opt.metrics} + self.write('metrics.jsonl', row) + self.event('step_complete', **{k: v for k, v in row.items() if k != 'ids'}) + if (step + 1) % self.args.save_every == 0 or step + 1 == steps: + save_start = time.monotonic() + state = train.save_state(name=f'step-{step+1:04d}').result().path + save_seconds = time.monotonic() - save_start + self.write('checkpoints.jsonl', {'step': step + 1, 'state_path': state, 'save_seconds': save_seconds}) + self.event('checkpoint', step=step + 1, state_path=state, save_seconds=save_seconds) + + # Benchmark. + + def sample_one(self, client, domain, row): + tokens = self.prompt(row['prompt']) + assert len(tokens) + self.args.max_tokens <= recipe.CONTEXT_LENGTH + start = time.monotonic() + # Multi-response requests leave the seed unset, as in the cookbook. + a = self.args + result = client.sample(tinker.ModelInput.from_ints(tokens), num_samples=a.bench_samples, + sampling_params=tinker.SamplingParams(max_tokens=a.max_tokens, temperature=a.bench_temperature, + top_p=1.0, top_k=-1, stop=self.stop)).result() + assert len(result.sequences) == a.bench_samples + records = [] + for j, seq in enumerate(result.sequences): + response = self.tokenizer.decode(seq.tokens, skip_special_tokens=True) + records.append({'domain': domain, 'index': row['_index'], 'sample': j, 'prompt_tokens': tokens, + 'tokens': seq.tokens, 'response': response, 'score': scoring.score(response, row['label']), + 'stop_reason': seq.stop_reason, 'request_seconds': time.monotonic() - start}) + return records + + def benchmark(self, client, domains, model): + a = self.args + start = time.monotonic() + protocol = {'examples': a.bench_examples, 'samples': a.bench_samples, 'temperature': a.bench_temperature} + self.event('benchmark_start', domains=domains, **protocol) + futures = [self.pool.submit(self.sample_one, client, d, row) + for d in domains for row in self.dev[d][:a.bench_examples]] + records = [] + for f in as_completed(futures): + for r in f.result(): + records.append(r) + self.write('benchmark-samples.jsonl', r) + results = {d: summarize(records, d, a.bench_samples) for d in domains} + seconds = time.monotonic() - start + latency = sorted(r['request_seconds'] for r in records if r['sample'] == 0) + sampled = sum(len(r['tokens']) for r in records) + timing = {'seconds': seconds, 'requests': len(latency), 'sampled_tokens': sampled, + 'sampled_tokens_per_second': sampled / seconds, + 'request_seconds': {'median': statistics.median(latency), 'p90': latency[int(0.9 * (len(latency) - 1))], + 'max': latency[-1]}} + with (self.out/'benchmark.json').open('x') as f: + json.dump({'model': model, 'protocol': protocol, 'max_tokens': a.max_tokens, **timing, **results}, f, indent=2) + self.event('benchmark', results=results, timing=timing) + return results + + def execute(self): + if self.args.mode == 'train': + self.train() + else: + if self.args.checkpoint: + client = self.backend.sampler(path=self.args.checkpoint) + else: + client = self.backend.sampler(model=MODEL) + self.benchmark(client, self.args.domains, self.args.checkpoint or MODEL) + self.event('complete') + + +def main(): + p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + p.add_argument('mode', choices=('train', 'bench')) + p.add_argument('--output', type=Path, required=True) + p.add_argument('--backend', choices=sorted(BACKENDS), default='tinker') + p.add_argument('--domain', choices=DOMAINS, help='train: the teacher domain') + p.add_argument('--domains', choices=DOMAINS, nargs='+', default=list(DOMAINS), help='bench: domains to benchmark') + p.add_argument('--checkpoint', help='train: optimizer-state path to resume; bench: sampler path (default: base)') + p.add_argument('--start-step', type=int, default=0, help='train: completed updates in --checkpoint') + p.add_argument('--seed', type=int, default=20260921) + p.add_argument('--max-tokens', type=int, default=recipe.MAX_RESPONSE_TOKENS) + p.add_argument('--concurrency', type=int, default=DEFAULTS['bench_examples'] * len(DOMAINS)) + for key, value in DEFAULTS.items(): + kind = type(value) + choices = ('linear', 'cosine', 'constant') if key == 'lr_schedule' else None + p.add_argument('--' + key.replace('_', '-'), type=kind, default=value, choices=choices, + help=f'default: {value}') + args = p.parse_args() + for key in ('batch_size', 'epochs', 'rank', 'save_every', 'bench_examples', 'bench_samples'): + if getattr(args, key) <= 0: p.error(f'--{key.replace("_", "-")} must be positive') + if not 1 <= args.min_correct <= 3: p.error('--min-correct is between 1 and 3') + if args.traces_per_prompt < 0 or args.max_rows < 0: p.error('--traces-per-prompt and --max-rows are >= 0') + if not (math.isfinite(args.learning_rate) and args.learning_rate > 0): p.error('--learning-rate must be positive') + if args.bench_temperature < 0: p.error('--bench-temperature must be >= 0') + if args.bench_examples > min(recipe.EVAL_EXAMPLES.values()): p.error('--bench-examples exceeds a dev split') + if args.mode == 'train' and not args.domain: p.error('train requires --domain') + if args.start_step and not (args.mode == 'train' and args.checkpoint): + p.error('--start-step requires a training --checkpoint') + if args.concurrency <= 0 or args.max_tokens <= 0: p.error('concurrency and max-tokens must be positive') + run = None + try: + run = Run(args) + run.execute() + except BaseException as exc: + if run: + run.event('failed', error_type=type(exc).__name__, message=str(exc)) + (run.out/'error.txt').write_text(traceback.format_exc()) + raise + finally: + if run: + run.pool.shutdown(wait=True, cancel_futures=True) + if hasattr(run, 'backend'): + run.backend.close() + + +if __name__ == '__main__': + main() diff --git a/hosted/opd/test_opd.py b/hosted/opd/test_opd.py new file mode 100644 index 0000000..e3c9458 --- /dev/null +++ b/hosted/opd/test_opd.py @@ -0,0 +1,71 @@ +"""Offline checks of the distillation math and prompt batches; no service calls.""" +import os +from types import SimpleNamespace +import unittest + +os.environ['HF_HUB_OFFLINE'] = '1' +from common import Experiment, advantages, percentile, policy_datum # noqa: E402 +from shared import recipe # noqa: E402 +from shared.puzzles import load_split # noqa: E402 + + +class DistillationTests(unittest.TestCase): + def test_advantage_is_negative_reverse_kl(self): + # The teacher likes token 0 more than the student does (positive) and token 1 less (negative). + self.assertEqual(advantages([-1.0, -3.0], [-2.0, -1.0]), [1.0, -2.0]) + with self.assertRaises(AssertionError): + advantages([-1.0], [-1.0, -2.0]) + with self.assertRaises(AssertionError): + advantages([float('nan')], [-1.0]) + + def test_datum_alignment(self): + d = policy_datum([10, 11, 12], [20, 21], [-0.5, -0.7], [1.5, -0.2]) + self.assertEqual(d.model_input.to_ints(), [10, 11, 12, 20]) + self.assertEqual(d.loss_fn_inputs['target_tokens'].data, [0, 0, 20, 21]) + self.assertEqual(d.loss_fn_inputs['advantages'].data[:2], [0.0, 0.0]) + + def test_opd_and_mopd_see_one_caesar_prompt_order(self): + import random + run = Experiment.__new__(Experiment) + run.args = SimpleNamespace(seed=20260921) + run.data = {d: {'train': load_split(d, 'train')} for d in recipe.DOMAINS} + run.order = {} + for d in recipe.DOMAINS: + run.order[d] = list(range(len(run.data[d]['train']))) + random.Random(run.args.seed + recipe.DOMAINS.index(d)).shuffle(run.order[d]) + # OPD takes 128 caesar prompts per update; MOPD takes 64 per domain. Caesar's order is the same. + opd = [r['_index'] for _, r in run.train_prompts('caesar_cipher', 0, 128)] + mopd = [r['_index'] for u in (0, 1) for _, r in run.train_prompts('caesar_cipher', u, 64)] + self.assertEqual(opd, mopd) + self.assertEqual(len(set(opd)), 128) + + def test_policy_lag(self): + """Async samples update u+1 on update u's sampler (lag 1); synchronous samples it on its own (lag 0).""" + from concurrent.futures import ThreadPoolExecutor + from itertools import count + + for lag, expected in ((1, ['w0', 'w0', 'w1', 'w2']), (0, ['w0', 'w1', 'w2', 'w3'])): + done = SimpleNamespace(result=lambda: SimpleNamespace(loss_fn_outputs=[{}], metrics={})) + train = SimpleNamespace(forward_backward=lambda *a, **k: done, optim_step=lambda *a, **k: done) + version = count(1) + run = Experiment.__new__(Experiment) + run.args = SimpleNamespace(policy_lag=lag, updates=4, start_update=0, seed=0, learning_rate=1e-5, + save_every=100, eval_every=100, no_eval=True, skip_initial_eval=True) + run.pool = ThreadPoolExecutor(4) + run.backend = SimpleNamespace(sync=lambda train, name: (f'w{next(version)}', name)) + run.start = lambda: (train, 'w0') + run.rollout = lambda student, *rest: {'student': student, 'prompt_tokens': [1], 'tokens': [2], + 'logprobs': [-1.0], 'advantages': [0.0]} + run.log = run.save_rollouts = run.append = run.checkpoint = lambda *a, **k: None + metrics = [] + run.update_metrics = lambda update, rollouts, *rest: metrics.append((rollouts[0]['student'], rest[-1])) + run.train_loop(lambda update: [('caesar_cipher', {})]) + self.assertEqual(metrics, [(w, 0 if i == 0 else lag) for i, w in enumerate(expected)]) + + def test_percentile(self): + self.assertEqual(percentile(list(range(10)), 0.9), 9) + self.assertEqual(percentile([5], 0.5), 5) + + +if __name__ == '__main__': + unittest.main() diff --git a/hosted/opd/test_sft.py b/hosted/opd/test_sft.py new file mode 100644 index 0000000..7e3eae1 --- /dev/null +++ b/hosted/opd/test_sft.py @@ -0,0 +1,102 @@ +"""Offline checks of the SFT datum, schedules, trace filters, rendering, and benchmark summary; no service calls. + +The rendering test needs the pinned dataset snapshot in the Hugging Face cache and is skipped otherwise. +""" +import os +from pathlib import Path +import random +from types import SimpleNamespace +import unittest + +os.environ['HF_HUB_OFFLINE'] = '1' # Before transformers is imported: use only the cached tokenizer. +from sft import (DEFAULTS, DOMAINS, MODEL, RENDERER, REVISION, Run, epoch_order, lr_at, select_traces, # noqa: E402 + sequence_nll, sft_datum, summarize) +from shared.puzzles import load_split # noqa: E402 + + +class SFTTests(unittest.TestCase): + def test_datum_trains_completion_only(self): + d = sft_datum([10, 11, 12], [20, 21, 22]) + self.assertEqual(d.model_input.to_ints(), [10, 11, 12, 20, 21]) + self.assertEqual(d.loss_fn_inputs['target_tokens'].data, [11, 12, 20, 21, 22]) + self.assertEqual(d.loss_fn_inputs['weights'].data, [0, 0, 1, 1, 1]) + + def test_sequence_nll_from_either_provider(self): + import tinker + tinker_output = {'logprobs': tinker.TensorData(data=[-9.0, -9.0, -2.0, -1.0, -0.5], dtype='float32', shape=[5])} + fireworks_output = {'loss': tinker.TensorData(data=[3.5], dtype='float32', shape=[1])} + self.assertAlmostEqual(sequence_nll(tinker_output, 3, 3), 3.5) + self.assertAlmostEqual(sequence_nll(fireworks_output, 3, 3), 3.5) + + def test_schedules(self): + self.assertEqual(lr_at(0, 10, 1e-3), 1e-3) + self.assertAlmostEqual(lr_at(5, 10, 1e-3), 5e-4) + self.assertGreater(lr_at(9, 10, 1e-3), 0) + self.assertAlmostEqual(lr_at(5, 10, 1e-3, 'cosine'), 5e-4) + self.assertGreater(lr_at(9, 10, 1e-3, 'cosine'), 0) + self.assertEqual(lr_at(9, 10, 1e-3, 'constant'), 1e-3) + + def test_first_epoch_is_the_released_teachers_shuffle(self): + # The released teachers shuffled once with Random(seed + domain index); epoch 0 must stay identical. + seed = 20260921 + 1 + expected = list(range(50)) + random.Random(seed).shuffle(expected) + orders = epoch_order(50, 2, seed) + self.assertEqual(orders[0], expected) + self.assertNotEqual(orders[1], expected) + self.assertEqual(sorted(orders[1]), list(range(50))) + + def test_trace_filters(self): + rows = [{'id': f'{p}-{s}', 'source_index': p, 'sample_index': s, 'correct_of_3': c} + for p, c in [(0, 3), (1, 1), (2, 2)] for s in range(c)] + self.assertEqual(select_traces(rows), rows) + self.assertEqual([r['id'] for r in select_traces(rows, min_correct=2)], ['0-0', '0-1', '0-2', '2-0', '2-1']) + self.assertEqual([r['id'] for r in select_traces(rows, traces_per_prompt=1)], ['0-0', '1-0', '2-0']) + subset = select_traces(rows, max_rows=3, seed=1) + self.assertEqual(len(subset), 3) + self.assertEqual(subset, select_traces(rows, max_rows=3, seed=1)) + + def test_defaults_reproduce_released_teachers(self): + self.assertEqual((DEFAULTS['revision'], DEFAULTS['epochs'], DEFAULTS['batch_size'], DEFAULTS['rank']), + ('b21bb3e75b1ca903f4b69bd609adc89bd99543f1', 1, 128, 64)) + self.assertAlmostEqual(DEFAULTS['learning_rate'], 0.0004990818286656736) + + def test_summary(self): + records = [{'domain': 'caesar_cipher', 'index': i, 'sample': j, 'score': float(i == 0 or j == 0), + 'tokens': [0] * 10, 'response': '', 'stop_reason': 'stop'} + for i in range(2) for j in range(3)] + s = summarize(records, 'caesar_cipher', 3) + self.assertEqual((s['problems'], s['samples'], s['correct']), (2, 6, 4)) + self.assertAlmostEqual(s['accuracy'], 4 / 6) + self.assertEqual(s['pass_at_3'], 1.0) + self.assertEqual(s['all_3_correct'], 0.5) + + def test_released_traces_render_to_prompt_plus_completion(self): + try: + import pyarrow.parquet as pq + from huggingface_hub import snapshot_download + root = snapshot_download(DEFAULTS['dataset'], repo_type='dataset', revision=DEFAULTS['revision']) + except Exception as exc: + self.skipTest(f'dataset snapshot not cached: {exc}') + from transformers import AutoTokenizer + from tinker_cookbook import renderers + run = Run.__new__(Run) + run.tokenizer = AutoTokenizer.from_pretrained(MODEL, revision=REVISION) + run.renderer = renderers.get_renderer(RENDERER, run.tokenizer, model_name=MODEL) + run.args = SimpleNamespace(max_tokens=0) + end = run.tokenizer.convert_tokens_to_ids('<|im_end|>') + for domain in DOMAINS: + path = sorted((Path(root)/'data'/domain).glob('train-*.parquet'))[0] + rows = pq.read_table(path, columns=['id', 'messages']).slice(0, 4).to_pylist() + dev_system = load_split(domain, 'dev')[0]['prompt'][0]['content'] + for row in rows: + prompt, completion = run.tokenize(row) + self.assertEqual(row['messages'][0]['content'], dev_system) + self.assertEqual(completion[-1], end) + text = run.tokenizer.decode(completion) + self.assertEqual(text.count(''), 1) + self.assertIn('', text.rsplit('', 1)[1]) + + +if __name__ == '__main__': + unittest.main() diff --git a/hosted/tinker/README.md b/hosted/tinker/README.md new file mode 100644 index 0000000..b19d8b1 --- /dev/null +++ b/hosted/tinker/README.md @@ -0,0 +1,221 @@ +# Tinker: hosted multi-teacher OPD + +This runs the shared MOPD recipe (`shared/recipe.py`: reasoning_gym `caesar_cipher` +and `simple_geometry`, thinking on) on the hosted Tinker service. It trains one +specialist teacher per domain, then distills a fresh student from both with routed +sampled-token OPD. Unlike the self-hosted frameworks, which distill the frozen +`recipe.TEACHERS`, this campaign builds its own teachers, by GRPO (default) or by SFT. +Model and optimizer checkpoints stay on Tinker; only the tokenizer, source and logs +are local. + +No student results exist yet for this campaign. [RESULTS.md](RESULTS.md) holds the +previous Countdown/Graph Coloring experiment (thinking off). Tinker OPD and MOPD runs +with the `hosted/opd` loop are in [../opd/RESULTS.md](../opd/RESULTS.md). + +| File | Purpose | +|---|---| +| `run.py` | One run: `eval`, `teacher` (GRPO) or `mopd` (student). | +| `launch.py` | The full campaign: account check, baseline, teachers, selection, student, report. | +| `report.py` | Audits the student and writes `final-audit.json`, `summary.json`, `REPORT.md`, `learning-curves.png`. | +| `test_run.py` | Offline tests: token alignment, data, prompt template, scoring, selection, launcher settings, pipelining. | +| [`../opd/sft.py`](../opd/sft.py) | SFT teachers (`train`) and the 100 × 3 teacher benchmark (`bench`); `launch.py --teachers sft` runs it. Its tests are in `../opd/test_sft.py`. | + +## Run + +Requirements: + +- Python 3.12 with `requirements.txt` installed (`matplotlib` is optional, for the plot). +- `reasoning-gym==0.1.25` installed, or its site-packages directory in `$CAMPAIGN_PYDEPS` + (see `shared/scoring.py`). +- The pinned tokenizer already in the Hugging Face cache; the launcher sets `HF_HUB_OFFLINE=1`. +- `TINKER_API_KEY` exported. To read the key from `~/.zprofile` instead, or to pin an + account, copy `config/tinker.example.json` to `config/tinker.local.json` and set + `api_key_profile_label` or `expected_email`/`expected_org`. With the guard set, the + launcher refuses any other account. See [config/README.md](../../config/README.md). + +```bash +# Full campaign into a new directory. +python hosted/tinker/launch.py --root /abs/new-campaign + +# Student only, reusing a finished campaign's selected teachers. +python hosted/tinker/launch.py --root /abs/new-student --teacher-campaign /abs/new-campaign + +# SFT teachers and their benchmark only (optionally --sft-config FILE), then a student from them. +python hosted/tinker/launch.py --root /abs/sft-teachers --teachers sft --teachers-only +python hosted/tinker/launch.py --root /abs/sft-student --teacher-campaign /abs/sft-teachers + +# One run directly. +python hosted/tinker/run.py mopd --teachers teachers.json --output /abs/student + +# Offline tests. +python -B -m unittest discover -s hosted/tinker -p 'test_*.py' -v +``` + +The launcher copies `hosted/tinker`, `hosted/opd`, `shared` and `data/` into +`/source-repo` and runs every stage from that snapshot. Progress is in +`status.json`, `campaign-events.jsonl` and `-console.log`. + +Each run directory keeps its config, package versions, source, scorer, data hashes, +every sample's tokens, logprobs and text, teacher and learner scores, metrics, +evaluations, and remote checkpoint paths. + +## Protocol + +- Base model `Qwen/Qwen3.6-35B-A3B`. The tokenizer is pinned at + `995ad96eacd98c81ed38be0c5b274b04031597b0`; the hosted weights are provider-controlled. + Prompts use the cookbook `qwen3_5` renderer, and its tokens (ending in the `` + prefill) must equal the pinned HF chat template's with `enable_thinking=True`. +- The exact `data/` train/dev splits and the shared scorer. Seed 20260921, with an + independent deterministic shuffle per domain. +- Rank-64 LoRA (attention, MLP, unembedding), LR `1e-5`. This is the cookbook's + documented 10× FullFT-to-LoRA conversion of the recipe's `1e-6`, which is also the + LR of the released GRPO teachers. It is a heuristic, not an equivalence. +- GRPO teachers use the training settings of the released GRPO teachers: one + independently initialized LoRA per domain; 32 prompts × 8 responses, temperature 1, + top-p 1; PPO clip 0.8/1.2; per-group mean/std reward normalization; Adam betas + 0.9/0.98, no weight decay, gradient clip 1. Each teacher is evaluated every 25 updates + and stops at the first evaluation that meets its goal (caesar_cipher 70%, + simple_geometry 95%), or after 200 updates. Selection takes the first checkpoint that + meets the goal, otherwise the best (earliest on ties). Two differences from the + released teachers' training: groups with no reward variance are kept (zero advantage), + and training is synchronous. +- Student: a fresh base model, 20 updates of 64 prompts per domain × 1 response (2,560 + trajectories). Sampled-token reverse KL (teacher minus sampler logprob) with native + importance sampling and zero task reward; Adam betas 0.9/0.999, no weight decay, + gradient clip 1. Losses sum over response tokens, so equal prompt counts do not mean + equal per-domain gradient weight. +- Up to 30,720 response tokens in a 32,768-token context. Generation stops only at the + model's end-of-turn token. A response scores 1 only if the text after its last + `` contains an `` that reasoning_gym accepts. +- Evaluation at updates 0, 10 and 20: the full dev split (126 caesar_cipher, 128 + simple_geometry), greedy. +- Multi-response teacher requests leave the sampling seed unset, as in the cookbook. + Rollouts are reproduced from the retained token records, not by regeneration. +- Optimizer calls are never retried by the application. To resume, start from a + recorded state checkpoint in a fresh directory. + +## Pipelining + +The launcher runs the student with `--pipeline-depth 1` (pass 0 for synchronous +training). It prepares the next batch, with teacher scoring overlapping generation, +while the current update trains. Policy lag is at most one update +(`recipe.MAX_POLICY_LAG`) and is audited; the budget stays exactly 20 × 128 responses. + +The advantage is still teacher minus behavior logprob under importance sampling, so +this is an off-policy variant of the synchronous objective, and its quality should be +measured. In pipelined runs, `sampling_seconds` is the exposed wait for the prepared +batch. + +## SFT teachers + +`hosted/opd/sft.py` builds a teacher by supervised fine-tuning instead of GRPO, and +benchmarks it. Its defaults reproduce the released SFT teachers below. Every setting is +a flag and is recorded in the run's `config.json` (`changed_from_defaults` lists what +differs). + +- Data: by default every trace of + [`semianalysisai/Qwen3.6-35B-A3B-teacher-sft-caesar-geometry`](https://huggingface.co/datasets/semianalysisai/Qwen3.6-35B-A3B-teacher-sft-caesar-geometry) + at `b21bb3e75b1ca903f4b69bd609adc89bd99543f1`: verified-correct traces (temperature 1, + thinking on) sampled from the frozen `recipe.TEACHERS`, 16,980 caesar_cipher and + 29,547 simple_geometry rows. No dev problem appears in it. Every row is checked + (roles, system prompt, one ``) before training. +- Rendering: the pinned HF template. Each prompt must equal the cookbook renderer's + generation prompt and be a prefix of the rendered trace. The loss covers only the + completion: thinking, answer and `<|im_end|>`. +- Training: one rank-64 LoRA per domain, on that domain's rows only. Summed token cross + entropy, one epoch, 128 sequences per update, final partial batch dropped (132 and 230 + updates), seeded shuffle. LR is the cookbook's `get_lr` for this model (about 5e-4) + with linear decay to zero; Adam betas 0.9/0.95, no weight decay or clipping. Optimizer + state is saved every 25 updates, sampler weights at the end. +- Benchmark: the first 100 problems of each dev split, 3 samples each at temperature 1, + top-p 1, up to 30,720 response tokens. `benchmark.json` reports avg@3 (`accuracy`), + pass@3, all-3-correct, lengths, truncations and unfinished thinking. Each teacher is + benchmarked on its own domain when training ends. + +| Flag | Default | Meaning | +|---|---|---| +| `--dataset`, `--revision` | the dataset above, pinned | Any dataset with the same layout (see the `sft.py` docstring) | +| `--min-correct` | 1 | Keep prompts the teacher solved at least this many times of 3 (3 = easiest only) | +| `--traces-per-prompt` | 0 (all) | At most this many traces per prompt, lowest sample index first | +| `--max-rows` | 0 (all) | Then a seeded random subset of this many rows | +| `--learning-rate`, `--lr-schedule` | `get_lr` (≈5e-4), `linear` | Peak LR; `linear`, `cosine` or `constant` | +| `--batch-size`, `--epochs`, `--rank` | 128, 1, 64 | Each epoch reshuffles; epoch 0 is the released order | +| `--save-every` | 25 | Optimizer-state checkpoint interval, in updates | +| `--bench-examples`, `--bench-samples`, `--bench-temperature` | 100, 3, 1.0 | Benchmark protocol | + +```bash +# Reproduce one released teacher, or benchmark a model (the base model by default). +python hosted/opd/sft.py train --domain caesar_cipher --output /abs/teacher-caesar-cipher +python hosted/opd/sft.py bench --output /abs/bench [--checkpoint tinker://.../sampler_weights/final] + +# A different teacher: one trace per prompt, only prompts the teacher always solved, two epochs. +python hosted/opd/sft.py train --domain caesar_cipher --output /abs/t \ + --traces-per-prompt 1 --min-correct 3 --epochs 2 + +# Resume from an optimizer-state checkpoint (see checkpoints.jsonl) into a fresh directory, same settings. +python hosted/opd/sft.py train --domain caesar_cipher --output /abs/t-resumed \ + --checkpoint tinker://.../weights/step-0100 --start-step 100 +``` + +`launch.py --teachers sft` runs the base-model benchmark alongside both teachers, then +writes `teachers.json` and `teacher-selection.json` so that `--teacher-campaign` can +reuse the teachers for a student. There is no selection: each teacher is its final +checkpoint. `--sft-config FILE` passes `sft.py` settings, for every stage or per domain, +and is saved as `sft-config.json` in the campaign: + +```json +{"learning_rate": 2e-4, "domains": {"caesar_cipher": {"epochs": 2}, "simple_geometry": {"max_rows": 5000}}} +``` + +### Released SFT teachers (2026-09-27) + +Built with `launch.py --teachers sft --teachers-only` and the defaults, on the SA RL +Infra Research account (campaign `tinker-sa-sft-teachers-20260927T115503Z`). The two +teachers trained concurrently, about 20 minutes each. + +| | caesar_cipher base | caesar_cipher SFT teacher | simple_geometry base | simple_geometry SFT teacher | +|---|---|---|---|---| +| avg@3 | 27.3% | 79.3% | 51.3% | 99.7% | +| pass@3 | 47% | 95% | 81% | 100% | +| all 3 correct | 12% | 57% | 19% | 99% | +| mean response tokens | 11.1k | 12.7k | 3.6k | 3.1k | +| truncated (of 300) | 13 | 13 | 1 | 0 | + +- Sampler paths: caesar_cipher + `tinker://02c4683b-edf3-526b-991d-90c9a0c1b183:train:0/sampler_weights/final`, + simple_geometry + `tinker://231067a0-323a-5424-b0f2-62353750d220:train:0/sampler_weights/final`. + They stay on the account that trained them. +- Training tokens: 199.0M and 93.5M, about $345 at the published $1.177/M, plus a few + dollars of sampling. This is a list-price estimate, not an invoice. +- Train NLL barely moves (0.316 → 0.308 and 0.342 → 0.331, first vs last 20 + updates). The traces come from a GRPO-trained copy of the same base, so little + per-token loss is reducible; accuracy is what changes. +- For comparison, the GRPO teachers score 73.2% and 99.5% avg@8 on the full dev splits. + The base scores here match the dataset card's (27.8% and 52.0%), but 100 problems × 3 + samples is a smaller sample. + +## Caveats + +- Thinking at 32k tokens costs far more than the previous 256-token protocol. Responses + in the released GRPO teachers' training averaged about 11k (caesar_cipher) and 4k + (simple_geometry) tokens, and a GRPO teacher can run 200 updates of 256 responses. + Watch the cost estimate in `REPORT.md` while the campaign runs. +- The GRPO teacher goals are operational, set just below the released GRPO teachers' + held-out scores (27.0% → 73.3% and 53.9% → 99.8%, sampled at temperature 1). Meeting + them does not show that the retrained teachers match `recipe.TEACHERS`. +- Equal steps and prompts do not make this a controlled comparison with the self-hosted + frameworks: Tinker uses LoRA, hosted scheduling and hardware, sampled-token OPD, and + its own teachers. Timings are client wall time, not GPU utilization. +- Report teacher training cost separately from the student, including failed attempts. + +## Sources + +- [Miles PR 3116](https://github.com/radixark/miles/pull/3116) at + `8f8e4dff55d80bd9d36c3490c333a119f38480ed`. +- Tinker docs: [distillation](https://tinker-docs.thinkingmachines.ai/cookbook/recipes/distillation/), + [LoRA](https://tinker-docs.thinkingmachines.ai/tinker/lora-primer/), + [PPO](https://tinker-docs.thinkingmachines.ai/tinker/losses/ppo/), + [importance sampling](https://tinker-docs.thinkingmachines.ai/tinker/losses/importance-sampling/). +- Tinker cookbook `recipes/rl_loop.py`, `distillation/train_on_policy.py` and + `hyperparam_utils.py` at `1e53aa3d1cdd6389b3290c2574641eccc0503242` (runtime 0.5.7). diff --git a/hosted/tinker/RESULTS.md b/hosted/tinker/RESULTS.md new file mode 100644 index 0000000..f2575ef --- /dev/null +++ b/hosted/tinker/RESULTS.md @@ -0,0 +1,107 @@ +# Tinker MOPD results, 2026-09-21 (previous experiment) + +These results are from the earlier Countdown/Graph Coloring campaign: thinking off, +256 response tokens, 40 student updates, 512 dev puzzles per domain. Its code is in git +history (commit `3e27ddb`, `frameworks/tinker/`). The current code runs the +reasoning_gym experiment in `shared/recipe.py`, which has no recorded results yet. + +Two specialist teachers were trained on Tinker, then frozen and used to distill a fresh +`Qwen/Qwen3.6-35B-A3B` student. The student ran 40 optimizer updates of 128 responses +(64 prompts per domain), 5,120 training responses in total. All scores use the 512 +development examples per domain, greedy decoding, thinking off, and a 256-token cap. +They are development results from one seed, not held-out test results. + +## Quality + +| Model/checkpoint | Countdown | Graph Coloring | +|---|---:|---:| +| Selected specialist (own domain), update 80 | 238/512 (46.48%) | 466/512 (91.02%) | +| Fresh student, update 0 | 59/512 (11.52%) | 130/512 (25.39%) | +| MOPD student, update 10 | 170/512 (33.20%) | 277/512 (54.10%) | +| MOPD student, update 20 | 202/512 (39.45%) | 317/512 (61.91%) | +| MOPD student, update 30 | 214/512 (41.80%) | 413/512 (80.66%) | +| MOPD student, update 40 | 215/512 (41.99%) | 436/512 (85.16%) | + +The specialist row covers two teachers. Each ran 80 updates of 32 prompts × 8 +responses; the 40-update checkpoints are also retained. The graph teacher met its +provisional 90% goal. The Countdown teacher missed its provisional 50% goal, so its best +scheduled checkpoint at the 80-update cap was used. The original teacher scores were +not available, so the goals are not recovered scores and teacher quality is not shown to +match the originals. Selection used development scores only; the student always uses +update 40. + +In the final student evaluation, 49/512 Countdown responses and no Graph Coloring +responses hit the token cap. All responses, including truncated ones, were scored and +kept. + +## Comparison with the self-hosted runs + +| Execution | Countdown baseline → final | Graph baseline → final | Observed student duration | +|---|---:|---:|---:| +| Miles, self-hosted | 10.55% → 45.51% | 26.17% → 88.28% | 21m 34s allocation | +| Prime-RL, self-hosted | 10.94% → 39.65% | 25.78% → 64.65% | 24m 22s allocation | +| Slime, self-hosted | 10.94% → 44.14% | 26.37% → 89.26% | 13m 25s allocation | +| Tinker, hosted | 11.52% → 41.99% | 25.39% → 85.16% | 31m 31s client elapsed | + +The self-hosted numbers and timing definitions come from +[PROVENANCE.md](../../docs/PROVENANCE.md). Every student had the same budget of 40 +updates and 5,120 responses, but this is not a controlled ranking: + +- The self-hosted runs used 16 B200 GPUs. Tinker's hardware and exact served weight + revision are provider-controlled. +- Tinker used rank-64 LoRA, LR 1e-5 (the cookbook's 10× conversion of the original + full-finetuning LR 1e-6), synchronous sampling, and sampled-token reverse-KL + distillation with no task reward. +- Teacher identities, objectives, scheduling and stopping rules differ. +- Tinker stops at EOS; the pinned Miles launcher defaults to an answer-tag stop. + +## Timing + +The Tinker student took 1,891.39 seconds, including 418.43 seconds of evaluation. The +update loops took 1,397.77 seconds (median 34.97 seconds per update). This excludes +teacher training and earlier attempts. The whole recorded campaign, including attempts, +gaps and overlapping teachers, took 96.02 minutes; it excludes setup before the first +recorded attempt. + +## Cost + +Billing had no settled usage rows when collected, which does not mean zero cost. +Estimates from recorded tokens are $2.57 (all prefill cached) to $3.74 (all uncached) +for the student, and $20.77 to $25.86 across all recorded attempts. These are not +invoices or cost bounds: they exclude unlogged retries, unfinished mutations, probes and +storage. The 40 retained remote checkpoints total 357.67 GB (decimal), about +$35.77/month at the documented storage rate if kept for a full month, depending on +billing units and retention time. + +## Issues found during the run + +The main problem was explicit seeding of multi-response teacher calls, which produced +repeated responses within groups. Both initial teacher attempts were invalidated (and +kept); the corrected teachers restarted from the base model with the multi-response +seed unset, as in the cookbook. Smaller fixes: explicit tokenizer `return_dict=False`, a +floating-point test tolerance, and disabling bytecode for scorer imports in the original +campaign template directory. The corrected teachers, their extensions and the student +all exited successfully. + +## Artifacts + +Full artifacts are kept outside Git in a private run archive, `opd-runs/tinker-20260921/`: + +- `REPORT.md`, `summary.json`, `learning-curves.png/.svg`: all attempts, evaluations, + timings and cost notes. +- `JOURNAL.md`: decisions, errors, fixes and teacher-selection amendments. +- Per-run source, config, package and data snapshots, console logs, `samples.jsonl`, + `learner-scores.jsonl`, `teacher-scores.jsonl`, `metrics.jsonl` and evaluations. +- `teacher-selection.json`, `teachers.json` and per-run `checkpoints.jsonl`: the private + remote model and optimizer-state paths. +- `final-audit.json`: all 40 updates, 5,120 training sequences and 5,120 evaluation + sequences verified, every task score recomputed, every routed distillation advantage + checked. Seven Tinker tests and six package tests passed. +- `service-metadata-20260921T104053Z/`: run and checkpoint metadata and a billing + snapshot. + +The final student sampler is +`tinker://e7a10aea-05ff-554d-bbe6-a1ba0511c480:train:0/sampler_weights/step-040`; the +optimizer state has the same run prefix with `weights/step-040`. No weights were +downloaded; the checkpoints remain private on Tinker. The self-hosted recipes were not +changed by this campaign. diff --git a/frameworks/tinker/UPSTREAM-LICENSE b/hosted/tinker/UPSTREAM-LICENSE similarity index 100% rename from frameworks/tinker/UPSTREAM-LICENSE rename to hosted/tinker/UPSTREAM-LICENSE diff --git a/hosted/tinker/launch.py b/hosted/tinker/launch.py new file mode 100644 index 0000000..e6f8258 --- /dev/null +++ b/hosted/tinker/launch.py @@ -0,0 +1,272 @@ +"""Run the hosted Tinker MOPD campaign. + +Full campaign: evaluate the base model, train one GRPO teacher per domain +(stopping at its goal), select the teachers, train a fresh MOPD student, and +write the report. + +--teachers sft: each teacher is instead a LoRA SFT on the released traces of the +frozen recipe.TEACHERS (hosted/opd/sft.py), and the base model and teachers get the +100-problem x 3-sample benchmark. --sft-config passes sft.py settings as JSON, for +every stage or per domain (see sft_arguments). +--teachers-only: stop once the teachers are selected. +--teacher-campaign: reuse that campaign's selected teachers and train only the student. + +The API key comes from $TINKER_API_KEY, or from the `~/.zprofile` line named by +api_key_profile_label in config/tinker.local.json. If that file sets expected_email and expected_org, +any other account is refused. Every stage runs from a snapshot of the code and +data in /source-repo; model weights stay remote. +""" +import argparse +from concurrent.futures import ThreadPoolExecutor +from datetime import datetime, timezone +import json +import os +from pathlib import Path +import shutil +import subprocess +import sys +import threading +import traceback + +REPO = Path(__file__).resolve().parents[2] +sys.path.insert(0, str(REPO)) +from shared.credentials import api_key, local_config # noqa: E402 + +# Operational own-domain dev accuracy goals (greedy, full dev split), set just below the released +# GRPO teachers' held-out scores (caesar_cipher 73.3%, simple_geometry 99.8%). Meeting them does not +# show a match with the frozen recipe.TEACHERS. A teacher stops at its first scheduled evaluation +# that meets its goal, or after run.py's TEACHER['steps'] updates. +TEACHER_GOALS = {'caesar_cipher': 0.70, 'simple_geometry': 0.95} +SFT_SCRIPT = '../opd/sft.py' # --teachers sft runs hosted/opd/sft.py on its Tinker backend. + + +def utc_now(): + return datetime.now(timezone.utc).isoformat() + + +def read_jsonl(path): + return [json.loads(line) for line in Path(path).read_text().splitlines()] + + +def verify_account(settings): + """Return the key and account.json record; with a configured guard, refuse any other account.""" + label = settings.get('api_key_profile_label') + key = api_key('TINKER_API_KEY', label) + import httpx + from tinker.cli.auth_api import TinkerAuthApi + with httpx.Client(timeout=30) as client: + identity = TinkerAuthApi(client).get_self_api_key(key) + email, organization = identity.details.user_details.email, identity.details.org_details.name + expected = settings.get('expected_email'), settings.get('expected_org') + guarded = any(expected) + if guarded and (email, organization) != expected: + raise RuntimeError(f'Account guard rejected {email} / {organization}') + return key, {'email': email, 'organization': organization, 'key_name': identity.name, + 'credential_label': label or 'TINKER_API_KEY', 'verified_utc': utc_now(), + 'account_guard_passed': guarded} + + +def choose(rows, domain, goal): + """First evaluated checkpoint meeting the goal; otherwise the best, earliest on ties.""" + qualifying = [r for r in rows if r['evaluation'][domain]['accuracy'] >= goal] + if qualifying: + chosen = min(qualifying, key=lambda r: r['step']) + else: + chosen = max(rows, key=lambda r: (r['evaluation'][domain]['accuracy'], -r['step'])) + return {**chosen, 'provisional_target': goal, 'target_met': bool(qualifying)} + + +def select_teachers(root): + """Write teacher-selection.json and teachers.json from the completed teacher runs.""" + candidates = {d: [] for d in TEACHER_GOALS} + for config_path in sorted(root.glob('*/config.json')): + path = config_path.parent + config = json.loads(config_path.read_text()) + if config['mode'] != 'teacher' or (path/'INVALIDATED.json').exists(): + continue + if read_jsonl(path/'events.jsonl')[-1]['event'] != 'complete': + raise RuntimeError(f'Teacher attempt is not complete: {path.name}') + checkpoints = {r['step']: r for r in read_jsonl(path/'checkpoints.jsonl')} + for evaluation in read_jsonl(path/'evaluations.jsonl'): + if evaluation['step'] in checkpoints: + candidates[config['domain']].append({'source_run': path.name, 'step': evaluation['step'], + 'evaluation': evaluation, 'checkpoint': checkpoints[evaluation['step']]}) + missing = [d for d, rows in candidates.items() if not rows] + if missing: + raise RuntimeError(f'No saved, evaluated teacher for {", ".join(missing)}') + selected = {d: choose(rows, d, TEACHER_GOALS[d]) for d, rows in candidates.items()} + routes = {d: r['checkpoint']['sampler_path'] for d, r in selected.items()} + assert len(set(routes.values())) == len(routes) + report = {'rule': 'First scheduled checkpoint reaching provisional goal; otherwise highest own-domain dev score, earliest on ties.', + 'original_teacher_quality_match_verified': False, + 'target_source': 'Operational goals below the released GRPO teachers\' held-out scores; not a verified quality match.', + 'selected': selected} + write_selection(root, report, routes) + + +def write_selection(root, report, routes): + for filename, value in [('teacher-selection.json', report), ('teachers.json', routes)]: + with (root/filename).open('x') as f: # Never overwrite a selection. + json.dump(value, f, indent=2) + + +def select_sft_teachers(root): + """Each domain's SFT teacher is its single, final checkpoint.""" + selected = {} + for d in TEACHER_GOALS: + teacher = json.loads((root/sft_name(d)/'teacher.json').read_text()) + assert teacher['domain'] == d + selected[d] = {'source_run': sft_name(d), 'step': teacher['steps'], 'benchmark': teacher['benchmark'], + 'checkpoint': {'sampler_path': teacher['sampler_path'], 'state_path': teacher['state_path']}} + routes = {d: r['checkpoint']['sampler_path'] for d, r in selected.items()} + assert len(set(routes.values())) == len(routes) + report = {'rule': 'SFT teachers: the final checkpoint of one epoch on the released traces; no selection.', + 'original_teacher_quality_match_verified': False, 'selected': selected} + write_selection(root, report, routes) + + +def sft_name(domain): + return 'teacher-' + domain.replace('_', '-') + + +def sft_arguments(settings, domain=None): + """sft.py flags from an --sft-config object: {"learning_rate": 2e-4, "domains": {"caesar_cipher": {...}}}. + + Top-level settings apply to every stage (the baseline too); a domain's section applies to its teacher only. + Keys are sft.py flag names with underscores; sft.py rejects unknown ones. + """ + merged = {k: v for k, v in settings.items() if k != 'domains'} + if domain: + merged.update(settings.get('domains', {}).get(domain, {})) + unknown = set(settings.get('domains', {})) - set(TEACHER_GOALS) + if unknown: + raise ValueError(f'Unknown domains in --sft-config: {sorted(unknown)}') + return [x for k, v in merged.items() for x in ('--' + k.replace('_', '-'), str(v))] + + +class Campaign: + def __init__(self, root): + self.root = root + self.scripts = root/'source-repo/hosted/tinker' + self.lock = threading.Lock() # Teachers train concurrently. + self.sft_settings = json.loads((root/'sft-config.json').read_text()) if (root/'sft-config.json').exists() else {} + + def record(self, event, **fields): + row = {'utc': utc_now(), 'event': event, **fields} + with self.lock: + with (self.root/'campaign-events.jsonl').open('a') as stream: + stream.write(json.dumps(row) + '\n') + print(json.dumps(row), flush=True) + + def status(self, stage, **fields): + temp = self.root/'status.tmp' + temp.write_text(json.dumps({'stage': stage, 'pid': os.getpid(), 'utc': utc_now(), **fields}, indent=2)) + temp.replace(self.root/'status.json') + + def execute(self, name, script, *arguments): + """Run one snapshot script; its console output goes to -console.log.""" + command = [sys.executable, '-u', str(self.scripts/script), *map(str, arguments)] + self.record('process_start', name=name, command=command) + with (self.root/f'{name}-console.log').open('w') as log: + code = subprocess.run(command, stdout=log, stderr=subprocess.STDOUT).returncode + self.record('process_exit', name=name, exit_code=code) + if code: + raise RuntimeError(f'{name} exited {code}; see retained console log') + + def teacher(self, domain): + name = 'teacher-' + domain.replace('_', '-') + self.execute(name, 'run.py', 'teacher', '--domain', domain, '--output', self.root/name, + '--target-score', TEACHER_GOALS[domain], '--skip-baseline') + + def sft_teacher(self, domain): + self.execute(sft_name(domain), SFT_SCRIPT, 'train', '--domain', domain, '--output', self.root/sft_name(domain), + *sft_arguments(self.sft_settings, domain)) + + def run(self, prior, pipeline_depth, method, teachers_only): + if prior: + self.record('reusing_teachers', teacher_campaign=str(prior)) + elif method == 'sft': + # The base-model benchmark runs alongside the teachers. + self.status('teachers') + with ThreadPoolExecutor(max_workers=1 + len(TEACHER_GOALS)) as pool: + baseline = pool.submit(self.execute, 'baseline', SFT_SCRIPT, 'bench', '--output', self.root/'baseline', + *sft_arguments(self.sft_settings)) + list(pool.map(self.sft_teacher, TEACHER_GOALS)) + baseline.result() + select_sft_teachers(self.root) + else: + self.status('baseline') + self.execute('baseline', 'run.py', 'eval', '--output', self.root/'baseline') + self.status('teachers') + # Leaving the pool waits for both teachers, even when one fails. + with ThreadPoolExecutor(max_workers=len(TEACHER_GOALS)) as pool: + list(pool.map(self.teacher, TEACHER_GOALS)) + select_teachers(self.root) + if teachers_only: + self.record('complete', teachers_only=True) + self.status('complete', teachers_only=True) + return + self.status('student') + self.execute('student', 'run.py', 'mopd', '--teachers', self.root/'teachers.json', + '--output', self.root/'student', '--pipeline-depth', pipeline_depth) + self.status('report') + self.execute('report', 'report.py', self.root) + self.record('complete') + self.status('complete', audit_passed=True) + + +def main(): + p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + p.add_argument('--root', type=Path, required=True, help='New campaign directory') + p.add_argument('--config', type=Path, help='Account settings (default: config/tinker.local.json, if present)') + p.add_argument('--teacher-campaign', type=Path, help='Reuse the selected teachers of this campaign') + p.add_argument('--pipeline-depth', type=int, choices=(0, 1), default=1, + help='Student lookahead (default 1: policy lag at most recipe.MAX_POLICY_LAG)') + p.add_argument('--teachers', choices=('grpo', 'sft'), default='grpo', + help='How to build the teachers: GRPO on the tasks (default), or SFT on the released traces') + p.add_argument('--teachers-only', action='store_true', help='Stop after the teachers are selected') + p.add_argument('--sft-config', type=Path, help='JSON of sft.py settings for --teachers sft (see sft_arguments)') + args = p.parse_args() + if args.sft_config and args.teachers != 'sft': + p.error('--sft-config requires --teachers sft') + sft_settings = json.loads(args.sft_config.read_text()) if args.sft_config else {} + for domain in TEACHER_GOALS: + sft_arguments(sft_settings, domain) # Fail on a malformed file before the campaign starts. + if args.teacher_campaign and (args.teachers_only or args.teachers != 'grpo'): + p.error('--teacher-campaign reuses teachers; it takes neither --teachers nor --teachers-only') + root = args.root.resolve() + prior = args.teacher_campaign.resolve() if args.teacher_campaign else None + if prior: + selection = json.loads((prior/'teacher-selection.json').read_text()) + routes = json.loads((prior/'teachers.json').read_text()) + if routes != {d: v['checkpoint']['sampler_path'] for d, v in selection['selected'].items()}: + raise RuntimeError('teachers.json does not match teacher-selection.json') + key, account = verify_account(local_config('tinker', args.config)) + os.environ['TINKER_API_KEY'] = key # This process and its stages only. + os.environ.pop('TINKER_CREDENTIAL_CMD', None) + os.environ['HF_HUB_OFFLINE'] = '1' # Reuse the cached pinned tokenizer only. + os.environ['TOKENIZERS_PARALLELISM'] = 'false' + os.environ['PYTHONDONTWRITEBYTECODE'] = '1' # Keep the snapshot unchanged. + root.mkdir(parents=True, exist_ok=False) + for package in ('hosted/tinker', 'hosted/opd', 'shared'): + shutil.copytree(REPO/package, root/'source-repo'/package, ignore=shutil.ignore_patterns('__pycache__')) + shutil.copytree(REPO/'data', root/'source-repo/data') + if prior: + for name in ('teachers.json', 'teacher-selection.json'): + shutil.copyfile(prior/name, root/name) + (root/'account.json').write_text(json.dumps(account, indent=2)) + if args.teachers == 'sft': + (root/'sft-config.json').write_text(json.dumps(sft_settings, indent=2)) + campaign = Campaign(root) + campaign.record('account_verified', email=account['email'], organization=account['organization']) + try: + campaign.run(prior, args.pipeline_depth, args.teachers, args.teachers_only) + except BaseException: + (root/'campaign-error.txt').write_text(traceback.format_exc()) + campaign.record('failed') + campaign.status('failed', error_file='campaign-error.txt') + raise + + +if __name__ == '__main__': + main() diff --git a/hosted/tinker/report.py b/hosted/tinker/report.py new file mode 100644 index 0000000..fc9a338 --- /dev/null +++ b/hosted/tinker/report.py @@ -0,0 +1,275 @@ +"""Audit the MOPD student and summarize every run of a Tinker campaign. + +Writes final-audit.json, summary.json, REPORT.md, and, when matplotlib is +installed, learning-curves.png. Only the retained run records are read. +""" +import argparse +from collections import Counter, defaultdict +import hashlib +import importlib.util +import json +import math +from pathlib import Path +import statistics +import sys + +sys.path.insert(0, str(Path(__file__).resolve().parents[2])) +from shared import recipe # noqa: E402 +from shared.puzzles import load_split, manifest # noqa: E402 + +# The fixed student protocol and the record counts it implies. +DOMAINS = recipe.DOMAINS +UPDATES, PROMPTS, EVAL_SIZE = recipe.UPDATES, recipe.PROMPTS_PER_UPDATE, recipe.EVAL_EXAMPLES # Full dev splits. +EVAL_STEPS = sorted({*range(0, UPDATES + 1, recipe.EVAL_INTERVAL), UPDATES}) +TRAIN_SEQUENCES = UPDATES * PROMPTS +DEV_SEQUENCES = len(EVAL_STEPS) * sum(EVAL_SIZE.values()) +# Published Tinker prices observed 2026-09-21, USD per million tokens. Illustrative, not billing. +PRICE = {'generated': 1.335, 'training': 1.177, 'prefill_cached': 0.108, 'prefill_uncached': 0.54} + + +def read_jsonl(path): + """Rows of a JSONL file. A live writer may leave only the final line incomplete.""" + if not path.exists(): + return [] + lines = path.read_text().splitlines() + rows = [] + for i, line in enumerate(lines): + try: + rows.append(json.loads(line)) + except json.JSONDecodeError: + if i != len(lines) - 1: + raise + return rows + + +def key(r): + return (r['step'], r['domain'], r['index'], r['sample']) + + +def digest(value): + return hashlib.sha256(json.dumps(value, sort_keys=True).encode()).hexdigest() + + +def sampled(r): + """The sampled fields that the teacher-score records must repeat unchanged.""" + return digest({k: r[k] for k in ('prompt_tokens', 'tokens', 'logprobs', 'response', 'score')}) + + +def audit(root): + """Check that the student ran the fixed protocol and that its records are consistent.""" + run = root/'student' + config = json.loads((run/'config.json').read_text()) + assert config['mode'] == 'mopd' and config['checkpoint'] is None and config['start_step'] == 0 + assert (config['steps'], config['prompts'], config['group_size'], config['max_tokens'], config['eval_every']) \ + == (UPDATES, PROMPTS, 1, recipe.MAX_RESPONSE_TOKENS, recipe.EVAL_INTERVAL) + pipelined = config.get('pipeline_depth', 0) == 1 + events = read_jsonl(run/'events.jsonl') + assert events[-1]['event'] == 'complete' and events[-1]['optimizer_updates'] == UPDATES + metrics = read_jsonl(run/'metrics.jsonl') + assert [m['step'] for m in metrics] == list(range(1, UPDATES + 1)) + for m in metrics: + assert m['samples'] == PROMPTS and 0 <= m['accepted_policy_lag'] <= int(pipelined) * recipe.MAX_POLICY_LAG + if pipelined: + p = m['pipeline'] + assert p['learner_version'] == m['step'] - 1 and p['policy_version'] == max(0, m['step'] - 2) + assert m['accepted_policy_lag'] == p['learner_version'] - p['policy_version'] + assert p['discarded_responses'] == 0 + + # Rescore every sample with the run's retained scorer on hash-checked data. + recorded = json.loads((run/'data-manifest.json').read_text()) + dataset = {} + for d in DOMAINS: + for split in ('train', 'dev'): + name = recipe.data_file(d, split) + assert manifest()[name]['sha256'] == recorded[name]['sha256'] + dataset[d, split] = load_split(d, split) + spec = importlib.util.spec_from_file_location('retained_scorer', run/'scoring.py') + scorer = importlib.util.module_from_spec(spec) + spec.loader.exec_module(scorer) + counts, correct, train, dev = Counter(), Counter(), {}, set() + for r in read_jsonl(run/'samples.jsonl'): + assert scorer.score(r['response'], dataset[r['domain'], r['split']][r['index']]['label']) == r['score'] + assert 0 < len(r['tokens']) <= recipe.MAX_RESPONSE_TOKENS and len(r['tokens']) == len(r['logprobs']) + assert all(math.isfinite(x) for x in r['logprobs']) and r['sample'] == 0 + counts[r['split'], r['step'], r['domain']] += 1 + correct[r['split'], r['step'], r['domain']] += r['score'] + if r['split'] == 'train': + assert key(r) not in train and (not pipelined or r['policy_version'] == max(0, r['step'] - 1)) + train[key(r)] = {'sampled': sampled(r), 'length': len(r['tokens'])} + else: + assert key(r) not in dev + dev.add(key(r)) + assert len(train) == TRAIN_SEQUENCES and sum(counts.values()) == TRAIN_SEQUENCES + DEV_SEQUENCES + assert all(counts['train', step, d] == PROMPTS // len(DOMAINS) for step in range(UPDATES) for d in DOMAINS) + assert all(counts['dev', step, d] == EVAL_SIZE[d] for step in EVAL_STEPS for d in DOMAINS) + assert dev == {(step, d, i, 0) for step in EVAL_STEPS for d in DOMAINS for i in range(EVAL_SIZE[d])} + evaluations = read_jsonl(run/'evaluations.jsonl') + assert [e['step'] for e in evaluations] == EVAL_STEPS + for e in evaluations: + for d in DOMAINS: + assert e[d]['correct'] == correct['dev', e['step'], d] + assert e[d]['accuracy'] == correct['dev', e['step'], d] / EVAL_SIZE[d] + + # Each training sample has one teacher score; its advantage is teacher minus sampler logprob. + advantages, gap_sums, gap_counts = {}, defaultdict(float), Counter() + for r in read_jsonl(run/'teacher-scores.jsonl'): + assert not pipelined or r['policy_version'] == max(0, r['step'] - 1) + k = key(r) + assert k in train and k not in advantages and sampled(r) == train[k]['sampled'] + advantages[k] = json.dumps(r['advantages']) + assert len(r['tokens']) == len(r['teacher_logprobs']) == len(r['advantages']) + for t, s, a in zip(r['teacher_logprobs'], r['logprobs'], r['advantages']): + assert math.isfinite(a) and math.isclose(a, t - s, rel_tol=0, abs_tol=1e-12) + gap_sums[r['step'], r['domain']] -= a + gap_counts[r['step'], r['domain']] += 1 + assert advantages.keys() == train.keys() + for m in metrics: + for d in DOMAINS: + k = (m['step'] - 1, d) + assert math.isclose(m['sampled_reverse_kl_by_domain'][d], gap_sums[k] / gap_counts[k], abs_tol=1e-10) + + # The learner trained on every sample once, with the teacher's advantages. + learner = read_jsonl(run/'learner-scores.jsonl') + assert len(learner) == TRAIN_SEQUENCES and {key(r) for r in learner} == train.keys() + for r in learner: + assert len(r['logprobs']) == len(r['advantages']) == train[key(r)]['length'] + assert all(math.isfinite(x) for x in r['logprobs']) and json.dumps(r['advantages']) == advantages[key(r)] + + # The student used the selected teachers and is not one of them. + routes = json.loads((run/'teachers.json').read_text()) + selection = json.loads((root/'teacher-selection.json').read_text()) + assert routes == {d: v['checkpoint']['sampler_path'] for d, v in selection['selected'].items()} + student_id = json.loads((run/'model-info.json').read_text())['model_id'] + assert all(student_id not in path for path in routes.values()) + return {'passed': True, 'optimizer_updates': UPDATES, 'training_sequences': TRAIN_SEQUENCES, + 'training_sequences_per_domain': TRAIN_SEQUENCES // len(DOMAINS), 'development_sequences': DEV_SEQUENCES, + 'exact_data_hashes': True, 'all_sample_scores_recomputed': True, + 'exact_teacher_minus_student_advantages': True, 'balanced_routing_each_update': True, + 'learner_and_teacher_scores_cover_all_training_sequences': True, + 'fresh_student': True, 'zero_accepted_policy_lag': all(m['accepted_policy_lag'] == 0 for m in metrics), + 'max_accepted_policy_lag': max(m['accepted_policy_lag'] for m in metrics), + 'pipeline_depth': int(pipelined), + 'source_sha256': hashlib.sha256((run/'source.py').read_bytes()).hexdigest()} + + +def summarize(path): + """Status, stage timing, evaluations, and recorded-token cost of one run directory.""" + config = json.loads((path/'config.json').read_text()) + events = read_jsonl(path/'events.jsonl') + steps = read_jsonl(path/'metrics.jsonl') + evaluations = read_jsonl(path/'evaluations.jsonl') + tokens = Counter() + for r in read_jsonl(path/'samples.jsonl'): + tokens['prompt'] += len(r['prompt_tokens']) + tokens['generated'] += len(r['tokens']) + tokens['training'] = sum(s['prompt_tokens'] + s['response_tokens'] - s['samples'] for s in steps) + tokens['teacher'] = sum(len(r['prompt_tokens']) + len(r['tokens']) for r in read_jsonl(path/'teacher-scores.jsonl')) + common = (tokens['generated']*PRICE['generated'] + tokens['training']*PRICE['training'])/1e6 + prefill = tokens['prompt'] + tokens['teacher'] + # Fractions of the update loops; the residual is weight publication and other client work. + stages = {k: sum(s[k] for s in steps) for k in ('sampling_seconds', 'teacher_seconds', 'train_seconds')} + stages['publication_and_other_seconds'] = sum(s['step_seconds'] for s in steps) - sum(stages.values()) + status = 'invalidated' if (path/'INVALIDATED.json').exists() else events[-1]['event'] if events else 'unknown' + return {'name': path.name, 'status': status, 'mode': config['mode'], 'domain': config['domain'], + 'learning_rate': config['learning_rate'], 'pipeline_depth': config.get('pipeline_depth', 0), + 'completed_updates': len(steps), 'tokens': dict(tokens), + 'elapsed_seconds': events[-1]['elapsed_seconds'] if events else None, + 'median_step_seconds': statistics.median(s['step_seconds'] for s in steps) if steps else None, + 'stage_seconds': stages, + 'evaluation_seconds': sum(e['seconds'] for e in evaluations), 'evaluations': evaluations, + 'cost_all_prefill_cached_usd': common + prefill*PRICE['prefill_cached']/1e6, + 'cost_all_prefill_uncached_usd': common + prefill*PRICE['prefill_uncached']/1e6} + + +def plot(root, runs): + """Teacher and student development accuracy per domain.""" + try: + import matplotlib + matplotlib.use('Agg') + import matplotlib.pyplot as plt + except ImportError: + print('matplotlib is not installed; skipping learning-curves.png') + return + baseline = next((r['evaluations'][0] for r in runs if r['mode'] == 'eval' and r['status'] == 'complete'), None) + fig, axes = plt.subplots(1, len(DOMAINS), figsize=(11, 4.6), sharey=True) + for ax, d in zip(axes, DOMAINS): + teacher = {0: baseline[d]['accuracy']*100} if baseline else {} + for r in runs: + if r['mode'] == 'teacher' and r['domain'] == d and r['status'] != 'invalidated': + teacher.update({e['step']: e[d]['accuracy']*100 for e in r['evaluations']}) + ax.plot(sorted(teacher), [teacher[x] for x in sorted(teacher)], '-o', color='#156f63', label='Specialist teacher') + for r in runs: + if r['mode'] == 'mopd' and r['status'] != 'invalidated' and r['evaluations']: + es = r['evaluations'] + ax.plot([e['step'] for e in es], [e[d]['accuracy']*100 for e in es], '-o', color='#a047a5', label=r['name']) + ax.set(title=d, xlabel='Optimizer updates', ylim=(0, 100)) + ax.legend(frameon=False, loc='lower right') + axes[0].set_ylabel('Development accuracy (%)') + fig.suptitle('Tinker: routed MOPD student (full dev split, greedy, thinking on)') + fig.tight_layout() + fig.savefig(root/'learning-curves.png', dpi=180, bbox_inches='tight') + plt.close(fig) + + +def report(root, runs, final_audit): + lines = ['# Tinker MOPD report', '', + '| Run | Status | Updates | LR | Median update (s) | Sampling | Teacher | Learner | Other | Evaluation (s) | Elapsed (s) |', + '|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|'] + for r in runs: + total = sum(r['stage_seconds'].values()) + fractions = [f'{v/total:.0%}' if total else '—' for v in r['stage_seconds'].values()] + median = f"{r['median_step_seconds']:.2f}" if r['median_step_seconds'] else '—' + elapsed = f"{r['elapsed_seconds']:.0f}" if r['elapsed_seconds'] is not None else '—' + lines.append(f"| {r['name']} | {r['status']} | {r['completed_updates']} | {r['learning_rate']:g} | {median} | " + + ' | '.join(fractions) + f" | {r['evaluation_seconds']:.0f} | {elapsed} |") + selection_path = root/'teacher-selection.json' + if selection_path.exists(): + lines += ['', '## Selected teachers', '', '| Domain | Run | Update | Own-domain dev accuracy | Goal | Met |', + '|---|---|---:|---:|---:|---|'] + for d, s in json.loads(selection_path.read_text())['selected'].items(): + lines.append(f"| {d} | {s['source_run']} | {s['step']} | {s['evaluation'][d]['accuracy']:.2%} " + f"| {s['provisional_target']:.0%} | {s['target_met']} |") + lines += ['', '## Development accuracy', '', '| Run | Update | ' + ' | '.join(DOMAINS) + ' |', + '|---|---:|' + '---:|' * len(DOMAINS)] + for r in runs: + for e in r['evaluations']: + lines.append(f"| {r['name']} | {e['step']} | " + + ' | '.join(f"{e[d]['correct']:g}/{e[d]['n']} ({e[d]['accuracy']:.1%})" for d in DOMAINS) + ' |') + lines += ['', '## Recorded-token cost estimate (USD)', '', '| Run | All prefill cached | All prefill uncached |', '|---|---:|---:|'] + lines += [f"| {r['name']} | {r['cost_all_prefill_cached_usd']:.3f} | {r['cost_all_prefill_uncached_usd']:.3f} |" for r in runs] + lines.append(f"| Total | {sum(r['cost_all_prefill_cached_usd'] for r in runs):.3f} " + f"| {sum(r['cost_all_prefill_uncached_usd'] for r in runs):.3f} |") + if final_audit: + lines += ['', f"Audit passed: {final_audit['optimizer_updates']} student updates, {final_audit['training_sequences']} " + f"training and {final_audit['development_sequences']} development sequences rescored; " + f"maximum accepted policy lag {final_audit['max_accepted_policy_lag']}. See final-audit.json."] + prices = ', '.join(f'{k} ${v:g}' for k, v in PRICE.items()) + lines += ['', '## Caveats', '', + '- Development-split scores from one seed, greedy, thinking on. Teacher goals are operational (just below the ' + 'released GRPO teachers\' held-out scores); the retrained teachers are not the frozen recipe teachers.', + '- Stage shares are client wall time within update loops (network and queueing included), not GPU utilization. ' + 'In pipelined runs, Sampling is the exposed wait for the prepared batch; teacher scoring overlaps it in the background.', + f'- Costs use recorded tokens at published per-million prices ({prices}); they are not invoices and exclude retries and storage.', + '- Tinker trains rank-64 LoRA with a 10x learning rate on provider-controlled hardware: a workload-matched observation, ' + 'not a controlled framework ranking.', ''] + (root/'REPORT.md').write_text('\n'.join(lines)) + + +def main(): + p = argparse.ArgumentParser(description=__doc__) + p.add_argument('root', type=Path) + root = p.parse_args().root.resolve() + final_audit = None + if (root/'student').exists(): + final_audit = audit(root) + (root/'final-audit.json').write_text(json.dumps(final_audit, indent=2)) + runs = [summarize(path.parent) for path in sorted(root.glob('*/config.json'))] + (root/'summary.json').write_text(json.dumps(runs, indent=2)) + report(root, runs, final_audit) + plot(root, runs) + print(json.dumps({'audit_passed': bool(final_audit), 'runs': [ + {k: r[k] for k in ('name', 'status', 'completed_updates')} for r in runs]}, indent=2)) + + +if __name__ == '__main__': + main() diff --git a/frameworks/tinker/requirements.txt b/hosted/tinker/requirements.txt similarity index 100% rename from frameworks/tinker/requirements.txt rename to hosted/tinker/requirements.txt diff --git a/hosted/tinker/run.py b/hosted/tinker/run.py new file mode 100644 index 0000000..f113d48 --- /dev/null +++ b/hosted/tinker/run.py @@ -0,0 +1,410 @@ +"""Remote reasoning_gym GRPO teachers and routed sampled-token OPD using the Tinker SDK, thinking on. + +Adapted from the first-party cookbook's recipes/rl_loop.py and +distillation/train_on_policy.py. No model weights are downloaded. +""" +import argparse +from concurrent.futures import ThreadPoolExecutor, as_completed +from datetime import datetime, timezone +import importlib.metadata +import json +import math +from pathlib import Path +import random +import statistics +import sys +import threading +import time +import traceback + +import tinker +from transformers import AutoTokenizer +from tinker_cookbook import renderers + +sys.path.insert(0, str(Path(__file__).resolve().parents[2])) +from shared import recipe, scoring # noqa: E402 +from shared.puzzles import REPO_DATA, load_split # noqa: E402 + +DOMAINS = recipe.DOMAINS +MODEL, REVISION = recipe.BASE_MODEL # The revision pins the tokenizer; hosted weights are provider-controlled. +RENDERER = 'qwen3_5' # The cookbook's thinking-enabled Qwen3.5/3.6 renderer. +EXPERIMENT_TAG = 'mopd-reasoning-gym-20260927' # Tinker user_metadata on every session. +SEED_STRIDE = 100_000 # Training prompt i of batch `step` samples with seed + step*SEED_STRIDE + i. + +# Teacher GRPO, with the training settings of the released GRPO teachers (recipe.TEACHERS): +# 32 prompts x 8 responses, at most 200 updates, dev evaluation every 25. The LR is set in main(). +TEACHER = {'prompts': 32, 'group_size': 8, 'steps': 200, 'eval_every': 25} +# Teacher GRPO uses clipped PPO. The student uses importance sampling on +# teacher-minus-behavior advantages, as in the cookbook distillation recipe. +LOSS = {'teacher': ('ppo', {'clip_low_threshold': 0.8, 'clip_high_threshold': 1.2}), + 'mopd': ('importance_sampling', None)} +ADAM = {'teacher': {'beta1': 0.9, 'beta2': 0.98, 'eps': 1e-8, 'weight_decay': 0.0, 'grad_clip_norm': 1.0}, + 'mopd': {'beta1': 0.9, 'beta2': 0.999, 'eps': 1e-8, 'weight_decay': 0.0, 'grad_clip_norm': 1.0}} + + +def centered_advantages(rewards): + # Match verifier GRPO: normalize within each prompt's response group. + mean = statistics.mean(rewards) + std = statistics.stdev(rewards) if len(rewards) > 1 else 0.0 + return [(v - mean) / (std + 1e-6) for v in rewards] + + +def routed_advantages(prompt_length, response_logprobs, full_teacher_logprobs): + """Teacher logprob index j predicts full_sequence[j], including prompt slots.""" + teacher = full_teacher_logprobs[prompt_length:] + assert len(teacher) == len(response_logprobs) + assert all(v is not None and math.isfinite(v) for v in teacher) + return teacher, [t - s for t, s in zip(teacher, response_logprobs)] + + +def teacher_advantages(teachers, r): + """Score one response with its own domain's frozen teacher.""" + lp = teachers[r['domain']].compute_logprobs(tinker.ModelInput.from_ints(r['prompt_tokens'] + r['tokens'])).result() + return routed_advantages(len(r['prompt_tokens']), r['logprobs'], lp) + + +def datum(prompt_tokens, response_tokens, logprobs, advantages): + """One training sequence; prompt positions get zero advantage.""" + assert response_tokens and len(response_tokens) == len(logprobs) == len(advantages) + assert all(math.isfinite(x) for x in logprobs + advantages) + prefix = len(prompt_tokens) - 1 + shape = [prefix + len(response_tokens)] + return tinker.Datum( + model_input=tinker.ModelInput.from_ints(prompt_tokens + response_tokens[:-1]), + loss_fn_inputs={ + 'target_tokens': tinker.TensorData(data=[0] * prefix + response_tokens, dtype='int64', shape=shape), + 'logprobs': tinker.TensorData(data=[0.0] * prefix + logprobs, dtype='float32', shape=shape), + 'advantages': tinker.TensorData(data=[0.0] * prefix + advantages, dtype='float32', shape=shape), + }, + ) + + +class Run: + def __init__(self, args): + self.args = args + self.out = args.output.resolve() + self.out.mkdir(parents=True, exist_ok=False) + self.started = time.monotonic() + self.write_lock = threading.Lock() + self.pool = ThreadPoolExecutor(max_workers=args.concurrency) + self.teacher_pool = ThreadPoolExecutor(max_workers=args.concurrency) + self.prepare_pool = ThreadPoolExecutor(max_workers=1) + config = {k: str(v) if isinstance(v, Path) else v for k, v in vars(args).items()} + self.event('start', config=config) + (self.out/'config.json').write_text(json.dumps(config, indent=2)) + packages = ('tinker', 'tinker-cookbook', 'transformers', 'torch', 'numpy') + (self.out/'versions.json').write_text(json.dumps({n: importlib.metadata.version(n) for n in packages}, indent=2)) + (self.out/'source.py').write_text(Path(__file__).read_text()) + (self.out/'scoring.py').write_text(Path(scoring.__file__).read_text()) # report.py rescores with it. + (self.out/'data-manifest.json').write_text((REPO_DATA/'manifest.json').read_text()) + self.tokenizer = AutoTokenizer.from_pretrained(MODEL, revision=REVISION) + self.renderer = renderers.get_renderer(RENDERER, self.tokenizer, model_name=MODEL) + self.stop = self.renderer.get_stop_sequences() + self.service = tinker.ServiceClient(user_metadata={'experiment': EXPERIMENT_TAG, 'stage': args.mode}) + caps = self.service.get_server_capabilities() + assert MODEL in [m.model_name for m in caps.supported_models] + (self.out/'capabilities.json').write_text(caps.model_dump_json(indent=2)) + self.rows = {d: {s: load_split(d, s) for s in ('train', 'dev')} for d in DOMAINS} + self.order = {} # An independent seeded shuffle of each domain's training split. + for d in DOMAINS: + self.order[d] = list(range(len(self.rows[d]['train']))) + random.Random(args.seed + DOMAINS.index(d)).shuffle(self.order[d]) + self.event('ready', stop=self.stop, tokenizer_revision=REVISION) + + def write(self, filename, row): + with self.write_lock, (self.out/filename).open('a') as f: + f.write(json.dumps(row, allow_nan=False) + '\n') + + def event(self, name, **data): + row = {'event': name, 'utc': datetime.now(timezone.utc).isoformat(), + 'elapsed_seconds': time.monotonic() - self.started, **data} + self.write('events.jsonl', row) + print(json.dumps(row), flush=True) + + def prompt(self, row): + # Use the pinned original HF template; fail if cookbook rendering differs. + ids = self.tokenizer.apply_chat_template(row['prompt'], tokenize=True, return_dict=False, + add_generation_prompt=True, enable_thinking=recipe.ENABLE_THINKING) + assert ids == self.renderer.build_generation_prompt(row['prompt']).to_ints(), \ + 'Pinned HF and cookbook prompt tokens differ' + assert len(ids) + self.args.max_tokens <= recipe.CONTEXT_LENGTH + return ids + + def batch_work(self, step): + """The (domain, row) prompts of training batch `step`; the student takes half from each domain.""" + a = self.args + domains = (a.domain,) if a.mode == 'teacher' else DOMAINS + count = a.prompts if a.mode == 'teacher' else a.prompts // 2 + return [(d, self.rows[d]['train'][self.order[d][(step*count + i) % len(self.order[d])]]) + for d in domains for i in range(count)] + + def sample_one(self, client, domain, row, group, temperature, seed): + tokens = self.prompt(row) + start = time.monotonic() + # Multi-response groups leave the seed unset, as in the cookbook. + seed = seed if group == 1 else None + result = client.sample(tinker.ModelInput.from_ints(tokens), num_samples=group, + sampling_params=tinker.SamplingParams(max_tokens=self.args.max_tokens, + temperature=temperature, top_p=1.0, top_k=-1, stop=self.stop, seed=seed)).result() + assert len(result.sequences) == group + records = [] + for j, seq in enumerate(result.sequences): + response = self.tokenizer.decode(seq.tokens, skip_special_tokens=True) + assert seq.logprobs is not None and len(seq.tokens) == len(seq.logprobs) + records.append({'domain': domain, 'index': row['_index'], 'sample': j, + 'prompt_tokens': tokens, 'tokens': seq.tokens, 'logprobs': seq.logprobs, + 'response': response, 'score': scoring.score(response, row['label']), + 'stop_reason': seq.stop_reason, 'request_seconds': time.monotonic() - start, 'seed': seed}) + return records + + def submit_samples(self, client, work, group, temperature, step): + return {self.pool.submit(self.sample_one, client, d, row, group, temperature, + self.args.seed + step*SEED_STRIDE + i): i for i, (d, row) in enumerate(work)} + + def sample_batch(self, client, work, group, temperature, step, split): + futures = self.submit_samples(client, work, group, temperature, step) + gathered = {} + for f in as_completed(futures): + gathered[futures[f]] = f.result() + for r in gathered[futures[f]]: + self.write('samples.jsonl', {'step': step, 'split': split, **r}) + return [gathered[i] for i in range(len(work))] + + def prepare_mopd(self, client, teachers, step, policy_version): + """One bounded batch; score finished responses while generation continues.""" + started = time.monotonic() + work = self.batch_work(step) + self.event('batch_prepare_start', batch_step=step, policy_version=policy_version) + sampling = self.submit_samples(client, work, 1, 1.0, step) + gathered, scoring_futures = {}, {} + + def score(record): + begin = time.monotonic() + return *teacher_advantages(teachers, record), time.monotonic() - begin + + for future in as_completed(sampling): + records = future.result() + assert len(records) == 1 + r = records[0] + r['policy_version'] = policy_version + self.write('samples.jsonl', {'step': step, 'split': 'train', **r}) + gathered[sampling[future]] = records + scoring_futures[self.teacher_pool.submit(score, r)] = r + sampling_finished = time.monotonic() + for future in as_completed(scoring_futures): + r = scoring_futures[future] + r['teacher_logprobs'], r['advantages'], r['teacher_request_seconds'] = future.result() + self.write('teacher-scores.jsonl', {'step': step, **r}) + finished = time.monotonic() + self.event('batch_prepare_complete', batch_step=step, policy_version=policy_version, + responses=len(gathered), seconds=finished - started) + return [gathered[i] for i in range(len(work))], { + 'policy_version': policy_version, + 'prepare_seconds': finished - started, + 'generation_phase_seconds': sampling_finished - started, + 'teacher_tail_seconds': finished - sampling_finished, + } + + def evaluate(self, client, step): + start = time.monotonic() + work = [(d, r) for d in DOMAINS for r in self.rows[d]['dev']] # The full dev split. + groups = self.sample_batch(client, work, 1, 0.0, step, 'dev') # Greedy. + metrics = {} + for d in DOMAINS: + rs = [r for g in groups for r in g if r['domain'] == d] + correct = sum(r['score'] for r in rs) + metrics[d] = {'n': len(rs), 'correct': correct, 'accuracy': correct / len(rs), + 'mean_tokens': statistics.mean(len(r['tokens']) for r in rs), + 'truncated': sum(r['stop_reason'] == 'length' for r in rs)} + self.write('evaluations.jsonl', {'step': step, 'seconds': time.monotonic() - start, **metrics}) + self.event('evaluation', step=step, metrics=metrics) + return metrics + + def checkpoint(self, train, step): + name = f'step-{step:03d}' + state = train.save_state(name=name).result() + sampler = train.save_weights_for_sampler(name=name).result() + row = {'step': step, 'state_path': state.path, 'sampler_path': sampler.path} + self.write('checkpoints.jsonl', row) + self.event('checkpoint', **row) + return row + + def execute(self): + a = self.args + if a.mode == 'eval': + model = {'model_path': a.checkpoint} if a.checkpoint else {'base_model': MODEL} + self.evaluate(self.service.create_sampling_client(**model), 0) + self.event('complete') + return + if a.checkpoint: + train = self.service.create_training_client_from_state_with_optimizer(a.checkpoint) + else: + train = self.service.create_lora_training_client(MODEL, rank=a.rank, seed=a.seed, + user_metadata={'domain': a.domain or 'mixed', 'run': self.out.name}) + (self.out/'model-info.json').write_text(train.get_info().model_dump_json(indent=2)) + self.event('training_client_created') + teachers = {} + if a.mode == 'mopd': + routes = json.loads(a.teachers.read_text()) + assert set(routes) == set(DOMAINS) + (self.out/'teachers.json').write_text(json.dumps(routes, indent=2)) + teachers = {d: self.service.create_sampling_client(model_path=routes[d]) for d in DOMAINS} + sample = train.save_weights_and_get_sampling_client() + if not a.skip_baseline: + self.evaluate(sample, a.start_step) + completed_updates = 0 + completion_reason = 'step_budget_reached' + # Pipelined MOPD keeps one batch in preparation, sampled by the policy + # version current when it was submitted. + pending = None + sample_version = a.start_step + if a.pipeline_depth: + pending = self.prepare_pool.submit(self.prepare_mopd, sample, teachers, a.start_step, sample_version) + for step in range(a.start_step, a.steps): + start = time.monotonic() + work = self.batch_work(step) + self.event('step_start', step=step+1, prompts=len(work)) + pipeline_info = None + if a.pipeline_depth: + groups, pipeline_info = pending.result() + lag = step - pipeline_info['policy_version'] + assert 0 <= lag <= recipe.MAX_POLICY_LAG, 'Refuse stale or future-policy batch' + # Fixed one-batch lookahead cannot exceed lag one or overshoot the budget. + pending = (self.prepare_pool.submit(self.prepare_mopd, sample, teachers, step+1, sample_version) + if step+1 < a.steps else None) + else: + groups = self.sample_batch(sample, work, a.group_size, 1.0, step, 'train') + sampling_seconds = time.monotonic() - start + flat = [r for g in groups for r in g] + teacher_start = time.monotonic() + if a.mode == 'teacher': + for g in groups: + for r, advantage in zip(g, centered_advantages([r['score'] for r in g])): + r['advantages'] = [advantage] * len(r['tokens']) + elif not a.pipeline_depth: + futures = {self.pool.submit(teacher_advantages, teachers, r): r for r in flat} + for f in as_completed(futures): + r = futures[f] + r['teacher_logprobs'], r['advantages'] = f.result() + self.write('teacher-scores.jsonl', {'step': step, **r}) + teacher_seconds = time.monotonic() - teacher_start + datums = [datum(r['prompt_tokens'], r['tokens'], r['logprobs'], r['advantages']) for r in flat] + self.event('forward_backward_start', step=step+1, samples=len(datums)) + learn_start = time.monotonic() + # Submit both before waiting so they share a clock cycle. Never + # application-retry a mutation: an ambiguous failure stops the run. + loss_fn, loss_fn_config = LOSS[a.mode] + fb_future = train.forward_backward(datums, loss_fn=loss_fn, loss_fn_config=loss_fn_config) + opt_future = train.optim_step(tinker.AdamParams(learning_rate=a.learning_rate, **ADAM[a.mode])) + fb = fb_future.result() + opt = opt_future.result() + response_deltas = [] + for r, output in zip(flat, fb.loss_fn_outputs, strict=True): + learner_lp = output['logprobs'].data[len(r['prompt_tokens'])-1:] + assert len(learner_lp) == len(r['tokens']) + differences = [float(x) - y for x, y in zip(learner_lp, r['logprobs'])] + assert all(math.isfinite(x) for x in differences) + response_deltas.extend(differences) + self.write('learner-scores.jsonl', {'step': step, 'domain': r['domain'], + 'index': r['index'], 'sample': r['sample'], 'logprobs': learner_lp, + 'advantages': r['advantages']}) + learn_seconds = time.monotonic() - learn_start + sample = train.save_weights_and_get_sampling_client() + sample_version = step + 1 + domains = (a.domain,) if a.mode == 'teacher' else DOMAINS + row = {'step': step+1, 'samples': len(flat), 'learning_rate': a.learning_rate, + 'sampling_seconds': sampling_seconds, 'teacher_seconds': teacher_seconds, + 'train_seconds': learn_seconds, 'step_seconds': time.monotonic() - start, + 'prompt_tokens': sum(len(r['prompt_tokens']) for r in flat), + 'response_tokens': sum(len(r['tokens']) for r in flat), + 'response_train_sample_logprob_delta_mean': statistics.mean(response_deltas), + 'response_train_sample_logprob_delta_abs_mean': statistics.mean(abs(x) for x in response_deltas), + 'mean_unique_responses_per_group': statistics.mean(len(set(r['response'] for r in g)) for g in groups), + 'scores': {d: statistics.mean(r['score'] for r in flat if r['domain'] == d) for d in domains}, + 'nonzero_advantage_samples': sum(any(v != 0 for v in r['advantages']) for r in flat), + 'forward_backward_metrics': fb.metrics, 'optimizer_metrics': opt.metrics} + if a.mode == 'mopd': + row['sampled_reverse_kl_by_domain'] = { + d: -statistics.mean(x for r in flat if r['domain'] == d for x in r['advantages']) + for d in domains} + row['accepted_policy_lag'] = lag if a.pipeline_depth else 0 + if pipeline_info is not None: + row['pipeline'] = {**pipeline_info, 'learner_version': step, + 'max_steps_off_policy': recipe.MAX_POLICY_LAG, 'lookahead_batches': 1, + 'discarded_responses': 0, + 'advantage_definition': 'teacher_minus_behavior_logprob', + 'timing_note': 'sampling_seconds is exposed preparation wait; preparation overlaps prior update/evaluation'} + self.write('metrics.jsonl', row) + self.event('step_complete', **row) + completed_updates += 1 + saved = (step+1) % a.save_every == 0 or step+1 == a.steps + if saved: + self.checkpoint(train, step+1) + if (step+1) % a.eval_every == 0 or step+1 == a.steps: + evaluation = self.evaluate(sample, step+1) + if a.mode == 'teacher' and a.target_score is not None and evaluation[a.domain]['accuracy'] >= a.target_score: + if not saved: + self.checkpoint(train, step+1) + completion_reason = 'teacher_target_reached' + break + self.event('complete', optimizer_updates=completed_updates, + total_optimizer_updates=a.start_step + completed_updates, reason=completion_reason) + + +def main(): + p = argparse.ArgumentParser(description=__doc__) + p.add_argument('mode', choices=('eval', 'teacher', 'mopd')) + p.add_argument('--output', type=Path, required=True) + p.add_argument('--domain', choices=DOMAINS) + p.add_argument('--teachers', type=Path) + p.add_argument('--checkpoint') + p.add_argument('--start-step', type=int, default=0) + p.add_argument('--steps', type=int, help=f'Default: {TEACHER["steps"]} for a teacher, {recipe.UPDATES} for MOPD') + p.add_argument('--prompts', type=int, help=f'Default: {TEACHER["prompts"]} for a teacher, {recipe.PROMPTS_PER_UPDATE} for MOPD') + p.add_argument('--group-size', type=int, help=f'Default: {TEACHER["group_size"]} for a teacher, 1 for MOPD') + # 10x the full-finetuning LR (1e-6 for the student and the released GRPO teachers): the + # cookbook's documented LoRA conversion. + p.add_argument('--learning-rate', type=float, default=10 * recipe.LEARNING_RATE) + p.add_argument('--target-score', type=float, help='Optional teacher-only dev threshold; stops after a scheduled evaluation') + p.add_argument('--rank', type=int, default=64) + p.add_argument('--seed', type=int, default=20260921) + p.add_argument('--max-tokens', type=int, default=recipe.MAX_RESPONSE_TOKENS) + p.add_argument('--eval-every', type=int, help=f'Default: {TEACHER["eval_every"]} for a teacher, {recipe.EVAL_INTERVAL} otherwise') + p.add_argument('--save-every', type=int, help='Default: every evaluation') + p.add_argument('--concurrency', type=int, default=32) + p.add_argument('--pipeline-depth', type=int, choices=(0, 1), default=0, + help='MOPD only: one-batch async lookahead with maximum policy lag recipe.MAX_POLICY_LAG') + p.add_argument('--skip-baseline', action='store_true') + args = p.parse_args() + defaults = TEACHER if args.mode == 'teacher' else {'prompts': recipe.PROMPTS_PER_UPDATE, 'group_size': 1, + 'steps': recipe.UPDATES, 'eval_every': recipe.EVAL_INTERVAL} + for key, value in defaults.items(): + setattr(args, key, getattr(args, key) or value) + args.save_every = args.save_every or args.eval_every + for key in ('prompts', 'group_size', 'max_tokens', 'rank', 'eval_every', 'save_every', 'concurrency'): + if getattr(args, key) <= 0: p.error(f'{key} must be positive') + if not math.isfinite(args.learning_rate) or args.learning_rate <= 0: p.error('learning rate must be finite and positive') + if args.mode != 'eval' and not 0 <= args.start_step < args.steps: p.error('require 0 <= start-step < steps') + if args.start_step and not args.checkpoint: p.error('--start-step requires an optimizer-state checkpoint') + if args.mode == 'teacher' and not args.domain: p.error('teacher requires --domain') + if args.target_score is not None and (args.mode != 'teacher' or not 0 < args.target_score <= 1): + p.error('--target-score is a teacher-only fraction in (0, 1]') + if args.mode == 'mopd' and (not args.teachers or args.prompts % 2 or args.group_size != 1): p.error('MOPD requires --teachers, even prompt count, and group size 1') + if args.pipeline_depth and args.mode != 'mopd': p.error('--pipeline-depth requires MOPD') + run = None + try: + run = Run(args) + run.execute() + except BaseException as exc: + if run: + run.event('failed', error_type=type(exc).__name__, message=str(exc)) + (run.out/'error.txt').write_text(traceback.format_exc()) + raise + finally: + if run: + for pool in (run.prepare_pool, run.pool, run.teacher_pool): + pool.shutdown(wait=True, cancel_futures=True) + + +if __name__ == '__main__': + main() diff --git a/hosted/tinker/test_run.py b/hosted/tinker/test_run.py new file mode 100644 index 0000000..c5e2bb1 --- /dev/null +++ b/hosted/tinker/test_run.py @@ -0,0 +1,196 @@ +"""Offline checks of token alignment, data, selection, and pipelining; no service calls. + +The scorer test needs reasoning-gym==0.1.25, for example CAMPAIGN_PYDEPS=. +""" +from concurrent.futures import ThreadPoolExecutor +import json +import os +from pathlib import Path +import tempfile +import threading +import time +from types import SimpleNamespace +import unittest + +os.environ['HF_HUB_OFFLINE'] = '1' # Before transformers is imported: use only the cached tokenizer. +from run import DOMAINS, MODEL, RENDERER, REVISION, Run, centered_advantages, datum, routed_advantages # noqa: E402 +from shared import recipe, scoring # run.py puts the repository on sys.path. +from shared.puzzles import load_split # noqa: E402 +from launch import choose, sft_arguments # noqa: E402 + + +class TrainingTests(unittest.TestCase): + def test_launcher_settings(self): + settings = {'learning_rate': 2e-4, 'domains': {'caesar_cipher': {'epochs': 2}}} + self.assertEqual(sft_arguments(settings), ['--learning-rate', '0.0002']) + self.assertEqual(sft_arguments(settings, 'caesar_cipher'), ['--learning-rate', '0.0002', '--epochs', '2']) + self.assertEqual(sft_arguments(settings, 'simple_geometry'), ['--learning-rate', '0.0002']) + with self.assertRaises(ValueError): + sft_arguments({'domains': {'countdown': {}}}) + + def test_teacher_selection_goal_and_fallback(self): + rows = [{'step': step, 'evaluation': {'caesar_cipher': {'accuracy': score}}} + for step, score in [(25, .3), (50, .5), (75, .6), (100, .5)]] + self.assertEqual(choose(rows, 'caesar_cipher', .5)['step'], 50) + selected = choose(rows, 'caesar_cipher', .7) + self.assertEqual(selected['step'], 75) + self.assertFalse(selected['target_met']) + + def test_next_token_alignment_and_prompt_mask(self): + d = datum([10, 11, 12], [20, 21], [-0.5, -0.7], [1.5, -0.2]) + self.assertEqual(d.model_input.to_ints(), [10, 11, 12, 20]) + self.assertEqual(d.loss_fn_inputs['target_tokens'].data, [0, 0, 20, 21]) + for actual, expected in zip(d.loss_fn_inputs['advantages'].data, [0, 0, 1.5, -0.2]): + self.assertAlmostEqual(actual, expected) + with self.assertRaises(AssertionError): + datum([10], [20], [-0.5], [float('nan')]) + + def test_group_advantages(self): + self.assertEqual(centered_advantages([0]*8), [0]*8) + self.assertEqual(centered_advantages([1]*8), [0]*8) + values = centered_advantages([1, 0, 0, 0, 0, 0, 0, 0]) + self.assertAlmostEqual(sum(values), 0) + self.assertGreater(values[0], 0) + self.assertLess(values[1], 0) + + def test_teacher_alignment_and_reverse_kl_sign(self): + teacher, adv = routed_advantages(3, [-2.0, -0.1], [None, -5, -8, -1.0, -0.5]) + self.assertEqual(teacher, [-1.0, -0.5]) + self.assertEqual(adv, [1.0, -0.4]) + with self.assertRaises(AssertionError): + routed_advantages(2, [-1], [None, -5, None]) + + def test_exact_data_and_no_dev_training_overlap(self): + for domain in DOMAINS: + train, dev = load_split(domain, 'train'), load_split(domain, 'dev') + self.assertEqual((len(train), len(dev)), (recipe.TRAIN_EXAMPLES_PER_DOMAIN, recipe.EVAL_EXAMPLES[domain])) + self.assertFalse({str(r['prompt']) for r in train}.intersection(str(r['prompt']) for r in dev)) + + def test_thinking_prompt_matches_pinned_template(self): + from transformers import AutoTokenizer + from tinker_cookbook import renderers + r = Run.__new__(Run) + r.args = SimpleNamespace(max_tokens=recipe.MAX_RESPONSE_TOKENS) + r.tokenizer = AutoTokenizer.from_pretrained(MODEL, revision=REVISION) + r.renderer = renderers.get_renderer(RENDERER, r.tokenizer, model_name=MODEL) + for domain in DOMAINS: + ids = r.prompt(load_split(domain, 'dev')[0]) # Asserts HF == renderer tokens and the length bound. + self.assertTrue(r.tokenizer.decode(ids).endswith('<|im_start|>assistant\n\n')) + + def test_scoring_needs_finished_thinking(self): + row = load_split('caesar_cipher', 'dev')[0] + answer = f"{json.loads(row['label'])['answer']}" + self.assertEqual(scoring.score(f'plan\n\n{answer}', row['label']), 1) + self.assertEqual(scoring.score(f'still thinking {answer}', row['label']), 0) # Cut off mid-thought. + self.assertEqual(scoring.score('plan\n\nWRONG', row['label']), 0) + + +def teacher_client(callback=lambda: None): + def compute(model_input): + callback() + return SimpleNamespace(result=lambda: [None, -3.0, -1.0]) + return SimpleNamespace(compute_logprobs=compute) + + +class PipelineTests(unittest.TestCase): + def make_run(self, directory, prompts=4): + r = Run.__new__(Run) + r.out = Path(directory) + r.started = time.monotonic() + r.write_lock = threading.Lock() + r.pool, r.teacher_pool = ThreadPoolExecutor(max_workers=2), ThreadPoolExecutor(max_workers=2) + r.prepare_pool = ThreadPoolExecutor(max_workers=1) + for pool in (r.pool, r.teacher_pool, r.prepare_pool): + self.addCleanup(pool.shutdown) + r.args = SimpleNamespace(prompts=prompts, seed=7, mode='mopd', checkpoint=None, + rank=64, domain=None, start_step=0, steps=3, pipeline_depth=1, + group_size=1, skip_baseline=False, learning_rate=1e-5, + save_every=2, eval_every=2, target_score=None) + r.rows = {d: {'train': [{'_index': i} for i in range(6)]} for d in DOMAINS} + r.order = {d: list(range(6)) for d in DOMAINS} + r.sample_one = lambda client, domain, row, group, temperature, seed: [{'domain': domain, + 'index': row['_index'], 'sample': 0, 'prompt_tokens': [10, 11], 'tokens': [20], 'logprobs': [-2.0], + 'response': 'x', 'score': 0, 'stop_reason': 'stop', 'request_seconds': 0.0}] + return r + + def test_teacher_starts_before_all_generation_finishes(self): + with tempfile.TemporaryDirectory() as directory: + r = self.make_run(directory, 2) + teacher_started = threading.Event() + original = r.sample_one + + def sample(client, domain, *args): + if domain == DOMAINS[1]: + self.assertTrue(teacher_started.wait(3), 'teacher scoring did not overlap generation') + return original(client, domain, *args) + r.sample_one = sample + groups, info = r.prepare_mopd(None, {d: teacher_client(teacher_started.set) for d in DOMAINS}, 0, 0) + self.assertEqual([g[0]['domain'] for g in groups], list(DOMAINS)) + self.assertTrue(all(g[0]['advantages'] == [1.0] for g in groups)) + self.assertEqual(info['policy_version'], 0) + + def test_lookahead_bound_versions_budget_and_request_order(self): + with tempfile.TemporaryDirectory() as directory: + r = self.make_run(directory) + r.args.teachers = r.out/'routes.json' + r.args.teachers.write_text(json.dumps({d: 'teacher/'+d for d in DOMAINS})) + updates, submitted = [], [] + prepared = {i: threading.Event() for i in range(3)} + prepare = r.prepare_mopd + + def wrapped(client, teachers, step, version): + prepared[step].set() + return prepare(client, teachers, step, version) + r.prepare_mopd = wrapped + + class Train: + version = 0 + + def get_info(self): + return SimpleNamespace(model_dump_json=lambda **kw: '{"model_id":"student"}') + + def save_weights_and_get_sampling_client(self): + return self.version + + def forward_backward(self, datums, **kw): + current = self.version + submitted.append(('fb', current)) + + def result(): + assert submitted[-1] == ('opt', current) + if current < 2: + assert prepared[current+1].wait(3), 'next batch did not start before learner wait' + return SimpleNamespace(loss_fn_outputs=[{'logprobs': SimpleNamespace(data=[0.0, -2.0])} + for _ in datums], metrics={}) + return SimpleNamespace(result=result) + + def optim_step(self, params): + submitted.append(('opt', self.version)) + + def result(): + self.version += 1 + updates.append(self.version) + return SimpleNamespace(metrics={}) + return SimpleNamespace(result=result) + train = Train() + r.service = SimpleNamespace(create_lora_training_client=lambda *a, **kw: train, + create_sampling_client=lambda **kw: teacher_client()) + evaluations, checkpoints = [], [] + r.evaluate = lambda client, step: evaluations.append((client, step)) + r.checkpoint = lambda client, step: checkpoints.append(step) + r.execute() + metrics = [json.loads(x) for x in (r.out/'metrics.jsonl').read_text().splitlines()] + samples = [json.loads(x) for x in (r.out/'samples.jsonl').read_text().splitlines()] + self.assertEqual(updates, [1, 2, 3]) + self.assertEqual([m['accepted_policy_lag'] for m in metrics], [0, 1, 1]) + self.assertEqual([m['pipeline']['policy_version'] for m in metrics], [0, 0, 1]) + self.assertEqual(len(samples), 12) + for step in range(3): + for domain in DOMAINS: + self.assertEqual(sum(x['step'] == step and x['domain'] == domain for x in samples), 2) + self.assertEqual(evaluations, [(0, 0), (2, 2), (3, 3)]) + self.assertEqual(checkpoints, [2, 3]) + + +if __name__ == '__main__': + unittest.main() diff --git a/shared/__init__.py b/shared/__init__.py new file mode 100644 index 0000000..3d15688 --- /dev/null +++ b/shared/__init__.py @@ -0,0 +1,10 @@ +"""Code shared by every framework and hosted provider. + +recipe the experiment settings and pinned models +puzzles packaged datasets and their checksums +scoring the puzzle verifier +slurm campaign paths, process supervision and the Slurm allocation +container the two-node Pyxis runtime used by Miles and Slime +capture telemetry sampler (GPU, host and Prometheus metrics) +credentials API keys for the hosted providers +""" diff --git a/shared/capture.py b/shared/capture.py new file mode 100644 index 0000000..ae92c2a --- /dev/null +++ b/shared/capture.py @@ -0,0 +1,135 @@ +"""Sample GPU, host and Prometheus metrics until the campaign stops. + + python -m shared.capture ROLE [HOST:PORT ...] [--engine-log LOG --engine-host HOST ...] [--vllm] + +Every five seconds, appends one row to `/-capture.jsonl.gz`. +Host counters cover the whole node. `--engine-log` follows a Miles or Slime +training log and adds each SGLang policy engine it announces on an `--engine-host` as a target. +`--vllm` adds the HTTP server of each vLLM engine that Ray starts on this node (verl's policy +and teacher servers): it probes only TCP ports that a `vLLMHttpServer` actor process listens on. +""" +import argparse +import concurrent.futures +import gzip +import json +import os +from pathlib import Path +import re +import signal +import subprocess +import threading +import time +import urllib.request + +from shared.slurm import STOP_FILE, result_dir + +INTERVAL_SECONDS = 5 +GPU_QUERY = ['nvidia-smi', '--format=csv', '--query-gpu=timestamp,index,uuid,utilization.gpu,utilization.memory,' + 'memory.used,memory.total,power.draw,temperature.gpu,clocks.sm,clocks.mem'] +PROC_FILES = ('stat', 'meminfo', 'net/dev', 'diskstats', 'loadavg') + + +def engine_ports(line, host): + """Ports of the policy engines on `host` announced in one Miles or Slime log line.""" + ports = set() + if 'sglang.launch_server' in line and '--model-path' in line: + ports.update(int(v) for v in re.findall(r'--port[ =]+(\d+)', line)) + ports.update(int(v) for v in re.findall(rf'Launch HttpServerEngineAdapter at: {re.escape(host)}:(\d+)', line)) + if 'Ports for engine' in line and f"'host': '{host}'" in line: + ports.update(int(v) for v in re.findall(r"'port': (\d+)", line)) + return ports + + +VLLM_PROCESS = b'vLLMHttpServer' # Ray titles each actor process after its class. +VLLM_PROBES = 3 + + +def listening_sockets(): + """{socket inode: 'IP:PORT'} for every IPv4 TCP socket listening on this node.""" + sockets = {} + for line in Path('/proc/net/tcp').read_text().splitlines()[1:]: + fields = line.split() + if fields[3] != '0A': # TCP_LISTEN + continue + address, port = fields[1].split(':') + ip = '.'.join(str(int(address[i:i + 2], 16)) for i in (6, 4, 2, 0)) + sockets[fields[9]] = f'{"127.0.0.1" if ip == "0.0.0.0" else ip}:{int(port, 16)}' + return sockets + + +def vllm_candidates(): + """'IP:PORT' of the listening sockets owned by vLLM server actor processes.""" + listening, found = listening_sockets(), set() + for pid in filter(str.isdigit, os.listdir('/proc')): + try: + if VLLM_PROCESS not in Path(f'/proc/{pid}/cmdline').read_bytes(): + continue + for fd in Path(f'/proc/{pid}/fd').iterdir(): + link = os.readlink(fd) + if link.startswith('socket:[') and link[8:-1] in listening: + found.add(listening[link[8:-1]]) + except OSError: # The process exited, or its file descriptors are not readable. + continue + return found + + +def read(function, *args, **kwargs): + """Call `function`, recording a failure in the row instead of stopping the capture.""" + try: + return function(*args, **kwargs) + except (OSError, subprocess.SubprocessError) as error: + return {'error': str(error)} + + +def prometheus(target): + with urllib.request.urlopen(f'http://{target}/metrics', timeout=2) as response: + return response.read().decode() + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument('role', help='names the output file, e.g. learner or generation') + parser.add_argument('targets', nargs='*', help='HOST:PORT Prometheus endpoints') + parser.add_argument('--engine-log', type=Path, help='training log to scan for policy-engine ports') + parser.add_argument('--engine-host', nargs='+', default=[], help='hosts whose engines to scrape from --engine-log') + parser.add_argument('--vllm', action='store_true', help="scrape the vLLM servers Ray starts on this node") + args = parser.parse_args() + result = result_dir() + targets, offset = set(args.targets), 0 + probed, next_probe = {}, 0.0 # --vllm: tries per candidate, and when to look for new ones + stop = threading.Event() + signal.signal(signal.SIGTERM, lambda *_: stop.set()) + signal.signal(signal.SIGINT, lambda *_: stop.set()) + with gzip.open(result / f'{args.role}-capture.jsonl.gz', 'at') as output, \ + concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool: + while not stop.is_set() and not (result / STOP_FILE).exists(): + started = time.monotonic() + if args.engine_log and args.engine_log.exists(): + with args.engine_log.open() as log: + log.seek(offset) + for line in log: + targets |= {f'{host}:{p}' for host in args.engine_host for p in engine_ports(line, host)} + offset = log.tell() + if args.vllm and started >= next_probe: + # A server's other listening sockets (e.g. its engine's RPC ports) do not serve vLLM metrics. + # A candidate is retried a few times in case a busy server misses the timeout. + candidates = read(vllm_candidates) + for candidate in candidates if isinstance(candidates, set) else (): + if candidate not in targets and probed.get(candidate, 0) < VLLM_PROBES: + probed[candidate] = probed.get(candidate, 0) + 1 + body = read(prometheus, candidate) + if isinstance(body, str) and 'vllm:' in body: + targets.add(candidate) + next_probe = started + 30 + ordered = sorted(targets) + row = {'time_unix': time.time(), 'role': args.role, 'host_scope': 'whole node', + 'gpu_csv': read(subprocess.check_output, GPU_QUERY, text=True, timeout=4), + **{name: read(Path('/proc', name).read_text) for name in PROC_FILES}, + 'prometheus': dict(zip(ordered, pool.map(lambda t: read(prometheus, t), ordered)))} + output.write(json.dumps(row) + '\n') + output.flush() + stop.wait(max(0.0, INTERVAL_SECONDS - (time.monotonic() - started))) + + +if __name__ == '__main__': + main() diff --git a/shared/container.py b/shared/container.py new file mode 100644 index 0000000..1c4f825 --- /dev/null +++ b/shared/container.py @@ -0,0 +1,210 @@ +"""Pyxis container campaigns for Miles and Slime: one trainer node and one or two generation nodes. + +`node.py` with no argument is the controller: it holds the Slurm allocation +and starts one container step per node, `node.py learner` on the trainer node +and `node.py generation` on the generation node (plus `node.py generation-2` on +the second generation node in the h100x3 layout). The roles coordinate through +ready files in the result directory: + + learner (trainer node) generation (each generation node) + ray head, head-started.json --> ray start --address=head + (waits) <-- generation-ready.json, generation-2-ready.json + placement-approved.json --> this node's frozen SGLang teachers + (waits) <-- teachers-ready.json, teachers-ready-2.json + framework training driver +""" +import importlib +import ipaddress +import os +from pathlib import Path +import shutil +import socket +import subprocess +import sys +import time +import urllib.request + +from shared import recipe +from shared.slurm import (CACHE_ENV, PYDEPS, ROOT, STOP_FILE, Allocation, Supervisor, exit_on_signals, + generation_hosts, result_dir, site, write_json) + +TEACHER_PORTS = {domain: 30000 + i for i, domain in enumerate(recipe.DOMAINS)} +# Generation roles in node order; role i runs on generation_hosts(site)[i]. +GENERATION_ROLES = ('generation', 'generation-2')[:recipe.GENERATION_NODES] +RAY_PORT = 6379 +BASE_MODEL = ROOT / 'models' / recipe.BASE_MODEL_DIR + + +def entry(framework, node_class): + """Run the controller for `node.py`, or the role for `node.py ROLE`.""" + return control(framework) if len(sys.argv) == 1 else node_class(sys.argv[1]).main() + + +def control(framework): + allocation = Allocation(framework) + allocation.start() + s = allocation.site + read_only = [s['assets'], str(Path(s['megatron_model']).parent), str(Path(s['base_model']).parent)] + mounts = ','.join([f'{ROOT}:{ROOT}', *(f'{path}:{path}:ro' for path in read_only)]) + # --mem=0: the whole node's memory. Slurm's per-CPU default can be too small for CPU-offloaded Adam. + srun = ['srun', '--exclusive', '-N1', '-n1', f'--gres=gpu:{recipe.GPUS_PER_NODE}', '--mem=0', + f'--cpus-per-task={recipe.CPUS_PER_NODE}', f'--container-image={s["container_image"]}', + '--container-writable', '--container-remap-root', '--no-container-mount-home', + f'--container-mounts={mounts}'] + nodes = {'learner': s['trainer_node'], + **{role: node for role, (node, _) in zip(GENERATION_ROLES, generation_hosts(s))}} + steps = {role: srun + ['-w', node, 'python3', str(ROOT / 'node.py'), role] for role, node in nodes.items()} + env = {**os.environ, 'CAMPAIGN_RESULT': str(allocation.result), 'CAMPAIGN_ROOT': str(ROOT), + 'NVIDIA_VISIBLE_DEVICES': 'all', 'NVIDIA_DRIVER_CAPABILITIES': 'all'} + return allocation.run(steps, env) + + +class ContainerNode: + """One role inside its container. Frameworks override the three hooks below.""" + + placement_module = None # Module that defines the framework's `_create_placement_group`. + env = {} # Extra environment for every process in the container. + + def __init__(self, role): + assert role == 'learner' or role in GENERATION_ROLES, role + self.role, self.site, self.result = role, site(), result_dir() + self.job = os.environ['SLURM_JOB_ID'] + self.generation_ips = [ip for _, ip in generation_hosts(self.site)] + # This generation node's index, which selects its teachers in recipe.TEACHER_PLACEMENT. + self.index = GENERATION_ROLES.index(role) if role != 'learner' else None + self.ip = self.site['trainer_ip'] if role == 'learner' else self.generation_ips[self.index] + self.suffix = '' if not self.index else f'-{self.index + 1}' # Ready-file suffix: '' or '-2'. + self.processes = Supervisor(self.result, prefix=f'{role}-') + + def setup(self): + """Adjust the container before anything starts.""" + + def placement(self, group): + """Return (sorted physical GPU ids, Ray placement group) for the framework's placement result.""" + raise NotImplementedError + + def train_command(self, teacher_urls): + """Return (command, extra environment) for the framework's training driver.""" + raise NotImplementedError + + def main(self): + os.chdir(ROOT / 'source') + # The placement check imports the framework in this process; use the pinned source, not the image's copy. + sys.path.insert(0, str(ROOT / 'source')) + self.setup() + os.environ.update({ + 'PYTHONPATH': f'{ROOT}:{ROOT}/source:{self.site["megatron_source"]}', 'PYTHONNOUSERSITE': '1', + 'PYTHONUNBUFFERED': '1', 'WANDB_MODE': 'disabled', 'NCCL_DEBUG': 'WARN', 'MASTER_ADDR': self.ip, + 'RAY_TMPDIR': f'/tmp/campaign-ray-{self.job}-{self.role}', 'RAY_memory_monitor_refresh_ms': '0', + 'RAY_DEDUP_LOGS': '0', 'TENSORBOARD_DIR': str(self.result / 'tensorboard'), 'CAMPAIGN_PYDEPS': str(PYDEPS), + # One cache per role: on NFS, a kernel one node rewrites becomes a stale file handle on another. + **{key: f'{path}/{self.role}' for key, path in CACHE_ENV.items()}, **self.env}) + if self.role == 'learner': + # Without this the Megatron trainer runs out of memory at 32k from fragmentation. + # Trainer node only; the SGLang engines on the generation node keep the default allocator. + os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'expandable_segments:True' + os.environ.pop('NETRC', None) + exit_on_signals() + (self.result / f'{self.role}-gpu-before.txt').write_bytes(subprocess.check_output(['nvidia-smi', '-q'])) + if self.role == 'learner': + capture = ['--engine-log', self.result / 'learner-train.out', '--engine-host', *self.generation_ips] + else: + capture = [f'127.0.0.1:{TEACHER_PORTS[d]}' for d in self.teachers()] + self.processes.launch([sys.executable, '-m', 'shared.capture', self.role, *capture], 'capture') + try: + return self.learner() if self.role == 'learner' else self.generation() + finally: + self.processes.stop_all() + for logs in Path('/tmp').glob(f'campaign-ray-{self.job}-*/ray/session_*/logs'): + if not logs.parent.is_symlink(): + shutil.copytree(logs, self.result / 'ray-logs' / self.role, dirs_exist_ok=True) + + def ray_start(self, *arguments, gpus, env=None): + self.processes.launch(['ray', 'start', f'--node-ip-address={self.ip}', f'--num-gpus={gpus}', + f'--num-cpus={recipe.RAY_CPUS_PER_NODE}', '--disable-usage-stats', '--block', + *arguments], 'ray', env) + + def teachers(self): + """Domains whose teacher this generation node serves.""" + return [d for d, (node, _) in recipe.TEACHER_PLACEMENT.items() if node == self.index] + + def learner(self): + self.ray_start('--head', gpus=recipe.TRAINER_GPUS) + write_json(self.result / 'head-started.json', {'ip': self.ip}) + ips = [self.processes.wait_for_file(f'{role}-ready.json')['ip'] for role in GENERATION_ROLES] + self.verify_placement(ips) + write_json(self.result / 'placement-approved.json', {'time': time.time()}) + for i in range(len(GENERATION_ROLES)): + self.processes.wait_for_file(f'teachers-ready{"" if not i else f"-{i + 1}"}.json') + urls = {domain: f'http://{ips[node]}:{TEACHER_PORTS[domain]}/generate' + for domain, (node, _) in recipe.TEACHER_PLACEMENT.items()} + command, env = self.train_command(urls) + self.processes.launch(command, 'train', env) + return self.processes.run_until_exit('train') + + def generation(self): + head = self.processes.wait_for_file('head-started.json')['ip'] + self.wait_until(lambda: socket.create_connection((head, RAY_PORT), timeout=2).close(), 'the Ray head') + self.ray_start(f'--address={head}:{RAY_PORT}', gpus=len(recipe.POLICY_GPUS), + env={'CUDA_VISIBLE_DEVICES': ','.join(map(str, recipe.POLICY_GPUS))}) + write_json(self.result / f'{self.role}-ready.json', {'ip': self.ip}) + self.processes.wait_for_file('placement-approved.json') + ports = [TEACHER_PORTS[domain] for domain in self.teachers()] + for domain in self.teachers(): + self.serve_teacher(domain, TEACHER_PORTS[domain]) + for port in ports: + self.wait_until(lambda: urllib.request.urlopen(f'http://127.0.0.1:{port}/health', timeout=5), + f'the teacher on port {port}', timeout=900) + write_json(self.result / f'teachers-ready{self.suffix}.json', {'ip': self.ip, 'ports': ports}) + while not (self.result / STOP_FILE).exists(): + self.processes.check() + time.sleep(3) + return 0 + + def serve_teacher(self, domain, port): + """Start one frozen teacher. It only prefills student responses to score them; it never generates.""" + gpus = recipe.TEACHER_PLACEMENT[domain][1] + self.processes.launch([ + sys.executable, '-m', 'sglang.launch_server', '--model-path', ROOT / 'teachers' / domain, + '--tokenizer-path', BASE_MODEL, '--host', '0.0.0.0', '--port', port, '--tp-size', len(gpus), + '--context-length', recipe.CONTEXT_LENGTH, '--mem-fraction-static', '0.8', + '--max-running-requests', '128', '--disable-cuda-graph', '--chunked-prefill-size', '-1', + '--disable-radix-cache', '--prefill-max-requests', '1', '--moe-runner-backend', 'triton', + '--enable-metrics'], f'teacher-{domain}', {'CUDA_VISIBLE_DEVICES': ','.join(map(str, gpus))}) + + def wait_until(self, attempt, what, timeout=180): + """Retry `attempt` until it stops raising OSError, failing fast if a child exits.""" + deadline = time.monotonic() + timeout + while True: + try: + return attempt() + except OSError: + self.processes.check() + if time.monotonic() > deadline: + raise TimeoutError(f'Timed out waiting for {what}') + time.sleep(2) + + def verify_placement(self, generation_ips): + """Check that the framework places the trainer and policy on the intended physical GPUs.""" + import ray + from ray.util import remove_placement_group + + total = recipe.TRAINER_GPUS + recipe.ROLLOUT_GPUS + ray.init(address=f'{self.ip}:{RAY_PORT}') + deadline = time.monotonic() + 180 + while int(ray.cluster_resources().get('GPU', 0)) != total: + self.processes.check() + assert time.monotonic() < deadline, ray.cluster_resources() + time.sleep(2) + nodes = {n['NodeManagerAddress']: n['Resources'].get('GPU') for n in ray.nodes() if n['Alive']} + assert nodes == {self.ip: float(recipe.TRAINER_GPUS), + **{ip: float(len(recipe.POLICY_GPUS)) for ip in generation_ips}}, nodes + # The frameworks sort bundles by node IP, so the trainer must have the lowest address. + assert all(ipaddress.ip_address(self.ip) < ipaddress.ip_address(ip) for ip in generation_ips), nodes + gpu_ids, group = self.placement(importlib.import_module(self.placement_module)._create_placement_group(total)) + try: + assert gpu_ids == list(range(recipe.TRAINER_GPUS)) + list(recipe.POLICY_GPUS) * len(generation_ips), gpu_ids + write_json(self.result / 'placement-verified.json', {'nodes': nodes, 'sorted_physical_gpu_ids': gpu_ids}) + finally: + remove_placement_group(group) + ray.shutdown() diff --git a/shared/credentials.py b/shared/credentials.py new file mode 100644 index 0000000..2c7c59e --- /dev/null +++ b/shared/credentials.py @@ -0,0 +1,41 @@ +"""API keys and local account settings for the hosted providers. + +By default the key comes from `_API_KEY`. Alternatively, +`config/.local.json` (gitignored; see `config/README.md`) can name a +`~/.zprofile` variable to read it from, so a key exported for another account +is not picked up by accident. Keys are never written or printed. +""" +import json +import os +from pathlib import Path +import re +import shlex + +CONFIG = Path(__file__).resolve().parents[1] / 'config' +PROFILE = Path.home() / '.zprofile' + + +def local_config(provider, path=None): + """Settings from `config/.local.json`, or {} when the file is absent.""" + path = Path(path) if path else CONFIG / f'{provider}.local.json' + return json.loads(path.read_text()) if path.exists() else {} + + +def api_key(env_name, profile_label=None): + """Read the key from the `profile_label=...` line in ~/.zprofile if given, otherwise from `$env_name`.""" + if profile_label: + return profile_secret(profile_label) + if not os.environ.get(env_name): + raise RuntimeError(f'Set {env_name}, or api_key_profile_label in config/, to provide the API key') + return os.environ[env_name] + + +def profile_secret(label, profile=PROFILE): + """Return the value of the single `[export] LABEL=value` line in `profile`, without executing it.""" + pattern = re.compile(r'^\s*(?:export\s+)?' + re.escape(label) + r'\s*=\s*(.*)$') + values = [shlex.split(match.group(1), comments=True) + for line in Path(profile).read_text().splitlines() + if (match := pattern.match(line))] + if len(values) != 1 or len(values[0]) != 1: + raise RuntimeError(f'Expected exactly one literal {label} assignment in {profile}') + return values[0][0] diff --git a/shared/puzzles.py b/shared/puzzles.py new file mode 100644 index 0000000..f200d86 --- /dev/null +++ b/shared/puzzles.py @@ -0,0 +1,39 @@ +"""Read the packaged puzzle datasets and check their recorded checksums. + +The checkout stores each dataset as `data/.gz` next to `data/manifest.json`. +A prepared campaign holds the verified, uncompressed files in its own `data/`. +""" +import gzip +import hashlib +import json +from pathlib import Path + +from shared import recipe + +REPO_DATA = Path(__file__).resolve().parents[1] / 'data' + + +def manifest(data_dir=REPO_DATA): + return json.loads((Path(data_dir) / 'manifest.json').read_text()) + + +def packaged_bytes(name, data_dir=REPO_DATA): + """Uncompressed bytes of one packaged dataset, after checking its size and SHA-256.""" + record = manifest(data_dir)[name] + data = gzip.decompress((Path(data_dir) / f'{name}.gz').read_bytes()) + if len(data) != record['bytes'] or hashlib.sha256(data).hexdigest() != record['sha256']: + raise ValueError(f'Dataset checksum mismatch: {name}') + return data + + +def load_split(domain, split, data_dir=REPO_DATA): + """Rows of one packaged split. Each row gains `_index`, its position in the file.""" + name = recipe.data_file(domain, split) + rows = [json.loads(line) for line in packaged_bytes(name, data_dir).splitlines()] + if len(rows) != manifest(data_dir)[name]['rows']: + raise ValueError(f'Dataset row count mismatch: {name}') + for index, row in enumerate(rows): + if row['metadata']['domain'] != domain: + raise ValueError(f'{name} row {index} belongs to {row["metadata"]["domain"]}') + row['_index'] = index + return rows diff --git a/shared/recipe.py b/shared/recipe.py new file mode 100644 index 0000000..177b22c --- /dev/null +++ b/shared/recipe.py @@ -0,0 +1,99 @@ +"""The experiment every framework runs: multi-teacher OPD (MOPD) on two reasoning_gym tasks. + +Each framework translates these values into its own configuration format; +anything framework-specific lives in that framework's directory. + +`tools/prepare.py --domains` can narrow a campaign to one task (for example +single-teacher OPD on `caesar_cipher`) by writing the campaign's `experiment.json`. +""" +import json +from pathlib import Path + +_EXPERIMENT = Path(__file__).resolve().parents[1] / 'experiment.json' +EXPERIMENT = json.loads(_EXPERIMENT.read_text()) if _EXPERIMENT.is_file() else {} + +# Tasks, each with its own frozen GRPO teacher. The rows in `data/` are reasoning_gym 0.1.25 +# problems from the default task config (train seed 20000, dev seed 10000, so the splits are +# disjoint) with reasoning_gym's default system prompt, which the teachers were trained with. +ALL_DOMAINS = ('caesar_cipher', 'simple_geometry') +DOMAINS = tuple(EXPERIMENT.get('domains', ALL_DOMAINS)) +MIXED_TRAIN_FILE = 'mixed-train.jsonl' # Both train files interleaved. +TRAIN_EXAMPLES_PER_DOMAIN = 7788 +EVAL_EXAMPLES = {'caesar_cipher': 126, 'simple_geometry': 128} # The whole dev split. + + +def data_file(domain, split): + """Name of one domain's split in `data/`, for example `caesar_cipher-dev.jsonl`.""" + return f'{domain}-{split}.jsonl' + + +# Training prompts: both tasks interleaved, or the single task's own file. +TRAIN_FILE = MIXED_TRAIN_FILE if DOMAINS == ALL_DOMAINS else data_file(DOMAINS[0], 'train') +assert set(DOMAINS) <= set(ALL_DOMAINS) and (len(DOMAINS) == 1 or DOMAINS == ALL_DOMAINS), DOMAINS + + +# Student training. Thinking is on, so responses are long. +UPDATES = 20 +PROMPTS_PER_UPDATE = 128 # Split equally between the domains. +SAMPLES_PER_PROMPT = 1 +LEARNING_RATE = 1e-6 +ENABLE_THINKING = True # The scorer only reads answers after a finished ``. +CONTEXT_LENGTH = 32_768 +MAX_RESPONSE_TOKENS = 30_720 +MAX_POLICY_LAG = 1 # Accepted optimizer updates between sampling and training. +SAVE_INTERVAL = 10 # Checkpoints after updates 10 and 20, so the benchmark also gets a mid-run point. + +# Evaluation: greedy decoding on the full dev split of each domain. +EVAL_INTERVAL = 10 + +# Trainer memory at 32k. No context parallelism: one micro-batch holds up to one full-length +# sequence per GPU, and log probabilities are computed in chunks of this many tokens. +MAX_TOKENS_PER_GPU = CONTEXT_LENGTH +LOG_PROBS_CHUNK_SIZE = 4096 + +# Pinned Hugging Face snapshots (repository, revision). Teachers are frozen. +BASE_MODEL = ('Qwen/Qwen3.6-35B-A3B', '995ad96eacd98c81ed38be0c5b274b04031597b0') +TEACHERS = { # GRPO teachers; the revisions are the tags step-125 and step-50. + 'caesar_cipher': ('semianalysisai/Qwen3.6-35B-A3B-caesar-cipher-GRPO-20260924', + 'dfedf677a5bf08a451733ec48c845c2661a6ab14'), + 'simple_geometry': ('semianalysisai/Qwen3.6-35B-A3B-simple-geometry-GRPO-20260925', + '679d4b66018c03249a3bfd0ecca3195ed9cd0e0d'), +} +# Model directory names inside a prepared campaign. +BASE_MODEL_DIR = 'Qwen3.6-35B-A3B' +MEGATRON_MODEL_DIR = 'Qwen3.6-35B-A3B_torch_dist' + +# Hardware, chosen per campaign with `tools/prepare.py --layout` (default `b200x2`). +# b200x2 Two 8 x B200 nodes. The trainer node trains on all eight GPUs. The generation node +# serves one frozen teacher per GPU on GPUs 0-1 (GPU 1 is idle in single-teacher runs) +# and the student policy on GPUs 2-7, two GPUs per engine. +# h100x3 Three 8 x H100 80 GB nodes, Miles and verl. A 35B teacher does not fit on one 80 GB GPU, +# so generation node i serves the teacher for DOMAINS[i] on GPUs 0-1 (TP2) and the policy +# on GPUs 2-7. The 12 policy GPUs double b200x2's six. The trainer node keeps Adam on the CPU +# and splits each sequence over two GPUs (Miles: context parallelism 2; verl: Ulysses +# sequence parallelism 2). verl places its own pools, so its GPU indices may differ. +# h100x2 Two 8 x H100 80 GB nodes, verl single-teacher OPD only: the trainer node as in h100x3, and one +# generation node with the teacher on GPUs 0-1 (TP2) and the policy on GPUs 2-7. +LAYOUTS = ('b200x2', 'h100x3', 'h100x2') +LAYOUT = EXPERIMENT.get('layout', 'b200x2') +assert LAYOUT in LAYOUTS, LAYOUT +H100 = LAYOUT in ('h100x3', 'h100x2') +assert LAYOUT != 'h100x2' or len(DOMAINS) == 1, 'h100x2 has room for one teacher' +GPUS_PER_NODE = 8 +TRAINER_GPUS = 8 +GENERATION_NODES = 2 if LAYOUT == 'h100x3' else 1 +TEACHER_GPU = {domain: gpu for gpu, domain in enumerate(DOMAINS)} # b200x2: one GPU per teacher +# Per domain: (generation node index, GPUs on that node). One engine per teacher. +TEACHER_GPUS_PER_ENGINE = 2 if H100 else 1 +TEACHER_PLACEMENT = ({domain: (i, (0, 1)) for i, domain in enumerate(DOMAINS)} if H100 else + {domain: (0, (gpu,)) for domain, gpu in TEACHER_GPU.items()}) +POLICY_GPUS = (2, 3, 4, 5, 6, 7) # On each generation node. +ROLLOUT_GPUS = len(POLICY_GPUS) * GENERATION_NODES +POLICY_GPUS_PER_ENGINE = 2 +TRAINER_OPTIMIZER_CPU_OFFLOAD = H100 # Adam states for 35B do not fit next to 32k activations in 80 GB. +TRAINER_TENSOR_PARALLEL = 2 if H100 else 1 # H100: TP2 with sequence parallelism, experts still EP8. +# H100: a full 32k-token sequence per GPU ran out of memory in the backward pass once responses grew +# (run 533, update 15), so two GPUs share each sequence. A micro-batch still holds MAX_TOKENS_PER_GPU tokens. +TRAINER_CONTEXT_PARALLEL = 2 if H100 else 1 +CPUS_PER_NODE = 128 if H100 else 80 +RAY_CPUS_PER_NODE = 64 diff --git a/shared/scoring.py b/shared/scoring.py new file mode 100644 index 0000000..4549b81 --- /dev/null +++ b/shared/scoring.py @@ -0,0 +1,42 @@ +"""reasoning_gym exact-match scoring for thinking-mode responses. + +The rule used by the teacher GRPO runs: take the text after the last `` +(a response cut off mid-thought scores 0), read its ``, and score 1 when +reasoning_gym's `score_answer` returns 1. In OPD this score is diagnostic only; +the training signal is the teachers' token log probabilities. + +Requires `reasoning-gym==0.1.25`. Campaigns may provide it as a directory in +`$CAMPAIGN_PYDEPS`, which is searched after the runtime's own packages. +""" +import json +import os +import sys + +if os.environ.get('CAMPAIGN_PYDEPS') and os.environ['CAMPAIGN_PYDEPS'] not in sys.path: + sys.path.append(os.environ['CAMPAIGN_PYDEPS']) + +import reasoning_gym # noqa: E402 +from reasoning_gym.utils import extract_answer # noqa: E402 + +_SCORERS = {} + + +def score_reply(reply, label): + """Score a final reply that no longer contains the thinking block.""" + row = json.loads(label) if isinstance(label, str) else label + answer = extract_answer(reply or '') + if answer is None: + return 0.0 + scorer = _SCORERS.setdefault(row['task'], reasoning_gym.get_score_answer_fn(row['task'])) + try: + return float(scorer(answer, row) >= 1.0) + except Exception: + return 0.0 + + +def score(response, label): + """Score a full response, thinking included.""" + text = (response or '').replace('<|im_end|>', '') + if '' not in text: + return 0.0 + return score_reply(text.rsplit('', 1)[1], label) diff --git a/shared/slurm.py b/shared/slurm.py new file mode 100644 index 0000000..1b1382a --- /dev/null +++ b/shared/slurm.py @@ -0,0 +1,190 @@ +"""Paths, site values, process supervision and the Slurm allocation for a prepared campaign. + +`tools/prepare.py` copies this package to `/shared/` and writes +`/site.json`. Processes coordinate through files in the result +directory: ready files announce each stage, and `STOP` tells every process to exit. +""" +import json +import os +from pathlib import Path +import signal +import subprocess +import time + +from shared import recipe + +ROOT = Path(__file__).resolve().parents[1] +DATA = ROOT / 'data' +PYDEPS = ROOT / 'pydeps' # reasoning-gym for the scorer; see shared/scoring.py. +STOP_FILE = 'STOP' +STOP_EXIT_CODE = 143 # 128 + SIGTERM, as for a scheduler cancellation. +# Compiled kernels (Triton, Inductor, FlashInfer, vLLM, SGLang), shared by both nodes and every run +# of the campaign, so only the first run pays for a cold compile. +CACHE = ROOT / 'runtime-cache' +CACHE_ENV = {'TRITON_CACHE_DIR': str(CACHE / 'triton'), 'TORCHINDUCTOR_CACHE_DIR': str(CACHE / 'inductor'), + 'FLASHINFER_WORKSPACE_BASE': str(CACHE), 'VLLM_CACHE_ROOT': str(CACHE / 'vllm'), + 'SGLANG_CACHE_DIR': str(CACHE / 'sglang')} +# Top-level campaign entries left out of each run's launch-source archive: outputs, weight links and caches. +NOT_ARCHIVED = {'results', 'checkpoints', 'models', 'teachers', 'wheels', 'uv-cache', 'runtime-cache', + 'allocation.out', 'submission.lock', 'active-run.json'} + + +def site(): + return json.loads((ROOT / 'site.json').read_text()) + + +def generation_hosts(s): + """(node, IP) of each generation node: the site's `generation_node`, then `generation_node_2` (h100x3).""" + suffixes = ['', '_2'][:recipe.GENERATION_NODES] + return [(s[f'generation_node{x}'], s[f'generation_ip{x}']) for x in suffixes] + + +def result_dir(): + return Path(os.environ['CAMPAIGN_RESULT']) + + +def write_json(path, value): + Path(path).write_text(json.dumps(value, indent=2)) + + +def exit_on_signals(code=STOP_EXIT_CODE): + """Turn SIGTERM and SIGINT into SystemExit so that `finally` cleanup runs.""" + def stop(*_): + raise SystemExit(code) + signal.signal(signal.SIGTERM, stop) + signal.signal(signal.SIGINT, stop) + + +def eval_config(metadata_overrides=None): + """Greedy dev-set evaluation in the Miles/Slime `--eval-config` format. + + Datasets inherit the training chat-template kwargs, so evaluation also runs with thinking on. + """ + datasets = [{'name': d, 'path': str(DATA / recipe.data_file(d, 'dev')), + **({'metadata_overrides': metadata_overrides} if metadata_overrides else {})} + for d in recipe.DOMAINS] + return {'eval': {'defaults': {'temperature': 0, 'n_samples_per_eval_prompt': 1, + 'max_response_len': recipe.MAX_RESPONSE_TOKENS}, + 'datasets': datasets}} + + +class Supervisor: + """Child processes, each in its own process group so that stopping one also stops its descendants. + + Each child's command is recorded in `-command.json`; its output goes to + `.out` unless `launch` is given another log. + """ + + def __init__(self, result, prefix='', base_env=None): + self.result, self.prefix, self.base_env = Path(result), prefix, base_env + self.children = {} + + def launch(self, command, name, env=None, log=None): + command = [str(part) for part in command] + write_json(self.result / f'{self.prefix}{name}-command.json', command) + base = os.environ if self.base_env is None else self.base_env + log = Path(log or self.result / f'{self.prefix}{name}.out') + self.children[name] = subprocess.Popen(command, env={**base, **(env or {})}, stdout=log.open('wb'), + stderr=subprocess.STDOUT, start_new_session=True) + return self.children[name] + + def check(self, allowed=()): + """Exit if the campaign asked to stop; fail if a required child exited.""" + if (self.result / STOP_FILE).exists(): + raise SystemExit(STOP_EXIT_CODE) + for name, process in self.children.items(): + if name not in allowed and process.poll() is not None: + raise RuntimeError(f'{name} exited with {process.returncode}; inspect its log') + + def wait_for_file(self, name, timeout=900): + """Wait for a ready file in the result directory and return its contents.""" + deadline = time.monotonic() + timeout + while not (self.result / name).exists(): + self.check() + if time.monotonic() > deadline: + raise TimeoutError(f'Readiness timed out for {name}') + time.sleep(2) + return json.loads((self.result / name).read_text()) + + def run_until_exit(self, name): + """Wait for one child while checking that the others stay alive.""" + while self.children[name].poll() is None: + self.check(allowed=(name,)) + time.sleep(3) + return self.children[name].returncode + + def stop_all(self, grace=0, timeout=8): + """Wait up to `grace` seconds for a clean exit, then SIGTERM, then SIGKILL after `timeout`.""" + processes = list(self.children.values()) + deadline = time.monotonic() + grace + while any(p.poll() is None for p in processes) and time.monotonic() < deadline: + time.sleep(1) + for process in reversed(processes): + if process.poll() is None: + _signal_group(process, signal.SIGTERM) + for process in processes: + try: + process.wait(timeout=timeout) + except subprocess.TimeoutExpired: + _signal_group(process, signal.SIGKILL) + process.wait() + + +def _signal_group(process, number): + try: + os.killpg(process.pid, number) + except ProcessLookupError: + pass + + +class Allocation: + """One campaign run inside a Slurm allocation: run records, per-role steps and shutdown. + + Two nodes, or three for the h100x3 layout or when the site file names a `teacher_node` (NeMo-RL). + In `run`, the learner's exit status becomes the run's status; every other role + only serves the learner, so an early exit there is a failure. + """ + + def __init__(self, framework): + self.site = site() + self.job = os.environ['SLURM_JOB_ID'] + self.result = ROOT / 'results' / f'{framework}-{self.job}' + + def start(self, **details): + """Create the result directory, check the nodes, and record the job and its launch source.""" + self.result.mkdir(parents=True, exist_ok=False) + nodes = subprocess.check_output(['scontrol', 'show', 'hostnames', os.environ['SLURM_JOB_NODELIST']], + text=True).split() + expected = [*(node for node, _ in generation_hosts(self.site)), self.site['trainer_node'], + *([self.site['teacher_node']] if 'teacher_node' in self.site else [])] + if sorted(nodes) != sorted(expected): # Slurm lists nodes in its own order; `srun -w` pins each role. + raise RuntimeError(f'Allocated nodes {nodes} differ from the site file {expected}') + write_json(ROOT / 'active-run.json', {'job_id': self.job, 'result': str(self.result), 'nodes': nodes}) + write_json(self.result / 'start.json', {'time': time.time(), 'job_id': self.job, 'nodes': nodes, **details}) + (self.result / 'slurm-start.txt').write_bytes(subprocess.check_output(['scontrol', 'show', 'job', self.job])) + entries = sorted(p.name for p in ROOT.iterdir() if p.name not in NOT_ARCHIVED) + subprocess.run(['tar', '--exclude=__pycache__', '--exclude=.git', '--exclude=.venv', '-czf', + str(self.result / 'launch-source.tar.gz'), '-C', str(ROOT), *entries], check=True) + + def run(self, steps, env, before=None): + """Run `steps` ({role: srun command}) until the learner exits, then stop everything.""" + exit_on_signals() + roles = Supervisor(self.result) + code = 1 + try: + if before: + before() + for role, command in steps.items(): + roles.launch(command, role, env=env, log=self.result / f'{role}-driver.out') + while roles.children['learner'].poll() is None: + for role, child in roles.children.items(): + if role != 'learner' and child.poll() is not None: + raise RuntimeError(f'{role} exited early with {child.returncode}') + time.sleep(5) + code = roles.children['learner'].returncode + finally: + (self.result / STOP_FILE).touch() + roles.stop_all(grace=40, timeout=20) + (self.result / 'exit-code.txt').write_text(f'{code}\n') + write_json(self.result / 'end.json', {'time': time.time(), 'exit_code': code}) + return code diff --git a/tests/test_package.py b/tests/test_package.py index a04e222..e1ce900 100644 --- a/tests/test_package.py +++ b/tests/test_package.py @@ -1,13 +1,14 @@ import ast -import gzip import hashlib import importlib.util import json from pathlib import Path import tempfile +import shlex import subprocess import sys import unittest +import unittest.mock ROOT = Path(__file__).resolve().parents[1] spec = importlib.util.spec_from_file_location('prepare', ROOT/'tools/prepare.py') @@ -16,61 +17,79 @@ class PackageTests(unittest.TestCase): - def test_all_templates_render_and_compile(self): - site = prepare.load_site(ROOT/'site.example.json') - values = prepare.substitutions(site, Path('/shared/test-campaign')) - for path in (ROOT/'frameworks').glob('*/campaign/**/*'): - if path.is_file(): - text = prepare.render(path.read_text(), values) - self.assertNotIn('/Users/joey', text, str(path)) - self.assertNotIn('clustermax-campaigns', text, str(path)) - if path.suffix == '.py': - ast.parse(text, filename=str(path)) - elif path.suffix == '.json': - json.loads(text) + def test_campaign_files_compile_without_machine_values(self): + for framework in prepare.FRAMEWORKS: + for source, target in prepare.campaign_files(framework): + text = source.read_text() + self.assertNotIn('/Users/', text, str(source)) + self.assertNotIn('clustermax-campaigns', text, str(source)) + if source.suffix == '.py': + ast.parse(text, filename=str(source)) + targets = {str(t) for _, t in prepare.campaign_files(framework)} + self.assertIn('node.py', targets) + self.assertIn('shared/slurm.py', targets) + self.assertFalse(any(t.startswith('upstream') for t in targets)) def test_dataset_integrity_and_counts(self): with tempfile.TemporaryDirectory() as temp: dest = Path(temp)/'data' prepare.materialize_data(dest) - manifest = json.loads((ROOT/'data/manifest.json').read_text()) + manifest = prepare.puzzles.manifest() for name, entry in manifest.items(): rows = [json.loads(line) for line in (dest/name).read_text().splitlines()] self.assertEqual(len(rows), entry['rows']) - self.assertEqual(manifest['mixed-train.jsonl']['rows'], 20000) - self.assertEqual(manifest['countdown4-dev.jsonl']['rows'], 512) - self.assertEqual(manifest['graph12-dev.jsonl']['rows'], 512) + self.assertEqual(manifest['mixed-train.jsonl']['rows'], 2 * prepare.recipe.TRAIN_EXAMPLES_PER_DOMAIN) + recipe = prepare.recipe + for domain in recipe.DOMAINS: + self.assertEqual(manifest[recipe.data_file(domain, 'dev')]['rows'], recipe.EVAL_EXAMPLES[domain]) + self.assertEqual(manifest[recipe.data_file(domain, 'train')]['rows'], recipe.TRAIN_EXAMPLES_PER_DOMAIN) + rows = [json.loads(line) for line in (dest/recipe.data_file(domain, 'dev')).read_text().splitlines()] + self.assertTrue(all(json.loads(r['label'])['task'] == r['metadata']['domain'] == domain for r in rows)) def test_patch_integrity(self): for name in prepare.FRAMEWORKS: - root = ROOT/'frameworks'/name + root = ROOT/'frameworks'/name/'upstream' manifest = json.loads((root/'manifest.json').read_text()) - self.assertEqual(hashlib.sha256((root/'source.patch').read_bytes()).hexdigest(), manifest['source_patch_sha256']) + if (root/'source.patch').exists() or manifest['source_patch_sha256'] is not None: + self.assertEqual(hashlib.sha256((root/'source.patch').read_bytes()).hexdigest(), manifest['source_patch_sha256']) self.assertEqual(len(manifest['revision']), 40) - def test_invalid_site_and_placeholders_fail(self): - with self.assertRaises(ValueError): - prepare.render('@MISSING@', {}) + def test_invalid_site_fails(self): with tempfile.TemporaryDirectory() as temp: path = Path(temp)/'bad.json' path.write_text('{}') with self.assertRaises(ValueError): prepare.load_site(path) - site = json.loads((ROOT/'site.example.json').read_text()) + site = json.loads((ROOT/'config/site.example.json').read_text()) site['base_model'] = '/tmp/model\";print(1)' path.write_text(json.dumps(site)) with self.assertRaises(ValueError): prepare.load_site(path) + def test_h100x3_site_needs_a_second_generation_node_above_the_trainer(self): + site = json.loads((ROOT/'config/site.example.json').read_text()) + with tempfile.TemporaryDirectory() as temp: + path = Path(temp)/'site.json' + path.write_text(json.dumps(site)) + with self.assertRaises(ValueError): + prepare.load_site(path, 'miles', 'h100x3') + path.write_text(json.dumps({**site, 'generation_node_2': 'generation-node-2', 'generation_ip_2': '192.0.2.5'})) + with self.assertRaises(ValueError): # Below the trainer's 192.0.2.10. + prepare.load_site(path, 'miles', 'h100x3') + path.write_text(json.dumps({**site, 'generation_node_2': 'generation-node-2', 'generation_ip_2': '192.0.2.30'})) + loaded = prepare.load_site(path, 'miles', 'h100x3') + with self.assertRaises(ValueError): + prepare.prepare('prime-rl', loaded, Path(temp)/'campaign', layout='h100x3') + def test_existing_destination_is_preserved(self): with tempfile.TemporaryDirectory() as temp: with self.assertRaises(ValueError): - prepare.prepare('slime', prepare.load_site(ROOT/'site.example.json'), Path(temp)) + prepare.prepare('slime', prepare.load_site(ROOT/'config/site.example.json'), Path(temp)) def test_submit_defaults_to_printing_only(self): with tempfile.TemporaryDirectory() as temp: root = Path(temp) - (root/'site.json').write_text((ROOT/'site.example.json').read_text()) + (root/'site.json').write_text((ROOT/'config/site.example.json').read_text()) (root/'package-provenance.json').write_text('{"framework":"slime"}') result = subprocess.run([sys.executable, str(ROOT/'tools/submit.py'), str(root), '--partition', 'test-only'], capture_output=True, text=True, check=True) @@ -79,6 +98,80 @@ def test_submit_defaults_to_printing_only(self): self.assertFalse((root/'submission.lock').exists()) self.assertFalse((root/'allocation.out').exists()) + def test_capture_finds_announced_engine_ports(self): + from shared.capture import engine_ports + ip = '192.0.2.20' + self.assertEqual(engine_ports('python -m sglang.launch_server --model-path /m --port 15000', ip), {15000}) + self.assertEqual(engine_ports(f'Launch HttpServerEngineAdapter at: {ip}:15001', ip), {15001}) + self.assertEqual(engine_ports(f"Ports for engine 0: {{'host': '{ip}', 'port': 15002}}", ip), {15002}) + self.assertEqual(engine_ports('Launch HttpServerEngineAdapter at: 10.0.0.1:15003', ip), set()) + if __name__ == '__main__': unittest.main() + + +class CredentialTests(unittest.TestCase): + def test_environment_by_default_and_profile_label_when_configured(self): + from shared import credentials + with tempfile.TemporaryDirectory() as temp: + profile = Path(temp)/'profile' + profile.write_text('export ORG_KEY="from-profile"\n') + self.assertEqual(credentials.local_config('missing', Path(temp)/'absent.json'), {}) + with unittest.mock.patch.dict('os.environ', {'DEMO_API_KEY': 'from-env'}): + self.assertEqual(credentials.api_key('DEMO_API_KEY'), 'from-env') + with unittest.mock.patch.object(credentials, 'PROFILE', profile): + self.assertEqual(credentials.profile_secret('ORG_KEY', profile), 'from-profile') + with unittest.mock.patch.dict('os.environ', {}, clear=True), self.assertRaises(RuntimeError): + credentials.api_key('DEMO_API_KEY') + + +@unittest.skipUnless(importlib.util.find_spec('reasoning_gym'), 'needs reasoning-gym==0.1.25') +class ScoringTests(unittest.TestCase): + def test_answer_after_thinking_is_required(self): + from shared import scoring + for domain in prepare.recipe.DOMAINS: + label = json.loads(prepare.puzzles.packaged_bytes(prepare.recipe.data_file(domain, 'dev')).splitlines()[0])['label'] + answer = f"{json.loads(label)['answer']}" + self.assertEqual(scoring.score('reasoning' + answer, label), 1.0) + self.assertEqual(scoring.score(answer, label), 0.0) # Cut off before the thought ended. + self.assertEqual(scoring.score('reasoningwrong', label), 0.0) + self.assertEqual(scoring.score_reply(answer, label), 1.0) + + +class ThinkingTests(unittest.TestCase): + """Every framework's generated launch configuration trains and evaluates with thinking on.""" + + def load(self, framework, module): + path = ROOT/'frameworks'/framework/f'{module}.py' + sys.path.insert(0, str(path.parent)) + self.addCleanup(sys.path.remove, str(path.parent)) + spec = importlib.util.spec_from_file_location(f'{framework}_{module}', path) + loaded = importlib.util.module_from_spec(spec) + spec.loader.exec_module(loaded) + return loaded + + def test_thinking_on_everywhere(self): + site = json.loads((ROOT/'config/site.example.json').read_text()) + thinking = json.dumps({'enable_thinking': True}) + with tempfile.TemporaryDirectory() as temp: + result = Path(temp) + prime = self.load('prime-rl', 'make_config').recipe_input(result, site)['orchestrator'] + self.assertTrue(prime['renderer']['enable_thinking']) + self.assertTrue(prime['eval']['sampling']['extra_body']['chat_template_kwargs']['enable_thinking']) + slime = self.load('slime', 'train') + with unittest.mock.patch.object(slime, 'site', return_value=site), \ + unittest.mock.patch.object(slime, 'model_args', return_value=[]): + argv = slime.build(result) + kwargs = argv[argv.index('--apply-chat-template-kwargs') + 1] + self.assertEqual(json.loads(kwargs), {'enable_thinking': True}) + miles = self.load('miles', 'node').MilesNode.__new__(self.load('miles', 'node').MilesNode) + miles.site, miles.result = site, result + with unittest.mock.patch.dict('os.environ', {'CAMPAIGN_PYDEPS': '/pydeps', 'PYTHONPATH': ''}): + command, _ = miles.train_command({d: 'http://teacher' for d in prepare.recipe.DOMAINS}) + extra = shlex.split(command[command.index('--extra-args') + 1]) + self.assertEqual(json.loads(extra[extra.index('--apply-chat-template-kwargs') + 1]), json.loads(thinking)) + verl = self.load('verl', 'train') + with unittest.mock.patch.object(verl, 'site', return_value=site): + verl_args = verl.overrides(result, result/'train.parquet', [result/'dev.parquet']) + self.assertIn('+data.apply_chat_template_kwargs.enable_thinking=True', verl_args) diff --git a/tools/benchmark.py b/tools/benchmark.py new file mode 100644 index 0000000..5d86b78 --- /dev/null +++ b/tools/benchmark.py @@ -0,0 +1,118 @@ +"""Benchmark one Hugging Face checkpoint on the packaged dev sets with vLLM, thinking on. + + python tools/benchmark.py MODEL OUTPUT_DIR [--domains ...] [--samples 8] [--seed 0] [--gpus 8] [--tp 1] + +Each dev prompt gets `--samples` responses at the training sampling settings: +temperature 1, top-p 1, up to `recipe.MAX_RESPONSE_TOKENS` tokens. Prompts are split +across `--gpus / --tp` vLLM engines of `--tp` GPUs each (`--tp 2` on 80 GB GPUs). The answer is read after the last ``, +so a response cut off mid-thought scores 0. + +Writes `responses.jsonl.gz` and `summary.json` (avg@k, pass@k, finished-thinking and +truncation rates, response lengths, output tokens per second). Needs vLLM, plus +reasoning-gym 0.1.25 installed or in `$CAMPAIGN_PYDEPS`. +""" +import argparse +import gzip +import json +import os +from pathlib import Path +import statistics +import subprocess +import sys +import time + +REPO = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(REPO)) +from shared import puzzles, recipe # noqa: E402 + + +def prompts(domains): + return [(domain, row) for domain in domains for row in puzzles.load_split(domain, 'dev')] + + +def worker(args): + from vllm import LLM, SamplingParams + + mine = prompts(args.domains)[args.worker::args.gpus // args.tp] + # Text only: some trainers export the language model without the unused vision tower. + llm = LLM(model=str(args.model), tensor_parallel_size=args.tp, max_model_len=recipe.CONTEXT_LENGTH, seed=args.seed, + language_model_only=True) + params = SamplingParams(n=args.samples, temperature=1.0, top_p=1.0, max_tokens=recipe.MAX_RESPONSE_TOKENS, + seed=args.seed) + started = time.time() + outputs = llm.chat([row['prompt'] for _, row in mine], params, + chat_template_kwargs={'enable_thinking': recipe.ENABLE_THINKING}) + seconds = time.time() - started + with gzip.open(args.output / f'worker-{args.worker}.jsonl.gz', 'wt') as stream: + for (domain, row), output in zip(mine, outputs): + for sample, completion in enumerate(output.outputs): + stream.write(json.dumps({'domain': domain, 'index': row['_index'], 'sample': sample, + 'label': row['label'], 'text': completion.text, + 'response_tokens': len(completion.token_ids), + 'finish_reason': completion.finish_reason, 'worker_seconds': seconds}) + '\n') + + +def summarize(args): + from shared.scoring import score + + responses = [] + for path in sorted(args.output.glob('worker-*.jsonl.gz')): + with gzip.open(path, 'rt') as stream: + responses += [json.loads(line) for line in stream] + for response in responses: + response['score'] = score(response['text'], response.pop('label')) + seconds = max(r['worker_seconds'] for r in responses) + summary = {'model': str(args.model), 'samples': args.samples, 'seed': args.seed, 'gpus': args.gpus, 'tp': args.tp, + 'generation_seconds': seconds, + 'output_tokens_per_second': sum(r['response_tokens'] for r in responses) / seconds, 'domains': {}} + for domain in args.domains: + mine = [r for r in responses if r['domain'] == domain] + per_prompt = {} + for r in mine: + per_prompt.setdefault(r['index'], []).append(r['score']) + assert len(per_prompt) == recipe.EVAL_EXAMPLES[domain] and len(mine) == len(per_prompt) * args.samples + lengths = [r['response_tokens'] for r in mine] + summary['domains'][domain] = { + f'avg@{args.samples}': statistics.fmean(r['score'] for r in mine), + f'pass@{args.samples}': statistics.fmean(max(s) for s in per_prompt.values()), + 'finished_thinking': statistics.fmean('' in r['text'] for r in mine), + 'truncated': statistics.fmean(r['finish_reason'] == 'length' for r in mine), + 'response_tokens_mean': statistics.fmean(lengths), 'response_tokens_median': statistics.median(lengths)} + with gzip.open(args.output / 'responses.jsonl.gz', 'wt') as stream: + for response in responses: + stream.write(json.dumps(response) + '\n') + for path in args.output.glob('worker-*.jsonl.gz'): + path.unlink() + (args.output / 'summary.json').write_text(json.dumps(summary, indent=2) + '\n') + print(json.dumps(summary, indent=2)) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument('model', type=Path, help='Hugging Face checkpoint directory') + parser.add_argument('output', type=Path, help='new directory for the results') + parser.add_argument('--domains', nargs='+', default=list(recipe.ALL_DOMAINS), choices=recipe.ALL_DOMAINS) + parser.add_argument('--samples', type=int, default=8, help='responses per prompt (default: 8)') + parser.add_argument('--seed', type=int, default=0) + parser.add_argument('--gpus', type=int, default=8, help='GPUs to use (default: 8)') + parser.add_argument('--tp', type=int, default=1, help='GPUs per vLLM engine (default: 1)') + parser.add_argument('--worker', type=int, help=argparse.SUPPRESS) + args = parser.parse_args() + if args.worker is not None: + return worker(args) + if args.gpus % args.tp: + parser.error('--gpus must be a multiple of --tp') + args.output.mkdir(parents=True) + workers = [subprocess.Popen([sys.executable, __file__, *sys.argv[1:], '--worker', str(i)], + env={**os.environ, 'CUDA_VISIBLE_DEVICES': ','.join( + str(gpu) for gpu in range(i * args.tp, (i + 1) * args.tp))}, + stdout=(args.output / f'worker-{i}.log').open('wb'), stderr=subprocess.STDOUT) + for i in range(args.gpus // args.tp)] + codes = [w.wait() for w in workers] + if any(codes): + raise SystemExit(f'Workers exited with {codes}; see {args.output}/worker-*.log') + summarize(args) + + +if __name__ == '__main__': + main() diff --git a/tools/case_study.py b/tools/case_study.py new file mode 100644 index 0000000..0602d5e --- /dev/null +++ b/tools/case_study.py @@ -0,0 +1,225 @@ +"""Case-study columns for one Miles or verl run, with the same definitions for both frameworks. + + tools/case_study.py RESULT_DIR OUT.json + +One row per update. Each update's window runs from the previous update's end (or the run start) to +its own end: for Miles, the trainer's per-update `perf` log line; for verl, the modification time of +`rollouts/.jsonl`, which verl writes at the end of update N. + +Columns (all per update): + step_s, train_s, wait_s the framework's own timers (verl logs update N at step N + 1 and + never logs its last update, so those are null for verl's update 20) + train_tokens prompt + response tokens of the trained batch + train_tok_s_gpu, mfu train_tokens / train_s / 8 trainer GPUs; MFU at 6 x ACTIVE_PARAMS FLOPs + per token against the H100 bf16 peak (attention scores not counted) + prompt_p50, response_p50 medians over the trained samples (verl: its dump, re-tokenized) + rollout_e2e_p50_s, rollout_requests, rollout_tok_s_gpu + from the policy servers' Prometheus metrics (SGLang or vLLM): + end-to-end request latency, finished requests, generated tokens per + second per policy GPU. Requests resumed after a weight sync count again. + filtered, filtered_ratio finished samples dropped as stale, and dropped / (dropped + 128) + effective_tok_s_gpu_policy train_tokens / window / policy GPUs + teacher_p50_s, teacher_tok_s_gpu, teacher_mfu + from the teacher servers' metrics: request latency, prompt (prefill) + tokens per second per teacher GPU, MFU at 2 x ACTIVE_PARAMS + cost_usd window x all GPUs of the allocation x H100_USD_PER_HOUR + train_score {task: mean verifier score of the trained batch} +""" +import datetime +import gzip +import json +from pathlib import Path +import re +import statistics +import sys + +ACTIVE_PARAMS = 3.0e9 # Qwen3.6-35B-A3B: about 3B parameters active per token (8 of 256 experts) +H100_PEAK_FLOPS = 989.4e12 # dense bf16 +H100_USD_PER_HOUR = 6.16 +GPUS_PER_NODE, TRAINER_GPUS, GPUS_PER_ENGINE, BATCH = 8, 8, 2, 128 +TASKS = ('caesar_cipher', 'simple_geometry') + + +def campaign(result): + root = result.parents[1] + experiment = json.loads((root / 'experiment.json').read_text()) if (root / 'experiment.json').exists() else {} + framework = json.loads((root / 'package-provenance.json').read_text())['framework'] + domains = experiment.get('domains', list(TASKS)) + layout = experiment.get('layout', 'b200x2') + nodes = len(json.loads((result / 'start.json').read_text())['nodes']) + policy_gpus = 12 if layout == 'h100x3' else 6 + return root, framework, domains, nodes, policy_gpus, GPUS_PER_ENGINE * len(domains) + + +# --- Prometheus captures (both frameworks) ------------------------------------------------------------ + +def metric_series(result, domains): + """[(time, {'policy'|'teacher': {metric: {le or '': value}}})], counters summed over servers.""" + rows = [] + for path in sorted(result.glob('*-capture.jsonl.gz')): + with gzip.open(path, 'rt') as stream: + try: # A capture still being written ends mid-stream. + for line in stream: + row = json.loads(line) + rows.append((row['time_unix'], row['prometheus'])) + except (EOFError, json.JSONDecodeError): + pass + rows.sort(key=lambda r: r[0]) + wanted = re.compile(r'^(vllm|sglang):(e2e_request_latency_seconds_bucket|prompt_tokens_total|generation_tokens_total)\b') + latest, out = {}, [] + for t, servers in rows: + for target, text in servers.items(): + if not isinstance(text, str): + continue + for line in text.splitlines(): + if not (m := wanted.match(line)): + continue + series, value = line.rsplit(' ', 1) + model = re.search(r'model_name="([^"]+)"', series) + role = 'teacher' if model and model[1].rstrip('/').rsplit('/', 1)[-1] in domains else 'policy' + le = b[1] if (b := re.search(r'le="([^"]+)"', series)) else '' + latest[(target, series)] = (role, m[2], le, float(value)) + total = {} + for role, metric, le, value in latest.values(): + bucket = total.setdefault(role, {}).setdefault(metric, {}) + bucket[le] = bucket.get(le, 0) + value + out.append((t, total)) + return out + + +def at(series, t): + best = {} + for time, value in series: + if time > t: + break + best = value + return best + + +def delta(h0, h1, role, metric): + a, b = h0.get(role, {}).get(metric, {}), h1.get(role, {}).get(metric, {}) + return {k: v - a.get(k, 0) for k, v in b.items()} + + +def p50(buckets): + """Median from cumulative histogram buckets, interpolated within the bucket; (median, count).""" + les = sorted((float('inf') if k == '+Inf' else float(k), c) for k, c in buckets.items()) + total = les[-1][1] if les else 0 + if total <= 0: + return None, 0 + prev_le, prev_c = 0.0, 0.0 + for le, c in les: + if c >= total / 2: + if le == float('inf'): + return prev_le, total + return prev_le + (le - prev_le) * (total / 2 - prev_c) / max(c - prev_c, 1e-9), total + prev_le, prev_c = le, c + + +# --- Per-framework update records --------------------------------------------------------------------- + +def miles_updates(result): + """{update: dict(end, step_s, train_s, wait_s, filtered, samples=[(task, prompt, response, score)])}.""" + perf = re.compile(r'\[(\d{4}-\d\d-\d\d \d\d:\d\d:\d\d\.\d+) actor_cell0_rank0\] train_metric_utils.py:\d+ - perf (\d+): (\{.*\})') + rollout = re.compile(r'rollout_executor\] metrics.py:\d+ - perf (\d+): (\{.*\})') + updates, stale = {}, {} + for line in open(result / 'learner-train.out', errors='replace'): + if m := perf.search(line): + end = datetime.datetime.strptime(m[1], '%Y-%m-%d %H:%M:%S.%f').replace(tzinfo=datetime.timezone.utc).timestamp() + p = eval(m[3]) # Miles logs a Python dict literal + updates[int(m[2]) + 1] = {'end': end, 'step_s': p.get('perf/step_time'), 'train_s': p.get('perf/train_time'), + 'wait_s': p.get('perf/train_wait_time'), 'samples': []} + elif m := rollout.search(line): + stale[int(m[1]) + 1] = eval(m[2]).get('rollout/fully_async/stale_groups_filtered', 0) or 0 + for line in open(result / 'train-samples.jsonl'): + s = json.loads(line) + if (u := s['rollout_id'] + 1) in updates: + updates[u]['samples'].append((s['domain'], s['prompt_tokens'], s['response_tokens'], s['score'])) + for u, row in updates.items(): + row['filtered'] = stale.get(u, 0) + return updates + + +def verl_updates(result, root, base_model): + from transformers import AutoTokenizer + tokenizer = AutoTokenizer.from_pretrained(base_model) + prompts = {} # user prompt -> task, from the packaged training data + for path in sorted((root / 'data').glob('*-train.jsonl*')): # .jsonl in a campaign, .jsonl.gz in the repo + task = path.name.split('-train.jsonl')[0] + if task in TASKS: + with (gzip.open(path, 'rt') if path.suffix == '.gz' else path.open()) as stream: + prompts.update((json.loads(l)['prompt'][-1]['content'].strip(), task) for l in stream) + logged = {} + for log in [result / 'learner-train.out', *sorted(result.glob('*trainer-worker.out'))]: + for line in log.read_text(errors='replace').splitlines() if log.exists() else (): + if 'timing_s/step:' in line: + step = int(line.split('step:', 1)[1].split(' ', 1)[0]) + fields = dict(f.strip().rsplit(':', 1) for f in line.split('step:', 1)[1].split(' - ')[1:] if ':' in f) + logged[step] = {k: float(v) for k, v in fields.items() if re.fullmatch(r'[-+.\deE]+', v.strip())} + updates, dropped_before = {}, 0 + for path in sorted((result / 'rollouts').glob('*.jsonl'), key=lambda p: int(p.stem)): + u, m = int(path.stem), logged.get(int(path.stem) + 1, {}) + samples = [] + for line in path.read_text().splitlines(): + row = json.loads(line) + user = row['input'].rsplit('\nuser\n', 1)[-1].rsplit('\nassistant\n', 1)[0].strip() + samples.append((prompts.get(user), len(tokenizer(row['input'])['input_ids']), + len(tokenizer(row['output'])['input_ids']), row['score'])) + dropped = m.get('fully_async/count/dropped_stale_samples', dropped_before) + updates[u] = {'end': path.stat().st_mtime, 'step_s': m.get('timing_s/step'), 'train_s': m.get('timing_s/update_actor'), + 'wait_s': m.get('timing_s/gen'), 'filtered': dropped - dropped_before, 'samples': samples, + 'train_tokens_logged': m.get('perf/total_num_tokens')} + dropped_before = dropped + return updates + + +def main(result, out): + result = Path(result) + root, framework, domains, nodes, policy_gpus, teacher_gpus = campaign(result) + site = json.loads((root / 'site.json').read_text()) + updates = miles_updates(result) if framework == 'miles' else verl_updates(result, root, site['base_model']) + series = metric_series(result, domains) + start = json.loads((result / 'start.json').read_text())['time'] + rows, previous = [], start + for u in sorted(updates): + r, t0, t1 = updates[u], previous, updates[u]['end'] + window = t1 - t0 + h0, h1 = at(series, t0), at(series, t1) + samples = r['samples'] + tokens = r.get('train_tokens_logged') or sum(p + q for _, p, q, _ in samples) or None + rollout_p50, finished = p50(delta(h0, h1, 'policy', 'e2e_request_latency_seconds_bucket')) + teacher_p50, _ = p50(delta(h0, h1, 'teacher', 'e2e_request_latency_seconds_bucket')) + generated = sum(delta(h0, h1, 'policy', 'generation_tokens_total').values()) + teacher_tokens = sum(delta(h0, h1, 'teacher', 'prompt_tokens_total').values()) + tok_s_gpu = tokens / r['train_s'] / TRAINER_GPUS if tokens and r['train_s'] else None + teacher_tok_s_gpu = teacher_tokens / window / teacher_gpus if teacher_tokens else None + scores = {} + for task, _, _, score in samples: + if task and score is not None: + scores.setdefault(task, []).append(score) + rows.append({ + 'update': u, 'window_s': window, 'step_s': r['step_s'], 'train_s': r['train_s'], 'wait_s': r['wait_s'], + 'train_tokens': tokens, 'train_tok_s_gpu': tok_s_gpu, + 'mfu': tok_s_gpu * 6 * ACTIVE_PARAMS / H100_PEAK_FLOPS if tok_s_gpu else None, + 'prompt_p50': statistics.median(p for _, p, _, _ in samples) if samples else None, + 'response_p50': statistics.median(q for _, _, q, _ in samples) if samples else None, + 'rollout_e2e_p50_s': rollout_p50, 'rollout_requests': finished, + 'rollout_tok_s_gpu': generated / window / policy_gpus if generated else None, + 'filtered': r['filtered'], 'filtered_ratio': r['filtered'] / (r['filtered'] + BATCH), + 'effective_tok_s_gpu_policy': tokens / window / policy_gpus if tokens else None, + 'teacher_p50_s': teacher_p50, 'teacher_tok_s_gpu': teacher_tok_s_gpu, + 'teacher_mfu': teacher_tok_s_gpu * 2 * ACTIVE_PARAMS / H100_PEAK_FLOPS if teacher_tok_s_gpu else None, + 'cost_usd': window * nodes * GPUS_PER_NODE * H100_USD_PER_HOUR / 3600, + 'train_score': {k: statistics.fmean(v) for k, v in sorted(scores.items())}, + 'samples': len(samples)}) + previous = t1 + meta = {'result': str(result), 'framework': framework, 'domains': domains, 'nodes': nodes, + 'policy_gpus': policy_gpus, 'teacher_gpus': teacher_gpus, 'active_params': ACTIVE_PARAMS, + 'h100_peak_flops': H100_PEAK_FLOPS, 'h100_usd_per_hour': H100_USD_PER_HOUR} + Path(out).write_text(json.dumps({'meta': meta, 'updates': rows}, indent=1) + '\n') + for row in rows: + print({k: (round(v, 3) if isinstance(v, float) else v) for k, v in row.items() if k != 'train_score'}) + + +if __name__ == '__main__': + main(*sys.argv[1:3]) diff --git a/tools/fuse_teacher.py b/tools/fuse_teacher.py new file mode 100644 index 0000000..dafcecc --- /dev/null +++ b/tools/fuse_teacher.py @@ -0,0 +1,94 @@ +"""Rewrite a downloaded teacher into the base checkpoint's fused-expert layout. + + python3 tools/fuse_teacher.py TEACHER_DIR OUTPUT_DIR BASE_MODEL_DIR + +The GRPO teachers on the Hub are Prime-RL weight broadcasts: each MoE expert is its +own module (`experts.N.{gate,up,down}_proj.weight`) and the untrained MTP head is +absent. The base Qwen3.6 checkpoint fuses each layer's experts into +`experts.gate_up_proj` [E, 2I, H] and `experts.down_proj` [E, H, I]. The output has +exactly the base's tensors: the teacher's weights with experts re-fused, plus the +base's MTP head. The teacher's other files (config, tokenizer) are copied as is. +Needs torch and safetensors. +""" +from collections import defaultdict +import json +from pathlib import Path +import re +import shutil +import sys + +import torch +from safetensors import safe_open +from safetensors.torch import save_file + +EXPERT = re.compile(r'^(.*\.mlp\.experts)\.(\d+)\.(gate_proj|up_proj|down_proj)\.weight$') +SHARD_BYTES = 5 * 1024**3 + + +def fuse(src, dst, base): + src_map = json.loads((src / 'model.safetensors.index.json').read_text())['weight_map'] + base_map = json.loads((base / 'model.safetensors.index.json').read_text())['weight_map'] + handles = {} + + def tensor(root, mapping, key): + if (root, mapping[key]) not in handles: + handles[root, mapping[key]] = safe_open(str(root / mapping[key]), 'pt') + return handles[root, mapping[key]].get_tensor(key) + + experts = defaultdict(dict) + for key in src_map: + if match := EXPERT.match(key): + experts[match.group(1)][int(match.group(2)), match.group(3)] = key + used, shards, buffer, size, total = set(), [], {}, 0, 0 + dst.mkdir(parents=True, exist_ok=False) + + def flush(): + nonlocal buffer, size + if buffer: + name = f'model-{len(shards) + 1:05d}.safetensors' + save_file(buffer, str(dst / name), metadata={'format': 'pt'}) + shards.append((name, list(buffer))) + buffer, size = {}, 0 + + for key in sorted(base_map): # Base order keeps related tensors together. + if key.startswith('mtp.'): + value = tensor(base, base_map, key) # Not trained by the teacher run; keep the base's. + elif key in src_map: + value = tensor(src, src_map, key) + used.add(key) + elif key.endswith(('.mlp.experts.gate_up_proj', '.mlp.experts.down_proj')): + prefix, projection = key.rsplit('.', 1) + group = experts[prefix] + count = 1 + max(i for i, _ in group) + parts = ('gate_proj', 'up_proj') if projection == 'gate_up_proj' else ('down_proj',) + stacked = [torch.stack([tensor(src, src_map, group[i, part]) for i in range(count)]) for part in parts] + value = torch.cat(stacked, dim=1) + used.update(group[i, part] for i in range(count) for part in parts) + else: + raise KeyError(f'Base tensor {key} has no source in the teacher') + expected = handles.get((base, base_map[key])) or safe_open(str(base / base_map[key]), 'pt') + if list(value.shape) != list(expected.get_slice(key).get_shape()): + raise ValueError(f'{key}: shape {tuple(value.shape)} differs from the base') + buffer[key] = value.contiguous() + size += value.numel() * value.element_size() + total += value.numel() * value.element_size() + if size >= SHARD_BYTES: + flush() + flush() + if unused := set(src_map) - used: + raise KeyError(f'{len(unused)} teacher tensors unused, for example {sorted(unused)[:3]}') + weight_map = {} + for i, (name, keys) in enumerate(shards, 1): + final = f'model-{i:05d}-of-{len(shards):05d}.safetensors' + (dst / name).rename(dst / final) + weight_map.update({k: final for k in keys}) + (dst / 'model.safetensors.index.json').write_text( + json.dumps({'metadata': {'total_size': total}, 'weight_map': weight_map}, indent=2)) + for path in src.iterdir(): + if path.is_file() and not path.name.endswith('.safetensors') and not (dst / path.name).exists(): + shutil.copy2(path, dst / path.name) + return {'tensors': len(weight_map), 'shards': len(shards), 'bytes': total} + + +if __name__ == '__main__': + print(json.dumps(fuse(*(Path(arg) for arg in sys.argv[1:4])))) diff --git a/tools/plot.py b/tools/plot.py new file mode 100644 index 0000000..aa305f0 --- /dev/null +++ b/tools/plot.py @@ -0,0 +1,256 @@ +"""Plot one finished (or running) Miles campaign run: timing, scores, lengths, GPUs and engines. + + python3 tools/plot.py RESULT_DIR OUTPUT_DIR + +Reads the files the run already writes (no extra instrumentation): the training log +(per-update metrics), `train-samples.jsonl` and `train-reward.jsonl` (frameworks/miles/opd_reward.py), +`teacher-latency-*.jsonl`, and every `-capture.jsonl.gz` (shared/capture.py). On the +generation nodes GPUs 0-1 serve the teachers and GPUs 2-7 the policy (both layouts in +shared/recipe.py); every learner GPU trains. Writes `dashboard.png` and one PNG per panel. +Needs matplotlib. +""" +import argparse +import json +from pathlib import Path +import statistics +import sys + +import matplotlib +matplotlib.use('Agg') +import matplotlib.pyplot as plt # noqa: E402 +from matplotlib.ticker import MaxNLocator # noqa: E402 + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +import report # noqa: E402 + +# Reference categorical palette (light mode), in fixed slot order. +BLUE, ORANGE, AQUA, YELLOW = '#2a78d6', '#eb6834', '#1baf7a', '#eda100' +SURFACE, INK, INK_2, GRID = '#fcfcfb', '#0b0b0b', '#52514e', '#e4e3df' +ROLE_COLOR = {'trainer': BLUE, 'policy': ORANGE, 'teacher': AQUA} +DOMAIN_COLOR = {'caesar_cipher': BLUE, 'simple_geometry': ORANGE} +TEACHER_GPUS = (0, 1) + + +def style(ax, title, ylabel, xlabel): + ax.set_facecolor(SURFACE) + ax.set_title(title, loc='left', fontsize=11, color=INK, fontweight='bold') + ax.set_ylabel(ylabel, color=INK_2, fontsize=9) + ax.set_xlabel(xlabel, color=INK_2, fontsize=9) + ax.tick_params(colors=INK_2, labelsize=8) + if xlabel == 'update': + ax.xaxis.set_major_locator(MaxNLocator(integer=True)) + ax.grid(True, color=GRID, linewidth=0.8) + ax.set_axisbelow(True) + for side in ('top', 'right'): + ax.spines[side].set_visible(False) + for side in ('left', 'bottom'): + ax.spines[side].set_color(GRID) + + +def legend(ax): + if ax.get_legend_handles_labels()[0]: + ax.legend(frameon=False, fontsize=8, labelcolor=INK_2) + + +def gpu_series(result, start): + """{role: [(minutes, mean util %, mean memory GiB, total power W)]} with GPUs mapped to trainer/policy/teacher.""" + series = {} + for path in sorted(result.glob('*-capture.jsonl.gz')): + node_role = path.name[:-len('-capture.jsonl.gz')] + for row in report.capture_rows(path): + if not isinstance(row['gpu_csv'], str): + continue + groups = {} + for line in row['gpu_csv'].splitlines()[1:]: + f = [x.strip() for x in line.split(',')] + if '[N/A]' in (f[3], f[5], f[7]): + continue + index = int(f[1]) + role = 'trainer' if node_role == 'learner' else 'teacher' if index in TEACHER_GPUS else 'policy' + groups.setdefault(role, []).append((float(f[3].split()[0]), float(f[5].split()[0]) / 1024, + float(f[7].split()[0]))) + for role, values in groups.items(): + series.setdefault(role, []).append(((row['time_unix'] - start) / 60, values)) + merged = {} + for role, rows in series.items(): + # Rows from several nodes: bucket by 5 s so a role's GPUs on two nodes average together. + buckets = {} + for minutes, values in rows: + buckets.setdefault(round(minutes * 12), []).extend(values) + merged[role] = [(k / 12, statistics.fmean(v[0] for v in vs), statistics.fmean(v[1] for v in vs), + sum(v[2] for v in vs)) for k, vs in sorted(buckets.items())] + return merged + + +def engine_rates(result, start): + """{'policy'|'teacher': [(minutes, tokens/s summed over engines)]}: policy generation, teacher prefill.""" + kinds = {'learner': ('policy', report.TOKEN_COUNTERS['generation']), + 'generation': ('teacher', report.TOKEN_COUNTERS['prompt'])} + rates = {} + for path in sorted(result.glob('*-capture.jsonl.gz')): + node_role = path.name[:-len('-capture.jsonl.gz')] + kind, names = kinds['learner' if node_role == 'learner' else 'generation'] + previous = {} + for row in report.capture_rows(path): + for target, text in row['prometheus'].items(): + if not isinstance(text, str): + continue + value = report.counter(text, names) or 0.0 + if target in previous: + t0, v0 = previous[target] + if row['time_unix'] > t0 and value >= v0: + bucket = round((row['time_unix'] - start) / 60 * 2) / 2 # 30 s buckets + rates.setdefault(kind, {}).setdefault(bucket, {}).setdefault(target, []).append( + (value - v0) / (row['time_unix'] - t0)) + previous[target] = (row['time_unix'], value) + return {kind: [(b, sum(statistics.fmean(v) for v in targets.values())) for b, targets in sorted(by.items())] + for kind, by in rates.items()} + + +def jsonl(paths): + return [json.loads(line) for path in paths for line in path.read_text().splitlines() if line.strip()] + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument('result', type=Path, help='run result directory (/results/)') + parser.add_argument('output', type=Path, help='directory for the PNG files') + args = parser.parse_args() + result, out = args.result, args.output + out.mkdir(parents=True, exist_ok=True) + start = json.loads((result / 'start.json').read_text())['time'] + framework = result.name.rsplit('-', 1)[0] + updates = report.framework_updates(result, framework) if (result / 'learner-train.out').exists() else {} + + def field(name): + return report.UPDATE_FIELDS[name][framework] + steps = sorted(u for u, v in updates.items() if field('step_s') in v) + samples = jsonl([result / 'train-samples.jsonl'] if (result / 'train-samples.jsonl').exists() else []) + latency = jsonl(sorted(result.glob('teacher-latency-*.jsonl'))) + gpus, engines = gpu_series(result, start), engine_rates(result, start) + panels = [] + + def panel(name, draw): + panels.append((name, draw)) + + def timing(ax): + x = [s + 1 for s in steps] + bottom = [0.0] * len(steps) + for key, label, color in (('train_s', 'Train', BLUE), ('wait_for_rollouts_s', 'Wait for rollouts', ORANGE), + ('weight_sync_s', 'Weight sync', AQUA)): + values = [updates[s].get(field(key)) or 0.0 for s in steps] + ax.bar(x, values, bottom=bottom, color=color, label=label, width=0.8, edgecolor=SURFACE, linewidth=1) + bottom = [b + v for b, v in zip(bottom, values)] + style(ax, 'Seconds per update', 'seconds', 'update') + legend(ax) + panel('timing', timing) + + def score(ax): + rows = jsonl([result / 'train-reward.jsonl']) if (result / 'train-reward.jsonl').exists() else [] + for domain, color in DOMAIN_COLOR.items(): + pts = [(r['update'], r['domains'][domain]['score']) for r in rows if domain in r['domains']] + if pts: + ax.plot(*zip(*pts), color=color, linewidth=2, marker='o', markersize=4, label=domain) + style(ax, 'Train-batch verifier score', 'mean score', 'update') + ax.set_ylim(0, 1) + legend(ax) + panel('score', score) + + def lengths(ax): + for domain, color in DOMAIN_COLOR.items(): + by = {} + for s in samples: + if s['domain'] == domain: + by.setdefault(s['rollout_id'] + 1, []).append(s['response_tokens']) + if by: + x = sorted(by) + ax.plot(x, [statistics.fmean(by[u]) for u in x], color=color, linewidth=2, label=f'{domain} mean') + ax.plot(x, [sorted(by[u])[int(0.9 * (len(by[u]) - 1))] for u in x], color=color, linewidth=1.2, + linestyle='--', label=f'{domain} p90') + if not samples and steps: # Older runs: the batch mean only. + ax.plot([s + 1 for s in steps], [updates[s].get(field('response_tokens_mean')) for s in steps], + color=BLUE, linewidth=2, label='all tasks mean') + style(ax, 'Trained response length', 'tokens', 'update') + legend(ax) + panel('response_length', lengths) + + def length_hist(ax): + for domain, color in DOMAIN_COLOR.items(): + values = [s['response_tokens'] for s in samples if s['domain'] == domain] + if values: + ax.hist(values, bins=40, range=(0, 30720), color=color, alpha=0.55, label=domain, + edgecolor=SURFACE, linewidth=0.5) + style(ax, 'Response length distribution (all updates)', 'samples', 'tokens') + legend(ax) + panel('response_length_hist', length_hist) + + def truncation(ax): + by = {} + for s in samples: + by.setdefault(s['rollout_id'] + 1, []).append(s['status'] == 'truncated') + if by: + x = sorted(by) + ax.plot(x, [100 * statistics.fmean(by[u]) for u in x], color=BLUE, linewidth=2, marker='o', markersize=4) + style(ax, 'Truncated at the response limit', '% of trained samples', 'update') + panel('truncation', truncation) + + def kl(ax): + pts = [(s + 1, updates[s][field('reverse_kl')]) for s in steps if field('reverse_kl') in updates[s]] + if pts: + ax.plot(*zip(*pts), color=BLUE, linewidth=2, marker='o', markersize=4) + style(ax, 'Reverse KL to the teacher (train batch)', 'nats / token', 'update') + panel('reverse_kl', kl) + + for metric, index, title, unit in (('gpu_util', 1, 'GPU utilization by role', 'mean %'), + ('gpu_memory', 2, 'GPU memory by role', 'mean GiB per GPU'), + ('gpu_power', 3, 'GPU power by role', 'total W')): + def draw(ax, index=index, title=title, unit=unit): + for role, color in ROLE_COLOR.items(): + if role in gpus: + ax.plot([r[0] for r in gpus[role]], [r[index] for r in gpus[role]], color=color, + linewidth=1.2, label=role) + style(ax, title, unit, 'minutes since start') + legend(ax) + panel(metric, draw) + + def policy_throughput(ax): + if 'policy' in engines: + ax.plot(*zip(*engines['policy']), color=ORANGE, linewidth=1.2) + style(ax, 'Policy generation throughput (all engines)', 'tokens / s', 'minutes since start') + panel('policy_throughput', policy_throughput) + + def teacher_throughput(ax): + if 'teacher' in engines: + ax.plot(*zip(*engines['teacher']), color=AQUA, linewidth=1.2) + style(ax, 'Teacher prefill throughput (all teachers)', 'tokens / s', 'minutes since start') + panel('teacher_throughput', teacher_throughput) + + def teacher_latency(ax): + for domain, color in DOMAIN_COLOR.items(): + pts = [(r['total_tokens'], r['seconds']) for r in latency if r.get('teacher') == domain and r['success']] + if pts: + ax.scatter(*zip(*pts), s=10, color=color, alpha=0.6, label=domain, edgecolors='none') + style(ax, 'Teacher scoring latency per sample', 'seconds', 'sequence tokens') + legend(ax) + panel('teacher_latency', teacher_latency) + + columns = 3 + rows = -(-len(panels) // columns) + figure, axes = plt.subplots(rows, columns, figsize=(6 * columns, 3.6 * rows), facecolor=SURFACE) + for ax, (name, draw) in zip(axes.flat, panels): + draw(ax) + single, single_ax = plt.subplots(figsize=(7, 4), facecolor=SURFACE) + draw(single_ax) + single.tight_layout() + single.savefig(out / f'{name}.png', dpi=130) + plt.close(single) + for ax in list(axes.flat)[len(panels):]: + ax.set_visible(False) + figure.suptitle(result.name, x=0.01, ha='left', fontsize=13, color=INK, fontweight='bold') + figure.tight_layout() + figure.savefig(out / 'dashboard.png', dpi=110) + print(out / 'dashboard.png') + + +if __name__ == '__main__': + main() diff --git a/tools/prepare.py b/tools/prepare.py index 5dac63e..cce7a05 100644 --- a/tools/prepare.py +++ b/tools/prepare.py @@ -1,6 +1,11 @@ -"""Prepare one isolated campaign from pinned source and recorded launch templates.""" +"""Prepare a self-contained campaign directory for one framework. Nothing is submitted. + +The directory gets the upstream source at its pinned revision with the patch applied, +the verified datasets, a copy of `shared/`, the framework's campaign files, `launch.py` +(a copy of tools/submit.py), links to the models, and `site.json`, which campaign +processes read at runtime. +""" import argparse -import gzip import hashlib import ipaddress import json @@ -8,112 +13,128 @@ import re import shutil import subprocess +import sys REPO = Path(__file__).resolve().parents[1] -FRAMEWORKS = ('miles', 'prime-rl', 'slime') -REQUIRED = ('assets', 'base_model', 'megatron_model', 'teachers', 'container_image', - 'megatron_source', 'uv', 'generation_node', 'trainer_node', 'generation_ip', 'trainer_ip') +sys.path.insert(0, str(REPO)) +from shared import puzzles, recipe # noqa: E402 + +FRAMEWORKS = ('miles', 'nemo-rl', 'prime-rl', 'slime', 'verl') +PACKAGE = REPO / 'shared' +PATH_FIELDS = ('assets', 'base_model', 'megatron_model', 'teachers', 'container_image', 'megatron_source', 'uv') +HOST_FIELDS = ('generation_node', 'trainer_node', 'generation_ip', 'trainer_ip') +REQUIRED = PATH_FIELDS + HOST_FIELDS +# NeMo-RL's MOPD reserves a whole node for the teacher and runs in its own NGC container. +NEMO_RL_FIELDS = ('teacher_node', 'teacher_ip', 'nemo_rl_container') +# The h100x3 layout's second generation node. +H100X3_FIELDS = ('generation_node_2', 'generation_ip_2') +LAYOUT_FRAMEWORKS = {'b200x2': FRAMEWORKS, 'h100x3': ('miles', 'verl'), 'h100x2': ('verl',)} +SAFE_VALUE = r'[A-Za-z0-9_./:+-]+' -def load_site(path): +def load_site(path, framework='miles', layout='b200x2'): site = json.loads(Path(path).read_text()) - missing = set(REQUIRED) - set(site) + required = REQUIRED + (NEMO_RL_FIELDS if framework == 'nemo-rl' else ()) + (H100X3_FIELDS if layout == 'h100x3' else ()) + missing = set(required) - set(site) if missing: raise ValueError(f'Missing site fields: {sorted(missing)}') - for key in REQUIRED: + for key in required: value = site[key] - if not isinstance(value, str) or not re.fullmatch(r'[A-Za-z0-9_./:+-]+', value): + if not isinstance(value, str) or not re.fullmatch(SAFE_VALUE, value): raise ValueError(f'{key} must contain only safe path, hostname or IP characters') - for key in REQUIRED[:7]: + for key in PATH_FIELDS + (('nemo_rl_container',) if framework == 'nemo-rl' else ()): if not Path(site[key]).is_absolute(): raise ValueError(f'{key} must be an absolute path') - if site['generation_node'] == site['trainer_node']: - raise ValueError('The two roles need different nodes') - if site['generation_node'] > site['trainer_node']: - raise ValueError('The recorded controller requires the generation node first in Slurm hostname order') - gen, trainer = (ipaddress.IPv4Address(site[k]) for k in ('generation_ip', 'trainer_ip')) - if trainer >= gen: - raise ValueError('The recorded Ray placement requires trainer IP < generation IP') + nodes = [site['generation_node'], site['trainer_node'], *([site['teacher_node']] if framework == 'nemo-rl' else []), + *([site['generation_node_2']] if layout == 'h100x3' else [])] + if len(set(nodes)) != len(nodes): + raise ValueError('Every role needs its own node') + trainer = ipaddress.IPv4Address(site['trainer_ip']) + generation = [ipaddress.IPv4Address(site[k]) for k in ('generation_ip', 'generation_ip_2')[:2 if layout == 'h100x3' else 1]] + # Miles and Slime sort Ray bundles by node IP; Prime-RL uses explicit addresses and verl places its own pools. + if framework in ('miles', 'slime') and any(trainer >= ip for ip in generation): + raise ValueError('The recorded Ray placement requires the trainer IP below every generation IP') for key in ('base_model', 'megatron_model', 'teachers'): if not Path(site[key]).is_relative_to(Path(site['assets'])): raise ValueError(f'{key} must be within assets so the container can access it') return site -def substitutions(site, output): - values = {k.upper(): v for k, v in site.items()} - values.update(CAMPAIGN=str(output), BASE_PARENT=str(Path(site['base_model']).parent), - MEGATRON_PARENT=str(Path(site['megatron_model']).parent), - GENERATION_IP_REGEX=site['generation_ip'].replace('.', r'\.')) - return values - - -def render(text, values): - for name, value in values.items(): - text = text.replace(f'@{name}@', value) - remaining = re.findall(r'@[A-Z_]+@', text) - if remaining: - raise ValueError(f'Unresolved template values: {remaining}') - return text - - def materialize_data(destination): destination.mkdir() - manifest = json.loads((REPO/'data/manifest.json').read_text()) - for filename, record in manifest.items(): - data = gzip.decompress((REPO/'data'/f'{filename}.gz').read_bytes()) - if len(data) != record['bytes'] or hashlib.sha256(data).hexdigest() != record['sha256']: - raise ValueError(f'Dataset checksum mismatch: {filename}') - (destination/filename).write_bytes(data) + for filename in puzzles.manifest(): + (destination / filename).write_bytes(puzzles.packaged_bytes(filename)) + + +def campaign_files(framework): + """(source, target relative to the campaign) for `shared/` and the framework's campaign files.""" + files = [(p, Path('shared') / p.name) for p in sorted(PACKAGE.glob('*.py'))] + framework_dir = REPO / 'frameworks' / framework + return files + [(p, p.relative_to(framework_dir)) for p in sorted(framework_dir.rglob('*')) + if p.is_file() and not {'upstream', '__pycache__'} & set(p.relative_to(framework_dir).parts)] -def prepare(framework, site, output, source_cache=None): +def prepare(framework, site, output, source_cache=None, domains=None, layout='b200x2'): output = output.resolve() - if not re.fullmatch(r'[A-Za-z0-9_./+-]+', str(output)): + if framework not in LAYOUT_FRAMEWORKS[layout]: + raise ValueError(f'The {layout} layout supports only {LAYOUT_FRAMEWORKS[layout]}') + if layout == 'h100x2' and len(domains or recipe.ALL_DOMAINS) != 1: + raise ValueError('The h100x2 layout has room for one teacher; pass one task with --domains') + if not re.fullmatch(SAFE_VALUE, str(output)): raise ValueError('The campaign path cannot contain whitespace or shell metacharacters') if output.exists(): raise ValueError('The campaign directory already exists. Select a new empty path.') - package = REPO/'frameworks'/framework - manifest = json.loads((package/'manifest.json').read_text()) - patch = package/'source.patch' - if hashlib.sha256(patch.read_bytes()).hexdigest() != manifest['source_patch_sha256']: + upstream = REPO / 'frameworks' / framework / 'upstream' + manifest = json.loads((upstream / 'manifest.json').read_text()) + patch = upstream / 'source.patch' # Absent when the framework runs unmodified. + if patch.exists() and hashlib.sha256(patch.read_bytes()).hexdigest() != manifest['source_patch_sha256']: raise ValueError('Source patch checksum mismatch') output.mkdir(parents=True) - source = output/'source' + source = output / 'source' clone_from = str(source_cache) if source_cache else manifest['repository'] subprocess.run(['git', 'clone', '--no-checkout', clone_from, str(source)], check=True) subprocess.run(['git', '-C', str(source), 'checkout', '--detach', manifest['revision']], check=True) if framework == 'prime-rl': subprocess.run(['git', '-c', 'url.https://github.com/.insteadOf=git@github.com:', '-C', str(source), 'submodule', 'update', '--init', '--recursive'], check=True) - subprocess.run(['git', '-C', str(source), 'apply', '--check', str(patch)], check=True) - subprocess.run(['git', '-C', str(source), 'apply', str(patch)], check=True) - values = substitutions(site, output) - for entry in sorted((package/'campaign').rglob('*')): - if not entry.is_file(): - continue - target = output/entry.relative_to(package/'campaign') + if patch.exists(): + subprocess.run(['git', '-C', str(source), 'apply', '--check', str(patch)], check=True) + subprocess.run(['git', '-C', str(source), 'apply', str(patch)], check=True) + for source_file, target in campaign_files(framework): + target = output / target target.parent.mkdir(parents=True, exist_ok=True) - target.write_text(render(entry.read_text(), values)) - materialize_data(output/'data') - shutil.copyfile(REPO/'tools/submit.py', output/'launch.py') - (output/'models').mkdir() - (output/'models/Qwen3.6-35B-A3B').symlink_to(site['base_model']) - (output/'models/Qwen3.6-35B-A3B_torch_dist').symlink_to(site['megatron_model']) - (output/'teachers').symlink_to(site['teachers']) - (output/'site.json').write_text(json.dumps(site, indent=2)+'\n') - (output/'package-provenance.json').write_text(json.dumps({'framework':framework, **manifest}, indent=2)+'\n') + # copyfile follows symlinks, so Prime-RL's link to the scorer becomes a regular file. + shutil.copyfile(source_file, target) + materialize_data(output / 'data') + shutil.copyfile(REPO / 'tools/submit.py', output / 'launch.py') + (output / 'models').mkdir() + (output / 'models' / recipe.BASE_MODEL_DIR).symlink_to(site['base_model']) + (output / 'models' / recipe.MEGATRON_MODEL_DIR).symlink_to(site['megatron_model']) + (output / 'teachers').symlink_to(site['teachers']) + (output / 'site.json').write_text(json.dumps(site, indent=2) + '\n') + experiment = {**({'domains': list(domains)} if domains else {}), **({'layout': layout} if layout != 'b200x2' else {})} + if experiment: # Read by the campaign's `shared/recipe.py`. + (output / 'experiment.json').write_text(json.dumps(experiment, indent=2) + '\n') + (output / 'package-provenance.json').write_text(json.dumps({'framework': framework, **manifest}, indent=2) + '\n') print(f'Prepared {output}. No GPU job was submitted. Complete docs/SETUP.md before submission.') def main(): parser = argparse.ArgumentParser(description=__doc__) parser.add_argument('framework', choices=FRAMEWORKS) - parser.add_argument('--site', type=Path, required=True) - parser.add_argument('--output', type=Path, required=True) - parser.add_argument('--source-cache', type=Path, help='Optional local Git clone; the recorded revision is still enforced') + parser.add_argument('--site', type=Path, default=REPO / 'config/site.local.json', + help='site file (default: config/site.local.json)') + parser.add_argument('--output', type=Path, required=True, help='campaign directory; must not exist') + parser.add_argument('--source-cache', type=Path, help='local Git clone to clone from; the pinned revision still applies') + parser.add_argument('--domains', nargs='+', choices=recipe.ALL_DOMAINS, + help='one task for single-teacher OPD, e.g. caesar_cipher (default: all)') + parser.add_argument('--layout', choices=recipe.LAYOUTS, default='b200x2', + help='hardware layout (default: b200x2); see shared/recipe.py') args = parser.parse_args() - prepare(args.framework, load_site(args.site), args.output, args.source_cache) + if args.domains and len(args.domains) != 1 and tuple(args.domains) != recipe.ALL_DOMAINS: + parser.error(f'--domains takes one task or all of {recipe.ALL_DOMAINS} in order') + prepare(args.framework, load_site(args.site, args.framework, args.layout), args.output, args.source_cache, + args.domains, args.layout) if __name__ == '__main__': diff --git a/tools/report.py b/tools/report.py new file mode 100644 index 0000000..54b4a45 --- /dev/null +++ b/tools/report.py @@ -0,0 +1,297 @@ +"""Summarize the telemetry of finished campaign runs. + + python3 tools/report.py RESULT_DIR [RESULT_DIR ...] [--output report.json] + +Reads each run's `start.json`, `end.json` and `-capture.jsonl.gz` files (from +`shared/capture.py`) and reports per role: utilization, memory, power and energy for +every GPU, host CPU utilization, network traffic, and the token throughput of every +captured SGLang or vLLM engine. A capture cut off by a cancelled run is read up to its +last complete row. + +Per-update metrics come from each framework's own output (the Miles, Slime and verl +training logs, Prime-RL's `monitors/file/metrics.jsonl`) and are mapped onto the common +fields in `UPDATE_FIELDS`. `train_score_by_domain` is the trained batch's verifier score per task: Prime-RL's +per-task metrics, Miles' `train-reward.jsonl`, Slime's `train-samples.jsonl` and verl's `rollouts/`. NeMo-RL runs get telemetry only. Standard library only. +""" +import argparse +import ast +import gzip +import json +from pathlib import Path +import re +import statistics +import zlib + +TOKEN_COUNTERS = {'generation': ('sglang:generation_tokens_total', 'vllm:generation_tokens_total'), + 'prompt': ('sglang:prompt_tokens_total', 'vllm:prompt_tokens_total')} + + +# Common per-update fields and the metric each framework logs for them. +UPDATE_FIELDS = { + 'step_s': {'miles': 'perf/step_time', 'slime': 'perf/step_time', 'verl': 'timing_s/step', 'prime-rl': 'time/step'}, + 'train_s': {'miles': 'perf/train_time', 'slime': 'perf/train_time', 'verl': 'timing_s/update_actor', + 'prime-rl': 'time/forward_backward'}, + 'wait_for_rollouts_s': {'miles': 'perf/train_wait_time', 'slime': 'perf/train_wait_time', 'verl': 'timing_s/gen', + 'prime-rl': 'time/wait_for_batch'}, + 'weight_sync_s': {'miles': 'perf/update_weights_time', 'slime': 'perf/update_weights_time', + 'verl': 'timing_s/timing_s/param_sync', 'prime-rl': 'time/broadcast_weights'}, + 'mfu': {'miles': 'perf/actor_train_mfu', 'slime': 'perf/actor_train_mfu', 'verl': 'perf/mfu/actor', + 'prime-rl': 'perf/mfu'}, + 'response_tokens_mean': {'miles': 'rollout/response_lengths', 'slime': 'rollout/response_lengths', + 'verl': 'response_length/mean', 'prime-rl': 'train/agg/all/num_output_tokens/mean'}, + 'reverse_kl': {'miles': 'train/opd_reverse_kl', 'slime': 'rollout/opd_reverse_kl', 'verl': 'actor/distillation/loss', + 'prime-rl': 'ref_kl/mean'}, + # Through the package's hooks, Miles and Slime log the task score as rollout/raw_reward. + # Miles logs no stale-drop count; Slime's one-rollout-ahead mode drops nothing. + 'train_score': {'miles': 'rollout/raw_reward', 'slime': 'rollout/raw_reward', 'verl': 'critic/score/mean', + 'prime-rl': 'train/agg/all/agent/reward/mean'}, + 'dropped_stale': {'verl': 'fully_async/count/dropped_stale_samples', + 'prime-rl': 'off_policy/dropped'}, +} +MILES_METRICS = re.compile(r'rank0\] \S+ - (?:perf|rollout|step) (\d+): (\{.*\})\s*$') +SLIME_METRICS = re.compile(r'\] \S+\.py:\d+ - (?:perf|rollout|step) (\d+): (\{.*\})\s*$') +TENSOR = re.compile(r"tensor\(\[?([-+\d.eE]+)\]?(?:, device='[^']*')?\)") +DATA = Path(__file__).resolve().parent.parent / 'data' +B200_PEAK_TFLOPS = 2250 # Dense bf16 peak Miles records as perf/mfu_peak_tflops; Slime logs TFLOPS only. + + +def framework_updates(result, framework): + """{update: {metric: value}} from the framework's own logs.""" + updates = {} + if framework in ('miles', 'slime'): + pattern = MILES_METRICS if framework == 'miles' else SLIME_METRICS + for line in (result / 'learner-train.out').read_text(errors='replace').splitlines(): + if match := pattern.search(line): + # Slime logs some one-element tensors, e.g. tensor([0.43], device='cuda:0'). + values = ast.literal_eval(TENSOR.sub(r'\1', match[2])) + updates.setdefault(int(match[1]), {}).update( + {k: v for k, v in values.items() if isinstance(v, (int, float))}) + if framework == 'slime': + for values in updates.values(): + if 'perf/actor_train_tflops' in values: + values['perf/actor_train_mfu'] = values['perf/actor_train_tflops'] / B200_PEAK_TFLOPS + elif framework == 'verl': + # The trainer actor's console metrics can stay in its Ray worker log on the trainer node; + # frameworks/verl/node.py copies that log to `-trainer-worker.out`. + logs = [result / 'learner-train.out', *sorted(result.glob('*trainer-worker.out'))] + lines = [line for log in logs if log.exists() for line in log.read_text(errors='replace').splitlines()] + for line in lines: + if 'timing_s/step:' in line: + fields = dict(f.strip().rsplit(':', 1) for f in line.split('step:', 1)[1].split(' - ')[1:] if ':' in f) + step = int(line.split('step:', 1)[1].split(' ', 1)[0]) + updates[step] = {k: float(v) for k, v in fields.items() if re.fullmatch(r'[-+.\deE]+', v)} + elif framework == 'prime-rl': + path = result / 'monitors' / 'file' / 'metrics.jsonl' + for line in path.read_text().splitlines() if path.exists() else (): + row = json.loads(line) + if row.get('step') is not None: + updates.setdefault(row['step'], {}).update(row) + return updates + + +def domain_of_prompt(): + """{user prompt: task} from the packaged training data; each prompt belongs to one task.""" + prompts = {} + for path in sorted(DATA.glob('*-train.jsonl.gz')): + if path.name.startswith(('mixed-', '.')): + continue + domain = path.name.removesuffix('-train.jsonl.gz') + with gzip.open(path, 'rt') as stream: + prompts.update((json.loads(line)['prompt'][-1]['content'].strip(), domain) for line in stream) + return prompts + + +def train_scores_by_domain(result, framework, updates): + """{update: {task: mean verifier score of the trained batch}}, keyed like framework_updates.""" + scores = {} # update -> task -> [score, ...] + if framework == 'prime-rl': + for step, values in updates.items(): + for key, value in values.items(): + match = re.fullmatch(r'train/(\w+)/effective/agent/reward/mean', key) + if match and match[1] != 'agg': # train/agg/... aggregates the tasks + scores.setdefault(step, {})[match[1]] = [value] + elif framework == 'miles' and (path := result / 'train-reward.jsonl').exists(): + for line in path.read_text().splitlines(): # written by frameworks/miles/opd_reward.log_rollout + row = json.loads(line) + scores[row['rollout_id']] = {d: [v['score']] for d, v in row['domains'].items()} + elif framework == 'slime' and (path := result / 'train-samples.jsonl').exists(): + with path.open() as stream: + for line in stream: + row = json.loads(line) + scores.setdefault(row['rollout_id'], {}).setdefault(row['domain'], []).append(row['task_score']) + elif framework == 'verl' and (directory := result / 'rollouts').is_dir(): + # verl's dump (trainer.rollout_data_dir) has the decoded chat prompt, not the task: + # "system\n...\nuser\n\nassistant\n\n". + prompts = domain_of_prompt() + for path in directory.glob('*.jsonl'): + for line in path.read_text().splitlines(): + row = json.loads(line) + user = row['input'].rsplit('\nuser\n', 1)[-1].rsplit('\nassistant\n', 1)[0].strip() + # verl names the dump after global_steps N and logs that update's metrics at step N + 1 + # (the scores and lengths match one step apart). + scores.setdefault(row['step'] + 1, {}).setdefault(prompts.get(user), []).append(row['score']) + return {u: {d: statistics.fmean(v) for d, v in by.items()} for u, by in scores.items()} + + +def update_summary(result, framework): + updates = framework_updates(result, framework) + by_domain = train_scores_by_domain(result, framework, updates) + rows = [{**{field: values.get(names.get(framework)) for field, names in UPDATE_FIELDS.items()}, + 'train_score_by_domain': by_domain.get(update)} + for update, values in sorted(updates.items())] + if framework == 'prime-rl': # Prime-RL reports MFU in percent; the others as a fraction. + for row in rows: + row['mfu'] = row['mfu'] / 100 if isinstance(row['mfu'], (int, float)) else row['mfu'] + rows = [r for r in rows if r['step_s'] is not None] + if not rows: + return None + if framework == 'miles' and not any(r['train_score'] for r in rows): + # Runs from before the reward hook set metadata['raw_reward'] logged the teacher reward, a constant 0. + for row in rows: + row['train_score'] = None + + def column(field): + return [r[field] for r in rows if isinstance(r[field], (int, float))] + summary = {'updates': len(rows), 'per_update': rows} + for field in ('step_s', 'train_s', 'wait_for_rollouts_s', 'weight_sync_s'): + if values := column(field): + summary[f'{field}_median'] = statistics.median(values) + summary[f'{field}_total'] = sum(values) + for field in ('mfu', 'response_tokens_mean'): + if values := column(field): + summary[f'{field}_mean'] = statistics.fmean(values) + for field in ('reverse_kl', 'train_score'): + if values := column(field): + summary[f'{field}_first'], summary[f'{field}_last'] = values[0], values[-1] + if values := column('dropped_stale'): + summary['dropped_stale_total'] = sum(values) + return summary + + +def capture_rows(path): + """Rows of one capture file, up to the last complete one.""" + rows = [] + try: + with gzip.open(path, 'rt') as stream: + for line in stream: + rows.append(json.loads(line)) + except (EOFError, zlib.error, json.JSONDecodeError): + pass # The run was stopped while the capture was writing. + return rows + + +def gpu_summary(rows): + samples = {} + for row in rows: + if not isinstance(row['gpu_csv'], str): + continue + for line in row['gpu_csv'].splitlines()[1:]: + fields = [f.strip() for f in line.split(',')] + index, util, memory, power = int(fields[1]), fields[3], fields[5], fields[7] + if '[N/A]' in (util, memory, power): + continue + samples.setdefault(index, []).append((row['time_unix'], float(util.split()[0]), + float(memory.split()[0]) / 1024, float(power.split()[0]))) + summary = {} + for index, values in sorted(samples.items()): + energy = sum((b[0] - a[0]) * a[3] for a, b in zip(values, values[1:])) / 3.6e6 + summary[index] = {'utilization_mean_pct': statistics.fmean(v[1] for v in values), + 'busy_fraction': statistics.fmean(v[1] > 0 for v in values), + 'memory_peak_gib': max(v[2] for v in values), + 'power_mean_w': statistics.fmean(v[3] for v in values), 'energy_kwh': energy} + return summary + + +def cpu_utilization(rows): + def totals(row): + fields = [int(x) for x in row['stat'].splitlines()[0].split()[1:]] + return sum(fields), fields[3] + fields[4] # All jiffies; idle + iowait. + rows = [r for r in rows if isinstance(r['stat'], str)] + if len(rows) < 2: + return None + (total_a, idle_a), (total_b, idle_b) = totals(rows[0]), totals(rows[-1]) + return 100 * (1 - (idle_b - idle_a) / (total_b - total_a)) + + +def network(rows): + def counters(row): + rx = tx = 0 + for line in row['net/dev'].splitlines()[2:]: + name, data = line.split(':', 1) + if name.strip() != 'lo': + fields = [int(x) for x in data.split()] + rx, tx = rx + fields[0], tx + fields[8] + return rx, tx + rows = [r for r in rows if isinstance(r['net/dev'], str)] + if len(rows) < 2: + return None + (rx_a, tx_a), (rx_b, tx_b) = counters(rows[0]), counters(rows[-1]) + seconds = rows[-1]['time_unix'] - rows[0]['time_unix'] + return {'rx_gb': (rx_b - rx_a) / 1e9, 'tx_gb': (tx_b - tx_a) / 1e9, + 'rx_gbit_s': 8 * (rx_b - rx_a) / 1e9 / seconds, 'tx_gbit_s': 8 * (tx_b - tx_a) / 1e9 / seconds} + + +def counter(text, names): + """Sum of every sample of the named Prometheus counters, or None if absent.""" + values = [float(line.rsplit(' ', 1)[1]) for line in text.splitlines() + if line.split('{')[0].split(' ')[0] in names] + return sum(values) if values else None + + +def engine_throughput(rows): + """Tokens per second of each captured engine, between its first and last successful scrape.""" + engines = {} + for row in rows: + for target, text in row['prometheus'].items(): + if isinstance(text, str): + engines.setdefault(target, []).append((row['time_unix'], text)) + summary = {} + for target, scrapes in sorted(engines.items()): + (t0, first), (t1, last) = scrapes[0], scrapes[-1] + entry = {'seconds': t1 - t0} + for kind, names in TOKEN_COUNTERS.items(): + # A labeled counter first appears on its first increment, so absent means zero. + a, b = counter(first, names) or 0.0, counter(last, names) + if b is not None and t1 > t0: + entry[f'{kind}_tokens'] = b - a + entry[f'{kind}_tokens_per_s'] = (b - a) / (t1 - t0) + summary[target] = entry + return summary + + +def summarize(result): + result = Path(result) + start = json.loads((result / 'start.json').read_text()) + end = json.loads((result / 'end.json').read_text()) if (result / 'end.json').exists() else {} + framework = result.name.rsplit('-', 1)[0] + report = {'run': result.name, 'job_id': start.get('job_id'), 'nodes': start.get('nodes'), + 'exit_code': end.get('exit_code'), 'wall_seconds': end['time'] - start['time'] if end else None, + 'updates': update_summary(result, framework), 'roles': {}} + for path in sorted(result.glob('*-capture.jsonl.gz')): + rows = capture_rows(path) + if not rows: + continue + role = path.name[:-len('-capture.jsonl.gz')] + gpus = gpu_summary(rows) + report['roles'][role] = { + 'samples': len(rows), 'seconds': rows[-1]['time_unix'] - rows[0]['time_unix'], + 'gpus': gpus, 'gpu_energy_kwh': sum(g['energy_kwh'] for g in gpus.values()), + 'cpu_utilization_pct': cpu_utilization(rows), 'network': network(rows), + 'engines': engine_throughput(rows)} + return report + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument('results', nargs='+', type=Path, help='run result directories (/results/)') + parser.add_argument('--output', type=Path, help='also write the JSON report here') + args = parser.parse_args() + reports = [summarize(r) for r in args.results] + text = json.dumps(reports, indent=2) + if args.output: + args.output.write_text(text + '\n') + print(text) + + +if __name__ == '__main__': + main() diff --git a/tools/reproduce.sh b/tools/reproduce.sh new file mode 100755 index 0000000..7894779 --- /dev/null +++ b/tools/reproduce.sh @@ -0,0 +1,272 @@ +#!/usr/bin/env bash +# Reproduce the OPD benchmark end to end: Qwen3.6-35B-A3B student, the caesar_cipher GRPO teacher +# (or both GRPO teachers for MOPD), the shared recipe (20 updates, thinking on, 32k context). +# +# tools/reproduce.sh all reference benchmarks, every framework, then the report +# tools/reproduce.sh miles verl chosen frameworks (miles, nemo-rl, prime-rl, slime, verl) +# tools/reproduce.sh references report base and teacher benchmarks; telemetry report over finished runs +# tools/reproduce.sh assets only download and fuse the models (before parallel runs) +# +# For each framework: assets -> prepare -> runtime -> run (blocks until the Slurm job ends) -> export +# Hugging Face weights of every saved checkpoint -> benchmark each (the final one as , earlier ones +# as -step). Steps whose output already exists are skipped, so a re-run resumes where the last +# one stopped. Run it on a Slurm login node with Pyxis/Enroot. +# +# Settings (environment): +# PARTITION Slurm partition (required) +# SITE site file for the two-node frameworks (default config/site.local.json) +# NEMO_RL_SITE site file for NeMo-RL, which needs a third node (default config/site-nemo-rl.local.json) +# RUNS campaigns and benchmark outputs (default /shared/opd-runs) +# NAME campaign suffix: campaigns are $RUNS/-$NAME (default caesar) +# DOMAINS tasks to train, each with its own teacher: "caesar_cipher" (default, single-teacher OPD) +# or "caesar_cipher simple_geometry" (MOPD) +# LAYOUT hardware layout from shared/recipe.py: b200x2 (default); h100x3 (Miles, verl), whose site +# file also names generation_node_2 and generation_ip_2; or h100x2 (verl single-teacher OPD) +# BENCH_TP GPUs per benchmark vLLM engine (default 1; 2 on 80 GB GPUs) +# RUNTIME container images, tool venv, caches, logs (default /shared/opd-runtime) +# SHARED shared filesystem mounted into single-node containers (default /shared) +# GPU_NODE node for single-node jobs (default: the site's trainer node) +# BENCH_CAMPAIGN verl campaign whose vLLM venv runs the benchmarks (default: $RUNS/verl-$NAME, built if needed) +# DEADLOCK_MIN NeMo-RL: cancel the run if its first weight sync has not finished after this many +# minutes (default 20); a hung run otherwise holds 24 GPUs until its time limit +set -euo pipefail +[[ $# -gt 0 ]] || { awk 'NR > 1 && !/^#/ {exit} NR > 1' "$0"; exit 2; } + +REPO=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) +: "${PARTITION:?Set PARTITION to your Slurm partition}" +SITE=${SITE:-$REPO/config/site.local.json} +NEMO_RL_SITE=${NEMO_RL_SITE:-$REPO/config/site-nemo-rl.local.json} +RUNS=${RUNS:-/shared/opd-runs} +NAME=${NAME:-caesar} +RUNTIME=${RUNTIME:-/shared/opd-runtime} +SHARED=${SHARED:-/shared} +DEADLOCK_MIN=${DEADLOCK_MIN:-20} +read -ra DOMAINS <<< "${DOMAINS:-caesar_cipher}" +LAYOUT=${LAYOUT:-b200x2} +BENCH_TP=${BENCH_TP:-1} +MILES_IMAGE='docker://radixark/miles@sha256:59a11219eae0defc6594ec678fafe4e897c16904263223f79968cd3e0209a502' +NEMO_RL_IMAGE='docker://nvcr.io#nvidia/nemo-rl:v0.7.0' +TOOLS=$RUNTIME/tools-venv +export UV_CACHE_DIR=$RUNTIME/uv-cache UV_PYTHON_INSTALL_DIR=$RUNTIME/python UV_LINK_MODE=copy +# Slurm sets NVIDIA_VISIBLE_DEVICES=void for job steps; Pyxis then exposes no GPUs or driver libraries. +export NVIDIA_VISIBLE_DEVICES=all NVIDIA_DRIVER_CAPABILITIES=all +mkdir -p "$RUNS/benchmarks" "$RUNTIME/logs" + +say() { echo "[$(date -u +%T)] $*"; } +site() { python3 -c "import json,sys; print(json.load(open(sys.argv[1]))[sys.argv[2]])" "$1" "$2"; } +recipe() { python3 -c "import sys; sys.path.insert(0, '$REPO'); from shared import recipe; print($1)"; } +node() { echo "${GPU_NODE:-$(site "$SITE" trainer_node)}"; } +on_node() { # on_node GPUS [srun args...] -- command...: run one exclusive step on GPU_NODE + local gpus=$1 args=(); shift + while [[ $1 != -- ]]; do args+=("$1"); shift; done; shift + srun -p "$PARTITION" -N1 -n1 -w "$(node)" --gres=gpu:"$gpus" --exclusive --time=04:00:00 "${args[@]}" bash -c "$*" +} +in_image() { # in_image IMAGE GPUS -- command...: on_node in a writable container with $SHARED mounted + local image=$1 gpus=$2; shift 3 + on_node "$gpus" --container-image="$image" --container-writable --container-remap-root \ + --no-container-mount-home --container-mounts="$SHARED:$SHARED" -- "$@" +} +campaign() { echo "$RUNS/$1-$NAME"; } +latest_run() { python3 -c "import json; print(json.load(open('$(campaign "$1")/active-run.json'))['result'])"; } +run_ok() { [[ -f $(campaign "$1")/active-run.json ]] && [[ $(cat "$(latest_run "$1")/exit-code.txt" 2>/dev/null) == 0 ]]; } + +tools() { # pinned uv and Hugging Face CLI; links the site's uv path to it if absent + [[ -x $TOOLS/bin/uv ]] || { python3 -m venv "$TOOLS"; "$TOOLS/bin/pip" install -q uv==0.11.33 'huggingface_hub[cli]==0.35.3'; } + local uv; uv=$(site "$SITE" uv) + [[ -e $uv ]] || { mkdir -p "$(dirname "$uv")"; ln -s "$TOOLS/bin/uv" "$uv"; } +} + +assets() { # download the pinned base and teacher, then fuse the teacher into the base layout + local base teachers hub + base=$(site "$SITE" base_model); teachers=$(site "$SITE" teachers); hub=$(site "$SITE" assets)/hub + [[ -f $base/config.json ]] || { say "downloading the base model"; "$TOOLS/bin/hf" download "$(recipe 'recipe.BASE_MODEL[0]')" \ + --revision "$(recipe 'recipe.BASE_MODEL[1]')" --local-dir "$base" >/dev/null; } + local d + for d in "${DOMAINS[@]}"; do + [[ -f $hub/$d/config.json ]] || { say "downloading the $d teacher"; "$TOOLS/bin/hf" download \ + "$(recipe "recipe.TEACHERS['$d'][0]")" --revision "$(recipe "recipe.TEACHERS['$d'][1]")" \ + --local-dir "$hub/$d" >/dev/null; } + [[ -f $teachers/$d/config.json ]] || { image miles; say "fusing the $d teacher"; \ + in_image "$(site "$SITE" container_image)" 1 -- python3 "$REPO/tools/fuse_teacher.py" "$hub/$d" "$teachers/$d" "$base"; } + done +} + +image() { # image miles|nemo-rl: import the digest- or tag-pinned container once + local path from + if [[ $1 == nemo-rl ]]; then path=$(site "$NEMO_RL_SITE" nemo_rl_container); from=$NEMO_RL_IMAGE + else path=$(site "$SITE" container_image); from=$MILES_IMAGE; fi + [[ -f $path ]] && return + say "importing $from" + on_node 0 -- "ENROOT_TEMP_PATH=/tmp enroot import -o $path.part '$from' && mv $path.part $path" +} + +megatron_checkpoint() { # Miles and Slime: convert the base model to Megatron torch_dist with Miles' converter + local out; out=$(site "$SITE" megatron_model) + [[ -f $out/latest_checkpointed_iteration.txt ]] && return + prepare miles + say "converting the base model to torch_dist" + in_image "$(site "$SITE" container_image)" 8 -- "set -euo pipefail; cd $(campaign miles)/source + read -ra args <<< \"\$(python3 miles/utils/external_utils/model_args_utils.py qwen3.6-35B-A3B)\" + PYTHONPATH=$(site "$SITE" megatron_source):\$PWD torchrun --nproc-per-node 8 tools/convert_hf_to_torch_dist.py \ + \"\${args[@]}\" --hf-checkpoint $(site "$SITE" base_model) --save $out --mtp-num-layers 1" +} + +prepare() { # create the campaign for DOMAINS with tools/prepare.py + local fw=$1 site=$SITE layout=b200x2 + [[ $fw == nemo-rl ]] && site=$NEMO_RL_SITE + [[ $fw == miles || $fw == verl ]] && layout=$LAYOUT # Other frameworks run b200x2. + [[ -d $(campaign "$fw") ]] && return + say "preparing $(campaign "$fw")" + python3 "$REPO/tools/prepare.py" "$fw" --site "$site" --output "$(campaign "$fw")" --domains "${DOMAINS[@]}" \ + --layout "$layout" +} + +flashinfer() { # install prebuilt FlashInfer kernels matching the venv's flashinfer-python version + local py=$1 uv v; uv=$(site "$SITE" uv) + v=$("$uv" pip list --python "$py" 2>/dev/null | awk '$1 == "flashinfer-python" {print $2}') + "$uv" pip list --python "$py" 2>/dev/null | grep -q '^flashinfer-jit-cache ' && return + "$uv" pip install -q --python "$py" --no-deps --index-url https://flashinfer.ai/whl/ "flashinfer-cubin==$v" + "$uv" pip install -q --python "$py" --no-deps --index-url https://flashinfer.ai/whl/cu130/ "flashinfer-jit-cache==$v+cu130" +} + +runtime() { # build what the campaign needs before submission (see docs/SETUP.md) + local fw=$1 c uv; c=$(campaign "$fw"); uv=$(site "$SITE" uv) + case $fw in + miles|slime) + image miles; megatron_checkpoint + # Install reasoning-gym with the container's Python 3.12: its pycosat dependency has no cp312 wheel. + [[ -d $c/pydeps/reasoning_gym ]] || in_image "$(site "$SITE" container_image)" 0 -- \ + "python3 -m pip install -q --target $c/pydeps reasoning-gym==0.1.25" + if [[ $fw == slime ]] && ! (cd "$c" && sha256sum -c --quiet wheels.sha256 2>/dev/null); then + "$TOOLS/bin/pip" download -q --only-binary=:all: --no-deps --platform manylinux2014_x86_64 --python-version 312 \ + --implementation cp --abi cp312 --dest "$c/wheels" numpy==1.26.4 scipy==1.15.3 + (cd "$c" && sha256sum -c wheels.sha256) + fi ;; + prime-rl|verl) + local extras=(--all-extras) + [[ $fw == verl ]] && extras=(--all-packages --extra vllm --extra fsdp) + [[ -x $c/source/.venv/bin/python ]] || { say "building the $fw venv"; (cd "$c/source" && "$uv" sync --frozen --managed-python --python 3.12 "${extras[@]}"); } + [[ $fw == prime-rl ]] && "$uv" pip install -q --python "$c/source/.venv/bin/python" --no-deps -e "$c/environments/rg_tasks" + flashinfer "$c/source/.venv/bin/python" + [[ -d $c/pydeps/reasoning_gym ]] || "$uv" pip install -q --target "$c/pydeps" --python "$c/source/.venv/bin/python" reasoning-gym==0.1.25 ;; + nemo-rl) + image nemo-rl ;; + esac +} + +nemo_rl_watchdog() { # cancel a NeMo-RL run whose first trainer->vLLM weight sync never completes + # That sync precedes the first update (async_grpo_train, grpo.py:3551-3554 at the pinned revision). + # Every Qwen3.6-35B-A3B run we tried hung there while rollouts kept going in the background. + local c previous result log started; c=$(campaign nemo-rl); previous=$1 + # The controller records the new run in active-run.json once Slurm grants the allocation. + until [[ -f $c/active-run.json ]] && [[ $(latest_run nemo-rl) != "$previous" ]]; do sleep 20; done + result=$(latest_run nemo-rl); log= + while [[ ! -f $result/end.json ]]; do + log=$(ls "$result"/*-logs/ray-driver.log 2>/dev/null | head -n 1) + if [[ -n $log ]] && grep -aq "Refitting policy generation" "$log"; then + started=${started:-$(date +%s)} + grep -aq "refit completed successfully" "$log" && return + if (( $(date +%s) - started > DEADLOCK_MIN * 60 )); then + { echo "first weight sync did not complete in $DEADLOCK_MIN min" + grep -a -o "Error in VllmInternalWorkerExtension[^[]*\|Worker failed to update weights[^[]*" "$log" | sort | uniq -c + } | tee "$result/DEADLOCK" + scancel "$(python3 -c "import json; print(json.load(open('$c/active-run.json'))['job_id'])")" + return + fi + fi + sleep 30 + done +} + +run() { # submit the campaign and wait for its Slurm allocation to end + local fw=$1 c; c=$(campaign "$fw") + run_ok "$fw" && { say "$fw: already finished ($(latest_run "$fw"))"; return; } + say "$fw: submitting" + [[ $fw == nemo-rl ]] && nemo_rl_watchdog "$( [[ -f $c/active-run.json ]] && latest_run nemo-rl )" & + python3 "$c/launch.py" "$c" --partition "$PARTITION" --submit || true + wait + run_ok "$fw" || { say "$fw: run $(latest_run "$fw") did not finish cleanly; see its results directory"; return 1; } +} + +export_hf() { # export_hf FRAMEWORK STEP: print the path of the Hugging Face weights saved after STEP updates, + # converting them first if the framework saves another format. Fails if there is no checkpoint at STEP. + local fw=$1 n=$2 c result name; c=$(campaign "$fw"); result=$(latest_run "$fw"); name=$(basename "$result") + case $fw in + miles|slime) [[ -d $c/checkpoints/$name/hf/iter_$(( n - 1 )) ]] && echo "$c/checkpoints/$name/hf/iter_$(( n - 1 ))" ;; + prime-rl) + local step=$c/checkpoints/$name/checkpoints/step_$n # Prime-RL nests checkpoints/ + [[ -d $step ]] || return 1 + [[ -f $step/weights/config.json ]] || on_node 8 -- "cd $c/source && \ + PRL_ATTEMPT_CONFIG_DIR=$(python3 -c "import json; print(json.load(open('$result/resolved-paths.json'))['config'])") \ + $(site "$SITE" uv) run --project $c/source --no-sync torchrun --nproc-per-node 8 tools/convert_dcp_to_bf16.py $step $step/weights" >&2 + echo "$step/weights" ;; + verl) + local actor=$c/checkpoints/$name/global_step_$n/actor + [[ -d $actor ]] || return 1 + [[ -f $actor/huggingface-merged/config.json ]] || on_node 0 -- "cd $c/source && CUDA_VISIBLE_DEVICES= .venv/bin/python -m \ + verl.model_merger merge --backend fsdp --local_dir $actor --target_dir $actor/huggingface-merged --trust-remote-code" >&2 + echo "$actor/huggingface-merged" ;; + nemo-rl) + # Follows NeMo-RL's README (examples/converters/convert_megatron_to_hf.py). Untested: no run has saved a checkpoint yet. + local step=$c/checkpoints/$name/step_$n + [[ -d $step ]] || return 1 + [[ -f $step/hf/config.json ]] || in_image "$(site "$NEMO_RL_SITE" nemo_rl_container)" 8 -- "cd /opt/nemo-rl && \ + uv run --extra mcore python examples/converters/convert_megatron_to_hf.py --config $step/config.yaml \ + --hf-model-name $(site "$NEMO_RL_SITE" base_model) --megatron-ckpt-path $(ls -d "$step"/policy/weights/iter_* | tail -n 1) \ + --hf-ckpt-path $step/hf" >&2 + echo "$step/hf" ;; + esac +} + +benchmark() { # benchmark NAME MODEL: run tools/benchmark.py on one node with the verl campaign's vLLM venv + local name=$1 model=$2 out=$RUNS/benchmarks/$1 v=${BENCH_CAMPAIGN:-$(campaign verl)} + [[ -f $out/summary.json ]] && { say "benchmark $name: done"; return; } + [[ -n ${BENCH_CAMPAIGN:-} ]] || { prepare verl; runtime verl; } + rm -rf "$out" + say "benchmarking $name" + on_node 8 -- "unset ROCR_VISIBLE_DEVICES; export CAMPAIGN_PYDEPS=$v/pydeps HF_HUB_OFFLINE=1 PYTHONNOUSERSITE=1 \ + VLLM_CACHE_ROOT=$v/runtime-cache/vllm FLASHINFER_WORKSPACE_BASE=$v/runtime-cache TRITON_CACHE_DIR=$v/runtime-cache/triton; \ + $v/source/.venv/bin/python $REPO/tools/benchmark.py $model $out --domains caesar_cipher simple_geometry --tp $BENCH_TP" +} + +framework() { + local fw=$1 + say "==== $fw" + tools; assets; prepare "$fw"; runtime "$fw"; run "$fw" + local name bench model n last; name=$(basename "$(latest_run "$fw")"); last=$(recipe recipe.UPDATES) + for n in $(recipe "' '.join(map(str, range(recipe.SAVE_INTERVAL, recipe.UPDATES + 1, recipe.SAVE_INTERVAL)))"); do + bench=$name; (( n == last )) || bench=$name-step$n + [[ -f $RUNS/benchmarks/$bench/summary.json ]] && { say "benchmark $bench: done"; continue; } + model=$(export_hf "$fw" "$n") || { say "$fw: no checkpoint after $n updates; skipping its benchmark"; continue; } + benchmark "$bench" "$model" + done +} + +references() { + tools; assets + benchmark base "$(site "$SITE" base_model)" + local d + for d in "${DOMAINS[@]}"; do # teacher-caesar, teacher-geometry + benchmark "teacher-$(sed 's/_cipher$//; s/^simple_//' <<< "$d")" "$(site "$SITE" teachers)/$d" + done +} + +report() { + local results=() + for fw in miles nemo-rl prime-rl slime verl; do + [[ -f $(campaign "$fw")/active-run.json ]] && results+=("$(latest_run "$fw")") + done + python3 "$REPO/tools/report.py" "${results[@]}" --output "$RUNS/benchmarks/telemetry.json" >/dev/null + say "report: $RUNS/benchmarks/telemetry.json; benchmarks: $RUNS/benchmarks/*/summary.json" +} + +[[ $1 == all ]] && set -- references miles prime-rl slime verl nemo-rl report +for target in "$@"; do + case $target in + miles|prime-rl|slime|verl|nemo-rl) framework "$target" || say "$target failed; continuing" ;; + assets) tools; assets ;; + references) references ;; + report) report ;; + *) echo "Unknown target: $target"; exit 2 ;; + esac +done diff --git a/tools/schema_verl.py b/tools/schema_verl.py new file mode 100644 index 0000000..95456f7 --- /dev/null +++ b/tools/schema_verl.py @@ -0,0 +1,173 @@ +"""Per-update case-study metrics for a verl run, in the columns of Miles run 535's schema file. + + /source/.venv/bin/python tools/schema_verl.py RESULT_DIR OUT.json + +Sources, all written by every verl run of this package: + - `*trainer-worker.out` / `learner-train.out`: verl's console metrics per step (timing, MFU, tokens, + dropped stale samples). verl logs update N at metric step N + 1 (see tools/report.py). + - `rollouts/.jsonl`: written at the end of update N, so its modification time ends the update's + window (it matches the trainer's progress bar to about a second). It holds the trained samples' decoded + prompt and response, re-tokenized here with the student's tokenizer. Decoding dropped special tokens, + so prompt lengths are a few tokens short. (verl's TensorBoard writer runs in the trainer's Ray worker, + which does not get TENSORBOARD_DIR, so the run's tensorboard/ directory stays empty.) + - `*-capture.jsonl.gz`: vLLM Prometheus histograms and counters of every policy and teacher server, + scraped every 5 s by shared/capture.py --vllm. A server is a teacher if its model path is under the + site's teachers directory. Policy requests resumed after a weight sync (partial rollout) count as + separate requests. +""" +import gzip +import json +from pathlib import Path +import re +import statistics +import sys + +TRAINER_GPUS, GPUS_PER_TEACHER, BATCH = 8, 2, 128 # h100x3 and h100x2 + + +def console_metrics(result): + """{logged step: {metric: value}} from verl's console lines.""" + steps = {} + logs = [result / 'learner-train.out', *sorted(result.glob('*trainer-worker.out'))] + for log in logs: + if not log.exists(): + continue + for line in log.read_text(errors='replace').splitlines(): + if 'timing_s/step:' not in line: + continue + step = int(line.split('step:', 1)[1].split(' ', 1)[0]) + fields = dict(f.strip().rsplit(':', 1) for f in line.split('step:', 1)[1].split(' - ')[1:] if ':' in f) + steps[step] = {k: float(v) for k, v in fields.items() if re.fullmatch(r'[-+.\deE]+', v.strip())} + return steps + + +def update_ends(result): + """{update: wall time it ended}, from the modification time of verl's per-update rollout dump.""" + return {int(p.stem): p.stat().st_mtime for p in (result / 'rollouts').glob('*.jsonl')} + + +def histograms(result, teachers_dir): + """[(time, {'policy'|'teacher': {metric: {le or '': value}}})] summed over servers, per capture row.""" + rows = [] + for path in sorted(result.glob('*-capture.jsonl.gz')): + with gzip.open(path, 'rt') as stream: + try: # A capture still being written ends mid-stream; keep the rows read so far. + for line in stream: + row = json.loads(line) + rows.append((row['time_unix'], row['prometheus'])) + except (EOFError, json.JSONDecodeError): + pass + rows.sort(key=lambda r: r[0]) + # Each capture file is one node; keep the latest value per (server, series) and sum over servers. + latest, out = {}, [] + names = ('vllm:e2e_request_latency_seconds_bucket', 'vllm:prompt_tokens_total', 'vllm:generation_tokens_total', + 'vllm:request_success_total') + for t, servers in rows: + for target, text in servers.items(): + if not isinstance(text, str): + continue + for line in text.splitlines(): + if not line.startswith(names): + continue + series, value = line.rsplit(' ', 1) + role = 'teacher' if teachers_dir in series else 'policy' + metric = series.split('{', 1)[0] + le = m[1] if (m := re.search(r'le="([^"]+)"', series)) else '' + latest[(target, series)] = (role, metric, le, float(value)) + total = {} + for role, metric, le, value in latest.values(): + bucket = total.setdefault(role, {}).setdefault(metric, {}) + bucket[le] = bucket.get(le, 0) + value + out.append((t, total)) + return out + + +def at(series, t): + """The cumulative values at time t (the last capture at or before t).""" + best = {} + for time, value in series: + if time > t: + break + best = value + return best + + +def delta(h0, h1, role, metric): + a, b = h0.get(role, {}).get(metric, {}), h1.get(role, {}).get(metric, {}) + return {k: v - a.get(k, 0) for k, v in b.items()} + + +def p50(buckets): + """Median from cumulative histogram buckets, interpolated within the bucket.""" + les = sorted((float('inf') if k == '+Inf' else float(k), c) for k, c in buckets.items()) + total = les[-1][1] if les else 0 + if total <= 0: + return None, 0 + prev_le, prev_c = 0.0, 0.0 + for le, c in les: + if c >= total / 2: + if le == float('inf'): + return prev_le, total + return prev_le + (le - prev_le) * (total / 2 - prev_c) / max(c - prev_c, 1e-9), total + prev_le, prev_c = le, c + + +def sample_lengths(result, tokenizer): + """{dump step: [(prompt tokens, response tokens), ...]} from verl's rollout dump.""" + lengths = {} + for path in sorted((result / 'rollouts').glob('*.jsonl')): + for line in path.read_text().splitlines(): + row = json.loads(line) + lengths.setdefault(row['step'], []).append( + (len(tokenizer(row['input'])['input_ids']), len(tokenizer(row['output'])['input_ids']))) + return lengths + + +def main(result, out): + result = Path(result) + site = json.loads((result.parents[1] / 'site.json').read_text()) + experiment = result.parents[1] / 'experiment.json' + experiment = json.loads(experiment.read_text()) if experiment.exists() else {} + domains = experiment.get('domains', [1, 2]) + POLICY_GPUS = 6 if experiment.get('layout') == 'h100x2' else 12 + teacher_gpus = GPUS_PER_TEACHER * len(domains) # 2 for single-teacher OPD, 4 for MOPD + from transformers import AutoTokenizer + tokenizer = AutoTokenizer.from_pretrained(site['base_model']) + metrics, ends = console_metrics(result), update_ends(result) + hist = histograms(result, site['teachers']) + lengths = sample_lengths(result, tokenizer) + start = json.loads((result / 'start.json').read_text())['time'] + rows, previous_end, previous_dropped = [], start, 0 + for update in sorted(ends): + # verl logs update N's metrics at step N + 1 and never logs the last update's. + m, t0, t1 = metrics.get(update + 1, {}), previous_end, ends[update] + h0, h1 = at(hist, t0), at(hist, t1) + rollout_p50, finished = p50(delta(h0, h1, 'policy', 'vllm:e2e_request_latency_seconds_bucket')) + teacher_p50, _ = p50(delta(h0, h1, 'teacher', 'vllm:e2e_request_latency_seconds_bucket')) + teacher_tokens = sum(delta(h0, h1, 'teacher', 'vllm:prompt_tokens_total').values()) + samples = lengths.get(update, []) + dropped_total = m.get('fully_async/count/dropped_stale_samples', previous_dropped) + dropped = dropped_total - previous_dropped + tokens = m.get('perf/total_num_tokens') + rows.append({ + 'update': update, 'step_s': m.get('timing_s/step'), 'train_s': m.get('timing_s/update_actor'), + 'wait_s': m.get('timing_s/gen'), + 'train_tok_s_gpu': tokens / m['timing_s/update_actor'] / TRAINER_GPUS + if tokens and m.get('timing_s/update_actor') else None, + 'mfu': m.get('perf/mfu/actor'), + 'prompt_p50': statistics.median(p for p, _ in samples) if samples else None, + 'response_p50': statistics.median(r for _, r in samples) if samples else None, + 'rollout_e2e_p50_s': rollout_p50, 'rollout_requests_finished': finished, + 'stale_filtered': dropped, 'filtered_ratio': dropped / (dropped + BATCH), + 'effective_tok_s_gpu_policy': sum(p + r for p, r in samples) / (t1 - t0) / POLICY_GPUS if samples else None, + 'teacher_p50_s': teacher_p50, + 'teacher_tok_s_gpu': teacher_tokens / (t1 - t0) / teacher_gpus if teacher_tokens else None, + 'window_s': t1 - t0, 'samples_dumped': len(samples)}) + previous_end, previous_dropped = t1, dropped_total + Path(out).write_text(json.dumps(rows, indent=1) + '\n') + for row in rows: + print({k: (round(v, 3) if isinstance(v, float) else v) for k, v in row.items()}) + + +if __name__ == '__main__': + main(*sys.argv[1:3]) diff --git a/tools/submit.py b/tools/submit.py index 5a53bd1..93e9414 100644 --- a/tools/submit.py +++ b/tools/submit.py @@ -1,26 +1,34 @@ -"""Print a Slurm command by default; submit only when --submit is explicit.""" +"""Print the salloc command for a prepared campaign; with --submit, check prerequisites and run it. + +Copied into each campaign as `launch.py`. +""" import argparse import hashlib import json from pathlib import Path import shlex import subprocess +import sys def main(): parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument('campaign', type=Path) - parser.add_argument('--partition', required=True) - parser.add_argument('--time', default='01:30:00') - parser.add_argument('--submit', action='store_true') + parser.add_argument('campaign', type=Path, help='prepared campaign directory') + parser.add_argument('--partition', required=True, help='Slurm partition') + # A 20-update run with 30k-token thinking takes about an hour after startup. + parser.add_argument('--time', default='04:00:00', help='Slurm time limit (default: 04:00:00)') + parser.add_argument('--submit', action='store_true', help='allocate GPUs instead of printing the command') args = parser.parse_args() root = args.campaign.resolve() site = json.loads((root/'site.json').read_text()) manifest = json.loads((root/'package-provenance.json').read_text()) name = 'opd-' + manifest['framework'] + '-' + hashlib.sha256(str(root).encode()).hexdigest()[:8] - command = ['salloc', f'--job-name={name}', f'--partition={args.partition}', '--nodes=2', - f'--nodelist={site["generation_node"]},{site["trainer_node"]}', '--exclusive', - '--gres=gpu:8', f'--time={args.time}', 'python3', str(root/'control.py')] + layout = json.loads((root/'experiment.json').read_text()).get('layout') if (root/'experiment.json').exists() else None + nodes = [site['generation_node'], site['trainer_node'], *([site['teacher_node']] if 'teacher_node' in site else []), + *([site['generation_node_2']] if layout == 'h100x3' else [])] + command = ['salloc', f'--job-name={name}', f'--partition={args.partition}', f'--nodes={len(nodes)}', + f'--nodelist={",".join(nodes)}', '--exclusive', '--mem=0', + '--gres=gpu:8', f'--time={args.time}', 'python3', str(root/'node.py')] print(shlex.join(command), flush=True) if not args.submit: print('Dry run only. Add --submit to allocate GPUs.') @@ -32,12 +40,16 @@ def main(): previous = json.loads((root/'active-run.json').read_text()) if not (Path(previous['result'])/'end.json').is_file(): raise RuntimeError('The previous run has no terminal record. Inspect it before submitting again.') - required = [Path(site['base_model'])/'config.json', Path(site['teachers'])/'countdown/config.json', - Path(site['teachers'])/'graph_color/config.json'] - if manifest['framework'] == 'prime-rl': - required += [root/'source/.venv/bin/python', Path(site['base_model'])/'prime/.prime-v1', Path(site['uv'])] + sys.path.insert(0, str(root)) # The campaign's own copy of the recipe. + from shared.recipe import DOMAINS + required = [Path(site['base_model'])/'config.json', + *(Path(site['teachers'])/domain/'config.json' for domain in DOMAINS)] + if manifest['framework'] == 'nemo-rl': # Own container; scores through NeMo Gym, so no pydeps. + required += [Path(site['nemo_rl_container'])] + elif manifest['framework'] in ('prime-rl', 'verl'): # Host virtual environments. + required += [root/'pydeps/reasoning_gym', root/'source/.venv/bin/python', Path(site['uv'])] else: - required += [Path(site['container_image']), Path(site['megatron_model'])] + required += [root/'pydeps/reasoning_gym', Path(site['container_image']), Path(site['megatron_model'])] if manifest['framework'] == 'slime': required += [root/line.split()[1] for line in (root/'wheels.sha256').read_text().splitlines()] missing = [str(p) for p in required if not p.exists()] diff --git a/tools/verify_sources.py b/tools/verify_sources.py index 6c1e133..6b9ce0a 100644 --- a/tools/verify_sources.py +++ b/tools/verify_sources.py @@ -1,4 +1,10 @@ -"""Apply patches to pinned Git archives and check their recorded file hashes.""" +"""Check that each framework's source.patch reproduces the recorded source. + + python3 tools/verify_sources.py [--miles CLONE] [--prime-rl CLONE] [--slime CLONE] + +For each local upstream clone given, applies the patch to a `git archive` of the +pinned revision and compares every changed file with its recorded SHA-256. +""" import argparse import hashlib import io @@ -12,7 +18,7 @@ def verify(name, repository): - package = REPO/'frameworks'/name + package = REPO/'frameworks'/name/'upstream' manifest = json.loads((package/'manifest.json').read_text()) raw = subprocess.check_output(['git', '-C', str(repository), 'archive', manifest['revision']]) with tempfile.TemporaryDirectory(prefix='opd-source-check-') as temp: @@ -29,9 +35,9 @@ def verify(name, repository): def main(): - parser = argparse.ArgumentParser(description=__doc__) - for name in ('miles', 'prime-rl', 'slime'): - parser.add_argument('--'+name, type=Path) + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + for name in ('miles', 'prime-rl', 'slime'): # verl and NeMo-RL run unmodified. + parser.add_argument('--'+name, type=Path, metavar='CLONE', help=f'local {name} Git clone') args = vars(parser.parse_args()) if not any(args.values()): parser.error('Provide at least one local upstream Git clone')