Skip to content

hyperloom custom workload: the result JSON names the binary but not the kernel or HSA runtime, so a number cannot be tied to a device stack #947

Closed

Description

@bong-water-water-bong

What

tools/hyperloom/custom/benchmark.py writes bench_summary.build = {onebit, bin_dir, sha256} (#933) but nothing about the device stack the measurement ran on. The pair-of-record artifacts of the long-context leg carry exactly these keys and no kernel field:

bench_summary: levers, longctx, placement, pp512_tok_s, runner_type, score_geomean, serve_tok_s, tg128_tok_s

(~/hyperloom-ws/window-20261102/arm_pair2_c20-{A,B}.json, the 14.5893 / 11.9825 tok/s pair.)

Why it matters

  • This box rebooted five times in 18 h on 2026-11-03/04 (journalctl --list-boots), and every long-context number of record is tied to its kernel by prose in ~/briefs/THYPERLOOM-STATE.md only.
  • The device-free window harness gates on the validated kernel (KERNEL_OK=7.3.0-rc4-1bit-perfopt+), but the tuner launcher has no kernel gate at all (grep -cE 'KERNEL|uname' hl-launch-26101.sh → 0), so a hyperloom optimize run on a test-kernel boot writes numbers into a report that cannot say which stack produced them.
  • placement does not cover it: it is null in those same JSONs (the counts only surface at the library's INFO level).

Same defect class as #932 (the report did not name the build), one field further out.

Fix

#: bench_summary.device_stack = {kernel, hsa_runtime, hsa_runtime_sha256} — uname -r, and the HSA runtime IREE_HAL_AMDGPU_LIBHSA_PATH names, whole-file hashed with sha256_head (the helper the build field already uses); hsa_runtime is null when there is no such file, so no number is attributed to a runtime it did not load.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions