Skip to content

hyperloom custom workload: a result does not record the box conditions it was measured in, so a contaminated arm and a clean one are the same JSON #996

Closed

Description

@bong-water-water-bong

Class: the artifact does not describe the measurement (same as #892, #930, #932, #947, #954, #977, #993).

A score is measured on a shared box, and the result JSON records nothing about the box: not the die temperature it ran at,
not the GTT in use by other tenants, not the memory pressure. Two arms of the same workload at the same configuration
therefore produce the same-shaped artifact while differing by 40 % — and this lane has produced exactly that pair:

arm date what the box was doing score
baseline of record 2026-11-01 23:07 quiet die (~44 C), 0 swap 44.69
"baseline" 2026-11-01 00:20 85 GB/122 GB in use, 37 GB of swap in play, die climbing 31.68 — retracted
long-context baseline 2026-11-02 16:33 die 56 C at the claim, quiet 12.94
long-context arm 2026-11-01 15:35 die 54 C at the claim, another lane loading the die aborted at 93 C after 108 s

Both of the first two passed the quality gate (KLD 0, texts identical) — the gate is a correctness gate and says
nothing about whether the measurement is comparable. The 31.68 arm was quotable in the artifact sense and wrong in
substance; it was only retracted because the same lane happened to hold the raw logs. The 15:35 arm produced no
measurement at all, and its reason (ABORT die at 93C) lives in a scriptable_stderr.log, not in any result.

The inputs are cheap and already exposed by this box, device-free to read:

/sys/class/hwmon/hwmon2/temp1_input          67750   label=Tctl
/sys/class/hwmon/hwmon4/temp1_input          68000   label=edge
/sys/class/drm/card0/device/mem_info_gtt_used  46098116648  (of mem_info_gtt_total 128849018880)

Wanted: every result records the conditions at the moment of measurement — die temperature (min/max over the arm, and at
start), GTT used by the whole box at start, and free system memory at start — with each field null on a box that does not
expose it, so a later reader can tell a clean arm from a contaminated one, and a report from a thermal abort from a report.

Not owner-gated, just sequenced: this lane already has three open PRs (#974, #978, #994) that all add a block to the same
test function, so the fourth branch is held until they merge rather than making the owner's merge order collide four ways.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions