Skip to content

AGKYLowW2019: initialise fForceNeuGenLimit (read ForceNeugenMultLimit) - #565

Open
AdamAurisano wants to merge 1 commit into
GENIE-MC:masterfrom
AdamAurisano:bugfix/agkylowW2019-forceneugenlimit
Open

AdamAurisano wants to merge 1 commit into
GENIE-MC:masterfrom
AdamAurisano:bugfix/agkylowW2019-forceneugenlimit

Conversation

@AdamAurisano

@AdamAurisano AdamAurisano commented Nov 16, 2026 •

Copy link
Copy Markdown

Problem

Runs with the same seed and configuration are not always reproducible. Depending on the heap byte left at the
member's address, a job's KNO
hadronization (AGKYLowW2019) caps the hadronic multiplicity at 10 for the whole job, so the events differ
from the first KNO event where the cap changes the drawn multiplicity onwards. This explains the rare
repeat-run divergence disclosed in #564; it is a bug in stock GENIE and unrelated to #562-#564.

Cause

AGKYLowW2019::MultiplicityProb() reads fForceNeuGenLimit
(src/Physics/Hadronization/AGKYLowW2019.cxx:410, if(fForceNeuGenLimit && maxmult>10) maxmult=10;),
but the member (declared at AGKYLowW2019.h:112) is set neither in the two constructors nor in
LoadConfig(). The GetParamDef("ForceNeugenMultLimit", fForceNeuGenLimit, false) read was dropped in the
2018 hadronization refactoring (366baff); the parameter is still documented in AGKYLowW2019.xml
(default false) but no configuration sets it. The flag therefore holds a stale heap byte at the object's
address, which changes between runs. Line numbers are those of master e2e04f1 (identical in R-3_06_00).

Instrumented repeat runs (numu Ar40, DIS only, 10k events, AR23_20i_00_000, fixed seed 1006) printed the raw
byte: non-zero in 3/36 runs with KeepLongLivedHyperons on (56, 176, 192) and in 3/16 runs with the stock
configuration (48, 96, 200). Exactly those runs diverged (at events 3847 and 4352 respectively, with KNO
pdfs of 9 instead of 10-11 bins); every zero-byte run was identical.

How often

The state depends on the build, environment and sample, and is stable for a given setup: in our repeat tests on a
development build, 3/36 and 3/16 jobs were capped; inside a fixed container image, the capped state was reproducible
run to run for some samples and absent for others (e.g. 41 of 64 multi-GeV generation blocks were capped). So it
can affect most jobs of a production, not just a few %.

Fix

Initialise fForceNeuGenLimit = false in both constructors and read ForceNeugenMultLimit (default false)
in LoadConfig(). This reproduces the zero-initialised (documented default false) behaviour; setting the
parameter to true reproduces the capped behaviour on purpose. fForceDecays (AGKYLowW2019.h:117) is never
set either, but it is never read (its uses are commented out), so it is left as it is.

No configuration in the tree sets ForceNeugenMultLimit as a <param>, so the default is what every
tune gets. Cross sections and splines are unaffected: KNOTunedQPMDISPXSec calls MultiplicityProb()
(cached path line 159, for W up to Wcut + 0.1; uncached path line 115, used only for W < Wcut), and with
Wcut <= 1.93 GeV in all tunes the maximum multiplicity there is at most 8, below the cap (the cap acts only
for W > M_n + 10 m_pi ~ 2.34 GeV).

Physics impact of a capped job

Measured in a separate analysis of 4M stock events (200 gevgen jobs x 20k, GENIE 3.06.00, AR23_20i_00_000): of DIS events with
2.34 < W < 3 GeV on the KNO path, 21 / 296,200 = 7.1e-5 have more than 10 primary hadrons. These are the
multiplicities the cap removes, so a capped job is statistically almost indistinguishable from a normal
one. Because the capped pdf is renormalised, a draw near a bin edge can also land one bin lower, and from
the first changed event onwards the random-number sequence differs, so a capped job differs event by event
from the uncapped job with the same seed. The visible effect is lost same-seed reproducibility rather than
a physics bias. (These numbers are for the AR23 tune; other tunes' KNO parameters give a different tail.)

Validation (master e2e04f1 + this commit, built in the GENIE 3.06 container toolchain)

Sample S6: numu Ar40, DIS only, 10k events, seed 1006. Comparisons are of the full GHEP event content
(all rows, all fields, event info).

  • Reference states: fixed build with the parameter absent or false = uncapped reference; fixed build with
    ForceNeugenMultLimit true = capped reference. Uncapped equals the earlier zero-byte runs (named
    anchor: stock run with byte 0); capped equals every earlier non-zero-byte run (bytes 48, 96, 200 with the
    stock configuration; 56, 176, 192 with hyperons on).
  • Stock master under MALLOC_PERTURB_ (85, 170, 85 with tcache off) = capped reference in all runs;
    MALLOC_PERTURB_=255 (allocations filled with 0) = uncapped. So the heap byte alone decides the state.
  • Fixed build under two MALLOC_PERTURB_ values (with and without tcache): identical to the unperturbed
    fixed output on S6 and on all-process numu and numubar Ar40 samples (10k each), for both option
    settings. No other uninitialised read showed up in these samples.
  • All 148 earlier stock-GENIE S6 runs (master and the Pythia 8 hadronization: optional hadronization history in the event record #562-Pythia 8 hadronization: option to leave long-lived hyperons undecayed #564 branches, several configurations) equal one
    of the two references: 138 uncapped, 10 capped, 0 unexplained.
  • Repeats: 36 (stock configuration) and 36 (with Pythia 8 hadronization: optional hadronization history in the event record #562-Pythia 8 hadronization: option to leave long-lived hyperons undecayed #564 and hyperons on) runs of the
    fixed build, all identical: 0 divergences.
  • No new compiler warnings (full build: same 56 warning lines as master). A G18 tune (G18_02a_00_000,
    500 CCDIS events with a small private spline) loads and runs; output identical to master.

🤖 Generated with Claude Code

https://claude.ai/code/session_01E3nCvREWPkWV3JvvxzenHN

AGKYLowW2019::MultiplicityProb() caps the KNO maximum multiplicity at 10
when fForceNeuGenLimit is set (AGKYLowW2019.cxx:410,
"if(fForceNeuGenLimit && maxmult>10) maxmult=10;"), but the member
(declared at AGKYLowW2019.h:112) is set neither in the constructors nor in
LoadConfig(): the GetParamDef("ForceNeugenMultLimit", ...) read was dropped
in the 2018 hadronization refactoring (366baff), although the parameter
is still documented in AGKYLowW2019.xml with default false.

The flag therefore holds whatever byte the heap has at the object's
address, and the heap layout differs between runs with the same seed.
In repeat runs of a numu Ar40 DIS-only sample (10k events, AR23_20i_00_000,
fixed seed) the byte was non-zero in 3 of 36 runs with the long-lived
hyperon option on (raw bytes 56, 176, 192) and in 3 of 16 runs with the
stock configuration (raw bytes 48, 96, 200). Exactly those runs diverged
from the others: the KNO multiplicity was capped at 10 for W > ~2.34 GeV
for the whole job. All runs with a zero byte were identical.

Initialise fForceNeuGenLimit to false in both constructors and read
ForceNeugenMultLimit (default false) in LoadConfig(). This reproduces the
zero-initialised (documented default false) behaviour; setting the
parameter to true gives the capped behaviour on purpose. Cross sections
are not affected: KNOTunedQPMDISPXSec calls MultiplicityProb() only for
W < Wcut + 0.1, where the maximum multiplicity is below 10.

fForceDecays (AGKYLowW2019.h:117) is never set either, but it is never
read (its uses are commented out), so it is left as it is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E3nCvREWPkWV3JvvxzenHN
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant