Repository navigation
Conversation
…replace_splines.py - construct_representative_spline.py: an exact tune entry now wins over the prefix rules; G18_10a-d use NievesQELCCPXSec/Dipole (not LwlynSmith); add AR23/AR25 20i sub-tunes, 20m, 22i/22m (CRPA), G24_12a (Martini), GEM26 (UnifiedQEL, proc EM,QES, --nucleon); unknown tunes stop with a message instead of a KeyError. GP: scale alpha for normalize_y and put a floor of 0.3 on the RBF length scale so the fit doesn't follow the noise. - average_splines.py: keep every non-free-nucleon spline (one parquet each, for electron jobs with N:2112 and N:2212); mean over runs, zeros included, instead of the median; check that knot energies match across runs. - replace_splines.py: rewritten to stream the base file line by line (.xml or .gz), take many revision files (-r ... / -l list), set nknots, and fail (no output) on unmatched, empty, duplicated or other-tune splines.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
These scripts average seeded QE spline runs, fit a smooth representative spline,
and put it back into the full spline file. Using them in a full production
(all six flavors, ~125 isotopes, 100–1000 seeds per pair, several tunes)
turned up the problems fixed here.
construct_representative_spline.py
by name, so a wrong model name means it is never used. An exact
modelsentryis now checked first: before, it was checked last, so entries such as
G18_10a_02_11bwere never reached. G18_10a/b/c/d now getNievesQELCCPXSec/Dipole, which is what theirModelConfiguration.xmluses,instead of
LwlynSmithQELCCPXSec/Dipole. Added, each taken from that tune'sModelConfiguration.xml:01_002,02_001), N24_20i_02_11b01_000/001/002sub-tunesHybridXSecAlgorithm/CRPA-QEL)HybridXSecAlgorithm/Martini-QEL)UnifiedQELPXSec/Dipole)KeyError.EM,QES, and the struck nucleon is givenwith
--nucleon 2112|2212, since electron jobs have one spline per nucleon.normalize_y=True, sklearn divides the targets by their standarddeviation but not
alpha, so the noise term was in the wrong units. It isnow
(err / std(mean))**2, with a floor so thaterr == 0can't break it.of a 250-knot, 0.01–1000 GeV grid). Without it the optimiser sometimes
settled on a length scale of a few knots and followed the noise.
average_splines.py
not only the first one, matched by name. With one spline the output is as
before; with several (electron jobs) there is one parquet per spline,
<outbase>_nu<pdg>_tgt<pdg>_N<pdg>.parquet.included. With one nucleon throw per run, many runs are exactly zero near
threshold. The median there is 0, or biased low, while the mean is the
unbiased estimate of the cross section.
missing a spline are reported and left out of its mean.
replace_splines.py
The old version parsed the whole base file into ElementTree. For a full neutrino
file (1.3 GB uncompressed) that is far more than 4 GB of memory, enough to stall
a grid job. It also took one revision file per run, re-parsing the base file
each time. It skipped any unmatched spline silently, and left
nknotsunchanged, although GENIE reads exactly
nknotsknots. The rewrite:.xmlor.xml.gz; output gzipped if-oends in
.gz)-r f1.xml f2.xml ...and/or-l list.The old one-file
-b/-r/-ousage still worksnknotsto the new knot count and prints a summary (replaced, notfound, knot count changed)
(
--allow-missingto accept that), has no knots, is in two revision files,or is for another tune (
--ignore-tune)It drops the unused
modelsdict and the commented-out renaming code.Testing
with a stand-in revision spline, it ran in about 14 s. Its output was
identical, line for line, to an independent awk implementation of the same
replacement. The not-found, wrong-tune and changed-knot-count cases behave
as described.
AR23_20i, AR25_20i_01_002, G18_02a and GEM26 productions. The smooth
splines are 4–300× smoother than the standard run away from threshold, and
differ from it by about the standard run's noise.
Note
Martini-QEL, CRPA-QEL and SuSAv2-QEL are integrated with
MECXSec(VEGAS overq0, q3), not with
NewQELXSec's nucleon throws. So the seed andNumNucleonThrowsdon't change them: every seeded run gives the same spline(checked for Martini-QEL), and averaging seeds does nothing for G24_12a,
AR23/AR25 22i/22m and G21_11*. Their model names are added anyway, so the
scripts don't stop on those tunes.