Repository navigation
Add CC-IN2P3 (Lyon) support to the grid production scripts - #557
Open
jtenavidal wants to merge 8 commits into
Open
jtenavidal wants to merge 8 commits into
jtenavidal wants to merge 8 commits into
Conversation
Add --grid-system LYON to the eA, vA, hadronA and XML grid submitters. Jobs are submitted to the CC-IN2P3 Slurm cluster through a generated lyon_slurm_submit.sh, where each production stage depends on the successful completion of the previous one (sbatch --dependency=afterok), replacing the FNAL jobsub DAG. - LyonGridUtils.py: job scripts running in the node local scratch, sbatch options and the stage-chained main submission script - setup_LYON.sh / setup_GENIE_LYON.sh: environment from an LCG view on CVMFS, with optional use of a prebuilt GENIE on /sps - Spline and event generation command builders gain LYON branches; FNAL output is unchanged - Fix XMLGridSubmitter using non-existent NuEMAXSPLINE/EEMAXSPLINE options - Document LYON usage in the README Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lyon jobs no longer clone and compile GENIE on the worker nodes, this is only done at FNAL. They use the compiled GENIE given by --genie-topdir (default: $GENIE). - The submitters abort for LYON if no GENIE top dir is given or it does not contain a compiled GENIE (bin/gevgen, bin/gmkspl), and warn if it is not in an area visible from the worker nodes - setup_GENIE_LYON.sh only sets up the environment of that build and defaults GALGCONF to $GENIE/config; LYON_GENIE_PREBUILT is removed - INCL tunes abort at LYON - Builders no longer fail at import time when $GENIE is not set - Update the README Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- compile_at_Lyon.sh: standalone script to compile GENIE at CC-IN2P3 from a given git location and version. The configuration follows the FSI model of the tune (hA/hN, INCL++ or Geant4). It checks the dependencies and saves the environment used, with the Pythia8 path, as <genie dir>/setup_LYON.sh - setup_LYON.sh: use the JUNO externals (ROOT, LHAPDF6, log4cpp) and Pythia8 from PYTHIA8PATH instead of the LCG view - Submitters: the jobs use the environment saved with the GENIE build when available, and the Lyon setup scripts are taken from the scripts directory, so that any GENIE build can be used - Document the compilation in the README Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Do not request local scratch with --tmp: the CC-IN2P3 Slurm
configuration rejects it ("Temporary disk specification can not be
satisfied"). --subjob-disk is not used at LYON
- lyon_slurm_submit.sh no longer uses set -e, which closed the login
shell when the script was sourced and a submission failed. On a
failed submission it now cancels the jobs already submitted and stops
- Update the README
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- At LYON the grouping jobs run gspladd directly in the spline directory, which is shared with the worker nodes, instead of copying the splines to the node. This also avoids copying the non-existent group_v*.xml file, which made the job fail. The intermediate per-target files are removed afterwards - Fix total_xsec.xml when several probes are requested: only the last probe of each target was added, and no total was produced for a single target. Single probe outputs are unchanged Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FastOnNuclei on a nucleus is the slowest spline job. By default its processes (CCQE, NCEL, CCRES, NCRES, CCDIS, NCDIS, CCDFR, NCDFR, CharmCCQE, LambdaCCQE, SingleKaon, NuEElastic, IMD) are now computed in separate jobs. Fast is still available through --vA-gen-list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only QE and DIS are slow on nuclei, so the rest of FastOnNuclei (RES, DFR, CharmCCQE, LambdaCCQE, SingleKaon, NuEElastic, IMD) is computed in a single FastOther job. gmkspl takes one event generator list, so this job runs gmkspl for each list and adds the splines with gspladd. Fast and the single-process jobs are still available through --vA-gen-list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--max-parallel-jobs N makes job k of each stage depend (afterany) on job k-N of the same stage, so that at most N jobs of a stage run at the same time. The limit can be changed when submitting with the MAX_PARALLEL_JOBS environment variable. The default, 0, keeps the previous behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add --grid-system LYON to the eA, vA, hadronA and XML grid submitters. Jobs are submitted to the CC-IN2P3 Slurm cluster through a generated lyon_slurm_submit.sh, where each production stage depends on the successful completion of the previous one (sbatch --dependency=afterok), replacing the FNAL jobsub DAG.