Repository navigation
feat(skills): ml-features — build a labelled, leakage-tiered ML training set on a cluster - #485
Conversation
SKILL.md with the six-section contract, references for the inputs and for the spec and tiers, and a receipt template that checks a training set's four outputs agree and binds them to the TSV and sidecar digests.
A chromosome-22 slice of the ClinVar x heart example without its expression axes, from public inputs plus a dbNSFP file the user brings: stage.sh, a positive and a negative control, check.py (M1-M5, grading through the skill's receipt template) and a placeholder reference.json until the control writes it.
hvantk download clinvar --version YYYYMMDD built vcf_<build>/archive/<file>, a directory NCBI does not have, so every dated download returned 404. A dated release is in archive_2.0/<year>/ once archived, in weekly/, and at the top level while current: probe them in that order with HEAD requests and download the first that exists (its .md5 and .tbi from the same place). The docs' example date was not a release date.
The README's skill count and badge (13), the workflow-skills guide, the plugin descriptions and a see-also link on the training-set page. The training-set example README gives the real prepare command per axis (the loop passed one placeholder input to every axis), the two missing expression summaries and the ClinVar pin. The gene-index skill's annotation-axes reference and one trap are brought up to date with #484 (training-set sections, matrix entries keyed by gene_id, tier).
…ldcard nullglob leaves a literal file name in place when no such file exists, so the INSIDER interfaces table counted as staged before it was downloaded.
A stage made on another day differs from the reference's, and the control's input check then asks for a re-baseline; say why.
The UniProt PTM build downloads the pinned Ensembl GTF into its raw dir and, with no --intermediate, writes its parse output there too. Both broke the next control run's raw-dir check, and the GTF download fails where compute nodes have no internet. stage.sh now copies the GTF from the gene-index stage, and the control writes the intermediate under its own output.
- the spec input grep missed list items (- input:), finding 2 of 5 inputs; - UniProt PTM: copy the pinned Ensembl GTF where compute nodes have no internet, keep the parse intermediate out of the raw dir, note the snapshot date; - dbNSFP: make the one .bgz from the per-chromosome files without bgzip; - Farah: the UCSC dataset is hoc/all-heart, listed with --list-datasets.
Written by the positive control on a cluster at c50bada; a second control run without WRITE_REFERENCE passed both arms with the same TSV digest.
…clear error _locate() keeps the URL it finds, so verify_md5() called on its own fetches the .md5 from where the release is, not from the first candidate. A read timeout in the HEAD probe raised a bare TimeoutError past the CLI's error handling; any OSError now reports the URL it could not reach. The CLI test runs the real probe (a 404 in archive_2.0/, then weekly/) instead of patching it out.
- the dataset does not record the ClinVar release (source_fingerprint is the upstream's state at build time): record the file, its sha256sum and ##fileDate in the run notes, and compare those; - training-set build checks that each layer1 axis is in the matrix, not its columns: prepare and compose again after any layer1 edit; the wrong-groups error comes from annotate prepare, now a trap and in the description; - traps for a GRCh37 ClinVar VCF and for CLNSIG/CLNREVSTAT commas; - the receipt exits 1 on a feature column typed str; - the dbNSFP .bgz is written by one pysam job, not a login-node cat; - eval: the ClinVar file is taken by version, a failed curl stops staging, raw/ and upstream/ are left read-only, the negative spec is generated from the staged one (spec_negative.yaml removed), stale comments fixed; - plugin descriptions: a double 'and'.
…name The login node's python3 can be too old to run hvantk/resources/ensembl_release.py.
builder_commit was always null on training-set sidecars, while every input dataset records it through resolve_builder_commit(); use the same function.
- copy the spec and gene lists to $OUT first; stage ClinVar with --verify-md5 and keep the VCF; how to find a release date; read ##fileDate with zgrep (no SIGPIPE under pipefail); - the dbNSFP .bgz job stops when no file matches and renames only when complete; the per-chromosome headers must be identical, with the check; - --hgnc for every uniprot_id- or symbol-keyed axis (farah and hca too); - expression summarize loads the whole .h5ad; Expression Atlas needs FTP; the UCSC download cancels on a non-empty directory; - the GRCh38 trap is for position-keyed inputs; the provenance loop also finds .h5ad sidecars.
…sion A job on a node without git records hvantk <version>; the commit is then only in the run notes.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
|
ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing |
Not up to standards ⛔🔴 Issues
|
| Category | Results |
|---|---|
| Documentation | 25 minor |
| Security | 2 minor 1 critical 3 medium |
| Complexity | 5 medium |
🟢 Metrics 174 complexity
Metric Results Complexity 174
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
|
Codacy triage (202521c):
|
Summary
Adds the
ml-featuresworkflow skill. It tells a coding agent how to build a labelled, leakage-tiered ML training set withhvantk training-set build(#484) on a Slurm cluster:Its references say how to adapt the spec, which tier to declare, and how to pick training columns free of label or allele-frequency leakage.
Running the skill's eval control found a toolkit bug, fixed here:
hvantk download clinvar --version YYYYMMDDfailed for every date.Changes
skills/ml-features/, followingskills/workflow-conventions/SKILL.md:SKILL.md, with the six sections;stage: analysis,scale: node;references/inputs.md: where each input of the example spec comes from and how it is built, including the two with no downloader (dbNSFP and the INSIDER variants BED). It also covers the dbNSFP rankscore-type check, making dbNSFP's single.bgzwithoutbgzip, and the UniProt PTM build's Ensembl GTF on clusters whose compute nodes have no internet;references/spec-and-tiers.md: adapting the spec, declaring tiers, and the four training arms with the selection code;templates/describe-training-set.py: the receipt. It exits 1 when the.ht, the TSV, the dictionary and the manifest disagree, or when a feature column is typedstr(a dbNSFP built by an older checkout passes text through without any error).skills/ml-features/eval/is a chromosome-22 slice of the ClinVar x heart example without its three expression axes:stage.shstages public inputs plus a dbNSFP file the user supplies, which is academic-licensed with no download URL. UniProt PTM is a live snapshot with no archived releases, so the reference records a digest of the staged inputs and the control asks for a re-baseline when they differ;control.shruns a positive and a negative control. The negative uses the staged spec withmin_review_stars: 0.raw/andupstream/are left read-only, so a run under test cannot change the stage;check.pygrades M1-M5 through the skill's own receipt template;reference.jsonwas written by the control on a cluster.vcf_<build>/archive/, a directory NCBI doesn't have, so every dated download returned 404. A dated release now resolves toarchive_2.0/<year>/,weekly/, or the top level while it is current. The candidates are probed with HEAD requests, and the.md5and.tbicome from the same place, also whenverify_md5()is called on its own. A timeout while probing is reported with the URL. The docs' example date was not a release date.builder_commitempty. It now records the commit that built it, through the sameresolve_builder_commit()every dataset build uses.skills/README.mdcatalog row and the README skill count and badge (12 → 13);preparecommand for each axis. The old loop passed one placeholder input to every axis. The steps also add the two missing expression summaries and say to pin ClinVar.references/annotation-axes.mdand one trap were stale since feat: hvantk training-set build — labelled, tiered ML training datasets #484. They now cover the training-set sections,matrix:entries keyed bygene_id(version suffixes dropped), andtier.Verification
Skill tests:
test_workflow_skill_conformance.pyandtest_no_site_data_in_skills.pypass, including the catalog/badge check.Eval on a cluster: the control ran four times: once with
WRITE_REFERENCE=1, then three times without it, the last two after the review fixes.content_okfalse,reported_oktrue).Receipt template:
ClinVar downloads:
--version 20261108(the current release) downloads fromweekly/, and the MD5 matches;20261004resolves toarchive_2.0/2026/;archive_2.0/falling through toweekly/for both the VCF and its index.Skill tested with agents: two fresh agents were given realistic tasks without the skill, then again with it. Without it, both downloaded ClinVar
latest, guessed job sizes 4-25x too large, and wrote ad-hoc verification scripts. With the skill, both pinned ClinVar, used the measured sizes and the receipt template, and B gave the four training arms with gene-grouped folds. They also found four gaps inreferences/inputs.md, all checked and fixed here:- input:list items;hvantk download ucsc --searchneeds--list-datasets;bgzipfor dbNSFP.After the whole-branch review's fixes, a third fresh agent re-ran scenario A. It passed every checklist item, and its refinements were checked against the code and fixed:
--hgncfor the symbol-keyed axes;--verify-md5;Whole-branch review by a fresh reviewer: no critical findings. Its important and minor findings are fixed here, apart from two toolkit checks left to the procedure (the build does not compare a matrix axis's columns with the spec, and the ClinVar dataset does not record its release; the skill says both).
Test suite (CI's selection,
pytest --require-round-trips): 3,592 passed. Four pqtl log-capture tests fail in a full local run; they pass when run alone and in CI, and fail the same way ondev.