pipe: track same-named concurrent batches; do not pipe sp jobs that reuse the opt output

[calvinp0]

Same-named pipe batches hid each other's results

PipeCoordinator.active_pipes was keyed by the pipe run_id. A second batch submitted under the same run_id (for example species_freq_batch) while the first was still running replaced the first entry. The first batch was then never polled or ingested, and its species were reported as not converged although every task had completed. In a 421-species run this dropped 402 completed frequency tasks.

• The registry, the poll-failure counters and the summary cache are now keyed by the unique pipe_root. The batch directories were already distinct (pipe_species_freq_batch_0, _1, …); only the key was not.
• The local-mode worker scratch directory is now the pipe root's path relative to calcs/ (e.g. Species/H2O/pipe_conf_opt_0, batches/pipe_species_freq_batch_1), which is unique for both per-species pipes and same-named cross-species batches.

Pipe mode ran sp jobs that reuse the opt output

When the sp level equals the opt level, run_sp_job does not run an sp job and parses the energy from the opt output. The deferred pipe sp flush sent every pending label to the pipe anyway, so those single points ran as batch jobs. The condition is now Scheduler.sp_reuses_opt_output, used by run_sp_job and by the flush; labels that reuse the opt output go through run_sp_job.

Tests

• Two same-named batches are tracked separately, and the first is ingested when it completes after the second was submitted.
• Local worker scratch directories differ between two species' per-species pipes and between two same-named cross-species batches.
• The pipe sp flush does not pipe labels that reuse the opt output; sp_reuses_opt_output matches the previous inline condition.
• Assertions that a finished pipe was removed from active_pipes now check its pipe_root.

arc/job/pipe/, arc/scheduler_pipe_test.py and arc/scheduler_test.py: 427 passed.

Searched before writing: the existing skip condition in run_sp_job (extracted, not duplicated) and every consumer of active_pipes in arc/.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.08%. Comparing base (a7c2666) to head (3619e02).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1060      +/-   ##
==========================================
+ Coverage   67.07%   67.08%   +0.01%     
==========================================
  Files         123      123              
  Lines       43663    43668       +5     
  Branches    11152    11152              
==========================================
+ Hits        29285    29293       +8     
+ Misses      11224    11223       -1     
+ Partials     3154     3152       -2     

Flag	Coverage Δ
functionaltests	67.08% <ø> (+0.01%)	⬆️
unittests	67.08% <ø> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.