Repository navigation
n2plain-improved-scaling-on-multithread - #256
HarshitNagpal29 wants to merge 9 commits into
Conversation
67664a9 to
cb4e276
Compare
|
Rebased into current head of main |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #256 +/- ##
==========================================
+ Coverage 83.14% 83.89% +0.74%
==========================================
Files 24 25 +1
Lines 1614 1720 +106
==========================================
+ Hits 1342 1443 +101
- Misses 272 277 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-authored-by: Mateusz Jakub Fila <37295697+m-fila@users.noreply.github.com>
| ordinary, single-threaded reconstruction: their allocation overhead is small, | ||
| and the returned result can be retained freely. | ||
| The ordinary `jet_reconstruct`, `plain_jet_reconstruct`, and | ||
| `tiled_jet_reconstruct` interfaces return an independently owned |
There was a problem hiding this comment.
Mention also ee_genkt_algorithm as well (gosh, those names look rather inconsistent now!)
| pattern is available through `N2PlainWorkspace`: | ||
|
|
||
| ```julia | ||
| workspace = N2PlainWorkspace(PseudoJet) |
There was a problem hiding this comment.
See comment below re. how to parameterise this on algorithm, not jet type
| end | ||
| ``` | ||
|
|
||
| `N2PlainWorkspace(EEJet)` provides the same reusable storage for e⁺e⁻ |
There was a problem hiding this comment.
See comment below re. how to parameterise this on algorithm, not jet type
| preprocess = preprocess_escheme, | ||
| γ::Union{Real, Nothing} = nothing, | ||
| β::Union{Real, Nothing} = nothing) | ||
| is_pp(algorithm) || |
There was a problem hiding this comment.
How could this error occur? _n2plain_reconstruct_with_workspace! is an internal function, not to be called by users.
| is_pp(algorithm) || | ||
| throw(ArgumentError("algorithm $algorithm requires an EEJet N2PlainWorkspace")) | ||
| isnothing(β) || | ||
| throw(ArgumentError("β is only supported by the Valencia algorithm")) |
There was a problem hiding this comment.
So here we have a parameter that had to be nothing, so then why does it exist as a parameter at all?
|
|
||
| Reusable full-semantics storage for one N2Plain reconstruction worker. | ||
|
|
||
| `J` selects the reconstruction family: use `PseudoJet` for pp algorithms and |
There was a problem hiding this comment.
OK, this is wrong - we should not parameterise on the jet type - a user could be working with any old jet type from their EDM, or, e.g., anything that supports LorentzVectorBase. PseudoJet and EEJet are not mandatory, we transform as needed.
Much better is to parameterise on the algorithm, which the user does know. I will make another comment re. how to do this generically.
|
Great, thanks @HarshitNagpal29! My main technical comment here is that we should use the algorithm to parameterise the workspace, not the jet type. One thing that seems to be missing, we have I would also say we should:
However, perhaps those last points can be a separate issue/PR, to keep this one clean and focused. |
graeme-a-stewart
left a comment
There was a problem hiding this comment.
Hello @HarshitNagpal29 - thanks for the changes.
Looking at this again after a few weeks I really don't like using the name N2PlainWorkspace as it ties much too closely to pp N2Plain. It's just confusing to use that for an EE algorithm. My suggestion is to switch to PlainWorkspace.
The source code organisation is also muddled now, with functions appearing in the structs source files, and EE things appearing in N2PlainStucts, which is wrong. I have made a suggestion to add a Workspace.jl for common workspace definitions and to move functions back to their natural source files. (Aside: if there's an ordering problem, and I think there might be, then add further source files, e.g., EEWorkspace.jl and N2PlainWorkspace.jl that can be included after the necessary definitions are made.)
It is worth reflecting on the consistency for the N2TiledWorkspace as well now, e.g., it would now be better named TiledWorkspace in light of the above.
| owning interface instead. | ||
|
|
||
| For pp events where the N2Plain strategy is appropriate, the same workspace | ||
| pattern is available through `N2PlainWorkspace`: |
There was a problem hiding this comment.
Rename to PlainWorkspace - see below
| [`with_ee_reconstruction`](@ref). The returned sequence borrows | ||
| `workspace.jets` and `workspace.history`. | ||
| """ | ||
| function _n2plain_reconstruct_with_workspace!(workspace::N2PlainWorkspace{A, EEJet}, |
There was a problem hiding this comment.
This name is misleading - it should probably be _reconstruct_with_workspace(), using argument dispatch to pick this specialisation.
|
|
||
| # Maps index from the compact array to the clusterseq jet vector | ||
| clusterseq_index::Vector{Int} = collect(1:N) | ||
| local kt2_array::Vector{Float64} |
There was a problem hiding this comment.
Remove the local keyword. We should only use that when it's really needed - we rightly don't use global variables that would accidentally cause confusion in functions.
| This is the pp implementation behind [`with_n2plain_reconstruction`](@ref). | ||
| The returned sequence borrows `workspace.jets` and `workspace.history`. | ||
| """ | ||
| function _n2plain_reconstruct_with_workspace!(workspace::N2PlainWorkspace{A, PseudoJet, |
There was a problem hiding this comment.
As above, I think _reconstruct_with_workspace() would do the trick here (even though it's less misleading in this context to use n2plain).
| and is overwritten by its next reconstruction. Copy values that must outlive | ||
| that operation. A workspace must not be used concurrently or reentrantly. | ||
| """ | ||
| mutable struct N2PlainWorkspace{A, J, S} |
There was a problem hiding this comment.
As this workspace can serve for pp N2Plain and the ee reconstructions it's just incorrect to have it live in PlainAlgoStructs which are tied to the N2Plain strategy.
I'd suggest that this moves to a new source file Workspace.jl.
I would also rename it as PlainWorkspace, to avoid N2Plain in the name, which is just going to cause some confusion to the user when it's used with e+e-.
| throw(ArgumentError("Unsupported jet algorithm: $algorithm")) | ||
| end | ||
|
|
||
| function _prepare_n2plain_recombination_jets!(jets::Vector{J}, |
| return jets | ||
| end | ||
|
|
||
| function prepare_n2plain_recombination_jets!(workspace::N2PlainWorkspace, |
| concurrently executing worker must own a distinct workspace, and the caller | ||
| must prevent concurrent or reentrant use of the same workspace. | ||
| """ | ||
| function with_n2plain_reconstruction(f::F, |
There was a problem hiding this comment.
This function should move to PlainAlgo.jl - it's a reconstruction entry point, not a stuct
| the next reconstruction using this workspace. Copy values that must outlive | ||
| that operation, and do not use one workspace concurrently or reentrantly. | ||
| """ | ||
| function with_ee_reconstruction(f::F, |
There was a problem hiding this comment.
This function is particularly bad to have in PlainAlgoStructs.jl should be in EEAlgorithm.jl
| @@ -0,0 +1,371 @@ | |||
| # Regression tests for reusable N2Plain reconstruction storage. | |||
There was a problem hiding this comment.
Bad name - how about test-plain-workspace.jl?
Hello @graeme-a-stewart Sir, Ok I will try to apply this asap. 😊 |
Summary
This PR adds an opt-in reusable workspace for full N2Plain reconstruction, covering both hadron-collider and electron-positron algorithms.
The workspace reduces repeated per-event allocations by reusing reconstruction scratch arrays, jet storage, and clustering history while preserving the complete
ClusterSequencesemantics.Changes
N2PlainWorkspacewith support for:PseudoJetstorage for pp algorithms;EEJetstorage for e⁺e⁻ algorithms.with_n2plain_reconstruction.StructArrayreconstruction storage.inclusive_jets!(output, clusterseq)Compatibility
This is an additive API change. Existing users continue receiving independently owned results. High-throughput applications can opt into reusable storage and copy only the values that must be retained.