Skip to content

External type generalisation - #247

Draft
graeme-a-stewart wants to merge 38 commits into
JuliaHEP:mainfrom
graeme-a-stewart:external-type-generalisation
Draft

graeme-a-stewart wants to merge 38 commits into
JuliaHEP:mainfrom
graeme-a-stewart:external-type-generalisation

Conversation

@graeme-a-stewart

Copy link
Copy Markdown
Member

This PR adds support so that the underlying numerical type of jets becomes parameterised, instead of being fixed to Float64. The parameterisation is T <: Real, which includes all float types and dual numbers (for potential autodiff applications).

One exception to parametrisation is the TiledJet structure used in the N2Tiled algorithm. This remains fixed to Float64, due to the use of a concrete singleton (noTiledJet) as the sentinel for linked lists (and to indicate tiles with no jets). As discussed in #91, this is tricky to optimise (I have a branch with a proof of concept that TiledJet can be parameterised, but it is not performance optimised as it allocates too many Ref{TiledJet}s).

Main Change Summary

Performance Report

The main optimisation steps of ensuring type consistency in mathematical operations have been done:

  • Internal numerical values in functions are typed as T
  • Function signatures for internal operations are uniformly typed to T when appropriate, to ensure that mixed parameters are not passed by accident

There has been some observation of a slight performance regression on pp reconstruction, but latest measurements show this as minimal O(1%) and not very consistent. Degradations also seem to be platform and phase of the moon (time) dependent, indicating no serious issue with the implementation.

Parametrisation

EEJet, PseudoJet and EERecoJet are now parameterised on T <: Real (also the common abstract FourMomentum type). The number of constructors is considerably expanded to cover cases when initialisation is explicit as well as implicit, and including mixed types, where promotion is applied. Conversion from LorentzVector, LorentzVectorCyl and anything supporting LorentzVectorBase are also covered. If the EDM4hep extension is loaded, construction from EDM4hep.ReconstructedParticle is supported.

ClusterSequence, HistoryElement and SoftKiller are likewise parameterised and will be initialised to the same T as the input particles.

For consistency T is used as the numerical type of the jet's parameters, and J is used for the jet type.

Invalid Values

When an invalid jet is returned from substructure functions, this is no longer a singleton (which cannot be parameterised), but is a PseudoJet{T} where isvalid(jet) == false. Technically this is implemented by setting the _cluster_hist_index to typemin(Int), but this is an internal detail - use isvalid()!

Reconstruction Interfaces

The common jet_reconstruction interface is left unchanged, but in the algorithm specific entry functions (ee_genkt_algorithm, plain_jet_reconstruct, tiled_jet_reconstruct) the numerical type of the input particle vector is interrogated to pick the appropriate jet type conversion. This code is also factorised, avoiding duplication (construct_reco_jets).

Jet Utility Interfaces

For substructure, soft killer and Lund plane, all interfaces are updated to parameterised jet types.

Robustness

Absurd R values (<=0 or >2pi) throws an error. Attempting reconstruction with no inputs returns an empty ClusterSequence.

Tests

Tests are updated to use parameterised values, when required. test-reco-numtypes.jl tests that reconstruction works for Float32 and Float16.

Examples

The instrumented-jetreco.jl and benchmark.sh scripts can now handle a numerical type parameter, allowing non-Float64 reconstruction to be tested (--numtype argument, second positional parameter respectively). Other examples are updated to adapt to parameterised types.

JetReconstruction Package Version

This change is quite intrusive and potentially breaks client code when types become important to specify (e.g., see the fixes in the various example code scripts). This PR should therefore be considered a major change, targeting v2.0.0.

Files Modified

Source Files (31 changed)

  • src/JetReconstruction.jl: Module file updated for new type parameters
  • src/CommonJet.jl: Base FourMomentum{T} type and utilities
  • src/PseudoJet.jl: Complete parametrisation of pp jet type
  • src/EEJet.jl: Complete parametrisation of e⁺e⁻ jet type
  • src/JetUtils.jl: Updated utility functions for parameterised types
  • src/AlgorithmStrategyEnums.jl: Minor updates for type consistency
  • src/ClusterSequence.jl: Parameterised history and cluster sequence types
  • src/PlainAlgo.jl: N2Plain strategy with type parameters
  • src/TiledAlgoLL.jl: N2Tiled strategy with type parameters
  • src/TiledAlgoUtils.jl: Tiled algorithm utilities updated
  • src/TiledAlgoLLStructs.jl: Struct definitions updated
  • src/GenericAlgo.jl: Main reconstruction entry point updated
  • src/EEAlgorithm.jl: e⁺e⁻ algorithms updated
  • src/Substructure.jl: All substructure functions updated
  • src/LundPlane.jl: Lund plane generation updated
  • src/SoftKiller.jl: Pileup mitigation updated
  • src/HepMC3.jl: HepMC3 reader updated
  • src/Utils.jl: Utility functions updated
  • src/JSONresults.jl: JSON output types updated

Documentation Files

  • README.md: Updated example to show explicit type parameter
  • docs/src/substructure.md: Minor updates for type signatures

Test Files

  • test/test-reco-numtypes.jl: New file with numerical type tests
  • test/_common.jl: Updated for new types
  • test/runtests.jl: Updated test runner
  • All existing test files updated for parameterised types

Example Files

  • examples/benchmark.sh: Added optional numtype as positional parameter
  • examples/instrumented-jetreco.jl: Added --numtype option
  • examples/constituents/jetreco-constituents.jl: Pass explicit numerical type
  • examples/constituents/jetreco-constituents.jl: Use own copy of JetReconstruction
  • examples/softkiller/softkiller_plots.jl: Use explicit numerical type
  • examples/softkiller/softkiller_runtime.jl: Use explicit numerical type
  • examples/visualisation/visualise-jets-nb.jl: Use own copy of JetReconstruction

Working parameterised reconstruction for pp with tiled algorithm.
These are now properly parameterised
With parameterised jet and cluster sequence types
We don't use underscores in any of the other test files
Improve type concretiser for jet selectors, when a user can pass an
un-parametrised type, and this will be concretised.

Fix parametrisation for jet preprocessors and adders.

Further parametrised constructors for PseudoJet, from (pt, rap, phi).

Fix softkiller function signatures for default types.

Fix HepMC3 reader to parameterise return types.

Read test files with set parametrised types, ensure tests use
parameters are needed.
Invalid PseudoJets are constructed now as zero(PseudoJet), which sets
the cluster_hist_index to typemin(Int).

Tests for validity specifically probe this field value.
FourMomentum, EEJet, PseudoJet are updated to take a numerical
parametrisation type, T <: Real.

Constructors are added to ensure that concrete jets can be constructed
without an explicit T (taking T from the input values) or with an
explicit T.

eltype() is defined to return the numerical type.

A useful helper function, construct_reco_jets() is added that converts
input particles into the starting "jets" for reconstruction, that
avoids significant code duplication in all of the reco algorithm
entry points.
Significant changes to correctly propagate parametrised numerical
type though the reconstuction algorithms.

Documentation is updated.
Fix a few small documentation errors.

Add some sanity checks in the generic reconstruction interface,
checking that we do have particles (disallow passing a zero length
vector) and that the R value is sane.
Allow for reconstruction to be run with Float16, Float32 or Float64.
Ensure that type T (the numerical type of the EEJet) is used at all
points in the reconstruction, including functional calls.

Parameterise functions to T to ensure that mismatched types do not get
used by accident.

Reduces regression by about 40%.
- pre this commit, reco time 35us
- post this commit, reco time 29us
- pre-parametrisation, reco time 21us
Ensure that all parameters are made consistent with the numerical type
of the underlying PseudoJet.

Add strict type specifiers to all internal functions, preventing
accidentally calling them with mismatched numerical types.
Also, reran formatting
Adapt unit tests to new paramaterisation.

Remove some unnecessarily harsh parameter typing and instead apply
conversions as needed.

Remove bogus "optimisation" for type of "p" in insert_new_jet!().
Also prettify output
Had not been parameterised correctly (fixes visualisation)
Ensure that jets of a parameterised numerical type can be constructed
correctly from an EDM4hep ReconstructedParticle. Support explicit
type declaration and taking the type from the ReconstructedParticle
(usually Float32).
@graeme-a-stewart
graeme-a-stewart force-pushed the external-type-generalisation branch from 4a63d7c to 1e058fa Compare June 11, 2026 19:27
@codecov

codecov Bot commented Jun 11, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.62500% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.36%. Comparing base (9bbdb71) to head (8a26d0b).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
src/ClusterSequence.jl 80.64% 6 Missing ⚠️
src/Utils.jl 78.26% 5 Missing ⚠️
ext/EDM4hepJets.jl 55.55% 4 Missing ⚠️
src/GenericAlgo.jl 71.42% 2 Missing ⚠️
src/JetUtils.jl 90.00% 2 Missing ⚠️
src/SoftKiller.jl 87.50% 1 Missing ⚠️
src/Substructure.jl 88.88% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #247      +/-   ##
==========================================
+ Coverage   82.94%   83.36%   +0.41%     
==========================================
  Files          23       23              
  Lines        1431     1485      +54     
==========================================
+ Hits         1187     1238      +51     
- Misses        244      247       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Important to preserve the "develop" path for JetReconstruction as a
relative one.
@graeme-a-stewart graeme-a-stewart added this to the 2.0.0 Release milestone Jun 12, 2026
@graeme-a-stewart graeme-a-stewart added enhancement New feature or request interface change Breaking changes to any public interfaces visualisation Visualisation of the reconstruction examples Related to example codes tests Related to testing the code labels Jun 12, 2026
@graeme-a-stewart
graeme-a-stewart requested a review from m-fila June 12, 2026 11:10
Comment thread src/JetUtils.jl
end

if isnothing(preprocess)
if P == J

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should compare the element type instead:

eltype(particles) == TargetJetType

With the current signature, P is the vector type, while J is the jet type.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request examples Related to example codes interface change Breaking changes to any public interfaces tests Related to testing the code visualisation Visualisation of the reconstruction

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants