Skip to content

Compare profiles along an axis in cosine and pearson - #73

Draft
smcolby wants to merge 1 commit into
pnnl:masterfrom
smcolby:rowwise-similarity
Draft

smcolby wants to merge 1 commit into
pnnl:masterfrom
smcolby:rowwise-similarity

Conversation

@smcolby

@smcolby smcolby commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

Description

deimos.deconvolution.cosine and deimos.deconvolution.pearson flatten their inputs and return one distance. Scoring many extracted-ion chromatograms against one elution profile, as a co-elution test does, then needs a Python loop over rows, so code with that need writes a separate row-wise correlation.

This PR adds an optional axis to cosine and pearson:

  • Without axis (the default), the functions flatten their inputs and return a float, unchanged, so MS2Deconvolution.apply and existing callers give the same results.
  • With axis, the arrays are broadcast together and compared along that axis, and the result has one distance per position of the other axes. pearson centers each array on its mean along the same axis before comparing, so each row of a 2D array is centered separately. A 2D array of profiles against a 1D profile with axis=-1 gives one distance per row.
  • A vector that is all zero (cosine) or constant (pearson) gives NaN with a RuntimeWarning, as the flattened form already does.

Verification

  • New tests in tests/test_deconvolution.py:
    • Hand-computed row distances for cosine along the last axis.
    • Hand-computed pearson row distances, including rows offset from each other, and NaN for a constant row.
    • On a random 3 x 4 x 5 array, for each of axis 0, 1, and -1 and each function, every entry equals a separate flattened call on the corresponding vector.
  • python -m pytest: 391 tests, 5 failing, the same 5 that fail on master in this environment (pandas 3.0.3, NumPy 2.5.2). They are get_1D_profiles and apply cases where this environment's SciPy raises _dfitpack.error for a spline with too few points. ruff check and ruff format --check pass, and pyright reports the same 7 errors in the two changed files as on master. I have not run the suite under pandas 2.3.3, the version CI installs.
  • On smoothed extracted-ion chromatograms from an LC-MS run (200 windows of 60 scans, 40 targets each, 8,000 rows of which 1,525 are flat), 1 - pearson(rows, profile, axis=-1) differs from a direct row-wise Pearson correlation ((a @ b) / sqrt(sum(a**2) * sum(b**2)) on centered rows) by at most 4.4e-16, once flat rows are mapped from NaN to the direct form's 0. No row crosses a 0.8 correlation threshold between the two.

Notes for reviewers

  • cosine and pearson are general similarity measures but live in deimos.deconvolution. I left them there. Moving them to a general module with aliases at the old names would be a separate change.
  • The axis form sums elementwise products with NumPy reductions, while the flattened form uses np.dot. The two can differ in the last bit of a distance.

No related issue.

Quality assurance and AI policy

To maintain project quality and respect maintainer bandwidth, please confirm the following about this pull request (PR):

  • Manual verification: I have manually reviewed and tested the code in this PR, and python -m pytest passes locally.
  • AI-assisted content: If artificial intelligence (AI) tools (for example, GitHub Copilot or ChatGPT) were used, I have personally verified the logic, edge cases, and compliance with the existing codebase. I confirm the code is not a "blind" AI generation.
  • Minimal review: I believe this PR is in a state that requires minimal intervention or correction from maintainers.
  • Scoped change: This PR addresses a single, well-scoped issue rather than multiple unrelated changes.

Status

  • Ready to go (checking this signals to maintainers that the PR is ready for final review)

Developer Certificate of Origin


Note to contributors: We reserve the right to close PRs without review if they appear to lack human validation or do not meet the quality standards described in our contributing guide.

cosine and pearson flatten their inputs and return one distance, so scoring many chromatograms against one profile takes a Python loop. An optional axis now broadcasts the arrays and compares them along it, returning one distance per position of the other axes, and pearson centers each array along the same axis. Without axis the functions flatten and return a float, as before.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant