[smcolby]

Description

deimos.slice and deimos.locate check whether the searched column is sorted and copy the searched columns on every call. On a 27.5-million-row LC-MS centroid table, that overhead is about 135 ms per call before any row is selected, so code that searches one large table thousands of times (one extracted-ion chromatogram per target, one lookup per peak) builds sorted NumPy arrays and searches them with np.searchsorted instead of calling deimos.

This PR adds deimos.subset.SortedFeatures, which sorts a table by one dimension once and stores its columns as NumPy arrays. Each lookup is then a binary search plus the rows it returns.

• window(low, high) returns the first position and one past the last position of the sorted rows within inclusive bounds, for one window or an array of them. Bounds are compared in float64, as slice compares them, including on float32 columns: float32(0.1) is outside [0.1, 0.1] under slice and under window.
• window_around(loc, tol, relative) gives the same windows from locations and a tolerance, with the bounds locate_asym computes (x * (1 - tol) and x * (1 + tol) for a relative tolerance).
• extract(loc, tol, relative, along, start, stop, value, how, fill) aggregates the rows of each window onto a dense integer axis and returns an array of shape (len(loc), stop - start). With by="mz" and along="scan", each row is an extracted-ion chromatogram. how is "max" or "sum", and positions without rows receive fill.
• weighted_mean(loc, tol, relative, of, weights, along, start, stop) averages a column (default by) over each window, weighted by another, optionally within a range of along. Windows without positive total weight give NaN.

A NaN in the sorted column raises ValueError, since it has no sorted position. No existing function changes.

Verification

• 14 new tests in tests/test_subset.py:

  • Sorting and index labels.
  • The NaN check.
  • Inclusive windows.
  • Agreement with slice on a float32 column at a bound just below and at the stored value.
  • Relative windows.
  • extract with max, with sum and a non-zero fill, with rows outside the axis range dropped, and with values below the fill under max.
  • extract against batch slice followed by collapse on the example MS1 data (frames numbered by retention time).
  • The errors for an unknown aggregation and a float axis.
  • No locations.
  • weighted_mean with an empty window and an axis range.

• python -m pytest: 397 tests, 5 failing, the same 5 that fail on master in this environment (pandas 3.0.3, NumPy 2.5.2). They are in tests/test_deconvolution.py, where this environment's SciPy raises _dfitpack.error for a spline with too few points. ruff check and ruff format --check pass. In this environment, pyright reports the same 10 errors in deimos/subset.py and tests/test_subset.py as on master, none in the new lines. CI installs pandas 2.3.3, under which pyright also checks the label arguments to DataFrame; its pyright check and its test runs on Ubuntu, macOS, and Windows pass.

• Equivalence on an LC-MS run (SCIEX, 27,466,493 MS1 centroids, 6,404 scans), with chromatograms within 10 ppm of 1 to 5,000 randomly drawn centroid m/z values, over all scans or scans 1,000-1,199:

  • extract(..., how="max") returns the same array, bit for bit, as batch slice followed by collapse(keep="scan", how="max") and as a per-window NumPy loop (np.searchsorted for the window, np.maximum.at per scan).
  • extract on a binned retention-time axis matches the loop on 0.05-minute bins.
  • weighted_mean matches np.average per window for 200 windows.

• Time per call on that run, best of 3 to 50 repeats. Building the SortedFeatures took 0.15 s here, on a table already in m/z order:

  Windows	Scans	batch slice + collapse	SortedFeatures.extract	NumPy loop
  1	6,404	136.7 ms	0.01 ms	0.01 ms
  1	200	135.2 ms	0.01 ms	0.01 ms
  20	6,404	137.9 ms	0.13 ms	0.13 ms
  20	200	136.1 ms	0.09 ms	0.08 ms
  1,000	6,404	327.0 ms	14.25 ms	10.05 ms
  1,000	200	206.7 ms	7.97 ms	4.61 ms
  5,000	6,404	1,315.2 ms	73.86 ms	59.94 ms
  5,000	200	574.0 ms	46.38 ms	25.78 ms

Notes for reviewers

• extract gathers the rows of every window at once and combines them with np.maximum.at or np.bincount. With many windows over a narrow axis range it gathers rows that the range then drops, which is why it is up to 1.8 times the per-window loop in the 200-scan rows above. Under max, cells start at fill when no value is below it, which is exact and avoids a separate mask; otherwise they start at -inf and a mask restores fill to positions without rows.
• along must be an integer column. A binned axis, such as retention time in fixed bins, is a column of bin numbers that the caller computes once.
• The class is not exported at the top level of deimos, unlike slice and locate. That is easy to add if preferred.

No related issue.

Quality assurance and AI policy

To maintain project quality and respect maintainer bandwidth, please confirm the following about this pull request (PR):

• Manual verification: I have manually reviewed and tested the code in this PR, and python -m pytest passes locally.
• AI-assisted content: If artificial intelligence (AI) tools (for example, GitHub Copilot or ChatGPT) were used, I have personally verified the logic, edge cases, and compliance with the existing codebase. I confirm the code is not a "blind" AI generation.
• Minimal review: I believe this PR is in a state that requires minimal intervention or correction from maintainers.
• Scoped change: This PR addresses a single, well-scoped issue rather than multiple unrelated changes.

Status

• Ready to go (checking this signals to maintainers that the PR is ready for final review)

Developer Certificate of Origin

• I certify that this contribution is covered by the BSD 3-Clause License and the Developer Certificate of Origin.

---

Note to contributors: We reserve the right to close PRs without review if they appear to lack human validation or do not meet the quality standards described in our contributing guide.