Skip to content

fix(rest_over_grpc): bound query field path nesting depth - #729

Merged
Evgenii (Vaiz) merged 4 commits into
mainfrom
u/vaiz/2026/09/04/rog-query-depth-limit
Sep 4, 2026
Merged

Evgenii (Vaiz) merged 4 commits into
mainfrom
u/vaiz/2026/09/04/rog-query-depth-limit

Conversation

@Vaiz

@Vaiz Evgenii (Vaiz) commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

🤖 Clawpilot here! Posted automatically by Clawpilot (an AI agent), not by a human. Please verify before acting.

Summary

A dotted query parameter key (?a.b.c=1) built one nesting level per .-separated segment, and both that build and the matching deserialization descended once per level. Neither had an upper bound, so the nesting depth was limited only by the input.

This caps the depth at 64 levels. A deeper field path is now rejected with Code::InvalidArgument instead of being decoded.

Effects

  • Query field paths nested more than 64 levels deep are rejected as an invalid request structure.
  • No change for any realistic message shape; proto nesting stays far below the bound.
  • No public API change.

Validation

  • just anvil-fmt, just anvil-clippy
  • cargo test for rest_over_grpc and rest_over_grpc_tests
  • Unit tests cover both sides of the bound, including through the public decode_request entry point.

Design note

The 64-level cap is deliberately fixed: it is not configurable, not runtime-tunable, and not exposed in any public API, because a security bound a caller can raise does not bound anything. The cap is documented on MAX_QUERY_FIELD_DEPTH as a known limitation — field paths nested deeper than it are rejected with a clear error, never silently truncated and never allowed to overflow the stack. Making the depth configurable is out of scope for this fix.

The limit is documented for crate users in the crate-level # Limitations section of lib.rs (and the generated README.md), because MAX_QUERY_FIELD_DEPTH is private and codegen_helpers is #[doc(hidden)], so a doc comment on either would never reach rustdoc.

Copilot AI lite review requested due to automatic review settings September 4, 2026 06:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The depth cap is enforced at the query-tree construction point with clear error behavior and targeted tests validating both sides of the bound.

Pull request overview

This PR hardens rest_over_grpc query-parameter decoding by bounding the nesting depth created from dotted query keys (e.g., ?a.b.c=1), preventing untrusted inputs from driving unbounded recursive descent during query tree construction and deserialization.

Changes:

  • Introduces a MAX_QUERY_FIELD_DEPTH constant (64) and enforces it during dotted query key insertion.
  • Returns a structured InvalidArgument error when the depth is exceeded instead of recursing further.
  • Adds unit tests covering both the rejection path (including via decode_request) and the “at bound” acceptance behavior; updates the crate changelog.
File summaries
File Description
crates/rest_over_grpc/src/transcode/overlay.rs Adds a max query-field nesting depth guard to query-tree insertion and tests the bound through internal and public decode paths.
crates/rest_over_grpc/CHANGELOG.md Documents the new nesting-depth limit under Unreleased fixes.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@codecov

codecov Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (40c705c) to head (e66d2bd).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #729   +/-   ##
=======================================
  Coverage   100.0%   100.0%           
=======================================
  Files         583      583           
  Lines       62891    62909   +18     
=======================================
+ Hits        62891    62909   +18     
Flag Coverage Δ
linux 80.5% <100.0%> (?)
linux-arm 80.5% <100.0%> (?)
windows 80.5% <100.0%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wukchung

Copy link
Copy Markdown
Contributor

I would expect the limitation get documented in README

Copilot AI review requested due to automatic review settings September 4, 2026 10:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It introduces a depth bound but still accepts empty path segments from dotted keys (e.g. a..b/.a), and the updated test module should follow the repo’s coverage exclusion convention for mod tests.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

crates/rest_over_grpc/src/transcode/overlay.rs:488

  • key.split('.') yields empty segments for leading/trailing dots or consecutive dots (e.g. "a..b" or ".a"), but insert_query_node currently accepts an empty segment and inserts a "" field name into the query tree. It’s safer to reject empty segments as invalid request structure to avoid confusing/ambiguous field paths.
    let key = path
        .next()
        .ok_or_else(|| TranscodeError::structure("query parameter has an empty field path"))?;
    let has_more = path.clone().next().is_some();

crates/rest_over_grpc/src/transcode/overlay.rs:605

  • Test modules in this repo are typically excluded from coverage measurement with #[cfg_attr(coverage_nightly, coverage(off))] above #[cfg(test)] mod tests (e.g. crates/tick/src/error.rs:103-105). Adding new tests here without that attribute can cause test-only lines to count toward the coverage gate.
  • Files reviewed: 3/3 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@Vaiz

Copy link
Copy Markdown
Contributor Author

Thanks — both points are noted, and both are declined for this PR. Answering here because they arrived as suppressed comments in review 5112095065 rather than as threads.

Empty path segments (a..b, .a) — out of scope, and not a hole. This PR fixes one CVE: unbounded recursion over nested query field paths. Empty-segment handling is pre-existing behaviour in code this PR does not change (your own note marks it "previously missed — in code that hasn't changed since the last review"). It is also not exploitable: an empty segment inserts a field named "", which matches no proto field, so the request is already rejected by the deserializer as an unknown field. There is no recursion, no truncation and no panic on that path. Tightening it is a separate, non-security change and belongs in its own PR.

#[cfg_attr(coverage_nightly, coverage(off))] on mod tests — incorrect for this crate. rest_over_grpc does not use that attribute on any of its ~25 #[cfg(test)] mod tests blocks; it applies coverage(off) only to individually unreachable functions such as expecting/visit_str in this very file (overlay.rs:220, 870, 887, 948, 953, 1061). The coverage gate passes on the crate as it stands, so the new tests need no exemption, and adding one to a single module would make this file inconsistent with the rest of its own crate. crates/tick follows a different local style; that is not this crate's convention.

The depth cap itself remains fixed and deliberately not configurable, for the reason given under "Design note" in the PR description.

Comment thread crates/rest_over_grpc/src/transcode/overlay.rs
Evgenii (Vaiz) and others added 3 commits September 4, 2026 12:42
Dotted query keys built one nesting level per `.`-separated segment with
no bound, so the build and the matching deserialization both descended
once per segment. Cap the depth at 64 levels and reject a deeper path
with `Code::InvalidArgument`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…known limitation

Add "recurse" to the repo word list so cargo-spellcheck accepts the sentence
that explains why the query field path depth is bounded, and state plainly on
MAX_QUERY_FIELD_DEPTH that an over-deep path is rejected rather than truncated,
and that the cap is deliberately fixed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The cap was documented only on the private MAX_QUERY_FIELD_DEPTH constant in
a private module, so it never reached rustdoc or the README. State it in the
crate-level Limitations section, where a user of the crate can see it, and
regenerate the README paragraph to match.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 4, 2026 11:44
@Vaiz
Evgenii (Vaiz) force-pushed the u/vaiz/2026/09/04/rog-query-depth-limit branch from 9b05ae6 to 5389189 Compare September 4, 2026 11:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The depth bound is applied at the recursion driver, maps to a clear InvalidArgument structure error, and is covered by targeted tests and documentation updates.

Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

anvil-readme-check regenerates the README with cargo-doc2readme pinned to
=0.6.4 and compares the input hash recorded in the dependencies_info blob.
The previous commit added the prose by hand, leaving the stale hash, so CI
reported "Input has changed". Regenerated with 0.6.4; only the blob moves.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 4, 2026 12:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The depth bound is enforced at the query-tree construction point, is exercised through the public decode entry point, and the documentation matches the behavioral change.

Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

⚠️ Potential breaking changes detected

cargo semver-checks flagged the following on this PR. This is informational -- breaking changes between commits are expected; the major-version bump happens at release time, not on every PR.

rest_over_grpc

     Cloning origin/main
    Building rest_over_grpc v0.2.0 (current)
       Built [  11.224s] (current)
     Parsing rest_over_grpc v0.2.0 (current)
error: unsupported rustdoc format v60 for file: /home/runner/work/oxidizer/oxidizer/target/semver-checks/local-rest_over_grpc-0_2_0-default-251561ef64dea3fc/target/doc/rest_over_grpc.json
(supported formats are v55, v56, v57)

rest_over_grpc_examples

     Cloning origin/main
    Building rest_over_grpc_examples v0.0.0 (current)
       Built [  23.860s] (current)
     Parsing rest_over_grpc_examples v0.0.0 (current)
error: unsupported rustdoc format v60 for file: /home/runner/work/oxidizer/oxidizer/target/semver-checks/local-rest_over_grpc_examples-0_0_0-default-01666ec060466c14/target/doc/rest_over_grpc_examples.json
(supported formats are v55, v56, v57)

rest_over_grpc_tests

     Cloning origin/main
    Building rest_over_grpc_tests v0.0.0 (current)
       Built [  23.491s] (current)
     Parsing rest_over_grpc_tests v0.0.0 (current)
error: unsupported rustdoc format v60 for file: /home/runner/work/oxidizer/oxidizer/target/semver-checks/local-rest_over_grpc_tests-0_0_0-default-01666ec060466c14/target/doc/rest_over_grpc_tests.json
(supported formats are v55, v56, v57)

@Vaiz
Evgenii (Vaiz) merged commit 461aa17 into main Sep 4, 2026
52 checks passed
@Vaiz
Evgenii (Vaiz) deleted the u/vaiz/2026/09/04/rog-query-depth-limit branch September 4, 2026 13:45
Zac Walker (ZacWalk) added a commit that referenced this pull request Sep 11, 2026
## Summary

Prepare 58 coordinated crate releases, including the maintainer's
macro-pinning and version-alignment commits. Initially rebased onto
`8624f907` on main, including #744 and the merged #733 runtime-recording
work. Subsequently merged main at `704eba82` in `b1b8dc17`, including
#722 (compressors) and #745 (metabench support identifiers), without
rewriting the PR history. All 79 pre-merge package versions are
preserved; the new compressors package retains upstream `0.1.0`. No new
release versions were chosen during the merge.

- Release `thread_aware 0.12.0` and affected consumers for
`thread_aware_core` adoption; retain `thread_aware_core 0.1.1` as a
compatible addition.
- Release `rest_over_grpc 0.2.1`, including the query-field
nesting-depth fix from #729.
- Preserve the observed family at `0.26.0`, fundle at `0.4.0`, and
data-privacy facade/macros at `0.13.0`.
- Preserve the later maintainer updates: exact macro pins; the ohno
family at `0.5.2`; internity and macros at `0.2.1`; metabench and macros
at `0.1.1`; multitude and macros at `0.10.0`; and templated-uri and
macros at `0.6.0`.
- Retain `data_privacy_core 0.1.2`, `plurality 0.2.2`, and `recoverable
0.2.0` without release bumps.
- Preserve the reviewed release-note cleanup: remove CI-only and
misattributed feature notes, restore preambles, normalize new
categories, preserve historical entries, and describe #729/#730 once
each. Retain the verified `http_path_template 0.2.0` publication date,
2026-07-22.
- Address remaining feedback in `18de2ac3`: correct the uniflight
`stampede` keyword and qualify optional dependency notes by their
features in rest_over_grpc and seatbelt_http.
- Reconcile registry lockfile resolution in `f3d71c7e`: restore seven
`windows-sys` references and the related `tempfile`/`getrandom`
reference to main, without changing any package versions.

Relative to current main, this is release metadata and generated
documentation, plus the bytesbuf rustdoc URL correction. The incoming
implementation changes from #733, #722, and #745 come from main and were
preserved. The root manifest conflict was resolved by retaining the
release's fetch package versions and main's new flate2 dependency.

## Required Release Review

The author confirmed on 2026-09-10 that `performables` and
`seismograph_cli` must remain at upstream `0.1.0`; do not add release
bumps for them. This is an intentional exception to the release
planner's normal dependent cascade. Their initial-publication handling
and dependency closure are specified below; keeping these versions does
not mean that their publication can be skipped.

The initial review compared 46 ordinary libraries against previous
version-bump commits with `cargo semver-checks --all-features
--release-type patch`. Manual evidence also covered the public
`ThreadAware` identity migration, public Syn 2-to-3 helper types, the
observed mutable-reference macro input restriction, and ohno's
warning-policy caveat.

The pre-merge SemVer report identifies 10 proc-macro packages and 26
ordinary consumers requiring manual compatibility review, with no
reported under-increment. Comparing the pre-rebase head `826feb24` with
`65cc1af8` shows no changes in the macro packages or `routerama_build`,
so the earlier macro-source evidence survives that rebase. The
subsequent merge brings in #745's metabench macro support-identifier fix
unchanged from main; its unit, expansion, and compile-contract tests
passed against the release versions. Prior evidence does not establish
compatibility of the consumer changes introduced by #733 or replace the
fresh merged-head report.

Final release review must confirm the macro/consumer classifications,
particularly the patch boundaries of `internity 0.2.1` and
`arty_executor 0.1.2`, routerama parser/macro compatibility, ohno's
generated-code warning policy, and the post-#733 public-type/runtime
implications. The automated SemVer warning is informational; passing
builds and resolved review threads do not replace this release signoff.

## Post-rebase Validation

- Passed at `65cc1af8`: locked workspace `cargo check --all-features`.
- Passed at `65cc1af8`: `just clippy` for the workspace, all features
and targets.
- Passed at `65cc1af8`: 811 selected tests across rallocator,
thread-aware, REST-over-gRPC, internity, and metabench packages (43
binaries).
- Passed at `65cc1af8`: pinned formatting checks across all 79 workspace
packages, README generation, whitespace checks, and editor diagnostics.
- Passed: comparison of surviving package versions against the exact
pre-rebase head `826feb24`; all 72 match. Current exact/caret workspace
dependency requirements match target versions.
- Confirmed: the old `thread_heap_usage_includes_remote_allocations`
test is absent after #733. No test was disabled or assertion weakened in
this release branch.
- The `65cc1af8` CI run completed with 58 successful checks and one
skipped check.
- Passed for the feedback fix `18de2ac3`: locked Cargo metadata
verification, `just anvil-cargo-sort`, `just anvil-clippy`, `just
anvil-fmt --fix`, `just anvil-readme --fix`, `just anvil-spellcheck`,
whitespace checks, and editor diagnostics. Regeneration produced no
additional source/README changes. The local spelling-tool prerequisite
is now installed.
- Passed for the lockfile fix `f3d71c7e`: full locked/offline Cargo
metadata, `just anvil-clippy` (all targets/features), `just anvil-fmt
--fix`, `just anvil-readme --fix`, `just anvil-spellcheck`, whitespace
checks, and editor diagnostics. All 568 registry-package lock entries
exactly match main; all 79 non-registry entries remain unchanged from
`18de2ac3`. `ring` retains its required `windows-sys 0.52.0` dependency.
- Fresh CI for the latest head must finish successfully; results from
preceding commits are not a substitute.

Before the rebase, selected runs passed 2,691 tests for release
consumers and 969 tests including thread-aware and ohno. Those results
are historical.

## Main-merge Validation

For `b1b8dc17`, merging main at `704eba82`:

- Passed: full locked/offline Cargo metadata for 80 workspace packages.
All 79 prior manifest versions are unchanged; compressors remains
`0.1.0`.
- Passed: all 576 registry-package lock entries exactly match incoming
main. The Windows dependency-resolution correction is preserved.
- Passed: 553 unit/integration tests across compressors, metabench,
metabench_macros, and metabench_macros_impl; none skipped.
- Passed: 26 doctests across those packages, with seven existing ignored
examples.
- Passed: generated Anvil workspace Clippy (all targets/features),
manifest sorting, formatting, README regeneration, spelling, whitespace
checks, and editor diagnostics.
- Regenerated the compressors README to link to the release's bytesbuf
`0.10.0`; incoming compressors and metabench macro implementation code
matches main.
- Confirmed: the release selector still selects 58 bumped packages
against current main. No release versions were added or changed.
- GitHub reports the conflict resolved. Fresh CI and approval of the
latest reviewable push are required; prior checks do not validate this
merge.

## Publication

This PR prepares releases only. No crates have been published or yanked
as part of this work.

Registry checks on 2026-09-10 found all seven new #733 packages
unpublished:

- `performables`
- `seismograph`
- `seismograph_protocol`
- `seismograph_io`
- `seismograph_cli`
- `seismograph_runtime`
- `seismograph_rallocator`

Preserve their initial versions. The normal comparison against current
main selects the 58 version-bumped packages and excludes these seven
initial packages and the newly merged compressors `0.1.0`. The existing
58 releases require `seismograph_protocol`, `seismograph`,
`performables`, and `seismograph_io`; shipping the CLI additionally
requires the runtime/rallocator helpers. Publish prerequisites before
their consumers in Cargo dependency order. The compressors
registry/publication status was not checked in this merge update.

Before merging, the publishing maintainer must confirm one of these
paths:

1. Complete and verify the #733 initial publications, then publish the
normal 58-package release set; or
2. Explicitly include the required initial packages in a catch-up
publication together with the 58 bumped packages. After merging #722,
the repository's `Get-PackagesWithVersionChanges` helper selects exactly
66 packages when compared against pre-#733 commit
`70bd5aafadb9b11fc45d8977bd808066e574b8de`: the 58 bumped packages, the
seven #733 packages, and compressors. Confirm actual registry
availability, equivalent selection, and dependency ordering in the
publishing pipeline before execution; the earlier 65-package count
applied before compressors joined main.

`.ado/pipelines/publish.yml` exposes `previousVersion` (default `HEAD^`)
but delegates publishing to the external `ox-sdk` template. Its catch-up
behavior has not been verified or executed here. The official pipeline
triggers on main and defaults `executePublish` to true, so publication
selection must be confirmed before merge, not deferred until after it.

Do not merge until required checks, final release review, and the
initial-publication plan are confirmed. Do not rerun the release driver
on the already-bumped branch merely to include these initial packages;
that would re-increment existing releases rather than preserve the
approved version plan.

## Tooling Follow-up

The release generator inserts sections ahead of preambles, combines
curated headings with list-style categories, and uses tag-based history
that can lag declared releases. The release artifacts have been
repaired; generator behavior and regression tests remain separate
follow-up work.

---------

Co-authored-by: Vaiz <4908982+Vaiz@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants