Repository navigation
path-wildcards: anchored trailing-* + per-endpoint port + R0040 args - #323
Conversation
Signed-off-by: entlein <einentlein@gmail.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughRefactors dynamicpathdetector: adds per-prefix CollapseConfig and pooled buffers, rewrites CompareDynamic (memoized DP for multi-*), changes process/collapse thresholds and wildcard emission, consolidates AnalyzeOpens results, and updates endpoint deduplication to fold/sweep wildcard vs specific ports with Internal isolation. Extensive tests and benchmarks added. ChangesDynamic Path Detector Refactoring
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@CodeRabbit review this please together with kubescape/node-agent#807 |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chainI'll fetch both PRs to conduct a combined review. 🏁 Script executed: Length of output: 144241 Now let me fetch the companion node-agent PR. 🏁 Script executed: Length of output: 29684 Let me verify the cross-repo dependency before writing the full review. 🏁 Script executed: Length of output: 632 🏁 Script executed: Length of output: 3221 🏁 Script executed: Length of output: 10791
🔴 Compilation Blockers (both PRs together won't build)1.
|
| # | Repo | Severity | File | Finding |
|---|---|---|---|---|
| 1 | storage#323 | 🔴 Blocker | — | CompareExecArgs not defined anywhere in the package |
| 2 | node-agent#807 | 🔴 Blocker | projection_types.go |
ExecsByPath field absent from ProjectedContainerProfile |
| 3 | storage#323 | 🟠 Design | types.go |
DefaultCollapseConfig exported mutable var |
| 4 | storage#323 | 🟠 Design | analyzer.go |
compareSegments exponential worst-case for adversarial * patterns |
| 5 | storage#323 | 🟠 Design | analyze_endpoints.go |
wildcardKey / getEndpointKey format coupling |
| 6 | storage#323 | 🟡 Minor | analyze_opens.go |
consolidateOpens O(n²) unacknowledged |
| 7 | node-agent#807 | 🟡 Minor | open.go |
Pattern false-negative gap not documented |
| 8 | node-agent#807 | 🟡 Minor | exec.go |
absent vs empty ExecsByPath asymmetry not commented |
| 9 | storage#323 | 🟢 Informational | analyze_endpoints.go |
removeEndpoint in-place mutation |
| 10 | storage#323 | 🟢 Informational | analyzer.go |
hasPrefixAtBoundary("") unguarded edge |
The storage PR's core logic (anchored trailing-*, per-endpoint port isolation, CollapseConfig per-prefix, zero-alloc processSegments) is solid and the test coverage is excellent. The main gap is that the R0040 args machinery in node-agent#807 is calling into infrastructure (CompareExecArgs, ExecsByPath) that hasn't been committed yet on either side.
|
@CodeRabbit also take into account kubescape/node-agent#805 and #322 kubescape/node-agent#808 kubescape/node-agent#806 #325 #324 . advise claude how to better split the code up, so all these PRs become more atomic (cough cough) and let it know the merge order, thx |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 19879
Dependency map across all 8 PRsgraph TD
s323["storage `#323`\npath-wildcards core\n(WildcardIdentifier, CollapseConfig API,\nanchored-*, per-port key)"]
s322["storage `#322`\nCompareExecArgs\n(compare_exec_args.go)"]
s324["storage `#324`\nnetworkmatch pkg\n(DNS/IP/CIDR)"]
s325["storage `#325`\nCollapseConfig CRD\n(+generated code, processors)"]
na805["node-agent `#805`\nresolveExecPath\n3-arg CEL parse.get_exec_path()"]
na806["node-agent `#806`\nnetwork wildcards CEL\n(DNS/CIDR/IP)"]
na807["node-agent `#807`\nR0040 args consumer\n(wasExecutedWithArgs,\nCompareExecArgs call,\nExecsByPath read)"]
na808["node-agent `#808`\nBig: ExecsByPath field +\ntamper detection (R1016) +\nnon-blocking fanout +\nNN resource support"]
s323 -->|"WildcardIdentifier"| s322
s323 -->|"FindConfigForPath,\nDefaultCollapseConfigs"| s325
s322 -->|"go.mod bump"| na805
s322 -->|"go.mod bump"| na807
s324 -->|"go.mod bump"| na806
s325 -->|"go.mod bump"| na808
na808 -->|"ExecsByPath on\nProjectedContainerProfile"| na807
What's wrong with atomicity today
Recommended atomic splitstorage sidenode-agent sideMerge order (safe, assuming the splits above)
Quick checklist for Claude
🐇 🗂️ |
…opy, nitpicks) Signed-off-by: entlein <einentlein@gmail.com>
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
pkg/registry/file/dynamicpathdetector/tests/coverage_test.go (1)
715-723: ⚡ Quick winAdd an explicit “different concrete ports stay distinct” assertion.
This case validates
:0wildcard folding, but it doesn’t pin the per-endpoint-port isolation contract (e.g.:8080vs:8443should not collapse). Please add a sibling assertion/test for two concrete ports to prevent regressions.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/registry/file/dynamicpathdetector/tests/coverage_test.go` around lines 715 - 723, The test TestBuildEndpointKey_SharedFormat currently verifies folding of a specific port into a wildcard but lacks an assertion that two distinct concrete ports remain separate; update this test to include inputs like ":8080/api/v1/users" and ":8443/api/v1/users" (or another different concrete port) passed to dynamicpathdetector.MergeDuplicateEndpoints and assert that the merged result contains two distinct entries for those concrete ports (using assert.Len/other assertions) to ensure concrete ports do not collapse into one another.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pkg/registry/file/dynamicpathdetector/tests/consolidate_opens_test.go`:
- Around line 109-113: Replace the non-fatal assertion with a fatal check so the
test stops before indexing: change the assert.Len(t, result, 1, ...) call to
require.Len(t, result, 1, ...) (keeping the same message) and leave the
subsequent checks (result[0].Path and result[0].Flags assertions) unchanged so
we don't risk panicking when accessing result[0].
In `@pkg/registry/file/dynamicpathdetector/tests/profile_test.go`:
- Around line 68-103: After successfully creating cpuFile and starting the CPU
profile (cpuFile, pprof.StartCPUProfile), add a deferred cleanup that calls
pprof.StopCPUProfile() and closes cpuFile (checking/handling the close error via
t.Logf or t.Fatalf as appropriate) so that any later t.Fatalf does not leak the
profiling state; ensure the defer is placed immediately after the successful
StartCPUProfile call and refers to the same cpuFile and pprof.StopCPUProfile/
cpuFile.Close operations used later in the test.
---
Nitpick comments:
In `@pkg/registry/file/dynamicpathdetector/tests/coverage_test.go`:
- Around line 715-723: The test TestBuildEndpointKey_SharedFormat currently
verifies folding of a specific port into a wildcard but lacks an assertion that
two distinct concrete ports remain separate; update this test to include inputs
like ":8080/api/v1/users" and ":8443/api/v1/users" (or another different
concrete port) passed to dynamicpathdetector.MergeDuplicateEndpoints and assert
that the merged result contains two distinct entries for those concrete ports
(using assert.Len/other assertions) to ensure concrete ports do not collapse
into one another.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 53c1c899-007e-4f39-b1e0-c666818f2f9d
📒 Files selected for processing (12)
pkg/registry/file/dynamicpathdetector/analyze_endpoints.gopkg/registry/file/dynamicpathdetector/analyze_endpoints_internal_test.gopkg/registry/file/dynamicpathdetector/analyze_opens.gopkg/registry/file/dynamicpathdetector/analyzer.gopkg/registry/file/dynamicpathdetector/tests/analyze_endpoints_test.gopkg/registry/file/dynamicpathdetector/tests/analyze_opens_test.gopkg/registry/file/dynamicpathdetector/tests/benchmark_test.gopkg/registry/file/dynamicpathdetector/tests/compare_dynamic_memoise_test.gopkg/registry/file/dynamicpathdetector/tests/consolidate_opens_test.gopkg/registry/file/dynamicpathdetector/tests/coverage_test.gopkg/registry/file/dynamicpathdetector/tests/profile_test.gopkg/registry/file/dynamicpathdetector/types.go
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 45421
✅ Confirmed Atomic
|
Signed-off-by: entlein <einentlein@gmail.com>
matthyx
left a comment
There was a problem hiding this comment.
Reviewed the dynamic path detector changes and the affected package tests pass. I did not find any blocking issues.
One non-blocking follow-up: operator-supplied CollapseConfig prefixes with a trailing slash (for example /etc/) currently do not match in hasPrefixAtBoundary, so they fall back to the default threshold instead of the configured one.
…s/op, 0 allocs/op on main to ~120 ns/op, 112 B/op, 2 allocs/op here." ┌───────────────────────────────────────────────────┬────────┬──────┬───────────┐ │ State │ ns/op │ B/op │ allocs/op │ ├───────────────────────────────────────────────────┼────────┼──────┼───────────┤ │ upstream main (2a689d8) — doesn't support * │ n/a │ n/a │ n/a │ ├───────────────────────────────────────────────────┼────────┼──────┼───────────┤ │ upstream-pr/sbob-path-wildcards BEFORE this patch │ 86–120 │ 112 │ 2 │ ├───────────────────────────────────────────────────┼────────┼──────┼───────────┤ │ AFTER this patch │ 14 │ 0 │ 0 │ └───────────────────────────────────────────────────┴────────┴──────┴───────────┘ 6× speedup, allocations eliminated. Reproduce on the branch with this patch applied: go test -mod=mod -run=^$ -bench='BenchmarkCompareDynamic/trailing_star$' -benchmem -count=3 -benchtime=200ms ./pkg/registry/file/dynamicpathdetector/tests/... --- Table 2 — Portable cases (apples-to-apples vs upstream main) These cases use only ⋯ — so upstream main can run them. Shows the residual cost of supporting * at all (the dispatch overhead). ┌─────────────────────────────────────────┬──────────────────────┬─────────────────────────┬──────────────────────┐ │ Case │ upstream main │ upstream-pr BEFORE │ upstream-pr AFTER │ ├─────────────────────────────────────────┼──────────────────────┼─────────────────────────┼──────────────────────┤ │ /api/⋯/⋯ vs /api/users/123 │ 16 ns, 0 B, 0 allocs │ 95 ns, 128 B, 2 allocs │ 24 ns, 0 B, 0 allocs │ ├─────────────────────────────────────────┼──────────────────────┼─────────────────────────┼──────────────────────┤ │ /api/⋯/⋯/⋯/⋯ vs /api/users/123/posts/42 │ 24 ns, 0 B, 0 allocs │ 158 ns, 192 B, 2 allocs │ 37 ns, 0 B, 0 allocs │ ├─────────────────────────────────────────┼──────────────────────┼─────────────────────────┼──────────────────────┤ │ /var/log/syslog (literal match) │ 19 ns, 0 B, 0 allocs │ 107 ns, 128 B, 2 allocs │ 27 ns, 0 B, 0 allocs │ ├─────────────────────────────────────────┼──────────────────────┼─────────────────────────┼──────────────────────┤ │ /var/log/syslog (literal mismatch) │ 20 ns, 0 B, 0 allocs │ 95 ns, 128 B, 2 allocs │ 28 ns, 0 B, 0 allocs │ ├─────────────────────────────────────────┼──────────────────────┼─────────────────────────┼──────────────────────┤ │ /etc/⋯ (one-segment wildcard) │ 12 ns, 0 B, 0 allocs │ 76 ns, 96 B, 2 allocs │ 19 ns, 0 B, 0 allocs │ └─────────────────────────────────────────┴──────────────────────┴─────────────────────────┴──────────────────────┘ All zero-allocation; within ~1.4–1.5× of upstream's bare two-pointer walk. The residual gap is the dispatch cost (counting * segments to pick fast vs memo path). --- Table 3 — Consecutive-* precollapse (the new win, plus your linter ask) These cases used to hit the memoised slice-descent unconditionally. The new collapseConsecutiveStars reduces them to a single * and routes them through the linear path. ┌───────────────────────────────────────────────┬───────────────────────────┬───────────────────────┬──────┐ │ Case │ BEFORE │ AFTER │ Δ │ ├───────────────────────────────────────────────┼───────────────────────────┼───────────────────────┼──────┤ │ /a/*/*/b (2 adjacent *) │ 472 ns, 1096 B, 5 allocs │ 67 ns, 8 B, 1 alloc │ ~7× │ ├───────────────────────────────────────────────┼───────────────────────────┼───────────────────────┼──────┤ │ /a/*/*/*/* (4 adjacent trailing *s) │ 870 ns, 2024 B, 5 allocs │ 49 ns, 16 B, 1 alloc │ ~18× │ ├───────────────────────────────────────────────┼───────────────────────────┼───────────────────────┼──────┤ │ /*/*/*/*/sentinel (adversarial 4-* + literal) │ 2050 ns, 3496 B, 5 allocs │ 125 ns, 24 B, 1 alloc │ ~16× │ └───────────────────────────────────────────────┴───────────────────────────┴───────────────────────┴──────┘ Non-adjacent multi-* still takes the memoised path (e.g. /a/*/b/*/c) — producer-generated patterns essentially never look like that, but it's the right algorithm for hand-authored cases that do. Producer linter (per your review) should still warn against */*/* at source — this matcher-side collapse is the safety net for legacy / hand-authored profiles already in the wild. Signed-off-by: entlein <einentlein@gmail.com>
matthyx
left a comment
There was a problem hiding this comment.
it's fine, every time you push something already approved, it dismisses the approval
analyze_endpoints: per-endpoint port handling (Internal/External merge key now includes the port so endpoints exposing the same path on different ports stay distinct)analyze_opens: anchored trailing-*only matches when the wildcard is at the end of the pattern (no recursive-*accidents)analyzer:DefaultCollapseConfigsaccessor andFindConfigForPathvalue-return (used by node-agent's CEL helpers)CompareDynamic perf note: the current implementation allocates two []string slices per call. matthyx's earlier review of PR #316 line 358 called this out and indicated he'd own a zero-alloc rewrite; remains untouched
Summary by CodeRabbit
Bug Fixes
Performance
New Features
Tests