You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Tracking issue for the weekly Third-Party Reviewer Scorecard. Each run posts a new comment below measuring the external agentic PR reviewers (Copilot, Gemini, Codex, CodeRabbit, SonarCloud) across the org.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Participation = distinct non-draft PRs a bot touched Γ· 187 review-eligible PRs. Latency = time from PR creation to the bot's first review/comment. Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
PRs reviewed
Participation
Reviews
Comments/PR
Latency p50
Latency p95
β / π
Rate-limited
Resolved
π/π
Gemini Code Assist
143
76%
85
1.2
63s
3m
0 / 0
9
86%
0/0
GitHub Copilot
106
57%
106
1.8
3m
3.0h
0 / 0
6
56%
0/0
SonarCloud
177
95%
0
0.0
6m
8.0d
0 / 0
0
n/a
0/0
Codex
160
86%
30
0.4
5s
4m
0 / 0
154
86%
0/0
CodeRabbit
177
95%
229
0.6
9s
22s
52 / 34
147
97%
0/0
β = APPROVED reviews Β· π = CHANGES_REQUESTED Β· Resolved = share of a bot's inline threads later resolved Β· Comments/PR = inline findings per reviewed PR.
Reliability & no-shows
Reviewer
Eligible PRs missed
Coverage gap
Rate-limited PRs
Gemini Code Assist
44
24%
9
GitHub Copilot
81
43%
6
SonarCloud
10
5%
0
Codex
27
14%
154
CodeRabbit
10
5%
147
Coverage gap = eligible PRs a bot did not touch. External SaaS reviewers only sample a subset of PRs; this is a participation signal, not necessarily a fault.
Coverage overlap
PRs reviewed by β₯2 bots: 177 of 187 eligible (95%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate requires judgment and is intentionally out of this deterministic report. It is a separate, human-approved LLM add-on.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
Gemini Code Assist
177
132 (β² +47)
0 / 0
8
42
42s
3m
GitHub Copilot
177
116 (β² +10)
0 / 0
11
56
3m
2.8h
SonarCloud
177
167 (β² +167)
0 / 0
0
12
7m
8.1d
Codex
177
19 (βΌ -11)
0 / 0
192
26
4m
10m
CodeRabbit
177
190 (βΌ -39)
53 / 23
148
13
5m
1.6h
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 168 of 177 eligible (95%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate requires judgment and is intentionally out of this deterministic report. It is a separate, human-approved LLM add-on.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
Gemini Code Assist
153
122
0 / 0
1
31
12s
2m
GitHub Copilot
153
119
0 / 0
2
34
4m
34m
SonarCloud
153
141
0 / 0
0
15
10m
7.2d
Codex
153
5
0 / 0
162
28
4m
8m
CodeRabbit
153
98
27 / 11
118
20
3m
84m
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 138 of 153 eligible (90%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate requires judgment and is intentionally out of this deterministic report. It is a separate, human-approved LLM add-on.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
Gemini Code Assist
206
180
0 / 0
3
26
7s
3m
GitHub Copilot
206
122
1 / 0
2
84
5m
32m
SonarCloud
206
189
0 / 0
0
17
8m
2.8d
Codex
206
0
0 / 0
185
24
β
β
CodeRabbit
206
156
31 / 12
156
30
4m
25m
Qodo Merge
206
84
0 / 0
3
144
62s
2m
CodeAnt
206
79
0 / 0
0
130
4s
3m
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 184 of 206 eligible (89%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate requires judgment and is intentionally out of this deterministic report. It is a separate, human-approved LLM add-on.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
Gemini Code Assist
35
30
0 / 0
1
5
50s
2m
GitHub Copilot
35
14
0 / 0
0
21
7m
13m
SonarCloud
35
30
0 / 0
0
5
7.0d
12.7d
Codex
35
0
0 / 0
30
5
β
β
CodeRabbit
35
53
8 / 4
17
5
56s
11.8d
Qodo Merge
35
21
0 / 0
4
20
57s
2m
CodeAnt
35
38
0 / 0
0
3
5s
11.8d
Graphite
35
2
0 / 0
0
33
4m
12.7d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 31 of 35 eligible (89%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 35
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
Gemini Code Assist
32
31
0 / 0
0
1
50s
2m
GitHub Copilot
32
9
0 / 0
0
23
7m
13m
SonarCloud
32
30
0 / 0
0
2
68m
16.2d
Codex
32
0
0 / 0
31
1
β
β
CodeRabbit
32
48
5 / 4
16
2
20s
7.0d
Qodo Merge
32
33
0 / 0
2
6
11s
14.1d
CodeAnt
32
41
0 / 0
0
2
5s
12.8d
Graphite
32
5
0 / 0
0
27
12.7d
14.1d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 31 of 32 eligible (97%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 32
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
GitHub Copilot
7
0
0 / 0
0
7
β
β
Gemini Code Assist
7
6
0 / 0
0
1
67s
2m
Codex
7
0
0 / 0
6
1
β
β
CodeRabbit
7
8
0 / 0
6
1
3m
9m
SonarCloud
7
7
0 / 0
0
0
24.4h
10.1d
Qodo Merge
7
6
0 / 0
0
1
9s
11s
CodeAnt
7
8
0 / 0
0
0
5s
7s
Graphite
7
1
0 / 0
0
6
16m
16m
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 8 of 7 eligible (114%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 8
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
GitHub Copilot
23
4
0 / 0
0
19
5m
13m
Gemini Code Assist
23
22
0 / 0
0
1
38s
2m
Codex
23
0
0 / 0
22
1
β
β
CodeRabbit
23
19
3 / 1
10
1
19s
2m
SonarCloud
23
21
0 / 0
0
2
1.6h
28.2d
Qodo Merge
23
24
0 / 0
0
3
7s
14.1d
CodeAnt
23
43
0 / 0
0
1
5s
12.7d
Graphite
23
2
0 / 0
0
21
9m
12.7d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 22 of 23 eligible (96%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 23
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
GitHub Copilot
6
0
0 / 0
0
6
β
β
Gemini Code Assist
6
5
0 / 0
0
1
30s
3m
Codex
6
0
0 / 0
5
1
β
β
CodeRabbit
6
6
0 / 0
2
1
19s
8m
SonarCloud
6
6
0 / 0
0
0
2.7h
3.3d
Qodo Merge
6
5
0 / 0
0
1
5s
6s
CodeAnt
6
11
0 / 0
0
0
5s
6s
Graphite
6
0
0 / 0
0
6
β
β
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 6 of 6 eligible (100%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 6
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
GitHub Copilot
15
0
0 / 0
0
15
β
β
Gemini Code Assist
15
14
0 / 0
0
1
38s
2m
Codex
15
0
0 / 0
14
1
β
β
CodeRabbit
15
11
0 / 0
7
1
33s
8m
SonarCloud
15
15
0 / 0
0
0
35m
10.2d
Qodo Merge
15
14
0 / 0
0
1
5s
6s
CodeAnt
15
33
0 / 0
0
0
5s
7s
Graphite
15
0
0 / 0
0
15
β
β
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 15 of 15 eligible (100%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 15
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
GitHub Copilot
3
0
0 / 0
0
3
β
β
Gemini Code Assist
3
2
0 / 0
0
1
5s
38s
Codex
3
0
0 / 0
2
1
β
β
CodeRabbit
3
4
0 / 0
1
0
31s
8m
SonarCloud
3
2
0 / 0
0
1
11m
9.4d
Qodo Merge
3
2
0 / 0
0
1
4s
4s
CodeAnt
3
6
0 / 0
0
0
4s
4s
Graphite
3
0
0 / 0
0
3
β
β
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 3 of 3 eligible (100%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 3
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Rate-limited
No response
Latency p50
Latency p95
GitHub Copilot
7
0
0 / 0
0
7
β
β
Gemini Code Assist
7
4
0 / 0
0
3
6s
4m
Codex
7
0
0 / 0
4
3
β
β
CodeRabbit
7
6
0 / 0
4
0
25s
8m
SonarCloud
7
4
0 / 0
0
3
6m
5.0d
Qodo Merge
7
4
0 / 0
0
3
4s
6s
CodeAnt
7
27
0 / 0
0
2
5s
2m
Graphite
7
0
0 / 0
0
7
β
β
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Rate-limited β count of out-of-quota / rate-limit refusals, each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 6 of 7 eligible (86%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 7
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
petry-projects/repo-template PR https://github.com/petry-projects/repo-template/pull/210 β comments truncated
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Refused
No response
Latency p50
Latency p95
GitHub Copilot
105
2
0 / 0
0
103
13m
14m
Gemini Code Assist
105
93
0 / 0
6
10
87s
3m
Codex
105
64
0 / 0
167
20
6m
10m
CodeRabbit
105
270
22 / 27
114
0
4m
7.2d
SonarCloud
105
89
0 / 0
0
22
22.2h
29.8d
Qodo Merge
105
94
0 / 0
0
11
5s
6s
CodeAnt
105
360
0 / 0
15
22
2m
3m
Graphite
105
16
0 / 0
2
52
9.0h
22.5d
cubic
105
69
0 / 0
29
71
10m
43.7d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Refused β count of refusal events: out-of-quota / rate-limit notices AND a check-run reporter's own decline (a "too large" / "did not run" skip), each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 101 of 105 eligible (96%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 108
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
petry-projects/repo-template PR https://github.com/petry-projects/repo-template/pull/205 β reviews truncated
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Refused
No response
Latency p50
Latency p95
GitHub Copilot
174
2
0 / 0
0
172
13m
14m
Gemini Code Assist
174
118
0 / 0
15
49
2m
4m
Codex
174
74
0 / 0
249
33
7m
11m
CodeRabbit
174
401
24 / 31
155
3
3m
7.2d
SonarCloud
174
116
0 / 0
0
59
6.2h
14.7d
Qodo Merge
174
143
0 / 0
0
31
5s
26s
CodeAnt
174
453
0 / 0
21
64
2m
3m
Graphite
174
22
0 / 0
8
101
3.3h
21.6d
cubic
174
222
0 / 0
105
67
10m
19.8d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Refused β count of refusal events: out-of-quota / rate-limit notices AND a check-run reporter's own decline (a "too large" / "did not run" skip), each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 161 of 174 eligible (93%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 179
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
petry-projects/repo-template PR https://github.com/petry-projects/repo-template/pull/233 β comments truncated
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Refused
No response
Latency p50
Latency p95
GitHub Copilot
246
1
0 / 0
0
245
14m
14m
Gemini Code Assist
246
190
0 / 0
13
51
85s
4m
Codex
246
131
0 / 0
365
20
12s
9m
CodeRabbit
246
507
40 / 44
243
0
2m
47.4h
SonarCloud
246
190
0 / 0
0
58
1.9h
6.8d
Qodo Merge
246
112
0 / 0
0
134
7s
10s
CodeAnt
246
549
0 / 0
14
26
3m
7m
Graphite
246
45
0 / 0
10
93
73m
21.6d
cubic
246
173
0 / 0
57
163
7m
20.7d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Refused β count of refusal events: out-of-quota / rate-limit notices AND a check-run reporter's own decline (a "too large" / "did not run" skip), each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 244 of 246 eligible (99%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 254
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
combsmith/repo-template PR https://github.com/combsmith/repo-template/pull/237 β comments truncated
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Refused
No response
Latency p50
Latency p95
GitHub Copilot
235
1
0 / 0
0
234
3m
3m
Gemini Code Assist
235
50
0 / 0
2
186
2m
8m
Codex
235
218
0 / 0
209
23
11s
5m
CodeRabbit
235
378
41 / 31
139
3
20s
3.7h
SonarCloud
235
231
0 / 0
0
30
14m
2.1d
Qodo Merge
235
16
0 / 0
0
219
5s
8s
CodeAnt
235
170
0 / 0
3
163
3m
5m
Graphite
235
21
0 / 0
5
166
7m
26.9d
cubic
235
27
0 / 0
5
223
14m
26.8d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Refused β count of refusal events: out-of-quota / rate-limit notices AND a check-run reporter's own decline (a "too large" / "did not run" skip), each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 228 of 235 eligible (97%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 238
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
combsmith/incubator PR https://github.com/combsmith/incubator/pull/188 β comments truncated
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Refused
No response
Latency p50
Latency p95
GitHub Copilot
268
1
0 / 0
0
267
5m
5m
Gemini Code Assist
268
22
0 / 0
1
248
83s
4m
Codex
268
282
0 / 0
253
21
9s
17s
CodeRabbit
268
493
40 / 59
181
0
29s
1.9h
SonarCloud
268
240
0 / 0
0
40
36m
3.0d
Qodo Merge
268
12
0 / 0
0
256
5s
8s
CodeAnt
268
79
0 / 0
1
240
3m
8m
Graphite
268
17
0 / 0
3
195
14m
17.3d
cubic
268
24
0 / 0
4
259
14m
28.7d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Refused β count of refusal events: out-of-quota / rate-limit notices AND a check-run reporter's own decline (a "too large" / "did not run" skip), each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 262 of 268 eligible (98%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 275
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
A check-run REST fetch failed on these PRs, so a check-run reporter clean pass may be missed and miscounted as No response (surfaced here, never silent β #1908):
combsmith/.github-private-ghsa-vxf5-2jf3-4jxr PR https://github.com/combsmith/.github-private-ghsa-vxf5-2jf3-4jxr/pull/1 β check-run fetch failed
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Refused
No response
Latency p50
Latency p95
GitHub Copilot
222
1
0 / 0
0
221
5m
5m
Gemini Code Assist
222
16
0 / 0
0
208
83s
4m
Codex
222
229
0 / 0
185
40
10s
18s
CodeRabbit
222
389
15 / 45
91
2
10s
7.0h
SonarCloud
222
180
0 / 0
0
50
3.1h
5.6d
Qodo Merge
222
8
0 / 0
0
214
5s
19m
CodeAnt
222
38
0 / 0
1
207
3m
5m
Graphite
222
19
0 / 0
4
182
43m
17.3d
cubic
222
12
0 / 0
4
216
14m
28.7d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Refused β count of refusal events: out-of-quota / rate-limit notices AND a check-run reporter's own decline (a "too large" / "did not run" skip), each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 197 of 222 eligible (89%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 226
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
A check-run REST fetch failed on these PRs, so a check-run reporter clean pass may be missed and miscounted as No response (surfaced here, never silent β #1908):
combsmith/TalkTerm-ghsa-pc5f-cw67-7cmm PR https://github.com/combsmith/TalkTerm-ghsa-pc5f-cw67-7cmm/pull/1 β check-run fetch failed
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
Reviews
β / π
Refused
No response
Latency p50
Latency p95
GitHub Copilot
262
0
0 / 0
0
262
β
β
Gemini Code Assist
262
22
0 / 0
1
242
73s
4m
Codex
262
274
0 / 0
292
35
9s
80s
CodeRabbit
262
498
25 / 43
76
4
9s
80m
SonarCloud
262
229
0 / 0
0
41
57m
5.9d
Qodo Merge
262
10
0 / 0
0
252
5s
19m
CodeAnt
262
52
0 / 0
1
241
2m
5m
Graphite
262
24
0 / 0
5
193
14m
32.5d
cubic
262
20
0 / 0
4
253
2.9d
28.7d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
Reviews β count of reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5). A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has that comment counted as its review. Rate-limit notices are never counted.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Refused β count of refusal events: out-of-quota / rate-limit notices AND a check-run reporter's own decline (a "too large" / "did not run" skip), each occurrence.
No response β eligible PRs the bot never engaged with at all (no review, comment, or refusal).
Coverage overlap
PRs reviewed by β₯2 bots: 248 of 262 eligible (95%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 267
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
This report was wrong and its figures have been removed. It counted 0 reviews for every bot.
Ignore this week's outage alerts, coverage matrix and quota timing as well.
Use the corrected report instead:#1220 (comment) (generated 2026-11-19, covering the 7 days to that date).
Why it was wrong.ubuntu-latest moved from Ubuntu 24.04 to 26.04 between the 2026-11-09 and 2026-11-16 runs. Ubuntu 26.04 ships jq 1.8, which parses the bot check in scripts/reviewer_report.sh differently, so no message was recognized as coming from a bot. The run itself succeeded, so nothing flagged the error. The fix is #2568 (issue #2547), merged 2026-11-19.
Window not re-run. The corrected report covers 2026-11-12 to 2026-11-19, not this report's 2026-11-09 to 2026-11-16. Next Monday's scheduled report resumes the weekly series.
Rewritten by Claude Code on behalf of the maintainer. The original all-zero report is in this comment's edit history.
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
GitHub Copilot β no PR reviewed on 8 consecutive days with PRs (117 PRs opened) in combsmith/.github, combsmith/.github-private, combsmith/broodly, combsmith/google-app-scripts, combsmith/software-factory, combsmith/software-factory-canary; first silent day 2026-11-12.
Codex β no PR reviewed on 3 consecutive days with PRs (6 PRs opened) in combsmith/.github-private, combsmith/broodly, combsmith/software-factory; first silent day 2026-11-15.
Qodo Merge β no PR reviewed on 8 consecutive days with PRs (117 PRs opened) in combsmith/.github, combsmith/.github-private, combsmith/broodly, combsmith/google-app-scripts, combsmith/software-factory, combsmith/software-factory-canary; first silent day 2026-11-12.
GitHub Copilot β 0 of 3 PRs opened in combsmith/.github reviewed in the whole window; first silent day 2026-11-13.
GitHub Copilot β 0 of 16 PRs opened in combsmith/.github-private reviewed in the whole window; first silent day 2026-11-12.
GitHub Copilot β 0 of 3 PRs opened in combsmith/broodly reviewed in the whole window; first silent day 2026-11-12.
GitHub Copilot β 0 of 84 PRs opened in combsmith/software-factory reviewed in the whole window; first silent day 2026-11-12.
GitHub Copilot β 0 of 10 PRs opened in combsmith/software-factory-canary reviewed in the whole window; first silent day 2026-11-13.
Codex β 0 of 10 PRs opened in combsmith/software-factory-canary reviewed in the whole window; first silent day 2026-11-13.
Qodo Merge β 0 of 3 PRs opened in combsmith/.github reviewed in the whole window; first silent day 2026-11-13.
Qodo Merge β 0 of 16 PRs opened in combsmith/.github-private reviewed in the whole window; first silent day 2026-11-12.
Qodo Merge β 0 of 3 PRs opened in combsmith/broodly reviewed in the whole window; first silent day 2026-11-12.
Qodo Merge β 0 of 84 PRs opened in combsmith/software-factory reviewed in the whole window; first silent day 2026-11-12.
Qodo Merge β 0 of 10 PRs opened in combsmith/software-factory-canary reviewed in the whole window; first silent day 2026-11-13.
Graphite β 0 of 10 PRs opened in combsmith/software-factory-canary reviewed in the whole window; first silent day 2026-11-13.
Rules and how to mark a cell off by design: docs/reviewer-report.md (Outage alerts).
Deterministic report β every figure is computed with jq/awk from GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh). Every bot message β review, inline comment, top-level comment, check run β gets exactly one class (finding, clean, refused, skipped, status or unclassified) from that bot's patterns (scripts/lib/reviewer-message-classes.tsv) first and the shared rate-limit pattern second. A PR counts as reviewed only when the bot left a finding or clean message on it; status notices and unclassified messages never count. Each eligible PR lands in exactly one bucket per bot, so the five PR columns sum to Total PRs. Latency = time from PR creation to the bot's first finding or clean message. Deltas (β²/βΌ) are vs the prior week and directional only.
Scorecard
Reviewer
Total PRs
With findings
Clean
Refused
Skipped
No response
Findings raised
Resolved %
Latency p50
Latency p95
GitHub Copilot
194
0
0
0
0
194
0
n/a
β
β
Gemini Code Assist
194
78
45
28
0
43
164
82%
2m
25.2h
Codex
194
35
0
53
0
106
149
88%
4m
47.4h
CodeRabbit
194
60
33
0
100
1
261
84%
16s
5.9d
SonarCloud
194
6
142
0
0
46
6
n/a
64m
7.7d
Qodo Merge
194
0
0
7
0
187
0
n/a
β
β
CodeAnt
194
91
0
0
0
103
143
79%
7m
21.5d
Graphite
194
28
96
3
0
67
49
90%
24m
21.5d
cubic
194
108
39
0
0
47
505
90%
4m
21.5d
Total PRs β review-eligible (non-draft) PRs in the window; the denominator each row is measured against.
With findings β PRs where the bot left at least one finding message (an inline thread, a review that requests changes, a body that reports issues).
Clean β PRs where the bot reviewed and reported nothing: at least one clean message and no finding.
Refused β PRs where the bot only declined for quota, rate limit, trial or plan reasons (no finding or clean message).
Skipped β PRs where the bot only reported that it skipped the review because of configuration (for example auto review turned off).
No response β every other eligible PR: the bot never appeared, or left only status or unclassified messages.
Findings raised β review threads the bot opened plus findings written in its review, comment or check-run bodies. CodeRabbit's outside-diff and nitpick items and Codex's body suggestions without a matching inline comment always count; any other finding body counts once, and only on a PR where the bot opened no thread (otherwise it summarizes those threads). The Details section splits the two.
Resolved % β threads the bot opened that are resolved, out of all threads it opened.
Details
Reviewer
Reviews
β / π
Refusal events
π / π
Status messages
Unclassified messages
Findings: threads / bodies
GitHub Copilot
0
0 / 0
0
0 / 0
0
0
0 / 0
Gemini Code Assist
144
0 / 0
37
0 / 0
0
102
164 / 0
Codex
88
0 / 0
147
0 / 0
154
0
149 / 0
CodeRabbit
393
1 / 28
13
0 / 0
189
225
211 / 50
SonarCloud
151
0 / 0
0
0 / 0
0
0
0 / 6
Qodo Merge
0
0 / 0
7
0 / 0
0
0
0 / 0
CodeAnt
168
0 / 0
0
0 / 0
498
165
112 / 31
Graphite
36
0 / 0
3
0 / 0
0
38
49 / 0
cubic
234
0 / 0
0
0 / 0
0
191
505 / 0
Reviews β formal reviews the bot submitted, each occurrence (a PR re-reviewed on 5 commits = 5), leaving out refusals, skips and status notices. A bot that posts no formal review but delivers its verdict as a comment (e.g. SonarCloud's quality-gate comment) has its finding / clean comments counted instead.
β / π β of those reviews, how many were APPROVED / CHANGES_REQUESTED (comment-only responses carry no verdict).
Refusal events β every refused message, each occurrence.
π / π β reactions on the bot's inline comments.
Status / Unclassified messages β message counts; neither ever makes a PR count as reviewed.
Findings: threads / bodies β the two parts of Findings raised: review threads the bot opened, and findings counted from its review, comment or check-run bodies.
Quota timing
Refusals (quota, rate limit, trial or plan) per bot per UTC day, and the time of the first one that day. Dependabot PRs are left out.
Reviewer
Day (UTC)
Refusals
First refusal (UTC)
Gemini Code Assist
2026-11-13
9
05:54
Gemini Code Assist
2026-11-14
3
01:28
Codex
2026-11-13
26
02:28
Codex
2026-11-14
6
03:06
CodeRabbit
2026-11-14
2
00:22
Graphite
2026-11-12
1
19:03
Graphite
2026-11-18
1
08:15
Unclassified messages
Bot messages that no pattern recognized. They count toward no column above. Add a row to scripts/lib/reviewer-message-classes.tsv for each new shape (see docs/reviewer-report.md).
Gemini Code Assist β 102 unclassified messages
NoteGemini is unable to generate a review for this pull request due to the file types involved not bβ¦
Code Review This pull request introduces a new check to verify that the pr-auto-review caller job skβ¦
Code Review This pull request updates the Node.js engine requirements to >=20.19.4 and introduces paβ¦
CodeRabbit β 225 unclassified messages
``
``
``
CodeAnt β 165 unclassified messages
``
``
Skipping PR review because a bot author is detected. If you want to trigger CodeAnt AI, comment @codβ¦
Graphite β 38 unclassified messages
``
``
``
cubic β 191 unclassified messages
1 issue found across 9 files Prompt for AI agents (unresolved issues) Check if these issues are valiβ¦
8 issues found across 4 files (changes from recent commits). Prompt for AI agents (unresolved issuesβ¦
All reported issues were addressed Reply with feedback, questions, or to request a fix.View guided dβ¦
Coverage matrix
PRs each bot reviewed (a finding or clean message) out of the non-draft PRs opened in each repo in the window. Dependabot PRs are left out. off (by design) marks a bot that is meant not to run on that repo (declared in scripts/lib/reviewer-message-classes.tsv).
Repo
GitHub Copilot
Gemini Code Assist
Codex
CodeRabbit
SonarCloud
Qodo Merge
CodeAnt
Graphite
cubic
combsmith/.github
0 / 3
3 / 3
2 / 3
3 / 3
3 / 3
0 / 3
3 / 3
2 / 3
3 / 3
combsmith/.github-private
0 / 16
13 / 16
6 / 16
16 / 16
13 / 16
0 / 16
11 / 16
9 / 16
16 / 16
combsmith/broodly
0 / 3
2 / 3
1 / 3
2 / 3
3 / 3
0 / 3
2 / 3
2 / 3
3 / 3
combsmith/google-app-scripts
0 / 1
1 / 1
1 / 1
1 / 1
1 / 1
0 / 1
1 / 1
0 / 1
1 / 1
combsmith/software-factory
0 / 84
65 / 84
11 / 84
off (by design)
84 / 84
0 / 84
39 / 84
81 / 84
83 / 84
combsmith/software-factory-canary
0 / 10
4 / 10
0 / 10
7 / 10
10 / 10
0 / 10
1 / 10
0 / 10
10 / 10
Coverage overlap
PRs reviewed by β₯2 bots: 173 of 194 eligible (89%)
Higher overlap = more redundant coverage; lower = reviewers are specializing or missing PRs.
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 198
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.
pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with jq from review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.
Miss rate: 0% β 0 missed vs 0 caught (accepted findings; miss = found first by a bot after approval).
Approvals on partial evidence: 0 β pr-review approved via the advisory-gate timeout fallback (before all bots reported).
Approved PRs in window: 0.
Known gaps
Cost is not measured here. These reviewers are external SaaS GitHub Apps and emit no token-usage artifacts, so per-review $ cost is not observable from our side. For our own Claude reviewer's spend, see the Token Cost Observatory.
Comment usefulness / false-positive rate for the third-party reviewers requires judgment and is intentionally out of this deterministic report (a separate, human-approved LLM add-on). Note the Agent comment noise section above is a different, deterministic metric β it scores OUR automation's no-action share from the markers each comment already carries.
Latency baseline is PR-creation time (v1). A refinement to last-human-push is tracked for v2.
Source: scripts/reviewer_report.sh Β· .github/workflows/reviewer-report.yml β no LLM in this pipeline.
Activity
github-actions commentedon Jul 14, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-07-14 Β· org
petry-projectsΒ· 11 repos scanned Β· 191 PRs active (187 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Participation = distinct non-draft PRs a bot touched Γ· 187 review-eligible PRs. Latency = time from PR creation to the bot's first review/comment. Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
β = APPROVED reviews Β· π = CHANGES_REQUESTED Β· Resolved = share of a bot's inline threads later resolved Β· Comments/PR = inline findings per reviewed PR.
Reliability & no-shows
Coverage gap = eligible PRs a bot did not touch. External SaaS reviewers only sample a subset of PRs; this is a participation signal, not necessarily a fault.
Coverage overlap
Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Jul 14, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-07-14 Β· org
petry-projectsΒ· 11 repos scanned Β· 181 PRs active (177 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Jul 20, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-07-20 Β· org
petry-projectsΒ· 11 repos scanned Β· 157 PRs active (153 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Jul 27, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-07-27 Β· org
petry-projectsΒ· 11 repos scanned Β· 207 PRs active (206 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Aug 3, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-08-03 Β· org
petry-projectsΒ· 12 repos scanned Β· 35 PRs active (35 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 35
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Aug 10, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-08-10 Β· org
petry-projectsΒ· 12 repos scanned Β· 32 PRs active (32 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 32
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Aug 17, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-08-17 Β· org
petry-projectsΒ· 12 repos scanned Β· 8 PRs active (7 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 8
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Aug 24, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-08-24 Β· org
petry-projectsΒ· 12 repos scanned Β· 23 PRs active (23 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 23
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Aug 31, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-08-31 Β· org
petry-projectsΒ· 12 repos scanned Β· 6 PRs active (6 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 6
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Sep 7, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-09-07 Β· org
petry-projectsΒ· 11 repos scanned Β· 15 PRs active (15 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 15
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Sep 14, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-09-14 Β· org
petry-projectsΒ· 11 repos scanned Β· 3 PRs active (3 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 3
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Sep 21, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-09-21 Β· org
petry-projectsΒ· 11 repos scanned Β· 7 PRs active (7 non-draft, review-eligible)Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Rate-limited count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 7
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.feat(#1894): lower the advisory quorum to 4 β drop three non-participβ¦
github-actions commentedon Sep 28, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-09-28 Β· org
petry-projectsΒ· 11 repos scanned Β· 108 PRs active (105 non-draft, review-eligible)Truncated connections
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
petry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1084βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1084βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1098βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1116βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1159βreviewThreads.commentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1159βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1167βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1730βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1742βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1742βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1770βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1770βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1793βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1794βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1796βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1799βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1873βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1873βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1887βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1920βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1928βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1961βreviewstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/499βcommentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/499βreviewThreads.commentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/499βreviewstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/509βreviewThreads.commentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/509βreviewstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/8βcommentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/8βreviewThreads.commentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/8βreviewstruncatedpetry-projects/bmad-bgreat-suitePRhttps://github.com/petry-projects/bmad-bgreat-suite/pull/472βcommentstruncatedpetry-projects/bmad-bgreat-suitePRhttps://github.com/petry-projects/bmad-bgreat-suite/pull/472βreviewThreads.commentstruncatedpetry-projects/bmad-bgreat-suitePRhttps://github.com/petry-projects/bmad-bgreat-suite/pull/472βreviewstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/435βcommentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/51βcommentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/521βcommentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/521βreviewThreads.commentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/521βreviewstruncatedpetry-projects/broodminder-exportPRhttps://github.com/petry-projects/broodminder-export/pull/135βreviewThreads.commentstruncatedpetry-projects/broodminder-exportPRhttps://github.com/petry-projects/broodminder-export/pull/135βreviewstruncatedpetry-projects/broodminder-exportPRhttps://github.com/petry-projects/broodminder-export/pull/60βcommentstruncatedpetry-projects/broodminder-exportPRhttps://github.com/petry-projects/broodminder-export/pull/60βreviewstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/548βreviewstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/584βreviewThreadstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/584βreviewstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/585βcommentstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/585βreviewThreads.commentstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/585βreviewstruncatedpetry-projects/incubatorPRhttps://github.com/petry-projects/incubator/pull/121βreviewThreadstruncatedpetry-projects/incubatorPRhttps://github.com/petry-projects/incubator/pull/121βreviewstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/188βcommentstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/190βcommentstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/205βcommentstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/205βreviewThreads.commentstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/205βreviewstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/210βcommentstruncatedDeterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 108
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Oct 5, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-10-05 Β· org
petry-projectsΒ· 12 repos scanned Β· 179 PRs active (174 non-draft, review-eligible)Truncated connections
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
petry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1098βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1116βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1116βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1124βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1167βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1167βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1196βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1196βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1238βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1238βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1239βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1730βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1742βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1742βreviewThreadstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1742βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1770βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1770βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1793βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1794βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1796βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1799βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1873βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1873βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1887βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1891βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1902βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1909βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1952βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1952βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1953βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1953βreviewThreadstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1953βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1972βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1983βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1983βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2009βreviewThreadstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2009βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2010βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2011βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2019βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2022βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2033βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2040βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2048βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2048βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2049βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2049βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2051βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2051βreviewstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/468βcommentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/499βcommentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/499βreviewThreads.commentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/499βreviewstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/509βreviewThreads.commentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/509βreviewstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/8βcommentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/8βreviewThreads.commentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/8βreviewstruncatedpetry-projects/bmad-bgreat-suitePRhttps://github.com/petry-projects/bmad-bgreat-suite/pull/472βcommentstruncatedpetry-projects/bmad-bgreat-suitePRhttps://github.com/petry-projects/bmad-bgreat-suite/pull/472βreviewThreads.commentstruncatedpetry-projects/bmad-bgreat-suitePRhttps://github.com/petry-projects/bmad-bgreat-suite/pull/472βreviewstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/36βcommentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/435βcommentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/51βcommentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/521βcommentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/521βreviewThreads.commentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/521βreviewstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/135βcommentstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/135βreviewThreads.commentstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/135βreviewstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/145βcommentstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/60βcommentstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/60βreviewstruncatedpetry-projects/empower-personal-dashboardPRhttps://github.com/petry-projects/empower-personal-dashboard/pull/37βreviewstruncatedpetry-projects/empower-personal-dashboardPRhttps://github.com/petry-projects/empower-personal-dashboard/pull/46βcommentstruncatedpetry-projects/empower-personal-dashboardPRhttps://github.com/petry-projects/empower-personal-dashboard/pull/46βreviewThreadstruncatedpetry-projects/empower-personal-dashboardPRhttps://github.com/petry-projects/empower-personal-dashboard/pull/46βreviewstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/548βreviewstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/584βcommentstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/584βreviewThreadstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/584βreviewstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/585βcommentstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/585βreviewThreads.commentstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/585βreviewstruncatedpetry-projects/incubatorPRhttps://github.com/petry-projects/incubator/pull/102βcommentstruncatedpetry-projects/incubatorPRhttps://github.com/petry-projects/incubator/pull/121βcommentstruncatedpetry-projects/incubatorPRhttps://github.com/petry-projects/incubator/pull/121βreviewThreadstruncatedpetry-projects/incubatorPRhttps://github.com/petry-projects/incubator/pull/121βreviewstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/205βcommentstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/205βreviewThreads.commentstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/205βreviewstruncatedDeterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 179
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Oct 12, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-10-12 Β· org
petry-projectsΒ· 14 repos scanned Β· 254 PRs active (246 non-draft, review-eligible)Truncated connections
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
petry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1116βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1116βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1124βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1167βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1167βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1196βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1196βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1237βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1245βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1269βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1273βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1273βreviewstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1275βcommentstruncatedpetry-projects/.githubPRhttps://github.com/petry-projects/.github/pull/1298βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1730βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1730βreviewThreads.commentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1742βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1742βreviewThreadstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1742βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1770βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1770βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1796βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1796βreviewThreads.commentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1796βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1799βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1902βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1902βreviewThreadstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1902βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1909βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1953βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1953βreviewThreadstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/1953βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2031βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2033βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2040βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2048βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2048βreviewThreadstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2048βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2049βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2049βreviewThreadstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2049βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2051βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2051βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2052βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2054βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2054βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2084βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2084βreviewThreads.commentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2084βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2085βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2135βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2135βreviewThreadstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2135βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2150βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2150βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2157βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2157βreviewstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2158βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2172βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2182βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2190βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2218βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2239βcommentstruncatedpetry-projects/.github-privatePRhttps://github.com/petry-projects/.github-private/pull/2308βcommentstruncatedpetry-projects/ContentTwinPRhttps://github.com/petry-projects/ContentTwin/pull/480βcommentstruncatedpetry-projects/ContentTwinPRhttps://github.com/petry-projects/ContentTwin/pull/480βreviewstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/468βcommentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/509βcommentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/509βreviewThreads.commentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/509βreviewstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/520βcommentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/521βcommentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/521βreviewstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/8βcommentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/8βreviewThreads.commentstruncatedpetry-projects/TalkTermPRhttps://github.com/petry-projects/TalkTerm/pull/8βreviewstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/36βcommentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/51βcommentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/521βcommentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/521βreviewThreads.commentstruncatedpetry-projects/broodlyPRhttps://github.com/petry-projects/broodly/pull/521βreviewstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/135βcommentstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/135βreviewThreads.commentstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/135βreviewstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/145βcommentstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/60βcommentstruncatedpetry-projects/broodminder-dataPRhttps://github.com/petry-projects/broodminder-data/pull/60βreviewstruncatedpetry-projects/empower-personal-dashboardPRhttps://github.com/petry-projects/empower-personal-dashboard/pull/67βcommentstruncatedpetry-projects/empower-personal-dashboardPRhttps://github.com/petry-projects/empower-personal-dashboard/pull/69βcommentstruncatedpetry-projects/empower-personal-dashboardPRhttps://github.com/petry-projects/empower-personal-dashboard/pull/69βreviewstruncatedpetry-projects/empower-personal-dashboardPRhttps://github.com/petry-projects/empower-personal-dashboard/pull/72βcommentstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/548βcommentstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/548βreviewstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/584βcommentstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/584βreviewThreadstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/584βreviewstruncatedpetry-projects/google-app-scriptsPRhttps://github.com/petry-projects/google-app-scripts/pull/594βcommentstruncatedpetry-projects/incubatorPRhttps://github.com/petry-projects/incubator/pull/121βcommentstruncatedpetry-projects/incubatorPRhttps://github.com/petry-projects/incubator/pull/121βreviewThreadstruncatedpetry-projects/incubatorPRhttps://github.com/petry-projects/incubator/pull/121βreviewstruncatedpetry-projects/marketsPRhttps://github.com/petry-projects/markets/pull/502βcommentstruncatedpetry-projects/marketsPRhttps://github.com/petry-projects/markets/pull/513βcommentstruncatedpetry-projects/marketsPRhttps://github.com/petry-projects/markets/pull/513βreviewstruncatedpetry-projects/marketsPRhttps://github.com/petry-projects/markets/pull/517βcommentstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/205βcommentstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/205βreviewThreads.commentstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/205βreviewstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/226βcommentstruncatedpetry-projects/repo-templatePRhttps://github.com/petry-projects/repo-template/pull/233βcommentstruncatedDeterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 254
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Oct 19, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-10-19 Β· org
combsmithΒ· 14 repos scanned Β· 238 PRs active (235 non-draft, review-eligible)Truncated connections
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
combsmith/.githubPRhttps://github.com/combsmith/.github/pull/1116βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1116βreviewstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1196βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1196βreviewstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1245βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1298βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βreviewThreads.commentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1796βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1796βreviewThreads.commentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1796βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2052βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βreviewThreadstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2217βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2218βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2308βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/468βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βreviewThreads.commentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βreviewstruncatedcombsmith/broodlyPRhttps://github.com/combsmith/broodly/pull/51βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/187βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/194βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/195βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/69βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/69βreviewstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/82βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/86βcommentstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/584βcommentstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/584βreviewThreadstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/584βreviewstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/627βcommentstruncatedcombsmith/incubatorPRhttps://github.com/combsmith/incubator/pull/173βcommentstruncatedcombsmith/marketsPRhttps://github.com/combsmith/markets/pull/513βcommentstruncatedcombsmith/marketsPRhttps://github.com/combsmith/markets/pull/513βreviewstruncatedcombsmith/repo-templatePRhttps://github.com/combsmith/repo-template/pull/237βcommentstruncatedDeterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 238
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Oct 26, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-10-26 Β· org
combsmithΒ· 15 repos scanned Β· 275 PRs active (268 non-draft, review-eligible)Truncated connections
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
combsmith/.githubPRhttps://github.com/combsmith/.github/pull/1116βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1116βreviewstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1196βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1196βreviewstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1245βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1298βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βreviewThreads.commentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βreviewThreadstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βreviewThreadstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2217βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2218βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2369βreviewstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/468βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βreviewThreadstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βreviewThreads.commentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βreviewstruncatedcombsmith/broodlyPRhttps://github.com/combsmith/broodly/pull/36βcommentstruncatedcombsmith/broodlyPRhttps://github.com/combsmith/broodly/pull/51βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/194βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/110βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/114βreviewstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/69βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/69βreviewstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/627βcommentstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/649βreviewThreadstruncatedcombsmith/incubatorPRhttps://github.com/combsmith/incubator/pull/173βcommentstruncatedcombsmith/incubatorPRhttps://github.com/combsmith/incubator/pull/182βreviewstruncatedcombsmith/incubatorPRhttps://github.com/combsmith/incubator/pull/188βcommentstruncatedDeterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 275
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Nov 2, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-11-02 Β· org
combsmithΒ· 17 repos scanned Β· 226 PRs active (222 non-draft, review-eligible)Truncated connections
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
combsmith/.githubPRhttps://github.com/combsmith/.github/pull/1116βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1116βreviewstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1196βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1196βreviewstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1245βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1298βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1378βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1383βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1411βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βreviewThreads.commentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βreviewThreadstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2217βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2218βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2260βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/468βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βreviewThreadstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βreviewThreads.commentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/8βreviewstruncatedcombsmith/broodlyPRhttps://github.com/combsmith/broodly/pull/51βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/194βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/207βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/225βreviewstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/110βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/128βcommentstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/627βcommentstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/649βcommentstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/649βreviewThreadstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/649βreviewstruncatedcombsmith/repo-templatePRhttps://github.com/combsmith/repo-template/pull/256βcommentstruncatedCheck-run fetch failures
A check-run REST fetch failed on these PRs, so a check-run reporter clean pass may be missed and miscounted as No response (surfaced here, never silent β #1908):
combsmith/.github-private-ghsa-fx78-wx9r-525pPRhttps://github.com/combsmith/.github-private-ghsa-fx78-wx9r-525p/pull/1β check-run fetch failedcombsmith/.github-private-ghsa-vxf5-2jf3-4jxrPRhttps://github.com/combsmith/.github-private-ghsa-vxf5-2jf3-4jxr/pull/1β check-run fetch failedDeterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 226
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Nov 9, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-11-09 Β· org
combsmithΒ· 20 repos scanned Β· 267 PRs active (262 non-draft, review-eligible)Truncated connections
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
combsmith/.githubPRhttps://github.com/combsmith/.github/pull/1116βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1116βreviewstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1196βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1196βreviewstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1245βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1298βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1299βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1378βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1383βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1418βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1421βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1437βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1463βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βreviewThreads.commentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βreviewThreadstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1796βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1796βreviewThreads.commentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1796βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βreviewThreadstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2217βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2218βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2260βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2308βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/468βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/551βcommentstruncatedcombsmith/broodlyPRhttps://github.com/combsmith/broodly/pull/51βcommentstruncatedcombsmith/broodlyPRhttps://github.com/combsmith/broodly/pull/624βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/194βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/207βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/110βcommentstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/627βcommentstruncatedcombsmith/repo-templatePRhttps://github.com/combsmith/repo-template/pull/256βcommentstruncatedCheck-run fetch failures
A check-run REST fetch failed on these PRs, so a check-run reporter clean pass may be missed and miscounted as No response (surfaced here, never silent β #1908):
combsmith/.github-ghsa-qpjq-vrh6-8wcgPRhttps://github.com/combsmith/.github-ghsa-qpjq-vrh6-8wcg/pull/1β check-run fetch failedcombsmith/.github-private-ghsa-fx78-wx9r-525pPRhttps://github.com/combsmith/.github-private-ghsa-fx78-wx9r-525p/pull/1β check-run fetch failedcombsmith/.github-private-ghsa-rw84-v39h-45cpPRhttps://github.com/combsmith/.github-private-ghsa-rw84-v39h-45cp/pull/1β check-run fetch failedcombsmith/TalkTerm-ghsa-pc5f-cw67-7cmmPRhttps://github.com/combsmith/TalkTerm-ghsa-pc5f-cw67-7cmm/pull/1β check-run fetch failedDeterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh); rate-limit/out-of-quota notices are detected by body text. Reviews and Refused count every event, not distinct PRs β a PR re-reviewed across N commits contributes N reviews. No response counts eligible PRs the reviewer never engaged with at all. Latency = time from PR creation to the bot's first real review (refusals excluded). Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
Coverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 267
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.github-actions commentedon Nov 16, 2026
π Third-Party Reviewer Scorecard β 7-day Report (2026-11-16): withdrawn
Warning
This report was wrong and its figures have been removed. It counted 0 reviews for every bot.
Ignore this week's outage alerts, coverage matrix and quota timing as well.
Use the corrected report instead: #1220 (comment) (generated 2026-11-19, covering the 7 days to that date).
Why it was wrong.
ubuntu-latestmoved from Ubuntu 24.04 to 26.04 between the 2026-11-09 and 2026-11-16 runs. Ubuntu 26.04 ships jq 1.8, which parses the bot check inscripts/reviewer_report.shdifferently, so no message was recognized as coming from a bot. The run itself succeeded, so nothing flagged the error. The fix is #2568 (issue #2547), merged 2026-11-19.Window not re-run. The corrected report covers 2026-11-12 to 2026-11-19, not this report's 2026-11-09 to 2026-11-16. Next Monday's scheduled report resumes the weekly series.
Rewritten by Claude Code on behalf of the maintainer. The original all-zero report is in this comment's edit history.
github-actions commentedon Nov 19, 2026
π Third-Party Reviewer Scorecard β 7-day Report
Generated 2026-11-19 Β· org
combsmithΒ· 20 repos scanned Β· 198 PRs active (194 non-draft, review-eligible)Truncated connections
These per-PR connections hit the GraphQL fetch cap, so the affected reviewer counts may undercount on these PRs (truncation is reported here, never silent β #1908):
combsmith/.githubPRhttps://github.com/combsmith/.github/pull/1116βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1116βreviewstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1117βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1218βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1298βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1378βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1383βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1426βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1498βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1503βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1503βreviewstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1506βcommentstruncatedcombsmith/.githubPRhttps://github.com/combsmith/.github/pull/1514βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βreviewThreads.commentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1730βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βreviewThreadstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1770βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1796βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1796βreviewThreads.commentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/1796βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βreviewThreadstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2135βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2217βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2218βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2260βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2290βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2308βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2308βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2319βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2328βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2368βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2483βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2488βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2488βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2494βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2505βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2505βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2516βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2517βcommentstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2517βreviewstruncatedcombsmith/.github-privatePRhttps://github.com/combsmith/.github-private/pull/2536βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/549βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/551βcommentstruncatedcombsmith/TalkTermPRhttps://github.com/combsmith/TalkTerm/pull/580βcommentstruncatedcombsmith/broodlyPRhttps://github.com/combsmith/broodly/pull/51βcommentstruncatedcombsmith/broodlyPRhttps://github.com/combsmith/broodly/pull/645βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/194βcommentstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/194βreviewstruncatedcombsmith/broodminder-dataPRhttps://github.com/combsmith/broodminder-data/pull/207βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/110βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/178βcommentstruncatedcombsmith/empower-personal-dashboardPRhttps://github.com/combsmith/empower-personal-dashboard/pull/180βcommentstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/627βcommentstruncatedcombsmith/google-app-scriptsPRhttps://github.com/combsmith/google-app-scripts/pull/680βcommentstruncatedcombsmith/repo-templatePRhttps://github.com/combsmith/repo-template/pull/256βcommentstruncatedOutage alerts
combsmith/.github,combsmith/.github-private,combsmith/broodly,combsmith/google-app-scripts,combsmith/software-factory,combsmith/software-factory-canary; first silent day 2026-11-12.combsmith/.github-private,combsmith/broodly,combsmith/software-factory; first silent day 2026-11-15.combsmith/.github,combsmith/.github-private,combsmith/broodly,combsmith/google-app-scripts,combsmith/software-factory,combsmith/software-factory-canary; first silent day 2026-11-12.combsmith/.githubreviewed in the whole window; first silent day 2026-11-13.combsmith/.github-privatereviewed in the whole window; first silent day 2026-11-12.combsmith/broodlyreviewed in the whole window; first silent day 2026-11-12.combsmith/software-factoryreviewed in the whole window; first silent day 2026-11-12.combsmith/software-factory-canaryreviewed in the whole window; first silent day 2026-11-13.combsmith/software-factory-canaryreviewed in the whole window; first silent day 2026-11-13.combsmith/.githubreviewed in the whole window; first silent day 2026-11-13.combsmith/.github-privatereviewed in the whole window; first silent day 2026-11-12.combsmith/broodlyreviewed in the whole window; first silent day 2026-11-12.combsmith/software-factoryreviewed in the whole window; first silent day 2026-11-12.combsmith/software-factory-canaryreviewed in the whole window; first silent day 2026-11-13.combsmith/software-factory-canaryreviewed in the whole window; first silent day 2026-11-13.Rules and how to mark a cell off by design:
docs/reviewer-report.md(Outage alerts).Deterministic report β every figure is computed with
jq/awkfrom GitHub review data; no LLM is involved. Bots are identified by their GraphQL App login (scripts/lib/advisory-review-gate.sh). Every bot message β review, inline comment, top-level comment, check run β gets exactly one class (finding,clean,refused,skipped,statusorunclassified) from that bot's patterns (scripts/lib/reviewer-message-classes.tsv) first and the shared rate-limit pattern second. A PR counts as reviewed only when the bot left afindingorcleanmessage on it; status notices and unclassified messages never count. Each eligible PR lands in exactly one bucket per bot, so the five PR columns sum to Total PRs. Latency = time from PR creation to the bot's firstfindingorcleanmessage. Deltas (β²/βΌ) are vs the prior week and directional only.Scorecard
findingmessage (an inline thread, a review that requests changes, a body that reports issues).cleanmessage and nofinding.statusorunclassifiedmessages.Details
finding/cleancomments counted instead.refusedmessage, each occurrence.Quota timing
Refusals (quota, rate limit, trial or plan) per bot per UTC day, and the time of the first one that day. Dependabot PRs are left out.
Unclassified messages
Bot messages that no pattern recognized. They count toward no column above. Add a row to
scripts/lib/reviewer-message-classes.tsvfor each new shape (seedocs/reviewer-report.md).NoteGemini is unable to generate a review for this pull request due to the file types involved not bβ¦Code Review This pull request introduces a new check to verify that the pr-auto-review caller job skβ¦Code Review This pull request updates the Node.js engine requirements to >=20.19.4 and introduces paβ¦Skipping PR review because a bot author is detected. If you want to trigger CodeAnt AI, comment @codβ¦1 issue found across 9 files Prompt for AI agents (unresolved issues) Check if these issues are valiβ¦8 issues found across 4 files (changes from recent commits). Prompt for AI agents (unresolved issuesβ¦All reported issues were addressed Reply with feedback, questions, or to request a fix.View guided dβ¦Coverage matrix
PRs each bot reviewed (a
findingorcleanmessage) out of the non-draft PRs opened in each repo in the window. Dependabot PRs are left out.off (by design)marks a bot that is meant not to run on that repo (declared inscripts/lib/reviewer-message-classes.tsv).combsmith/.githubcombsmith/.github-privatecombsmith/broodlycombsmith/google-app-scriptscombsmith/software-factorycombsmith/software-factory-canaryCoverage overlap
Agent comment noise
Agent comments: 0
No-action comments: 0 (n/a of agent comments)
Active PRs (window): 198
PRs with agent comments: 0
Affected PRs (β₯1 no-action comment): 0
No-action comments per PR: 0.00
Noise = agent comments that ask nothing of a human β a reported no-change, a no-op engine run, or a clean/repeat approval. Actionable agent comments (applied fixes, escalated findings, human-attention flags) are excluded. Classification is deterministic (
scripts/lib/comment-noise.sh), computed from the automation markers each comment already carries β no LLM.pr-review miss rate
The false-NEGATIVE signal (#1596): accepted defects a trusted advisory bot found on a PR pr-review had already approved. Deterministic β computed with
jqfrom review data, no LLM. Only findings resolved as accepted count; refuted false positives and ambiguous dispositions do not.Known gaps
Source:
scripts/reviewer_report.shΒ·.github/workflows/reviewer-report.ymlβ no LLM in this pipeline.feat(reviewer-report): declare Copilot and Qodo off on every repo witβ¦