Repository navigation
chore(specs): gardener checkbox sync - #870
Merged
Merged
Conversation
Deploying mcpproxy-docs with
|
| Latest commit: |
9be3158
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://e00ac674.mcpproxy-docs.pages.dev |
| Branch Preview URL: | https://claude-spec-gardener.mcpproxy-docs.pages.dev |
Member
Author
|
The failing This looks like a version-skew regression introduced by #863 ( Generated by Claude Code |
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Contributor
📦 Build ArtifactsWorkflow Run: View Run Available Artifacts
How to DownloadOption 1: GitHub Web UI (easiest)
Option 2: GitHub CLI gh run download 30239412615 --repo smart-mcp-proxy/mcpproxy-go
|
Dumbris
force-pushed
the
claude/spec-gardener
branch
from
July 27, 2026 05:18
6e2030a to
9be3158
Compare
Dumbris
marked this pull request as ready for review
July 29, 2026 03:04
2 of 3 tasks
Dumbris
added a commit
that referenced
this pull request
Aug 31, 2026
…und 3 All four verified against GitHub and the tree before applying. 1. windows-tray: "no Windows-tray PR is open or merged" was false. #74 ("feature: windows tray support") merged 2025-10-23 and the cross-platform Go/systray build still ships on Windows. The claim now scopes to what the epic actually tracks — the native WebView2 replacement — for which no PR exists and native/windows/ is a README placeholder. 2. analytics-dashboard: "can never tick from code" conflated two things. A human can run spec 069's T023 Playwright sweep and tick the box; what can never exist is code evidence for it. Reworded so the distinction is explicit. 3. hygiene-spec-gardener: "three gardener runs ... auto-approved by #842" was wrong. Only #999 carries a github-actions APPROVED review. #824 has none and merged ~24 minutes BEFORE the auto-approve workflow landed; #870 records no approving review either. 4. token-bench-public: calling the MCP-specific third "integrated" overstated completion. ToolRet and LiveMCPTool are static retrieval/token corpora and lap-score is a linter — none is a public agent-loop task suite, which is what the task ultimately owes. Also noted for the record: the linux-arm64 Build Binaries failure on the previous commit is a flaky test, not a regression from this docs-only change. TestServiceStartScanDeepOnRunsSourceResolutionAndPass2 (internal/security/scanner) passes locally with -count=3, and two runs of this same branch four minutes apart — 33360429077 and 33360648947, both docs-only — disagreed on it.
Dumbris
added a commit
that referenced
this pull request
Aug 31, 2026
…ithub warnings (#1136) * chore(roadmap): truth-sync statuses against code, clear all --check-github warnings An evidence-based re-derivation of every non-done roadmap.yaml leaf. Each flip below was verified against the tree, git/gh state and the rendering sites, then independently re-checked by an adversarial pass; claims that did not survive that pass were dropped rather than applied. Statuses that were wrong in the "already shipped" direction: - upgrade-nudge-delta todo -> done (#1085, closes #1081). The note asserted TODO(spec-079/FR-002) markers exist and that the checker calls only /releases/latest; both are false. internal/updatecheck/delta.go now computes the delta and all six surfaces render it. - upgrade-nudge epic in_progress -> done (all five tasks now done). - analytics-dashboard in_progress -> done. Both children shipped; the default-landing half is live (router '/' -> Dashboard, guarded by dashboard-default-landing.spec.ts). Spec 069's one open task is a local Playwright sweep that leaves no artifact and can never tick from code. - discovery-eval-harness todo -> done. Both halves ship: cmd/scan-eval for security recall/FP, and the eval.yml retrieval-d1 gate plus bench/'s own RecallAtK/NDCGAtK + recall@5 parity gate for discovery quality. - hygiene-spec-gardener in_progress -> done (#824, #870, #999, #842). Its recorded blocker is cleared: check-spec-evidence.py is on main. Statuses that were wrong in the "claimed progress it does not have" direction: - windows-tray and windows-tray-window in_review -> todo. No Windows-tray PR is open or merged and native/windows/ is a README placeholder, so "in review" pointed at nothing. The section header is renamed to match. - planning-hygiene todo -> in_progress and action-log-transparency todo -> in_progress; both already have done children. Notes corrected where they would misdirect the next implementer: - token-bench-harness claimed it absorbs discovery-eval-harness by measuring retrieval recall. Recall is already measured, and bench/ already covers token counting, encoding arms (incl. direct_deferred), break-even and reporting. The note now records what is genuinely open: activity-log session replay, measured first-call success/retry rate, and tokens-per-successful-task. - token-bench-public: the MCP-specific third already ships via spec 083 (ToolRet, LiveMCPBench snapshot, lap-score); only agent-loop suites remain. specs/098-tools-preflight/tasks.md: tick T001, T024, T031, each confirmed against the tree (doc.go states both invariants T001 requires; toolslist_snapshot_test.go is labelled T024 and green; README carries all three FR-017 examples). T025/T027/T032/T033 are left unchecked — they are one-time process acts or did not survive verification. Deliberately NOT changed: no epics were added for shipped specs (084-100). roadmap.yaml is the working set by design and ROADMAP.md already renders a generated per-spec progress table, so absence of an epic is not drift. scripts/gen-roadmap.py --check-github: 3 warnings -> 0 errors, 0 warnings. * fix(roadmap): correct six claims flagged by cross-model review All six findings from the opencode gpt-5.6-sol review were independently re-verified against the tree before being applied; none were false positives. (The reviewer's initial claim that spec-gardener-auto-approve.yml is missing WAS a false positive from its glob tool and it self-corrected — the file exists at .github/workflows/spec-gardener-auto-approve.yml.) 1. Un-tick specs/098 T024. T024 asks for a golden snapshot test proving tools/list is BYTE-IDENTICAL across all three modes. Spec 099 (FR-014/015) later converted that test from a no-delta gate into an ENUMERATED-delta gate (toolsListAllowedDelta permits describe_tool and quarantine_security to move), so at HEAD the test no longer demonstrates the property T024 states. It may well have held on 098's own branch, but that is not checkable from the tree, and an unverifiable tick is exactly what this PR exists to remove. Badge 29/33 -> 28/33. 2. discovery-eval-harness note: drop the "SUPERSEDED — folded into token-bench-harness" framing. It contradicted itself once the epic was marked done — token-bench-harness is still unbuilt, so nothing could have been folded into it. Both halves shipped independently. 3. Same note: the eval.yml retrieval-d1 job is REPORT-ONLY on pull requests (continue-on-error, guarding against npx/uvx fetch flake) and blocks only on the nightly schedule. Promoting it to PR-blocking is still open (MCP-742). 4. Same note: the SC-003 recall@5 = 0.68 +/- 0.05 parity gate is asserted in bench/armindex_test.go, not armindex.go — the latter supplies the production index wiring, not the assertion. 5. token-bench-harness note: the literature-derived retry defaults live in bench/session.go (armRetryRates, research D8). bench/report.go only renders and labels them as estimates. The substantive point stands: retry rates are assumed, not observed. 6. token-bench-public note: "CI-wired" overstated the LiveMCPBench snapshot. bench.yml's regular job passes only -corpus-v2; the snapshot is runnable via -livemcptool but not run in CI, and its loader has no retrieval golden set, so it measures token/scale rather than recall. ToolRet does have its own bench.yml job. gen-roadmap.py --check and --check-github both still clean. * fix(roadmap): tighten two claims from cross-model review round 2 Both findings verified against the tree before applying. 1. The retrieval-D1 blocking scope was stated too narrowly. eval.yml sets continue-on-error only for github.event_name == 'pull_request', and the workflow triggers on pull_request, workflow_dispatch and schedule — so the job is report-only on PRs and BLOCKING on both nightly and manual dispatch runs, not "only on the nightly schedule". The note now says so, and records that the workflow's own inline comment carries the same omission. 2. discovery-eval-harness is now done but still declared depends_on: [token-bench-harness], which is unbuilt — a done node cannot depend on an open one, and the edge contradicted the corrected note saying both halves shipped independently. Edge removed; the id remains a valid depends_on target for anything that needs it. gen-roadmap.py --check and --check-github both clean. * fix(roadmap): scope four over-broad claims from cross-model review round 3 All four verified against GitHub and the tree before applying. 1. windows-tray: "no Windows-tray PR is open or merged" was false. #74 ("feature: windows tray support") merged 2025-10-23 and the cross-platform Go/systray build still ships on Windows. The claim now scopes to what the epic actually tracks — the native WebView2 replacement — for which no PR exists and native/windows/ is a README placeholder. 2. analytics-dashboard: "can never tick from code" conflated two things. A human can run spec 069's T023 Playwright sweep and tick the box; what can never exist is code evidence for it. Reworded so the distinction is explicit. 3. hygiene-spec-gardener: "three gardener runs ... auto-approved by #842" was wrong. Only #999 carries a github-actions APPROVED review. #824 has none and merged ~24 minutes BEFORE the auto-approve workflow landed; #870 records no approving review either. 4. token-bench-public: calling the MCP-specific third "integrated" overstated completion. ToolRet and LiveMCPTool are static retrieval/token corpora and lap-score is a linter — none is a public agent-loop task suite, which is what the task ultimately owes. Also noted for the record: the linux-arm64 Build Binaries failure on the previous commit is a flaky test, not a regression from this docs-only change. TestServiceStartScanDeepOnRunsSourceResolutionAndPass2 (internal/security/scanner) passes locally with -count=3, and two runs of this same branch four minutes apart — 33360429077 and 33360648947, both docs-only — disagreed on it. * fix(roadmap): ToolRet is not MCP-specific (cross-model review round 4) Verified: bench/corpusio/toolret.go documents ToolRet as a general tool-retrieval corpus whose documentation fields are "heterogeneous JSON/text, NOT MCP input schemas" (Schema stays nil). Grouping it under "the MCP-specific third" was wrong. The token-bench-public note now says public corpus/linter groundwork is integrated while NO public agent-loop task suite is, and names the ToolRet caveat explicitly so the next implementer does not re-inherit the error. Round 4 found no other issues; the round-3 windows-tray, analytics T023 and gardener corrections were all confirmed accurate against GitHub and the tree. * fix(roadmap): discovery-eval-harness is in_progress, not done (review round 5) The final review round caught an error in this PR's own flip, and it is the same failure mode the PR was written to fix: an epic marked done while a governing FR is unmet. Spec 065 FR-009 and SC-005 require CI to FAIL on a discovery regression beyond tolerance. The retrieval-D1 job sets continue-on-error for pull_request (D1 depends on npx/uvx fetches, a known flake source), so on the PR path CI does not fail — and eval.yml's own comment records the promotion to PR-blocking as still planned (MCP-742). The HARNESS shipped; the GATE did not. Reverted to in_progress and added the remaining work as a child task (discovery-eval-pr-blocking, MCP-742) so the epic's status has something in the DAG to point at. The reviewer also read CN-002 (frozen corpus, never live) as violated because D1 boots a live mcpproxy serving seven reference servers. Recorded as a weaker tension to adjudicate rather than a violation: #931 pinned all seven upstreams to freeze-era versions and the job gates on the exact corpus ID set, so the corpus is reproducible in practice. Whether that satisfies CN-002 or a committed snapshot is required is a judgement for the spec owner. gen-roadmap.py --check and --check-github both clean (0 errors, 0 warnings).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spec-gardener run: 2026-07-20. Applied 15 checkbox ticks, proposing 1 un-tick candidate (NOT applied), across 34 specs with tick/un-tick candidates surfaced by
scripts/check-spec-evidence.py.Applied ticks
All 15 were independently re-verified by the gardener session (reading the cited files directly) in addition to the sub-agent that originally proposed each.
Proposed un-ticks (NOT applied)
All other 85 UNRESOLVED/REMOVED/RELOCATED findings from the checker were investigated and found to be RELOCATED-NOT-ABSENT — the cited evidence path was stale (file renamed/consolidated/moved), but the actual behavior exists elsewhere in the current codebase (e.g.
internal/contracts/config.go→ consolidated intointernal/contracts/types.go;cmd/mcpproxy/teams_register.go→ renamed toserveredition_register.goas part of a deliberate Teams→Server-Edition rename). These are correctly left ticked and are not proposed as un-ticks.Dropped by verification
130
possibly_builtcandidates (unchecked tasks where every cited file happens to exist) were investigated and dropped — the file existed but the specific promised behavior was absent, differently implemented without meeting the task's bar, an unwired component, a stub, or an untested claim. Grouped by spec:001-code-execution— 12 dropped001-oas-endpoint-documentation— 6 dropped001-update-version-display— 6 dropped003-tool-annotations-webui— 3 dropped004-management-health-refactor— 9 dropped006-oauth-extra-params— 8 dropped007-oauth-e2e-testing— 5 dropped008-oauth-token-refresh— 6 dropped009-proactive-oauth-refresh— 19 dropped010-release-notes-generator— 1 dropped011-resource-auto-detect— 1 dropped014-cli-output-formatting— 1 dropped016-activity-log-backend— 1 dropped017-activity-cli-commands— 9 dropped019-activity-webui— 1 dropped021-request-id-logging— 1 dropped026-pii-detection— 3 dropped028-agent-tokens— 1 dropped029-mcpproxy-teams— 1 dropped040-server-ux— 4 dropped042-telemetry-tier2— 12 dropped044-diagnostics-taxonomy— 10 dropped044-retention-telemetry-v3— 3 dropped056-output-schema-validation— 1 dropped057-in-proxy-profiles— 1 dropped073-activity-size-retention— 1 dropped076-deterministic-tool-scanner— 1 dropped077-scanner-simplification— 2 dropped083-discovery-profiler— 1 droppedMethodology note
Given the volume (147
possibly_builtcandidates + 88 UNRESOLVED/REMOVED/RELOCATED findings across 34 specs), verification was fanned out across 8 parallel research sub-agents (one per group of specs), each applying the tick test from this repo's spec-gardener instructions: state the promised behavior, cite file:line, confirm it's real (not a stub/comment/unwired component/skipped test), and default to not-ticking when uncertain. The gardener session then independently re-verified every candidate the sub-agents recommended ticking by reading the cited source directly before applying any checkbox change, and dropped two of a sub-agent's proposed ticks where the task named a specific artifact (a named computed property) that did not actually exist in the code even though the surrounding behavior worked via a different mechanism (009-proactive-oauth-refresh T054, T060 — dropped despite sibling tasks T055/T056/T061/T063 for the same feature area being ticked, since those name outcomes rather than specific artifacts).ROADMAP.mdwas regenerated viascripts/gen-roadmap.pyin a separate commit after the checkbox sync, per the pre-commit hook's staleness check.Note: the branch was rebased onto a newer
mainafter this PR opened; the diff content is unchanged (same 15 checkbox ticks, same files) — verified against the current file list.