Skip to content

fix(backup): reject repeated Drive collaboration page tokens - #1080

Closed
SebTardif wants to merge 1 commit into
openclaw:mainfrom
SebTardif:fix/backup-collab-page-token
Closed

SebTardif wants to merge 1 commit into
openclaw:mainfrom
SebTardif:fix/backup-collab-page-token

Conversation

@SebTardif

Copy link
Copy Markdown
Contributor

What Problem This Solves

gog backup push --drive-collaboration lists Drive permissions, comments, and revisions for each file. The three helpers in internal/cmd/backup_drive_collaboration.go copied nextPageToken into the next request with no seen-set.

When Drive repeats a continuation token, those loops never terminate. Backup collection keeps requesting the same permission, comment, or revision page and never finishes the collaboration snapshot.

The same hang class is already closed for Chat and Classroom backup (#1063), Drive sync push listing (#1065), Drive audit permission listing (#1066), calendar and Gmail listing (#1004), and People/Gmail-from-contact/contacts-export listing (#1044, #1045, #1046). Drive backup file listing is covered separately in #1078. Collaboration listing was still on the unguarded loop.

Evidence

terminal output from the compiled internal/cmd listing binary after the patch. A stuck continuation token is rejected after two list calls, with no third request:

$ gogcli-backup-collab-paging.exe -test.run TestFetchBackupDriveCollaborationRejectsRepeatedPageToken -test.v
    backup_drive_collaboration_test.go:139: err = drive file file-1 comments: pagination loop: repeated page token "stuck" after 2 list calls
    backup_drive_collaboration_test.go:139: err = drive file file-1 revisions: pagination loop: repeated page token "stuck" after 2 list calls
    backup_drive_collaboration_test.go:139: err = drive file file-1 permissions: pagination loop: repeated page token "stuck" after 2 list calls

Before the patch, the same stuck token made a third list request and returned HTTP 400 from the safety cap (unexpected extra ... page request after 3 list calls) instead of stopping on the repeated token.

Real behavior proof

  • Behavior or issue addressed: Drive backup collaboration listing hung when Google repeated a page token. gog backup push --drive-collaboration never finished permission, comment, or revision pages.
  • Real environment tested: Windows, Go 1.27.1, branch fix/backup-collab-page-token at current HEAD, compiled internal/cmd listing binary against a local httptest Drive endpoint.
  • Exact steps or command run after this patch: Compiled the listing binary with go, then ran gogcli-backup-collab-paging.exe with -test.v on the permission, comment, and revision hang-guard cases.
  • Evidence after fix: terminal output from that binary shows pagination loop: repeated page token "stuck" after 2 list calls for permissions, comments, and revisions.
  • Observed result after fix: A repeated Drive collaboration page token now fails closed with the shared paginator error after two requests. Distinct pages still concatenate. Comments reuse listDriveComments with all: true. Permission listing still sends supportsAllDrives=true and pageSize=100. Revision listing still uses pageSize=200.
  • What was not tested: A live Google Drive account with a real repeated token. Encrypted shard write after the listing error.

Summary

Route fetchBackupDrivePermissions and fetchBackupDriveRevisions through existing collectAllPages. Reuse listDriveComments with all: true for comments. Keep the original field lists, page sizes, and shared-drive flags.

Related: #1063, #1065, #1066, #1004, #1044, #1045, #1046, #1078.

Introduced in efc3df2e (2026-04-27, feat(backup): expand google backup coverage).

Route permission and revision backup listing through collectAllPages.
Reuse listDriveComments with all:true for comments so a repeated Google
page token fails closed instead of hanging gog backup push
--drive-collaboration.

Signed-off-by: Sebastien Tardif <SebTardif@ncf.ca>
@clawsweeper

clawsweeper Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Sep 5, 2026
@clawsweeper

clawsweeper Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Codex review: blocked before merge. Reviewed September 5, 2026, 3:07 AM ET / 07:07 UTC.

ClawSweeper review

What this changes

Guard Drive backup permission, comment, and revision pagination using existing collectors, with regression coverage for repeated tokens, successful pagination, and recorded errors.

Merge readiness

⛔ Blocked before merge - 3 items remain

The fix remains necessary: current main and v0.39.0 still contain the unguarded collaboration loops. The implementation preserves existing error handling, and the supplied production-client fault injection supports the repeated-token fix.

Priority: P2
Reviewed head: f57392f1d8f6304a19e2b2e3008737300610ceaf

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused, source-supported repair with appropriate production-client fault evidence and bounded documentation follow-up.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (terminal): The captured Windows terminal run exercises all three changed production listing helpers through Google’s generated Drive HTTP client against an injected repeated-token server response, observing rejection after two requests. This is sufficient for the internal pagination repair; it does not claim live-account or encrypted-shard coverage.
Patch quality 🐚 platinum hermit (4/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The captured Windows terminal run exercises all three changed production listing helpers through Google’s generated Drive HTTP client against an injected repeated-token server response, observing rejection after two requests. This is sufficient for the internal pagination repair; it does not claim live-account or encrypted-shard coverage.
Evidence reviewed 10 items Applicable repository policy: Read the complete root AGENTS.md; no nested AGENTS.md or matching maintainer notes were found. The repository requires user-visible fixes in Unreleased with references and contributor credit.
Current main still needs the fix: All three collaboration helpers on the pinned main revision advance continuation tokens without remembering previously visited tokens.
Latest release comparison: The v0.39.0 tag resolves to b058dbb. Its collaboration, comment-listing, and paginator sources are unchanged on pinned main, so the latest release does not contain this fix.
Findings 1 actionable finding [P3] Add the required Unreleased changelog entry
Security None None.

How this fits together

Drive backup collects collaboration metadata for each discovered file through Google’s Drive client. Successful records and per-file error rows then feed encrypted backup shards.

flowchart TD
  A[Backup push and Drive file IDs] --> B[Collaboration collector]
  B --> C[Google Drive HTTP lists]
  C --> D{Pagination result}
  D -->|Another unique token| C
  D -->|Complete| E[Collaboration metadata]
  D -->|Repeated token or page limit| F[Per-file error row]
  E --> G[Encrypted backup shards]
  F --> G
Loading

Before merge

  • Add the required Unreleased changelog entry (P3) - This changes user-visible backup failure behavior, but the branch contains no CHANGELOG.md update. The applicable repository policy requires an Unreleased entry with a PR reference and contributor thanks when the fix lands. Add that entry, including the inherited page limit, without editing a published release section.
  • Resolve merge risk (P1) - A valid per-file listing needing more than 10,000 pages now loses its collected metadata in favor of an error row while the backup can continue; this additional limit is not disclosed in the PR’s user-facing documentation.
  • Complete next step (P2) - Add an Unreleased changelog entry with contributor credit and document the inherited 10,000-page limit and per-file error-row behavior.

Findings

  • [P3] Add the required Unreleased changelog entry — internal/cmd/backup_drive_collaboration.go:150
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test delta Production +22/−37, net −15; tests +247 Existing paginator reuse reduces production code while adding repeated-token, ordered-success, and aggregate-error coverage.

Merge-risk options

Maintainer options:

  1. Disclose the inherited collection limit (recommended)
    Retain the established shared guard and document that exceeding 10,000 pages records a per-file error rather than complete metadata.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Add the required Unreleased changelog entry with contributor credit and document Drive collaboration’s repeated-token and 10,000-page limits, per-file error rows, and continued sibling collection in docs/backup.md; preserve runtime behavior and the existing shard format.

Technical review

Best possible solution:

Keep the shared pagination guards and existing per-file error contract, with explicit documentation of the inherited page ceiling and partial-backup outcome.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: returning the same nonempty continuation token repeatedly keeps each current-main helper requesting pages. The contributor supplies bounded before/after HTTP fault-injection evidence; this review did not execute it.

Is this the best way to solve the issue?

Yes. Reusing the existing collector and the comments all-pages path avoids duplicate guard logic while preserving request and ordinary-error semantics; the additional page ceiling needs disclosure.

Full review comments:

  • [P3] Add the required Unreleased changelog entry — internal/cmd/backup_drive_collaboration.go:150
    This changes user-visible backup failure behavior, but the branch contains no CHANGELOG.md update. The applicable repository policy requires an Unreleased entry with a PR reference and contributor thanks when the fix lands. Add that entry, including the inherited page limit, without editing a published release section.
    Confidence: 0.97

Overall correctness: patch is correct
Overall confidence: 0.93

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 25703c789da5.

Labels

Label changes:

  • add P2: This bounded backup reliability fix prevents repeated provider tokens from keeping collaboration collection running indefinitely.
  • add merge-risk: 🚨 compatibility: Previously unbounded valid listings now inherit a 10,000-page ceiling and can finish with an error row instead of complete metadata.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The captured Windows terminal run exercises all three changed production listing helpers through Google’s generated Drive HTTP client against an injected repeated-token server response, observing rejection after two requests. This is sufficient for the internal pagination repair; it does not claim live-account or encrypted-shard coverage.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The captured Windows terminal run exercises all three changed production listing helpers through Google’s generated Drive HTTP client against an injected repeated-token server response, observing rejection after two requests. This is sufficient for the internal pagination repair; it does not claim live-account or encrypted-shard coverage.

Label justifications:

  • P2: This bounded backup reliability fix prevents repeated provider tokens from keeping collaboration collection running indefinitely.
  • merge-risk: 🚨 compatibility: Previously unbounded valid listings now inherit a 10,000-page ceiling and can finish with an error row instead of complete metadata.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The captured Windows terminal run exercises all three changed production listing helpers through Google’s generated Drive HTTP client against an injected repeated-token server response, observing rejection after two requests. This is sufficient for the internal pagination repair; it does not claim live-account or encrypted-shard coverage.
  • proof: sufficient: Contributor real behavior proof is sufficient. The captured Windows terminal run exercises all three changed production listing helpers through Google’s generated Drive HTTP client against an injected repeated-token server response, observing rejection after two requests. This is sufficient for the internal pagination repair; it does not claim live-account or encrypted-shard coverage.

Evidence

Acceptance criteria:

  • [P1] git diff --check.

What I checked:

  • Applicable repository policy: Read the complete root AGENTS.md; no nested AGENTS.md or matching maintainer notes were found. The repository requires user-visible fixes in Unreleased with references and contributor credit. (AGENTS.md:38, f57392f1d8f6)
  • Current main still needs the fix: All three collaboration helpers on the pinned main revision advance continuation tokens without remembering previously visited tokens. (internal/cmd/backup_drive_collaboration.go:149, 25703c789da5)
  • Latest release comparison: The v0.39.0 tag resolves to b058dbb. Its collaboration, comment-listing, and paginator sources are unchanged on pinned main, so the latest release does not contain this fix. (internal/cmd/backup_drive_collaboration.go:149, b058dbbaeb44)
  • Introduced behavior and compatibility boundary: The verified main-to-head diff preserves request fields, page sizes, context, and shared-drive support. All three listings newly inherit the shared collector’s 10,000-page ceiling and nil-on-error behavior. (internal/cmd/paging.go:24, f57392f1d8f6)
  • Existing partial-backup contract: The unchanged aggregate converts listing failures into file-specific error rows, continues other metadata listings, and counts collaboration errors. The snapshot builder serializes those rows into the existing shard format. (internal/cmd/backup_drive_collaboration.go:120, f57392f1d8f6)
  • After-fix production-client evidence: The supplied body snapshot, sourceRevision da14f339fa5da744d14fbf7f4512998bb2e7132d4cffbd2919308d0e91c1d97f, records a Windows compiled-test-binary run returning the repeated-token error after exactly two requests for all three production helpers. The inspected fixture uses drive.NewService and a real HTTP client against a local server, validates token handoff, and bounds an unwanted third request. This supports injected-fault pagination behavior, without claiming a live Google failure or encrypted-shard execution. (internal/cmd/backup_drive_collaboration_test.go:100, f57392f1d8f6)

Likely related people:

  • salmonumbrella: Raw commit 3371e3f adds internal/cmd/paging.go:26 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 3371e3f3ad6d; files: internal/cmd/paging.go)
  • Peter Steinberger: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • SebTardif: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Add the Unreleased entry and document the 10,000-page ceiling and per-file error-row outcome.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete

steipete commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

Superseded by #1087, which incorporates Drive collaboration cycle protection without adding a new page ceiling. Encrypted per-file error records and continued collection were verified. Thanks @SebTardif. Superseded by #1087, which incorporates Calendar, Contacts, and Tasks cycle protection while retaining fetched ACL rows on ordinary API errors. Backup failure and encrypted-row behavior were verified. Thanks @SebTardif. Superseded by #1087, which incorporates Workspace and Forms pagination protection without introducing new page limits. Stock-CLI runs verified fatal inventory failures and embedded Forms error records. Thanks @SebTardif.

@steipete steipete closed this Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants