Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

## 0.39.1 - Unreleased

### Highlights

- Backups: stop repeated page tokens across Drive, Calendar, Contacts, Tasks, Groups, Admin, Keep, Workspace, and Apps Script without adding new page limits; retain earlier ACL and membership rows on ordinary API failures. (#1078, #1079, #1080, #1081, #1082, #1087) — thanks @SebTardif.
- Calendar: stop cyclic recurring-instance lookups while preserving success as soon as the requested occurrence is found. (#1084, #1087) — thanks @SebTardif.
- Gmail: stop cyclic query scans before bulk message changes or automatic replies, including empty result pages. (#1086, #1087) — thanks @SebTardif.
- Drive and Contacts: stop tree, inventory, disk-usage, and contact-dedupe scans on repeated page tokens before emitting a partial report or applying contact changes. (#1083, #1085, #1087) — thanks @SebTardif.
- Docs: stop open-comment scans when Drive repeats a page token, preserving first-open-page results. (#1087) — thanks @SebTardif.
- Dependencies: refresh Google API support modules, tracking worker tooling and Undici, pnpm, and the Pages deployment action while retaining Go 1.26 compatibility.

## 0.39.0 - 2026-09-03
Expand Down
8 changes: 8 additions & 0 deletions docs/backup.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,14 @@ Best-effort optional services record encrypted `errors` shards and let the rest
of the backup finish. The Gmail cache is only a local acceleration/resume cache;
encrypted backup shards remain the source of truth once a push completes.

Backup listings reject repeated page tokens without imposing new limits on
distinct pages. Drive, Calendar, Contacts, and Tasks collection errors stop the
final snapshot; optional service errors follow `--best-effort`. Per-file Drive
collaboration errors and per-form response errors remain embedded error rows.
Calendar ACL and Groups membership listings retain fetched rows on an ordinary
later API error, but discard the affected resource's rows when pagination cycles;
both cases record an error and continue with the next calendar or group.

## Files

Local config:
Expand Down
3 changes: 3 additions & 0 deletions docs/contacts-dedupe.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ read_when:
`gog contacts dedupe` finds likely duplicate personal contacts and prints a
merge plan. Preview is the default; `--apply` performs the reviewed plan.

Repeated page tokens stop the scan before a plan or contact update. `--max 0`
still scans every distinct page; a positive `--max` stops at that contact count.

## Command Page

- [`gog contacts dedupe`](commands/gog-contacts-dedupe.md)
Expand Down
4 changes: 4 additions & 0 deletions docs/docs-editing.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,10 @@ Use `--occurrence N` when an anchor is ambiguous and `--match-case` when case
must be exact. `docs comments locate` applies the same matching rules to a
comment's quoted text and reports its tab plus UTF-16 range.

By default, `docs comments list` skips resolved-only pages until it finds open
comments or reaches the end. A repeated page token stops the scan with a
pagination error instead of hanging or reporting an incomplete result.

`docs comments list --locate` does the same for every listed comment from a
single document fetch, adding a `location` object (`matches`, `orphaned`) and a
`TAB` column. `--tab` implies `--locate`, adds the resolved tab to JSON output
Expand Down
3 changes: 3 additions & 0 deletions docs/drive-audits.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ aggregates those placements independently and sorts by `size`, `path`, or
`files`. Shortcuts count as file placements with zero content bytes; scans do
not follow shortcut targets.

These reports reject repeated folder-list page tokens before emitting output.
Successful scans retain their existing size and depth limits.

## Inventory Export

Export a read-only item inventory:
Expand Down
4 changes: 4 additions & 0 deletions docs/gmail-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ page by page and retains only exact name or email matches. Multiple matching
contacts require a more specific selector; a repeated page token stops with a
pagination error instead of leaving the command stuck.

Query-based bulk message commands and autoreply also reject repeated page
tokens, including empty pages, before modifying messages or sending replies.
Their existing `--max` limit still stops the scan as soon as it is reached.

For agents, logs, or issue reports, prefer sanitized content:

```bash
Expand Down
118 changes: 61 additions & 57 deletions internal/cmd/backup_directory_keep.go
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,11 @@ func buildKeepBackupSnapshot(ctx context.Context, flags *RootFlags, shardMaxRows
func fetchBackupCloudIdentityGroups(ctx context.Context, svc *cloudidentity.Service, account string) ([]*cloudidentity.GroupRelation, error) {
var out []*cloudidentity.GroupRelation
pageToken := ""
var guard pageTokenGuard
for {
if err := guard.check(pageToken); err != nil {
return nil, err
}
call := svc.Groups.Memberships.SearchTransitiveGroups("groups/-").
Query(searchTransitiveGroupsQuery(account)).
PageSize(1000).
Expand Down Expand Up @@ -171,7 +175,13 @@ func fetchBackupCloudIdentityGroupMembers(ctx context.Context, svc *cloudidentit
continue
}
pageToken := ""
var guard pageTokenGuard
groupStart := len(out)
for {
if err := guard.check(pageToken); err != nil {
out = append(out[:groupStart], groupsBackupMember{GroupEmail: groupEmail, Error: err.Error()})
break
}
call := svc.Groups.Memberships.List(groupName).PageSize(1000).Context(ctx)
if pageToken != "" {
call = call.PageToken(pageToken)
Expand Down Expand Up @@ -199,48 +209,47 @@ func fetchBackupCloudIdentityGroupMembers(ctx context.Context, svc *cloudidentit
return out
}

func fetchBackupAdminUsers(ctx context.Context, svc *admin.Service, domain string) ([]*admin.User, error) {
var out []*admin.User
pageToken := ""
for {
call := svc.Users.List().Domain(domain).MaxResults(500).Context(ctx)
if pageToken != "" {
call = call.PageToken(pageToken)
}
resp, err := call.Do()
if err != nil {
return nil, err
}
out = append(out, resp.Users...)
if resp.NextPageToken == "" {
break
}
pageToken = resp.NextPageToken
func fetchSortedBackupPages[T any](fetch func(string) ([]T, string, error), less func(T, T) bool) ([]T, error) {
rows, err := collectUnboundedPages("", fetch)
if err != nil {
return nil, err
}
sort.Slice(out, func(i, j int) bool { return out[i].PrimaryEmail < out[j].PrimaryEmail })
return out, nil
sort.Slice(rows, func(i, j int) bool { return less(rows[i], rows[j]) })
return rows, nil
}

func fetchBackupAdminUsers(ctx context.Context, svc *admin.Service, domain string) ([]*admin.User, error) {
return fetchSortedBackupPages(
func(pageToken string) ([]*admin.User, string, error) {
call := svc.Users.List().Domain(domain).MaxResults(500).Context(ctx)
if pageToken != "" {
call = call.PageToken(pageToken)
}
resp, err := call.Do()
if err != nil {
return nil, "", err
}
return resp.Users, resp.NextPageToken, nil
},
func(a, b *admin.User) bool { return a.PrimaryEmail < b.PrimaryEmail },
)
}

func fetchBackupAdminGroups(ctx context.Context, svc *admin.Service, domain string) ([]*admin.Group, error) {
var out []*admin.Group
pageToken := ""
for {
call := svc.Groups.List().Domain(domain).MaxResults(200).Context(ctx)
if pageToken != "" {
call = call.PageToken(pageToken)
}
resp, err := call.Do()
if err != nil {
return nil, err
}
out = append(out, resp.Groups...)
if resp.NextPageToken == "" {
break
}
pageToken = resp.NextPageToken
}
sort.Slice(out, func(i, j int) bool { return out[i].Email < out[j].Email })
return out, nil
return fetchSortedBackupPages(
func(pageToken string) ([]*admin.Group, string, error) {
call := svc.Groups.List().Domain(domain).MaxResults(200).Context(ctx)
if pageToken != "" {
call = call.PageToken(pageToken)
}
resp, err := call.Do()
if err != nil {
return nil, "", err
}
return resp.Groups, resp.NextPageToken, nil
},
func(a, b *admin.Group) bool { return a.Email < b.Email },
)
}

func fetchBackupAdminGroupMembers(ctx context.Context, svc *admin.Service, groups []*admin.Group) []adminBackupMember {
Expand Down Expand Up @@ -286,25 +295,20 @@ func fetchAllBackupAdminGroupMembers(ctx context.Context, svc *admin.Service, gr
}

func fetchBackupKeepNotes(ctx context.Context, svc *keepapi.Service) ([]*keepapi.Note, error) {
var out []*keepapi.Note
pageToken := ""
for {
call := svc.Notes.List().PageSize(1000).Context(ctx)
if pageToken != "" {
call = call.PageToken(pageToken)
}
resp, err := call.Do()
if err != nil {
return nil, err
}
out = append(out, resp.Notes...)
if resp.NextPageToken == "" {
break
}
pageToken = resp.NextPageToken
}
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out, nil
return fetchSortedBackupPages(
func(pageToken string) ([]*keepapi.Note, string, error) {
call := svc.Notes.List().PageSize(1000).Context(ctx)
if pageToken != "" {
call = call.PageToken(pageToken)
}
resp, err := call.Do()
if err != nil {
return nil, "", err
}
return resp.Notes, resp.NextPageToken, nil
},
func(a, b *keepapi.Note) bool { return a.Name < b.Name },
)
}

func domainFromAccount(account string) string {
Expand Down
Loading