rpk: Add new Schema Registry option to Shadow Link command space - #30860
Merged
Conversation
The core admin/v2 proto turned SchemaRegistrySyncOptions into a oneof and added a shadow_schema_registry_api mode that replicates a source Schema Registry over its HTTP API. Add the local types with JSON tags matching the proto field names, which the drift check and the update field-mask paths both rely on.
Wire the new oneof branch and its nested auth, TLS, filter, destination, and policy messages. Intervals are only set when positive so unset values fall back to server defaults. Output-only fields (effective_* and password_set*) are never sent. The cloud control plane reuses the same adminv2 sync option types, so cloud requests get the new mode through the same function.
Add the reverse oneof branch and helpers for its auth, TLS, filter, destination, and policy fields. TLS is decoded only when present so a link without it round-trips to a nil setting rather than an empty struct. The policy enum maps UNSPECIFIED to the empty string, mirroring the forward direction so the default survives a round-trip.
r-vasquez
force-pushed
the
sr-sl-new-opts
branch
2 times, most recently
from
June 19, 2026 22:55
3904b4f to
375c0c4
Compare
r-vasquez
marked this pull request as ready for review
June 20, 2026 00:04
r-vasquez
requested review from
Copilot and
weeco
and removed request for
a team
June 20, 2026 00:04
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends rpk shadow to support the new Shadow Link Schema Registry sync mode shadow_schema_registry_api, aligning rpk’s config/types/mappers and CLI output with the updated Core Admin v2 proto.
Changes:
- Add config/types + (un)marshal coverage for
schema_registry_sync_options.shadow_schema_registry_api, including auth/TLS/filter/destination/policy fields. - Extend mapping logic (config ⇄ adminv2 proto) and diffing to handle the new oneof branch and related fields.
- Add describe/status CLI rendering for schema registry API config and schema registry sync status, plus tests and proto dependency bumps.
Reviewed changes
Copilot reviewed 14 out of 16 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/go/rpk/pkg/cli/shadow/update.go | Redact password placeholders for SCRAM and schema registry basic auth during update editing. |
| src/go/rpk/pkg/cli/shadow/update_test.go | Add diffing tests for schema registry API fields / mode switches. |
| src/go/rpk/pkg/cli/shadow/types.go | Introduce schema registry API mode config types (auth/TLS/filter/destination/policy). |
| src/go/rpk/pkg/cli/shadow/types_test.go | Add YAML/JSON unmarshal tests for schema registry API mode; drift ignore updates. |
| src/go/rpk/pkg/cli/shadow/status.go | Add schema registry sync status view + printing and adminv2 conversion. |
| src/go/rpk/pkg/cli/shadow/status_test.go | Verify adminv2 → view conversion for schema registry sync status. |
| src/go/rpk/pkg/cli/shadow/mapper.go | Map schema registry API mode between config and adminv2 proto; add enum mapping. |
| src/go/rpk/pkg/cli/shadow/mapper_test.go | Add mapper and round-trip tests for schema registry API mode and policies. |
| src/go/rpk/pkg/cli/shadow/describe.go | Print schema registry API mode details in describe output; add TLS helper and formatting. |
| src/go/rpk/pkg/cli/shadow/create.go | Validate schema registry API oneof constraints; extend cloud secret validation to SR API fields. |
| src/go/rpk/pkg/cli/shadow/create_test.go | Add validation coverage for schema registry mode/TLS/destination and cloud secret rules. |
| src/go/rpk/pkg/cli/shadow/BUILD | Include new status tests and timestamppb dependency. |
| src/go/rpk/go.mod | Bump core protocolbuffers dependency to pick up new schema registry fields. |
| src/go/rpk/go.sum | Update sums for the bumped core protocolbuffers dependency. |
| src/go/rpk/gen/protocomments/admin/v2/comments.pb.go | Regenerated comments map for new schema registry API/status/policy proto fields. |
| buf.gen.yaml | Update buf module reference for regenerated core protos. |
Files not reviewed (1)
- src/go/rpk/gen/protocomments/admin/v2/comments.pb.go: Generated file
Collaborator
CI test resultstest results on build#86062
test results on build#86106
test results on build#86113
|
r-vasquez
force-pushed
the
sr-sl-new-opts
branch
from
June 22, 2026 18:52
375c0c4 to
cde49b6
Compare
Mateoc
previously approved these changes
Jun 22, 2026
c-julin
reviewed
Jun 22, 2026
Creating or updating a shadow link did not check the new shadow_schema_registry_api mode, so a config that set both shadowing modes, both destination mappings, or plain cloud secrets was sent straight to the server. Add client-side validation mirroring the existing client-option rules, and redact the basic-auth password in the update editor the way the SCRAM password already is. validateParsedShadowLinkConfig now binds the client options once and nil-guards every access, since a cloud config may omit client_options entirely; previously such a config panicked. The cloud secret checks no longer return early when client options are absent, so the schema registry secrets are still validated, and the four plain secret checks collapse into one loop over all secret-bearing fields. File-based TLS is rejected for the schema registry API under cloud options too, since the agent cannot read local paths.
Render the new SR fields in both the text and the JSON/YAML output, reusing the reverse mappers and showing effective interval and rate values plus auth metadata without the password. The client TLS printing is extracted into a shared helper since the schema registry API carries the same core TLS settings. The schema registry section was also added to the cloud section header map.
The admin status now carries a schema registry sync status, so surface its inventory counts, the running sync, the last full sync, cumulative totals, and the last error. A new print-registry flag selects the section, and it joins the default set. The dataplane API does not expose this status, so it is populated only on the self-hosted path; cloud output omits it for now.
We allow empty client options in Cloud configs as you just need the Redpanda ID. However, this print here could cause a panic. In reality this was never reported as Cloud always require ClientOpts.TLS.Enabled to be true and it was provided in the config template.
r-vasquez
force-pushed
the
sr-sl-new-opts
branch
from
June 22, 2026 22:23
cde49b6 to
cd9d82a
Compare
Collaborator
Retry command for Build#86113please wait until all jobs are finished before running the slash command |
c-julin
approved these changes
Jun 23, 2026
Contributor
Author
|
/ci-repeat 1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for the new Schema Registry option in shadow link:
shadow_schema_registry_apiThe template
The new option is defined in Core's proto: https://github.com/redpanda-data/redpanda/blob/dev/proto/redpanda/core/admin/v2/shadow_link.proto#L418 but here is how it will look like in our config file (a 1:1)
This PR also handle the whole command life cycle:
create,describeandstatusDescribe
Status
Backports Required
Release Notes
Features
shadow_schema_registry_apiin Shadow Links.