Repository navigation
fix(api,event,gateway): preserve encoded upstream paths in gateway routes - #837
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The consumer controller test lacks an encoded GatewayUrl case and a persisted upstream-path assertion, leaving the Route mapping vulnerable to undetected regressions.
1 open issue
What changed in this PR
This PR fixes upstream URL mapping in the API controller so gateway Routes preserve percent-encoded paths, preventing provider URL construction failures.
Changes:
- Use
url.URL.EscapedPath()when mapping upstream URLs. - Reuse the corrected mapper for remote API consumer Routes.
- Add mapper and persisted-Route regression tests, and document encoded-path forwarding.
| File | Description |
|---|---|
docs/docs/user-journey/exposing-apis.mdx |
Documents preservation of encoded upstream paths. |
api/internal/handler/util/gateway.go |
Preserves path encoding with EscapedPath(). |
api/internal/handler/util/gateway_test.go |
Tests URL components and several encoding cases. |
api/internal/handler/util_real_route_test.go |
Verifies encoded paths survive Route persistence. |
api/internal/handler/remoteapisubscription/consumer_scenario.go |
Uses the shared upstream mapper for consumer Routes. |
Review effort: Max
5e158fc to
75dd2cd
Compare
|
I would suggest to use some central function for this which the gateway-api module. Alternative could also be that this encoding logic is inside of the gateway-operator only. So no CP changes. Related but out-of-scope I guess: How is this done for event-callback-urls? |
|
@ron96g I agree with the central function, I'll move it into a shared helper in gateway and have all modules use it. I'd rather not do it only in the operator, once the path is decoded, %2F can't be restored and a path separator cannot be distinguished from an encoded slash. The event callback query param isn't encoded in eventsubscription/handler.go (spectre does it right). Separate issue or should this PR cover that too? |
|
With the new helper in gateway/api, we have a new test suite. For now, the gateway Makefile runs it from its test target ($(MAKE) -C api test), the same way approval runs approval/api. So they run in CI whenever something under gateway/ changes. I didn't find a standard approach for this. Most nested API modules with tests (api/api, rover/api, event/api, admin/api, organization/api) aren't tested in CI at all, and none has an entry in |
AI-assisted investigation: URL-encoding compatibilityThis comment documents a local investigation performed by an AI assistant using Copilot SDK in VS Code. Reviewed PR head: Result: no encoding-related downstream incompatibility found in the tested scenarios. Fully encoded callback URLThe fully encoded subscriber URL is the value of the gateway callback query parameter, not the destination string passed directly to Jumper: Kong parses that query parameter and the request-transformer expression The outer Runtime setup and verification
Scope and rollout notesThis was local runtime/HTTP verification, not a full deployed Kafka/Horizon/identity integration test or a test of the deployed gateway version. No deployed/shared gateway instance was used; temporary containers were stopped and removed. No repository code changes were made. Existing encoded upstreams will change to their intended semantics when reconciled: for example, Recommendation: retain downstream wire-level regression coverage, since parser/configuration assertions alone cannot establish consumer compatibility. |
The subscriber callback URL was appended to the gateway callback URL without encoding. A subscriber URL with its own query string, such as "?a=1&b=2", lost every parameter after the first "&" because the gateway read them as separate parameters. Build the query with url.Values, as Spectre already does. The gateway decodes the "callback" query parameter before it uses it as upstream. Existing callback subscriptions get the encoded URL on their next reconcile, which updates their Horizon subscription once.
2b61e2f to
1a1381c
Compare
Resolve conflicts with #864 (callback query param renamed to "url") by keeping the query-encoded subscriber URL under the new "url" parameter. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
## [0.48.3](v0.48.2...v0.48.3) (2026-11-18) ### Bug Fixes * **api,event,gateway:** preserve encoded upstream paths in gateway routes ([#837](#837)) ([551f3be](551f3be)) * **discovery-server:** expose icto and psiid of applications ([#858](#858)) ([1237c46](1237c46)) * **rover,event:** additional publishers get a client ([#872](#872)) ([af3d8e2](af3d8e2)) ### Performance Improvements * **rover-server:** stop resolving secrets when listing rovers ([#870](#870)) ([50a5f62](50a5f62))

Intent
Gateway Routes must preserve percent-encoded characters in upstream paths. Before this change, an upstream such as
https://provider.example.com/api/url%20encoded%20pathproduced a Route upstream path with raw spaces. The gateway then could not construct the provider URL and returned HTTP 500. Other encodings were silently changed:%2Fbecame an extra path segment and%2520lost one encoding layer.Approach
All modules now convert an upstream URL with one shared helper,
gatewayv1.ParseUpstreamingateway/api/v1. It storesurl.URL.EscapedPath()instead of the decodedurl.URL.Path.EscapedPath()preserves canonical encodings such as%20.RawPathalone is insufficient because Go can leave it empty for these encodings. TheUpstream.Pathfield documentation now states that the path is percent-encoded.The helper replaces these per-module copies:
apiAsUpstreamagenticAsUpstreamadmineventparseUpstreamspectreparseSSEUpstreamfileThe Kong client URL parser in
gateway/pkg/kong/clientalso usesEscapedPath().Re-encoding only in the gateway operator was considered. It cannot restore
%2F, because a decoded/is indistinguishable from a path separator.The user documentation now states that encoded upstream path characters are forwarded unchanged.
Commits
fix: preserve encoded upstream paths in gateway routes: the fix and the regression tests in each module.refactor: share upstream URL parsing in gateway api: the shared helper, as suggested in review.test(gateway): run gateway/api tests in make test:gateway/apigets a Makefile, and thegatewaymodule'smake testruns it, likeapprovaldoes forapproval/api.fix(event): query-encode subscriber URL in gateway callback URL: the subscriber URL is added to the gateway callback URL withurl.Values, as Spectre already does. Before, a subscriber URL with its own query string, such as?a=1&b=2, lost every parameter after the first&. The gateway decodes thecallbackquery parameter before it uses it as upstream.Verification
gateway/api/v1/util_test.gocovers the helper:%20,%2F,%25,%2520, a plain path, default ports, and parse errors. WithEscapedPath()replaced byPath, the four encoding cases fail./api/url encoded path.make testpassed inapi,agentic,admin,event,file,spectre,gateway, andgateway/api.callbackparameter. Before the fix, this test and the encoded expectations in the handler and callback lifecycle tests failed. An end-to-end delivery through the gateway was not tested.make lintreports no findings in changed files. It fails inapi,agentic,gateway, andspectrebecause of findings in unchanged files.The docs site build was not run.
Reviewer guidance
Start with
gateway/api/v1/util.go. The module changes replace local parsing withParseUpstream. Callers that need a weight set it after the call.Existing Routes keep their decoded paths until their controllers reconcile them after deployment.
Rollout impact of commit 4: every existing callback subscription gets a new callback URL on its next reconcile, for example
?callback=https://…becomes?callback=https%3A%2F%2F…. Pubsub then updates each Horizon subscription once. The delivered events go to the same subscriber URL.