Skip to content

feat: experimental catalog search through pathfinder - #442

Open
leodurandfr wants to merge 3 commits into
devgianlu:masterfrom
leodurandfr:feat/pathfinder-search
Open

leodurandfr wants to merge 3 commits into
devgianlu:masterfrom
leodurandfr:feat/pathfinder-search

Conversation

@leodurandfr

Copy link
Copy Markdown
Contributor

Part of #311 (search only).

Adds an experimental, opt-in catalog search. It goes through pathfinder (api-partner.spotify.com/pathfinder), the GraphQL service the official clients search with, using the session's login5 token, not the rate-limited public Web API. Off by default (search.enabled: false); disabled, nothing is sent to Spotify.

API

Shaped after /library: one endpoint per type, each returning one page {total, offset, limit, items}.

  • GET /search/tracks, /search/albums, /search/artists, /search/playlists, /search/shows and /search/episodes, with q, offset and limit (1–50, default 20).
  • Albums and artists are libraryAlbum and libraryArtist, so an album carries its year. Tracks and episodes share searchTrack, named after the fields of track; an episode names its show the way track does. Playlists and shows get small schemas of the same kind. Images follow server.image_size.
  • GET /search/suggestions?q=&limit= (1–30, default 10): the complete queries Spotify suggests for what was typed so far, and its first results by type, each list in Spotify's order.
  • 204 without a session, 400 on bad input (nothing is sent), 404 when disabled, 429 with Retry-After. Other failures answer like the library endpoints: 500, logged.

How it is built

  1. Spclient.PathfinderQuery sends a persisted query (operation name + sha256 hash) through innerRequest, so the token, the 401 renewal, network errors and 500/502/503/504 are handled as for every other spclient request. It reads the result like PutConnectStateRaw: a 429 is a RateLimitedError (now documented for any spclient 429), any other failed status a PathfinderError, 412 meaning the hash is outdated. A 200 whose data holds nothing but nulls, an operation that failed as a whole, is run again, at most twice.
  2. Pathfinder publishes no schema, so, as you suggested in feat: pathfinder gql service #311, its search responses are described in pathfinder-spec.yml and the models are generated into pathfinder/ with oapi-codegen, by go generate, next to api-spec.yml. The spec only covers the fields that are read.
  3. daemon/search.go maps those models straight to the Api* types, the way the library endpoints map extended metadata, and runs off the player loop through goDetached, as they do.

Things to know

  • Persisted-query hashes rotate. They are the web player's, valid on 2026-11-10. Pathfinder answers 412 "Invalid query hash" to a hash it does not know, which is what an old one becomes once Spotify rebuilds a query. That search then fails until go-librespot is updated, and the log names it: pathfinder searchTracks failed with status 412, persisted query hash outdated: Invalid query hash. The other searches keep working. There is no config override in this PR; I can add one if you would rather let users fix a rotation without a release.
  • User-Agent. Pathfinder checks it before the query hash. It answers 403 Client/request not allowed to Go's default (which spclient requests send) and to UserAgent(), and accepts no User-Agent at all, abcd (but not abcde), or one starting with Mozilla or Spotify. Pathfinder requests, and only those, send SpotifyLikeUserAgent(), Spotify/127700358 <OS>/0 (go-librespot/<version>): the desktop client go-librespot presents to the access point, then its own name. libspot's pathfinder client, for comparison, sends the full desktop app UA. This is the one choice I'd like your call on: is Spotify/… OK with you, or would you rather send no User-Agent header at all (which works too), or Mozilla/5.0 (compatible; go-librespot/<version>)?
  • Paging. Spotify's result count is an estimate, at times lower than what it then pages through. So total is raised to reach into the next page while there is one, and on the last page it is the offset plus what Spotify returned. Pathfinder only pages through the first 1000 results and refuses offset + limit above that, so the handlers answer 400 there and total stops at 1000. A page can hold fewer items than limit before the last one, and an item can come back on two pages.
  • 401. A pathfinder 401 goes through the shared renewal: one forced login5 login per attempt, until the 15 s search timeout, as for every spclient request today. If pathfinder ever refused login5 tokens outright, each search would cost several forced logins. I kept the shared behavior rather than give one endpoint its own policy.
  • Network failures take 15 s. Through the shared transport, a closed port, a DNS failure or a black hole is retried until the search timeout, which then answers 500.
  • Not in this PR: Spotify's cross-type "top result" (suggestions keep each type's order, not the order between types), and cancelling a search when its HTTP client goes away (no endpoint does that today).

Testing

Unit and integration tests cover:

  • every outcome of PathfinderQuery: refusals, 412, 429, failed operations with and without GraphQL errors, partial data, oversized and non-JSON bodies, cancellation;
  • each search type, mapped from recorded responses trimmed to the fields that are read;
  • skipped and unknown results, missing result nodes, paging totals, image sizes;
  • the handlers' forwarding, defaults and bounds, and the 429 Retry-After.

On a Raspberry Pi 5, with this branch running as a separate daemon (null audio output, nothing played):

  • Queries. 10 queries (accents, CJK, emoji, 1 to 200 characters; their first 6 characters for suggestions) across the 6 types and suggestions: 70/70 OK. The expected artist came first for the 6 queries that name one.
  • Limits. limit 1 and 50 on every type; shows return 44 at 50.
  • Paging. Tracks, albums and playlists, 50 at a time, up to the 1000 cap: 20 pages each, with 116, 39 and 68 items repeated across pages.
  • Bad input. Rejected with 400 in 1–2 ms, including a page past the cap.
  • Cost. Median latency at the default limit 311–489 ms per type, episodes 1.1 s (0.5 s in another run). About 5 ms of CPU per search at typing rate, 12 ms when paging 50 at a time. RSS stays around 26 MiB.
  • Typing rate. 600 searches: suggestions at 10 and 20 per second, tracks at 5 per second. All OK, no 429, and GET /status answered in 22 ms or less throughout.
  • Variants.
    • Disabled, with a session: 404 on all 7 endpoints.
    • No User-Agent: OK. Go's default and UserAgent(): 403, logged as above.
    • No client-token: OK.
    • A corrupted token on the first search: one login5 renewal, and the searches after it OK.
    • A hash pathfinder does not know (all zeros): that search fails at once with the 412 above; albums, searched meanwhile, are unaffected.
    • Closed port, DNS failure, black hole: 500 after 15 s, with /status answering in under 3 ms meanwhile.

It is also used by a client that runs four typed searches and suggestions as the user types. One search there came back without its searchV2; when the data holds nothing but nulls like that, this branch now runs it again.

🤖 Generated with Claude Code

leodurandfr and others added 3 commits November 10, 2026 18:30
Pathfinder (api-partner.spotify.com/pathfinder/v2/query) is the GraphQL
service the official clients search through (devgianlu#311). It accepts the
login5 token spclient already holds, and only runs persisted queries: an
operation name and the sha256 hash of its query document.

PathfinderQuery sends one such query through innerRequest, so token, 401
renewal, network errors and 5xx are handled as for every spclient
request, and reads the result the way PutConnectStateRaw does: a 429 is
a RateLimitedError left to the caller, any other failed status a
PathfinderError, 412 being an outdated hash. A 200 whose data holds
nothing but nulls, such as {"searchV2": null}, is an operation that
failed as a whole; a client has seen a search come back without its
searchV2 once, and such a result is run again, at most twice. Responses
are capped at 8 MiB. RateLimitedError, which only connect-state returned
so far, is now documented for any spclient 429.

Pathfinder checks the User-Agent before the query hash and refuses Go's
default, which the other spclient requests send, and UserAgent() with
403 "Client/request not allowed". Measured with the login5 token, it
accepts no User-Agent at all, "abcd" (but not "abcde"), or one starting
with "Mozilla" or "Spotify". Its requests carry SpotifyLikeUserAgent(),
Spotify/<SpotifyVersionCode> <OS>/0 (go-librespot/<version>): the
desktop client go-librespot already presents to the access point, whose
client ID it also logs in to login5 with, then its own name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pathfinder publishes no schema, so its search responses were described
by hand from what it answers, in pathfinder-spec.yml, and the Go models
are generated from it with oapi-codegen into pathfinder/, as discussed
in devgianlu#311. go generate runs it next to api-spec.yml.

The spec only covers what the search endpoints read: the data of
searchTracks, searchAlbums, searchArtists, searchPlaylists,
searchPodcasts, searchFullEpisodes and searchSuggestions. Results are a
oneOf told apart by __typename, so an unknown or error type (NotFound,
GenericError...) can be skipped item by item. A missing result node
stays nil, so it can be told from an empty page.

coverage.sh now leaves out every *_gen.go file, not only api_gen.go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An experimental, opt-in catalog search through pathfinder, behind
search.enabled (off by default; nothing is sent to Spotify then).

The endpoints follow /library: GET /search/tracks, /search/albums,
/search/artists, /search/playlists, /search/shows and /search/episodes
each return one page, {total, offset, limit, items}, of one type. Albums
and artists are the libraryAlbum and libraryArtist of /library, so an
album carries its year. Tracks and episodes share searchTrack, named
after the fields of track, an episode naming its show as track does.
Images are picked by server.image_size. GET /search/suggestions returns
the queries Spotify suggests for what was typed so far, and its first
results by type, each list in Spotify's order.

Spotify's result count is an estimate, at times below the results it
then pages through, so total is raised to reach into the next page while
there is one, and on the last page is the offset plus what Spotify
returned there. Pathfinder pages through the first 1000 results only and
refuses an offset plus limit beyond them, so the handlers answer 400
there and total stops at 1000. A result Spotify cannot show (a NotFound
or GenericError in its place) is skipped; a response that lacks the
operation's results altogether is an error rather than an empty page, so
a change on Spotify's side does not read as nothing found.

The pathfinder models are mapped straight to the API types in the
daemon, as the library endpoints map extended metadata. A search runs
off the player loop like them, and fails like them; a 429 answers 429,
with a Retry-After from any spclient RateLimitedError, which CORS now
exposes to browsers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant