Collect Claude Code's provider bodies, with repeats taken out by wu-sheng · Pull Request #10 · apache/skywalking-ai-sessionizer

wu-sheng · 2026-09-15T10:46:12Z

What

Claude Code writes every request it sends its model provider, and every response, when OTEL_LOG_RAW_API_BODIES=file:<dir> is set. A request holds what no transcript records: the system prompt, the tool schemas, the injected reminders and the system messages. This PR collects those files, lands them without the bytes a session already holds, and links them from the view.

Format. A new Session Data kind, provider_body, under <session>/provider_body/. A record keeps only new pieces. Its last part, a provider_body/1 manifest, rebuilds the body byte for byte from literals, its own parts, earlier pieces by digest, and a copy of the front of an earlier body by record id. The manifest keys are the model's words, not Claude Code's. The .sd and .sf formats are unchanged.
Collector. A new adapter, claude-code-provider, on by default. It finds each body's session, lands a session's new bodies once per pass, cut at collect.max_delta_bytes, and keeps a derived table at _provider/seen.state.
Server side. The kind stays out of the fold and out of every index lookup. asz verify rebuilds every body. In asz.view, an llm.call lists provider_bodies: [{role, ref}], and the summary counts provider_bodies and captured_prompts. A reader loads the session's provider_body files up to the ref and rebuilds on demand. asz.view stays 1.0; only keys are added.
Removal. A scenario removal deletes the body files with the session and drops its lines from the provider table.
Scenarios. Scenarios can write provider bodies in both formats. Three scenarios were added: provider-bodies, provider-bodies-late-child, provider-bodies-errors. Two properties were added: provider_bodies_rebuild, which also runs on the repacked root, and provider_bodies_leave_the_fold.
Docs. New page docs/en/setup/claude-code-provider-bodies.md. Session Data, storage root, asz.view, configuration, command line, OTLP export and scenario guide pages are updated.

Measured

Two captures of Claude Code 2.1.260:

Capture	Bodies	Landed / written	Rebuilt exactly
1	12	25.5%	all
2	30	21.6%	all 28 landed

For comparison, cutting each file on its own with no references across files gives 43.9% and 50.8%.

On capture 2, the title and compaction responses wait, because no request names their call. All 13 calls had both bodies joined.

Compatibility with the OAP

The OAP stores every .sd file it receives and decodes only changes, so it already skips provider_body in analysis. It must keep storing them, because rounds chain file digests.

Follow-up in apache/skywalking:

FileNames.dataFile has no provider_body case.
The OAP view does not yet list provider_bodies on llm.call, or the new summary counts.

Tests

go vet, golangci-lint (0 issues), license-eye
go test -race ./...
every scenario in both formats

The Docker collector end-to-end run was not run locally.

The implementation went through two external review rounds (8 and 7 findings), and all findings are fixed.

Claude Code writes every request it sends its model provider, and every response, when OTEL_LOG_RAW_API_BODIES=file:<dir> is in its environment. A request holds what no transcript records: the system prompt, the tool schemas, the injected reminders and the system messages between the others. Every call sends its chain's whole message list again, so most of a body is bytes the session already holds. - pkg/providerbody cuts a body into pieces, each tool definition and every string of 1 KiB or more, and copies the front a body shares with the previous body of its chain. A record keeps only what is new; its last part, a provider_body/1 manifest in the model's words, rebuilds the body byte for byte. References are record ids and digests, which asz repack leaves valid, and they may reach into earlier files. Two captures of Claude Code 2.1.260 landed at 25.5% and 21.6% of the bytes written, and all 42 bodies rebuilt exactly. - The claude-code-provider adapter reads the flat directory, finds each body's session from the body and the landed transcripts, and lands a session's new bodies under <session>/provider_body/, cut at max_delta_bytes. What it knows of Claude Code's names stays in the adapter. A root table, _provider/seen.state, is derived from the landed records. Each record is checked through the writer and reader before it lands. - The kind stays out of the fold and out of every index lookup. storage.LandedFiles lists the session's provider_body directory, and asz verify rebuilds every body. In asz.view an llm.call lists the role and landed ref of its request and response, joined by the bodies' own ids; a reader loads the provider_body files up to that ref and rebuilds. summary.provider_bodies and summary.captured_prompts count them. On the second capture all 13 calls had both joined. - A scenario removal deletes the body files with the session's other files and drops its lines from the provider table. - Scenarios can write provider bodies in both formats. An expectation's collect.max_delta_bytes goes into the build's configuration and the check follows it. provider_bodies_rebuild reads each body the way an on-demand reader does, on the root and its repacked copy, and provider_bodies_leave_the_fold compares fold identities.

The view reads the call nodes from a map and sorts them by stream and landed position. Two calls on one record kept the map's order, and a request joins the call its order puts first, so the same files could give two documents. The call id now breaks the tie, as the OAP's view does.

Provider bodies in the conversation view. The AI Sessionizer can land the request and response bodies an agent runtime exchanged with its model provider, apache/skywalking-ai-sessionizer#10. They arrive as Session Data files of kind provider_body, one directory per session: <session>/provider_body/provider_body-<stamp>-<seq>.sd. The OAP already stored them like any other file, but gave them a wrong name, <session>/runs//..., which the raw-file query could not parse back. - FileNames names the provider_body directory and parses it back. - The asz.view document follows the Sessionizer's. An llm.call step lists its request and then its response under provider_bodies, each as a role and the ref of the landed record, never the body. summary.provider_bodies and summary.captured_prompts count them. The join uses the bodies' own ids: a response by its message id, a request by the previous call's response request id and its prompt, and only when exactly one request and one call carry them. A synthetic call takes part in no join, and no request joins in a stream whose landed transcript lines have a gap. Calls on one record are ordered by id, as the Sessionizer now orders them. - What the join reads is decoded as the Sessionizer decodes it. A manifest whose known keys, or its segments' keys, have the wrong JSON type is no body. A record whose call or run is not a string is gone to the join. An ord is read from the raw line: the digits after a leading {"ord":, as an unsigned 64-bit number, or else the decoded value, where null is 0. A transcript whose records end at a line that does not decode has a gap. SessionDataFile keeps each record's leading ord digits and whether its records stopped early. - The test data is refreshed from the Sessionizer: the two existing sets gain the two summary counts, and the provider-bodies and provider-bodies-errors sets are added. Both documents equal the Sessionizer's key for key. A session without its provider_body file folds to the same nodes, a stream with a gap joins responses only, and a manifest or an ord the Sessionizer reads differently counts as it does; each of those cases gave the same counts from asz conversation. - The e2e builds the Sessionizer's provider-bodies scenario into the same root and checks the file, every call's bodies and the export by name. The views, list, sender and token expectations count the new session. The Sessionizer is pinned to the commit that lands provider bodies, since its view now carries the two summary counts. Conversation files over HTTP, not GraphQL. The query protocol drops getConversationRawFiles, apache/skywalking-query-protocol#175. A conversation's stored Session Data files are read from a second HTTP route beside the view, so a page loads what a step points at, such as an llm.call's provider bodies, only when a reader opens it: GET /ai-agent/conversations/{conversation}/v1/files ?service=&instance=&session=&seq=[&seq=...][&coldStage=true] - A file is chosen by its session and its landed seq, the two columns the storage reads it by; the Sessionizer assigns a seq once per file in a session. One to 32 seqs a request: a file is cut at 2 MiB, so a response holds about 64 MiB. There is no read of every file. - The body is application/vnd.skywalking.asz.files+ndjson: for each file a naming line {file, seq, lines, bytes, digest}, exactly that many bytes, and a newline after a non-empty file that does not end with one. Files come in seq order, the order provider bodies are read in. - The read takes its time range from the newest intact round, reading the rounds down from the head only until one is intact, then reads one storage window of files and hands it to the response before reading the next. Windows are produced one at a time and cannot overflow. - The route compresses with gzip itself, a chunk at a time. Armeria's encoder keeps every compressed chunk of a response in one growing buffer until the response ends. The view route still uses it. - Both routes require the service and the sender's instance, as a list row names them, so every read is a full series lookup; serviceId goes. A bad coldStage gets a problem document, not Armeria's plain-text 400. - The e2e reads files through swctl by session and seq taken from the document, and passes the instance on every view and route call. The raw-files case counts the three Session Data files of the first conversation, since rounds are no longer served. The e2e pins swctl to the merge commit of apache/skywalking-cli#235, which adds swctl ai-agent files with --session and --seqs.

…TTP (#14084) Provider bodies in the conversation view. The AI Sessionizer can land the request and response bodies an agent runtime exchanged with its model provider, apache/skywalking-ai-sessionizer#10. They arrive as Session Data files of kind provider_body, one directory per session: <session>/provider_body/provider_body-<stamp>-<seq>.sd. The OAP already stored them like any other file, but gave them a wrong name, <session>/runs//..., which the raw-file query could not parse back. - FileNames names the provider_body directory and parses it back. - The asz.view document follows the Sessionizer's. An llm.call step lists its request and then its response under provider_bodies, each as a role and the ref of the landed record, never the body. summary.provider_bodies and summary.captured_prompts count them. The join uses the bodies' own ids: a response by its message id, a request by the previous call's response request id and its prompt, and only when exactly one request and one call carry them. A synthetic call takes part in no join, and no request joins in a stream whose landed transcript lines have a gap. Calls on one record are ordered by id, as the Sessionizer now orders them. - What the join reads is decoded as the Sessionizer decodes it. A manifest whose known keys, or its segments' keys, have the wrong JSON type is no body. A record whose call or run is not a string is gone to the join. An ord is read from the raw line: the digits after a leading {"ord":, as an unsigned 64-bit number, or else the decoded value, where null is 0. A transcript whose records end at a line that does not decode has a gap. SessionDataFile keeps each record's leading ord digits and whether its records stopped early. - The test data is refreshed from the Sessionizer: the two existing sets gain the two summary counts, and the provider-bodies and provider-bodies-errors sets are added. Both documents equal the Sessionizer's key for key. A session without its provider_body file folds to the same nodes, a stream with a gap joins responses only, and a manifest or an ord the Sessionizer reads differently counts as it does; each of those cases gave the same counts from asz conversation. - The e2e builds the Sessionizer's provider-bodies scenario into the same root and checks the file, every call's bodies and the export by name. The views, list, sender and token expectations count the new session. The Sessionizer is pinned to the commit that lands provider bodies, since its view now carries the two summary counts. Conversation files over HTTP, not GraphQL. The query protocol drops getConversationRawFiles, apache/skywalking-query-protocol#175. A conversation's stored Session Data files are read from a second HTTP route beside the view, so a page loads what a step points at, such as an llm.call's provider bodies, only when a reader opens it: GET /ai-agent/conversations/{conversation}/v1/files ?service=&instance=&session=&seq=[&seq=...][&coldStage=true] - A file is chosen by its session and its landed seq, the two columns the storage reads it by; the Sessionizer assigns a seq once per file in a session. One to 32 seqs a request: a file is cut at 2 MiB, so a response holds about 64 MiB. There is no read of every file. - The body is application/vnd.skywalking.asz.files+ndjson: for each file a naming line {file, seq, lines, bytes, digest}, exactly that many bytes, and a newline after a non-empty file that does not end with one. Files come in seq order, the order provider bodies are read in. - The read takes its time range from the newest intact round, reading the rounds down from the head only until one is intact, then reads one storage window of files and hands it to the response before reading the next. Windows are produced one at a time and cannot overflow. - The route compresses with gzip itself, a chunk at a time. Armeria's encoder keeps every compressed chunk of a response in one growing buffer until the response ends. The view route still uses it. - Both routes require the service and the sender's instance, as a list row names them, so every read is a full series lookup; serviceId goes. A bad coldStage gets a problem document, not Armeria's plain-text 400. - The e2e reads files through swctl by session and seq taken from the document, and passes the instance on every view and route call. The raw-files case counts the three Session Data files of the first conversation, since rounds are no longer served. The e2e pins swctl to the merge commit of apache/skywalking-cli#235, which adds swctl ai-agent files with --session and --seqs.

wu-sheng added this to the 0.4.0 milestone Sep 15, 2026

wu-sheng added the enhancement New feature or request label Sep 15, 2026

wu-sheng merged commit 7373315 into main Sep 15, 2026
18 checks passed

wu-sheng deleted the provider-bodies branch September 15, 2026 13:10

wu-sheng mentioned this pull request Sep 15, 2026

AI agent conversations: provider bodies in the view, and files over HTTP apache/skywalking#14084

Merged

Navigation Menu

Collect Claude Code's provider bodies, with repeats taken out - #10

Collect Claude Code's provider bodies, with repeats taken out#10
wu-sheng merged 2 commits into
mainfrom
provider-bodies

wu-sheng commented Sep 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant