feat: preserve opaque_body provider state on prompt messages and LLM results - #261
Open
hpiclaranet wants to merge 5 commits into
Open
hpiclaranet wants to merge 5 commits into
hpiclaranet wants to merge 5 commits into
Conversation
…results Plugin LLM results carry provider round-trip state in opaque_body (e.g. signed/encrypted thinking blocks, Gemini thought_signature), matching the dify_plugin SDK entities. Graphon dropped that state in two places: - AssistantPromptMessage / PromptMessageContent had no opaque_body field, so daemon -> core Pydantic validation silently discarded it. - _LLMChunkAccumulator and _StreamingInvokeAccumulator rebuilt the assistant message from content and tool calls only. Add the field (same name, type and default as the plugin SDK) and carry the last non-None snapshot through both accumulators. Refs: langgenius#260
|
All contributors on this pull request have signed the CLA. |
Author
|
I have read the CLA Document and I hereby sign the CLA |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
!Related Issue
Closes #260
Summary
Provider reply snapshots were discarded when Graphon parsed assistant messages and when its LLM wrappers aggregated chunks.
A mixed reply such as a text chunk, a content block carrying a signature, and another text chunk also lost the content block during non-stream aggregation.
This change preserves the SDK's existing
opaque_body: JsonValue | Nonecontract on assistant messages and content blocks.Both LLM accumulators retain the last non-
Noneassistant snapshot within each invocation, including falsy JSON values, while later empty chunks leave it intact.Non-stream aggregation preserves mixed strings and content blocks in order, and assistant messages with opaque state are no longer considered empty.
Compatibility: payloads without the field remain valid, and pure-text responses still return strings.
Serialized content blocks now include
opaque_body: nullwhen unset, matching the SDK.Mixed responses retain their content arrays instead of discarding them.
This covers message serialization,
LargeLanguageModelaggregation including stream callbacks, and ordinary/structured-output blocking calls throughSlimLLM.Slim reuses the existing runtime normalizer while preserving structured output, tool-call fragments, usage, and fingerprint semantics.
LLM prompt filtering retains state on supported content blocks instead of collapsing signed text blocks into strings, and preserves empty-content assistant messages carrying state.
Cross-session storage and result-envelope history serialization remain separate boundaries; this PR alone does not establish deployed Dify replay.
The SDK aggregation counterpart is langgenius/dify-plugin-sdks#390, and the daemon transport counterpart is langgenius/dify-plugin-daemon#828.
Related reports and earlier work: dify#41092, official-plugins#3715, and dify#31566.
Validation:
just test(956 passed) andjust checkpassed, including lockfile, formatting, lint, type, and import checks.The public invocation tests cover JSON values and types at both message and content-block level, trailing empty chunks, consecutive calls, and both stream modes.
Independent test, knowledge, and final naming reviews completed for the added Slim and prompt-filter paths.
The new regressions failed on the previous implementation, and all tests/checks passed after the final naming changes.
The branch includes the latest upstream repository-knowledge review, so the documentation freshness check also passes.
No provider API requests or deployed Dify environment were used.
Checklist
!Dify 0.7 integration
Dify currently depends on the 0.7 runtime API.
The same reviewed production fix is available as the 0.7 backport, with 759 tests passing, for Dify #41095.
The backport also removes the older filter branch that discarded empty content arrays before inspecting assistant state.
It is an immutable review dependency; no new package version has been published.