Skip to content

feat: preserve opaque_body provider state on prompt messages and LLM results - #261

Open
hpiclaranet wants to merge 5 commits into
langgenius:mainfrom
hpiclaranet:feat/opaque-body-prompt-messages
Open

hpiclaranet wants to merge 5 commits into
langgenius:mainfrom
hpiclaranet:feat/opaque-body-prompt-messages

Conversation

@hpiclaranet

@hpiclaranet hpiclaranet commented Aug 22, 2026 •

Copy link
Copy Markdown

Important

  1. Make sure you have read our contribution guidelines
  2. Search existing issues and pull requests to confirm this change is not a duplicate
  3. Open or identify the issue this pull request resolves or advances
  4. Use a Conventional Commits title for this pull request, and mark breaking changes with !
  5. Remember that the pull request title will become the squash merge commit message
  6. If CLA Assistant prompts you, sign CLA.md in the pull request conversation

Related Issue

Closes #260

Summary

Provider reply snapshots were discarded when Graphon parsed assistant messages and when its LLM wrappers aggregated chunks.
A mixed reply such as a text chunk, a content block carrying a signature, and another text chunk also lost the content block during non-stream aggregation.

This change preserves the SDK's existing opaque_body: JsonValue | None contract on assistant messages and content blocks.
Both LLM accumulators retain the last non-None assistant snapshot within each invocation, including falsy JSON values, while later empty chunks leave it intact.
Non-stream aggregation preserves mixed strings and content blocks in order, and assistant messages with opaque state are no longer considered empty.

Compatibility: payloads without the field remain valid, and pure-text responses still return strings.
Serialized content blocks now include opaque_body: null when unset, matching the SDK.
Mixed responses retain their content arrays instead of discarding them.

This covers message serialization, LargeLanguageModel aggregation including stream callbacks, and ordinary/structured-output blocking calls through SlimLLM.
Slim reuses the existing runtime normalizer while preserving structured output, tool-call fragments, usage, and fingerprint semantics.
LLM prompt filtering retains state on supported content blocks instead of collapsing signed text blocks into strings, and preserves empty-content assistant messages carrying state.
Cross-session storage and result-envelope history serialization remain separate boundaries; this PR alone does not establish deployed Dify replay.
The SDK aggregation counterpart is langgenius/dify-plugin-sdks#390, and the daemon transport counterpart is langgenius/dify-plugin-daemon#828.
Related reports and earlier work: dify#41092, official-plugins#3715, and dify#31566.

Validation: just test (956 passed) and just check passed, including lockfile, formatting, lint, type, and import checks.
The public invocation tests cover JSON values and types at both message and content-block level, trailing empty chunks, consecutive calls, and both stream modes.
Independent test, knowledge, and final naming reviews completed for the added Slim and prompt-filter paths.
The new regressions failed on the previous implementation, and all tests/checks passed after the final naming changes.
The branch includes the latest upstream repository-knowledge review, so the documentation freshness check also passes.
No provider API requests or deployed Dify environment were used.

Checklist

  • This pull request links the issue it resolves or advances
  • This pull request title follows Conventional Commits, and any breaking change is marked with !
  • If CLA Assistant prompted me, I signed CLA.md in the pull request conversation

Dify 0.7 integration

Dify currently depends on the 0.7 runtime API.
The same reviewed production fix is available as the 0.7 backport, with 759 tests passing, for Dify #41095.
The backport also removes the older filter branch that discarded empty content arrays before inspecting assistant state.
It is an immutable review dependency; no new package version has been published.

…results

Plugin LLM results carry provider round-trip state in opaque_body
(e.g. signed/encrypted thinking blocks, Gemini thought_signature),
matching the dify_plugin SDK entities. Graphon dropped that state in
two places:

- AssistantPromptMessage / PromptMessageContent had no opaque_body
  field, so daemon -> core Pydantic validation silently discarded it.
- _LLMChunkAccumulator and _StreamingInvokeAccumulator rebuilt the
  assistant message from content and tool calls only.

Add the field (same name, type and default as the plugin SDK) and
carry the last non-None snapshot through both accumulators.

Refs: langgenius#260
@dosubot dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. enhancement New feature or request labels Aug 22, 2026
@github-actions

github-actions Bot commented Aug 22, 2026 •

Copy link
Copy Markdown

All contributors on this pull request have signed the CLA.
Posted by the CLA Assistant Lite bot.

@hpiclaranet

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@WH-2099 WH-2099 self-assigned this Sep 23, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provider round-trip state (opaque_body) is dropped on assistant prompt messages and LLM results

2 participants