Skip to content

feat: Dapr 1.19 building blocks — catalog refresh + workflow data references - #149

Merged
marcduiker merged 16 commits into
mainfrom
feat/dapr-1-19-building-blocks
Nov 4, 2026
Merged

marcduiker merged 16 commits into
mainfrom
feat/dapr-1-19-building-blocks

Conversation

@marcduiker

@marcduiker marcduiker commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Dapr 1.19 added three alpha building blocks: Binary Store, Search and Vector. The .NET and JS SDKs also gained workflow data references: a large workflow payload is offloaded to a binary store, and workflow history holds a small {"$daprRef":{…}} envelope in its place. This PR makes the dashboard aware of both.

Design spec: docs/specs/2026-11-04-dapr-1-19-building-blocks-design.md

Component catalog

  • The embedded catalog is refreshed to components-contrib v1.19.0. It adds 6 binarystore, 2 search and 2 vector components, plus bindings.sqlserver.
  • The builder and Components list are catalog-driven, so the new categories need no UI change. Guard tests are added, and the 14 category chips still wrap correctly at the 768px minimum width.
  • ⚠️ Upstream renamed the pub/sub solace.amqp component to amqp. Nothing in this repo references the old name.

Workflow data references

  • On Workflow detail, a $daprRef envelope at any of the five payload sites (workflow input, output, custom status, event input and output) is shown as a card with store, key and size, plus a Load payload button.
  • Loading goes through the workflow app's own sidecar via a new read-only endpoint, GET /api/apps/{appId}/binarystore/{store}/file?key=…&size=…. The new pkg/binarystore package calls daprd's gRPC GetBinaryFileAlpha1. HTTP isn't usable here: daprd's route takes the file name as a single, un-unescaped path segment, and SDK keys contain /. It reuses the existing SidecarPool connections.
  • The payload is capped at 1 MiB, checked both on the declared size and while streaming.
  • The SHA-256 is verified in the browser and shown as verified, hash mismatch or unverified. unverified appears on non-secure origins, where crypto.subtle is missing. Content is shown as JSON, plain text, or "Binary payload (N bytes) — not shown".
  • Clear messages cover these cases: app not running ("Start to load this payload"), store not loaded by the app, file not found, sidecar older than 1.19, staged (compressed or encrypted) envelopes, and malformed envelopes.
  • Envelope detection follows the upstream normative rules (Workflow Data References at the SDK-level dapr/proposals#109). Only a top-level object with exactly one $daprRef property counts as an envelope.

Docs: AGENTS.md (repo layout) and ARCHITECTURE.md (read path). The endpoint is read-only, so the mutating-operations list is unchanged.

Out of scope

  • Browsing Search indexes and Vector collections (follow-up). Binary Store has no List API, so it can't be browsed.
  • Any write or delete against a binary store, including cleanup on purge.
  • Envelope handling in Replay, the Workflows list and Stats.

Test plan

  • make test: Go unit tests, and web 142 files / 1438 tests.
  • make build (tsc)
  • make test-integration
  • make lint: 0 errors. The 13 warnings are pre-existing and in files this PR doesn't touch.
  • Manual e2e against Dapr 1.19.0 with a binarystore.in-memory component:
    • the endpoint returns 200 for both a flat key and a slashed key (workflow-payloads/demo-1/<sha256>);
    • it returns 404 for a missing key and 400 for a non-binarystore store and for a missing key parameter;
    • for a real workflow (go-sdk) whose input is a $daprRef envelope, the card renders and Load shows verified and the JSON. Checked in light theme and in dark theme (via the Theme toggle).
  • e2e (CI, 6 jobs green): https://github.com/diagridio/dev-dashboard/actions/runs/41916567378
    • New TestComposeDiscovery/DataReference passes on Dapr 1.19.1. datarefapp stores a payload through SetBinaryFileAlpha1 under a slashed key and starts a workflow with the $daprRef envelope as input. The dashboard API shows that envelope as the input, serves the exact bytes (sha256 matches), and returns 404 for a missing key. On 1.18.10 the subtest is skipped and the compose project is unchanged.
    • Go SDK pairing: wfapp now uses the go-sdk v1.15 workflow API, and CI builds it with v1.15.0 for Dapr 1.18 and v1.16.0 for 1.19, resolved by scripts/latest-dapr-versions.sh.

Small fixes added after review

  • A sidecar time-out now returns 504 with "sidecar didn't return the file from within 10s" instead of a raw 502.

  • formatBytes moves up a unit after rounding (1048575 → "1.0 MiB", not "1024.0 KiB").

  • The payload cache key now includes sha256.

  • The disabled Load button is linked to its reason with aria-describedby.

  • Catalog sort is now a total order (type, status, title, then name, version), so v1/v2 pairs no longer reshuffle the golden summary on bundle refreshes. One-time golden change: postgresql and sqlserver now list v1 before v2. The state-store connection form used to pick the first stable entry, which only meant v2 by accident. It now explicitly picks stable first, then the highest version, so it stays on v2.

  • The sha256 cache-key change has its own test.

  • The e2e tests assert that wfapp was built with the paired go-sdk: it reports the compiled-in version and must match GO_SDK_VERSION. Run: https://github.com/diagridio/dev-dashboard/actions/runs/41919051193

Known follow-ups

  • None from review. Search/Vector browse pages are a separate feature.

🤖 Generated with Claude Code

marcduiker and others added 12 commits November 4, 2026 22:10
Adds binarystore (6), search (2), vector (2) and bindings/sqlserver.
Upstream renamed pubsub solace.amqp to amqp (no code references).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ument shared Conn

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tyling)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- binarystore: map gRPC DeadlineExceeded / read timeout to a new
  KindTimeout (HTTP 504) with a readable message instead of a raw 502
- formatBytes: promote to the next unit after rounding, so 1048575
  reads "1.0 MiB" instead of "1024.0 KiB"
- useBinaryFile: include sha256 in the query key so a rewritten key
  never shows stale cached content
- PayloadView: link the disabled Load button to its reason with
  aria-describedby

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@marcduiker
marcduiker force-pushed the feat/dapr-1-19-building-blocks branch from d247b34 to 29dece5 Compare November 4, 2026 21:12
marcduiker and others added 4 commits November 4, 2026 22:29
…line

Data references (Dapr 1.19+): a new compose profile "dataref" adds a
second sidecar (app id datarefapp) with a binarystore.in-memory
component and the datarefapp fixture (go-sdk v1.16.0). It stores a JSON
payload through SetBinaryFileAlpha1 under a slashed key and starts a
workflow whose input is the $daprRef envelope. TestComposeDiscovery's
DataReference subtest enables the profile when DAPR_RUNTIME_VERSION is
1.19+ and checks, through the dashboard's HTTP API, that the workflow
input is the envelope, that the binary store endpoint returns the exact
bytes (sha256 matches) and that a missing key is a 404. Below 1.19 the
subtest skips and the compose project is unchanged.

SDK pairing: wfapp moves from go-sdk v1.12.0 (Dapr 1.15 era) to the
v1.15 workflow API (durabletask-go/workflow + client.NewWorkflowClient),
pinned at v1.15.0. Its Dockerfile takes a GO_SDK_VERSION build arg, and
scripts/latest-dapr-versions.sh now also resolves go_sdk: the newest
go-sdk release whose go.mod requires that dapr/dapr minor (1.18 ->
v1.15.0, 1.19 -> v1.16.0). The e2e workflow exports it for compose and
testcontainers builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two references that differ only in sha256 must fetch separately, and
switching back must hit the cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wfapp reports the go-sdk version compiled into it (from build info) via
a WFAPP_GO_SDK log line and a -sdk-version flag. The compose and
testcontainers tests log it and, when GO_SDK_VERSION is set (CI),
require it to match, so a silently ignored build arg fails the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sortComponents tied on type, status and title, so v1/v2 of one
component (postgresql, sqlserver) could swap on any bundle refresh and
reshuffle the golden summary. Break ties by name, then version. The
golden summary changes once (those two pairs now list v1 first).

The state-store connection form picked the first stable catalog entry,
which only meant v2 because of that incidental order. It now picks
stable first, then the highest version, regardless of catalog order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@marcduiker
marcduiker merged commit e520c4c into main Nov 4, 2026
17 checks passed
@marcduiker
marcduiker deleted the feat/dapr-1-19-building-blocks branch November 4, 2026 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant