Repository navigation
feat(session): expose EGFX compositing - #2120
Marc-André Moreau (mamoreau-devolutions) merged 3 commits into
Conversation
|
Validation at 31a3331:
Where it's used: a macOS client I work on runs the pipeline on its own thread and calls this after each EGFX message. Against a headless GNOME Remote Desktop with 50 ms and 150 ms of artificial decode delay per update, all of a 17 s tone arrived; with the pipeline on the session loop, audio stopped after 3 s and the server dropped the connection. Note Human-tuned, LLM-assisted content. |
There was a problem hiding this comment.
🟡 Changes recommended
The usage documentation incorrectly routes tunneled EGFX replies through the TCP-only encoder.
1 open finding
What changed in this PR
Exposes EGFX compositing so pipelines can run outside the session loop without blocking other channels.
Changes:
- Adds public compositing results and helper API.
- Reuses the helper within
ActiveStage. - Adds integration coverage and marks EGFX as public.
| File | Description |
|---|---|
crates/ironrdp-session/src/active_stage.rs |
Adds and integrates the compositing API. |
crates/ironrdp-session/src/lib.rs |
Re-exports the new public API. |
crates/ironrdp-session/Cargo.toml |
Marks EGFX as a public dependency. |
crates/ironrdp-testsuite-core/tests/session/active_stage.rs |
Tests reset, compositing, and damage behavior. |
🧠 Review effort: Balanced
There was a problem hiding this comment.
PR #2120 extracts ActiveStage's private EGFX drain logic into a public composite_graphics_pipeline plus a #[non_exhaustive] CompositedGraphics result, making ironrdp-egfx a public dependency so off-loop callers (e.g., a dedicated decode thread) can composite without reimplementing reset/compositing subtleties. Verified behaviorally equivalent to the old inline path for all cases (no pipeline, reset only, updates only, both), including the Ok(None) channel-absent case. Two new tests cover the public path and the preserved ActiveStage path (reset widening, damage recording). No correctness or protocol defects found. Published one low-severity doc gap (docs omit the failure mode of compositing a pipeline ActiveStage still owns via the public get_dvc_mut), one low-severity control-flow style note on the rewritten drain method, and one low-severity test-fixture duplication note; rejected the suggestion to drop CompositedGraphics::region because the union helper is private and the field is …
- [code-compressor] Rewritten drain_graphics_pipeline chains Option combinators where a flat let-else reads smaller — low 🟡 — crates/ironrdp-session/src/active_stage.rs
The rewrite replaces the old map(...).unwrap_or_default() with map(...).transpose()? plus a let-else, keeping the combinator chain only to thread the channel-absent case. Since get_dvc_mut already returns Option<DynamicChannelMut<T>>, the same behavior is shorter and flat: bind the channel with a let-else returning Ok(None), then call composite_graphics_pipeline(gfx.processor_mut(), image)? directly. Error propagation, the None return, and lifetimes are identical. Optional style refinement; the current form is correct.
Push a commit after addressing these findings. If no code change is needed, you may resolve inline threads and comment @github-actions review-ready to request human review.
| /// [`ActiveStage`] does this after each PDU for a [`GraphicsPipelineClient`] | ||
| /// registered on its dynamic channels. Call it directly for one that runs | ||
| /// elsewhere, such as on a thread of its own so that a slow H.264 decode | ||
| /// doesn't hold up the session's other channels. The caller then also: | ||
| /// | ||
| /// - registers in its place a [`DvcClientProcessor`] named | ||
| /// [`ironrdp_egfx::CHANNEL_NAME`] that forwards each message to the | ||
| /// pipeline; | ||
| /// - sends the pipeline's replies (capabilities, frame acknowledgements): | ||
| /// wraps them with [`ironrdp_dvc::encode_dvc_messages`] for the channel, | ||
| /// then, if [`ActiveStage::dvc_tunnel_for_channel`] names a Soft-Sync | ||
| /// tunnel, sends each one's [`SvcMessage::encode_unframed_pdu`] through | ||
| /// it, and otherwise encodes them for TCP with | ||
| /// [`ActiveStage::encode_dvc_messages`]; | ||
| /// - shares `image` with the session under a lock, since other updates | ||
| /// (bitmaps, the pointer) write to it too; | ||
| /// - tracks the regions itself: they don't reach | ||
| /// [`ActiveStage::take_damage_regions`]. |
There was a problem hiding this comment.
[skeptical] Docs omit the failure mode of compositing a pipeline that ActiveStage still owns — low 🟡 — ActiveStage::get_dvc_mut::<GraphicsPipelineClient>() is already public (active_stage.rs:493), so a caller can fetch the in-stage pipeline and pass it to the now-public composite_graphics_pipeline. That drains take_output_reset() and drain_output() before ActiveStage::process runs, so the in-stage drain sees no reset and no updates: graphics_output_needs_full_refresh is never set, widen_to_full_refresh never fires, and regions never reach take_damage_regions. The image is cleared and painted while the caller repaints only the returned region, leaving pre-reset content as silent visual corruption. The documented caller obligations cover the intended off-loop setup but do not warn that one pipeline must not be driven from both modes; a one-sentence doc guard (or debug-time detection) would close this.
| }; | ||
| let fill = |frame_id, rect: ExclusiveRectangle| { | ||
| [ | ||
| GfxPdu::StartFrame(StartFramePdu { | ||
| timestamp: Timestamp { | ||
| milliseconds: 0, | ||
| seconds: 0, | ||
| minutes: 0, | ||
| hours: 0, | ||
| }, | ||
| frame_id, | ||
| }), | ||
| GfxPdu::SolidFill(SolidFillPdu { | ||
| surface_id: 1, | ||
| fill_pixel: Color { | ||
| b: 0x30, | ||
| g: 0x20, | ||
| r: 0x10, | ||
| xa: 0xFF, | ||
| }, | ||
| rectangles: vec![rect], | ||
| }), | ||
| GfxPdu::EndFrame(EndFramePdu { frame_id }), | ||
| ] |
There was a problem hiding this comment.
[code-compressor] Two new tests rebuild identical EGFX PDU fixtures instead of sharing a helper — low 🟡 — Both new tests inline the same StartFrame/SolidFill/EndFrame triple, the all-zero Timestamp, the same fill color, and the same ResetGraphics/CreateSurface/MapSurfaceToOutput sequence; test 2 builds the fill triple via a local closure while test 1 repeats it inline. Hoisting fill (and optionally the surface-setup list) to file scope as small helpers returning Vec<GfxPdu> would cut roughly 40 lines and keep the fixture in one place. Test 2's gfx() frame wrapper is transport-specific and stays per-test.
ActiveStage composites the EGFX output after each PDU. That only works when the GraphicsPipelineClient lives inside it. Why run it somewhere else: - H.264 decoding can be slow (a busy GPU). - While it decodes, the session loop doesn't read the socket. - Audio waits behind video, and GNOME Remote Desktop drops audio older than 50 ms. - Running the pipeline on its own thread fixes that, but then nothing composites its output: the reset and composite steps were private. What this adds: - composite_graphics_pipeline(gfx, image): follows a pending ResetGraphics, then applies the finished output regions. - CompositedGraphics: whether it reset, the union region, each region, and the RemoteApp window updates. - ActiveStage calls it too, so its behavior doesn't change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The function's docs now say what running the pipeline outside ActiveStage takes: a forwarding channel processor, sending the replies, locking the image, painting the window updates, and tracking damage. After a reset the caller repaints the whole image, as ActiveStage does. ironrdp-egfx is marked public, since GraphicsPipelineClient is in the signature, and CompositedGraphics is non_exhaustive without the unused Default. A test drives a registered pipeline through ActiveStage and checks the widened update after a reset and the recorded damage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The steps for a caller of composite_graphics_pipeline sent the pipeline's replies with ActiveStage::encode_dvc_messages, which always encodes for TCP, so they went to the wrong transport once Soft-Sync moved the channel onto a tunnel. Name the tunnel route, the ironrdp_dvc::encode_dvc_messages step the replies need first, and that a reset must be remembered until the next paint. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3c4c95c to
397fc69
Compare
|
Rebased onto What the rebase changed
Validation at
Lines changed against
32 of the added production lines are rustdoc. Note LLM-assisted content (no human feedback). |
c25dba1
into
Devolutions:master
…ktop-scale-factor * upstream/master: (92 commits) fix(server): advertise horizontal mouse wheel input (Devolutions#2236) fix(pdu): reject RSA moduli below 512 bits (Devolutions#2225) ci(ffi): verify generated bindings (Devolutions#2166) build(ffi)!: move to Diplomat 0.16.1-devo.2 (Devolutions#2165) build: make the client's video, camera and decoder crates publishable (Devolutions#2239) ci(release): release the newly published crates (Devolutions#2218) perf(graphics): replace the RLGR decoder's bit reader (Devolutions#2150) chore(graphics): add RemoteFX decode benchmarks (Devolutions#2149) fix(server)!: pass a GfxContext to the GfxServerFactory build methods (Devolutions#2104) fix(pdu): check remaining length when skipping padding and reading client core data (Devolutions#2130) fix(session): avoid hang on empty QOIZ updates (Devolutions#2147) feat(session): expose EGFX compositing (Devolutions#2120) fix(server): send RDP as the Demand Active source descriptor (Devolutions#2119) feat(activex)!: broaden mstscax compatibility (Devolutions#2211) feat(viewer): run RemoteApp programs in native windows (Devolutions#2201) fix(graphics): reject RLGR3 values larger than their remainder (Devolutions#2132) feat(egfx)!: composite surfaces mapped to RemoteApp windows (Devolutions#2197) feat: redirect USB devices over URBDRC with nusb and RemoteFX USB backends (Devolutions#2181) build: mark exposed dependencies as public (Devolutions#2213) chore(release): prepare for publishing (Devolutions#2212) ... # Conflicts: # crates/ironrdp-testsuite-extra/tests/e2e.rs

ActiveStagecomposites the EGFX output after each PDU. That only works when theGraphicsPipelineClientlives inside it.Why run it somewhere else:
What this adds:
composite_graphics_pipeline(gfx, image): follows a pendingResetGraphics, applies the finished output regions, and takes the RemoteApp window regions.CompositedGraphics: whether it reset, the union region, each region, and the window updates.ActiveStagecalls it too, so its behavior doesn't change.ironrdp-egfxis now a# publicdependency, sinceGraphicsPipelineClientis in the signature.CompositedGraphicsis#[non_exhaustive].Using it off the session loop, the caller also:
ironrdp_egfx::CHANNEL_NAMEthat forwards each message to the pipeline;dvc_tunnel_for_channelnames, or over TCP withActiveStage::encode_dvc_messages;ActiveStagereturns them inWindowSurfaceUpdates;The function's docs say the same.
🤖 Generated with Claude Code