Skip to content

feat(activex): present EGFX dirty regions - #1874

Merged
Marc-André Moreau (mamoreau-devolutions) merged 9 commits into
masterfrom
copilot/activex-accelerated-graphics
Sep 16, 2026
Merged

Marc-André Moreau (mamoreau-devolutions) merged 9 commits into
masterfrom
copilot/activex-accelerated-graphics

Conversation

@mamoreau-devolutions

@mamoreau-devolutions Marc-André Moreau (mamoreau-devolutions) commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Propagate validated ResetGraphics extents into the session framebuffer and deliver exact packed desktop damage to ActiveX without rebuilding full image snapshots.

Coalesce only fully covered regions, preserve sparse updates with a 64-event and 256 MiB pixel-data budget, and backpressure without evicting accepted frame or static-channel payloads. Update retained GDI and RPC surfaces in place. Reuse framebuffer storage with fallible growth, and retain software cursor shape, position, visibility, and clipping across graphics resets. Keep V8 non-AVC negotiation and full-frame output fallback unchanged. Rejected oversized resets still destroy prior surfaces and are reported as unsupported without allocating a session framebuffer.

Copilot AI balanced review requested due to automatic review settings September 1, 2026 18:36
@github-actions github-actions Bot added kind/protocol Affects RDP or related protocol behavior needs-review A human reviewer is the current next actor risk/medium Behavioral change that does not substantially alter a core public API scope/core Touches the core architectural tier scope/cross-cutting Spans multiple architectural boundaries scope/ffi Affects native or .NET bindings size/XL Size: up to 1299 counted lines and 49 files; exceeds L in either measure labels Sep 1, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds dirty-region delivery for ActiveX while retaining full-frame fallback.

Changes:

  • Propagates validated EGFX reset extents into session framebuffers.
  • Adds packed desktop updates and retained partial-frame presentation.
  • Introduces bounded damage queuing and related tests/documentation.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
crates/ironrdp-testsuite-extra/tests/client/output_channel.rs Tests desktop-update validation.
crates/ironrdp-session/src/active_stage.rs Resizes framebuffers after EGFX resets.
crates/ironrdp-egfx/src/compositor.rs Validates output dimensions and memory limits.
crates/ironrdp-egfx/src/client.rs Reports validated reset extents.
crates/ironrdp-client/src/rdp.rs Adds packed dirty-region delivery.
crates/ironrdp-activex/src/rpc.rs Applies partial updates to retained RPC frames.
crates/ironrdp-activex/src/control.rs Queues, merges, presents, and invalidates dirty regions.
crates/ironrdp-activex/README.md Documents the dirty-region pipeline.

Comment thread crates/ironrdp-activex/src/control.rs
Comment thread crates/ironrdp-client/src/rdp.rs Outdated
Comment thread crates/ironrdp-session/src/active_stage.rs Outdated
@github-actions github-actions Bot added breaking-change Includes a breaking change, and requires special scrutiny at the boundaries risk/high Substantial core public API impact, or fail-closed triage; needs maintainer-level scrutiny risk/unknown Risk could not be determined automatically; needs maintainer-level scrutiny size/XXL Size: 1300 or more counted lines or 50 or more files and removed risk/medium Behavioral change that does not substantially alter a core public API size/XL Size: up to 1299 counted lines and 49 files; exceeds L in either measure breaking-change Includes a breaking change, and requires special scrutiny at the boundaries risk/high Substantial core public API impact, or fail-closed triage; needs maintainer-level scrutiny labels Sep 1, 2026
@mamoreau-devolutions

Copy link
Copy Markdown
Contributor Author

The size/XXL warning is acknowledged. The diff has two conceptual review boundaries—EGFX reset/session propagation and ActiveX packed-damage presentation—but they form the end-to-end graphics path validated by this PR. This recovery was explicitly constrained to the existing PR branch and to preserving valid pushed work, so I did not create stacked branches, discard either boundary, or rewrite the existing commits. Both boundaries received separate protocol, implementation, documentation, and skeptical review passes; the retained findings are fixed at 82d445e88.

Note

Human-tuned, LLM-assisted content.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Direct frame dispatch can violate lifecycle ordering, and differing framebuffer extents still evict accepted updates.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 11/11 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread crates/ironrdp-activex/src/control.rs Outdated
Comment thread crates/ironrdp-activex/src/control.rs
@github-actions github-actions Bot removed the breaking-change Includes a breaking change, and requires special scrutiny at the boundaries label Sep 4, 2026
@github-actions github-actions Bot added risk/unknown Risk could not be determined automatically; needs maintainer-level scrutiny risk/high Substantial core public API impact, or fail-closed triage; needs maintainer-level scrutiny and removed risk/high Substantial core public API impact, or fail-closed triage; needs maintainer-level scrutiny risk/unknown Risk could not be determined automatically; needs maintainer-level scrutiny labels Sep 4, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dirty-region pipeline is sound: packed updates are validated, covered-union coalescing preserves ordering, backpressure replaces payload eviction, and pointer-preserving reset is well tested. Two availability questions remain open: queue producers now block the RDP worker indefinitely on a UI-drained queue, and an oversized but protocol-valid ResetGraphics becomes a hard PDU error consumed only by capture-replay. The RPC retained frame can go stale after an extent change, and legacy oversized-frame discards need a reachability decision. One independent question covers damage regions cleared per ActiveStage call while packed delivery consumes them once per iteration. Remaining candidates are valid deduplication cleanups; the PR-scope concern is rejected as non-actionable and already answered.

  1. [skeptical] Producers block the RDP worker thread indefinitely when the STA stops draining the event queue — medium 🟠 ❓ — crates/ironrdp-activex/src/control.rs
    queue_worker_event now parks Image, StaticChannelData, and AutoReconnecting producers on space_available with no timeout; the only exits are a UI drain, queue close, or a failed PostMessageW. Producers execute on the RDP worker's current-thread runtime, so a stalled or non-pumping STA wedges transport reads, input encoding, keepalives, and reconnect decisions where payloads were previously dropped or failed fast. The diff does not show that every teardown path reaches WorkerEventQueue::close, so a teardown deadlock cannot be ruled out.
  2. [skeptical] Oversized but protocol-valid ResetGraphics now returns a PDU error that plausibly terminates live sessions — medium 🟠 ❓ — crates/ironrdp-egfx/src/client.rs
    handle_reset_graphics returns Err once materializable_output_size rejects the extent (a dimension above 32766 or output above the 256 MiB budget), after clearing surfaces and pending deltas. The new on_reset_graphics_rejected callback is implemented only by capture-replay; no ironrdp-client or ActiveX handler consumes it, so live clients observe a session-failing PDU error where oversized output previously degraded to dropped deltas. Confirm the intended live-client outcome and that no supported deployment negotiates such extents.
  3. [skeptical] Discarded partial update leaves a stale previous-extent frame under new desktop-size properties — low 🟡 — crates/ironrdp-activex/src/rpc.rs
    retain_frame_region_with_limit inserts desktopwidth/desktopheight before validation, but the partial-update-without-matching-base path returns without touching live.frame, unlike the oversized path which clears it. After an EGFX reset changes the extent, a partial update arriving before the guaranteed full frame leaves a screenshot-visible frame of the old dimensions advertised with the new desktop size. Clearing live.frame on this discard path keeps the snapshot coherent.
  4. [skeptical] Legacy full-frame updates above the 256 MiB queue budget are discarded with only a warning — low 🟡 ❓ — crates/ironrdp-activex/src/control.rs
    The Image arm drops any single update with more than MAX_PENDING_FRAME_PIXELS (~67M) pixels, logging only a warn. EGFX outputs are pre-validated to the same 256 MiB budget, and conformant legacy desktops cap at 8192x8192, which exactly equals the budget and passes the strictly-greater check, so the path may be unreachable defensive code. If larger extents are negotiable through the control, every frame is silently discarded and the display goes stale or black with no session-level signal; confirm reachability and prefer failing or signaling over a per-frame silent discard.

Comment thread crates/ironrdp-activex/src/control.rs
Comment thread crates/ironrdp-client/src/rdp.rs
Comment thread crates/ironrdp-activex/src/control.rs
Comment thread crates/ironrdp-activex/src/rpc.rs
Comment thread crates/ironrdp-session/src/image.rs Outdated
Comment thread crates/ironrdp-client/src/rdp.rs
@github-actions github-actions Bot added ai-reviewed/2 Two automated reviews completed and removed ai-reviewed/1 One automated review completed labels Sep 10, 2026
Propagate validated ResetGraphics extents into the session framebuffer and deliver packed desktop damage to ActiveX without rebuilding full image snapshots.

Coalesce only fully covered regions, preserve disjoint updates with bounded backpressure, and update retained GDI and RPC surfaces in place. Keep V8 non-AVC negotiation and full-frame output fallback unchanged.
Destroy prior surfaces when a ResetGraphics output exceeds local allocation limits, matching protocol reset semantics.

Notify handlers about the rejected output so capture replay retains its unsupported-path classification and fuzz models stay synchronized.
Preserve exact compositor regions so sparse updates do not expand into full bounding-box copies, and cap queued pixel payloads at 256 MiB with backpressure.

Retain software cursor shape, position, and visibility when ResetGraphics replaces the session framebuffer.
Invalidate retained RPC frames when a resized desktop exceeds the screenshot budget, and consume the EGFX client's accepted reset state directly during capture replay.
Keep exact frame and static-channel updates ordered when the UI queue is saturated, and make dispatch recovery fail instead of waiting without an outstanding message.

Reuse framebuffer storage across graphics resets and clip retained software cursors without source-rectangle underflow.
Integrate partial frame presentation with the modern screen-update suspension contract by snapshotting one bounded full-frame base and applying every suspended dirty region before resume.
Route desktop updates through the same ordered output-event channel
used by Connected and other events, instead of invoking a
side-channel closure directly from the output loop. The closure
bypassed channel ordering, letting the UI observe a desktop update
before Connected on a separate task, since Connected was merely
enqueued rather than synchronously flushed.

Replace `with_desktop_update_handler(closure)` with
`with_desktop_updates()`, a flag consumed by `active_session`, and
add `RdpOutputEvent::DesktopUpdate`, classified as `MustDeliver`
since it carries a diff against the prior frame and cannot be safely
dropped like a full snapshot.

ActiveX now consumes `DesktopUpdate` from its ordered output
receiver instead of a raw callback. The viewer does not opt in and
adds a no-op match arm.
`queue_worker_event` merges a same-generation frame in place when
extents match. When a resize landed between frames, the fallback
path replaced the queued frame at a different extent instead of
appending, silently discarding an already-accepted transitional
frame and its pixels.

Append instead of overwriting. The preceding budget check already
sizes for queued pixels plus the incoming frame as an addition, so
this does not change pending-pixel/event budget behavior.
@github-actions github-actions Bot added the breaking-change Includes a breaking change, and requires special scrutiny at the boundaries label Sep 16, 2026
`reset_preserving_pointer` called `resize(len, 0)` then `fill(0)`,
zero-filling the whole buffer twice on every ResetGraphics: resize
only zero-fills the grown tail, so the pre-existing prefix was
written by both calls. Clear the buffer first so resize zero-fills
the entire (now-empty) buffer in one pass; capacity reserved just
above is preserved by `clear`, so no extra allocation occurs.
@mamoreau-devolutions
Marc-André Moreau (mamoreau-devolutions) merged commit 2b44c62 into master Sep 16, 2026
41 checks passed
@mamoreau-devolutions
Marc-André Moreau (mamoreau-devolutions) deleted the copilot/activex-accelerated-graphics branch September 16, 2026 02:26
Anton Mostovoy (antonmos) added a commit to antonmos/IronRDP that referenced this pull request Sep 20, 2026
…ting-session-default

* upstream/master: (22 commits)
  ci(agents): sync shared review skills (Devolutions#1973)
  [DEVOPS-4678] chore(ci): drop the Artifactory cache refresh step (Devolutions#1972)
  feat(activex): present EGFX dirty regions (Devolutions#1874)
  fix(pdu): add missing error info codes (Devolutions#1966)
  ci(pr-automation): hand off after a clean review (Devolutions#1959)
  ci(pr-automation): link resolved PR in run summary (Devolutions#1960)
  ci(pr-automation): show automation mode in the run name (Devolutions#1958)
  ci(pr-automation): make overlap detection advisory (Devolutions#1957)
  feat(server): add ConnectionPolicy to reject a second connection (Devolutions#1913)
  feat(rdpeai): add RdpeaiServer orchestration for AUDIO_INPUT (Devolutions#1945)
  ci(pr-automation): clarify review repair failures (Devolutions#1956)
  ci(pr-automation): clarify workflow name and run title (Devolutions#1950)
  ci(pr-automation): delay maintainer handoff (Devolutions#1937)
  ci(pr-automation): reserve turns a repair can spend (Devolutions#1939)
  fix(egfx): do not track frames a suspended client cannot acknowledge (Devolutions#1912)
  feat(rdpeudp): MS-RDPEUDP version 1/2 reliable data transfer (Devolutions#1919)
  fix(viewer): map native physical keycodes (Devolutions#1929)
  ci(pr-automation): classify after final review (Devolutions#1936)
  fix(egfx): decode AVC420 with full-range BT.709 (Devolutions#1923)
  test(core): add real-world progressive RemoteFX difference fixtures from Haven (Devolutions#1848)
  ...

# Conflicts:
#	crates/ironrdp-server/src/builder.rs
#	crates/ironrdp-server/src/server.rs

This branch was previously deployed

1 inactive deployment
llm-providers — 0610017d Deployed Sep 16, 2026 by mamoreau-devolutions via Classify pull request #420
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-reviewed/2 Two automated reviews completed breaking-change Includes a breaking change, and requires special scrutiny at the boundaries kind/protocol Affects RDP or related protocol behavior needs-review A human reviewer is the current next actor risk/high Substantial core public API impact, or fail-closed triage; needs maintainer-level scrutiny scope/core Touches the core architectural tier scope/cross-cutting Spans multiple architectural boundaries scope/ffi Affects native or .NET bindings size/XXL Size: 1300 or more counted lines or 50 or more files

Development

Successfully merging this pull request may close these issues.

3 participants