Skip to content

Raise data-source delegates at test lifecycle stages - #7521

Merged
thomhurst merged 2 commits into
mainfrom
issue-7473-lifecycle
Nov 3, 2026
Merged

thomhurst merged 2 commits into
mainfrom
issue-7473-lifecycle

Conversation

@thomhurst

Copy link
Copy Markdown
Owner

Data-source subscriptions to registration, start, end, skipped and retry events were retained in row contexts but never dispatched. Raise them alongside the matching lifecycle receivers, with start/end delegates in the late stage and retry delegates receiving the upcoming one-based attempt after backoff and cancellation checks.

Registration tracks subscription identity across the pre/post property-injection passes, preserving duplicate subscriptions. Dispatch uses AsyncEvent's immutable published list as a snapshot. End failures join the teardown exception collection; other event failures propagate, and failing retry handlers preserve accepted cancellation. Reuses the merged ownership/context foundation from #7472 without changing scope ownership or shipped signatures.

Closes #7473

Validation:

  • Targeted EventReceiverRegistrationTests and RetryDelegateCancellationTests in generated and reflection modes.
  • Targeted DelegateLifecycleEventTests, ClassDelegateLifecycleEventTests and PropertyRegistrationDelegateTests in generated and reflection modes (one intentional skip per lifecycle run).
  • Engine.Tests left to hosted CI per repository instructions. No generator output or reflection paths changed.

@thomhurst

Copy link
Copy Markdown
Owner Author

@codex review

@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 25 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 8 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 73c7f9d1-b19d-420a-abd2-ca2f83fe5cfc
Commits

Reviewing files that changed from the base of the PR and between 149ee33 and 8ea91bd.

Files selected for processing: 10
  • docs/docs/execution/retrying.md
  • src/TUnit.Core/Interfaces/ITestEvents.cs
  • src/TUnit.Core/TestContextEvents.cs
  • src/TUnit.Engine/Services/EventReceiverOrchestrator.cs
  • src/TUnit.Engine/Services/TestExecution/RetryHelper.cs
  • src/TUnit.Engine/Services/TestFilterService.cs
  • tests/TUnit.TestProject/DelegateLifecycleEventTests.cs
  • tests/TUnit.TestProject/PropertyRegistrationDelegateTests.cs
  • tests/TUnit.UnitTests/EventReceiverRegistrationTests.cs
  • tests/TUnit.UnitTests/RetryDelegateCancellationTests.cs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-03T20:24:04.680585Z 8ea91bd Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Review: Raise data-source delegates at test lifecycle stages

I read the diff. The Claude review skill failed to run, so this was a manual read and I did not run any tests.

What looks right

  • Delegates now fire for registration, start, end, skipped and retry, which is what Raise data-source delegate handlers at test lifecycle stages #7473 asks for.
  • Start and end delegates run only in the non-Early stage, so they run once. The un-staged end calls in TestCoordinator also reach them.
  • Every fast-path early return was updated to account for delegate subscribers. Without that, delegate-only tests would have been skipped silently.
  • Registration dedupes by invocation identity across the pre- and post-injection passes. This keeps duplicate subscriptions while avoiding double-firing. Iterating InvocationList as a snapshot is a sound way to tolerate re-entrant subscription.
  • End failures join the teardown exception list, and retry handler failures preserve an accepted cancellation. Both are consistent with the documented behaviour.
  • Tests cover class-level and method-level subscriptions, retry, skip and property injection.

Suggestions (non-blocking)

  1. The start, skipped and end dispatch loops are near-duplicates. A small helper such as InvokeDelegatesAsync(AsyncEvent<TestContext>?, context) would keep the failure semantics (propagate vs collect) in one place.
  2. RetryHelper uses .ConfigureAwait(false) but EventReceiverOrchestrator does not. The surrounding orchestrator code already omits it, so this is harmless, but it is inconsistent within the same PR.
  3. The OnTestRegistered handlers throw in the registration path. A thrown handler aborts registration of that test, and the docs say so. Check that the failure surfaces as a test failure and not a discovery-wide error.

Overall this is a clean change with good test coverage. CI on the engine tests will be the real check.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8f885cfc79

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@greptile-apps

greptile-apps Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the previous cancellation issue is fixed.

Summary

Raises data-source delegates at registration, start, end, skip, and retry stages.

  • Registration callbacks run as data-source values are registered.
  • Lifecycle callbacks run alongside their matching test events.
  • Retry callbacks receive the upcoming attempt number.

Reviews (2) · Last reviewed commit: "Stop retrying after successful handlers ..." · Reviewed by Greptile

@thomhurst

Copy link
Copy Markdown
Owner Author

Addressed both retry-cancellation findings in 8ea91bd. The regression failed before the fix and all 3 retry-delegate cancellation cases now pass in generated and reflection modes (net10.0). Engine.Tests remains for CI per repository instructions.

Claude's nonblocking suggestions: keeping the small dispatch loops explicit preserves their different propagation/teardown collection semantics; keeping the orchestrator's existing await convention avoids unrelated style changes. I checked registration exception propagation: the existing receiver path can abort the whole request, so I recorded that separate engine failure-isolation problem as #7525. It does not block this lifecycle dispatch change.

@thomhurst

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 8ea91bd0a1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Review of #7521

I read the diff and the retry loop in RetryHelper. I didn't build or run the tests. The skill-based review failed to run, so this is a manual read.

Overall: The change looks correct and the scope is tight. It dispatches the start, end, skipped, registration and retry delegate events that were previously stored but never raised.

What I checked

  • Fast paths: The early-return guards now also check whether a delegate event exists, so tests with only delegate subscribers aren't short-circuited.
  • Stage handling: Start and end delegates run only in the non-Early stage, so they fire once rather than in both passes.
  • Snapshots: Iterating InvocationList gives a snapshot, so handlers added during dispatch apply to the next dispatch. The unit tests cover this.
  • End failures: They are collected and the remaining handlers still run, matching how end receivers behave.
  • Registration: HashSet<Invocation> identity keeps duplicate subscriptions while skipping handlers already run in the pre-injection pass. The second pass is guarded by argumentsRegistered.
  • Retry: The handler runs after backoff and the cancellation check, and receives attempt + 1. A throwing handler preserves an accepted cancellation. A second IsTestCancellationRequested check follows the handler, which covers a handler that cancels without throwing.

Suggestions (non-blocking)

  1. Duplicated dispatch loops. Start, skipped, registration and retry each repeat a foreach (var handler in ev.InvocationList) await handler.InvokeAsync(...) loop. A small helper such as AsyncEvent<T>.InvokeAllAsync(sender, arg) would remove the repetition. The end path would need a variant that collects exceptions. It would also be the one place to document the snapshot semantics, rather than repeating them in comments.
  2. Retry attempt number. attempt + 1 relies on attempt being the zero-based index of the failed attempt. The docs and tests state the intended one-based value of the upcoming retry, and Retried asserts retry == 1. A named local, e.g. var upcomingRetry = attempt + 1, would make that intent clear in the code.
  3. Handler failures stop remaining handlers. The remarks say start, skipped and registration failures "stop the remaining handlers". That is consistent with receivers, but it means one subscriber can starve others. This is acceptable, and it is documented.

No blocking issues. Engine tests are left to CI as per repo policy.

@thomhurst

Copy link
Copy Markdown
Owner Author

Reviewed all review bodies, inline threads and conversation comments for 8ea91bd. Claude and Codex both report no blocking findings; both cancellation threads have pushed fixes and are resolved.

Disposition of Claude's latest nonblocking suggestions: retain the explicit loops because the end path collects failures while the other paths propagate them; a shared helper can be considered separately. Keep attempt + 1: the documented one-based retry number and the targeted retry assertion already make the behavior explicit. Stopping subsequent handlers on failure is intentional and documented. Registration failure isolation is tracked separately in #7525 as already recorded above.

Marking ready to start hosted CI. Auto-merge must wait for the Claude review triggered by promotion to complete successfully; a later unit will read that review and arm it.

@thomhurst
thomhurst marked this pull request as ready for review November 3, 2026 20:21
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Review

The PR wires up the delegate subscriptions (OnTestRegistered, OnTestStart, OnTestEnd, OnTestSkipped, OnTestRetry) that were previously stored but never dispatched. I read the diff only and did not run the tests. I found no blocking issues.

What looks right

  • The fast-path checks now also consider delegate subscribers, so tests with no subscribers keep the allocation-free early return. The Early/Late stage handling is consistent: delegates run once, in the late stage, and the new unit test covers duplicate subscriptions.
  • Iterating InvocationList takes a snapshot, so a handler subscribing another handler mid-dispatch applies from the next dispatch. The tests cover that.
  • Registration deduplicates by invocation identity across the pre/post property-injection passes, and keeps duplicate subscriptions of the same delegate.
  • End-handler failures are collected into the teardown exceptions. Start, skipped and registered failures propagate, and the docs and XML remarks say so.
  • In RetryHelper, the retry handler runs after backoff and cancellation checks, and the attempt number is one-based. A throwing handler preserves an already-accepted cancellation through CompleteTestCancellation(). The re-check of IsTestCancellationRequested after the handlers is a good guard.

Suggestions, non-blocking

  1. OnTestStart, OnTestEnd and OnTestSkipped each repeat the "foreach over InvocationList → InvokeAsync(context, context)" loop, and the condition InternalEventsIfCreated?.X is null is repeated across the fast paths. A small helper such as InvokeDelegatesAsync(AsyncEvent<TestContext>?, TestContext), with a collecting variant for End, would keep the three paths from drifting apart.
  2. Retry handlers use ConfigureAwait(false), but the orchestrator loops do not. Pick one convention within this change.
  3. The registration dispatch is a local function that allocates the HashSet lazily. That is fine, but a short comment on why the second pass is gated on argumentsRegistered would help, since the skip path deliberately avoids the second pass.

Tests cover generated and reflection modes, retry, skip, class-level data and property injection. Nice work.

@thomhurst

Copy link
Copy Markdown
Owner Author

The earlier blocker is cleared: Claude run 41711875402 succeeded for 8ea91bd. I re-read the review bodies, inline comments, resolved threads and conversation comments. Claude and Codex both have positive verdicts on this head, and no reviewer has reopened the cancellation findings after their fixes.

Disposition of the promotion review's nonblocking suggestions: retain explicit dispatch loops to keep propagation and teardown collection visible, and retain the existing orchestrator await convention. The registration second-pass guard intentionally avoids dispatch after the skip path; existing property-injection and skip tests cover that behavior, so an extra comment is optional. Earlier dispositions still apply; no new blocking findings remain.

The final merge gate passed: CLEAN, all 39 checks green, no unresolved threads. Merging this reviewed head.

This was referenced Nov 18, 2026

This branch was previously deployed

1 inactive deployment
Pull Requests — 8ea91bd0 Deployed Nov 3, 2026 by thomhurst via modularpipeline (windows-latest) #21033
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Raise data-source delegate handlers at test lifecycle stages

1 participant