Skip to content

Test coverage: telemetry, tools & prompt mop-up to >=95% - #650

Merged
Alexandre Zollinger Chohfi (azchohfi) merged 3 commits into
mainfrom
azchohfi-tests-uia-mopup
Jul 16, 2026
Merged

Alexandre Zollinger Chohfi (azchohfi) merged 3 commits into
mainfrom
azchohfi-tests-uia-mopup

Conversation

@azchohfi

Copy link
Copy Markdown
Collaborator

What

Raises per-file unit-test coverage to ≥95% for the remaining "mop-up" telemetry / tools / prompt files, part of the CLI coverage initiative (#630):

File Coverage
ConsoleTasks/PromptConfirmationTask.cs 100%
Telemetry/Telemetry.cs 96.6%
Telemetry/Events/CommandInvokedEvent.cs 97.9%
Telemetry/Events/CommandCompletedEvent.cs 100%
Telemetry/Events/EventBase.cs 100%
Telemetry/TelemetryEventSource.cs 100%
Tools/MakeAppxTool.cs 100%
Tools/MakePriTool.cs 100%

How

  • Adds focused unit tests for PromptConfirmationTask, Telemetry (event emission + sensitive-string scrubbing), the telemetry event types, and the MakeAppxTool / MakePriTool SDK-tool wrappers.
  • Two minimal, behavior-preserving product test-seams:
    • CommandInvokedEvent: extracted internal static string CreateContext(IEnumerable<SymbolResult>) and made GetValue static so the context serialization (and its error fallback) can be unit-tested directly.
    • PromptConfirmationTask: the input-read catch (InvalidOperationException) (which maps a console EOF/no-input to a cancelled prompt) is scoped to just the ReadKeyAsync call, so unrelated failures elsewhere in the loop still surface instead of being silently swallowed.

Policy

No [ExcludeFromCodeCoverage], no coverage.runsettings edits. Only innermost native/environment boundaries remain uncovered.

Review

Reviewed with the repo pr-review skill (7 specialists + GPT multi-model cross-check) before opening. Three findings were fixed on this branch:

  • Narrowed the over-broad InvalidOperationException catch (correctness).
  • Rewrote a vacuous AddWellKnownSensitiveStrings test to assert the machine name is actually scrubbed from telemetry payloads.
  • Replaced a misleading "cancel while waiting" test (which passed via the empty-console EOF path) with an honest EOF test and a genuine cancellation test that blocks in ReadKeyAsync until the token fires — which pushed PromptConfirmationTask from 96.4% to 100%.

Validation

  • Debug (product + tests) 0W/0E; Release both projects individually (-p:UseSharedCompilation=false -nodeReuse:false) 0W/0E each.
  • Affected test classes: 35 passed / 0 failed.

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

Add focused tests for prompt confirmation, telemetry events, telemetry event source, and makeappx/makepri error handling.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… non-vacuous

Addresses three pr-review findings (correctness + test-coverage, confirmed by the
GPT multi-model cross-check) on the mop-up coverage work:

- [correctness/medium] PromptConfirmationTask.WaitForInputAsync wrapped the whole
  input loop in `catch (InvalidOperationException)`, so an InvalidOperationException
  from state updates / _onUpdate callbacks would be silently swallowed as a cancelled
  prompt. Narrow the catch to just the ReadKeyAsync call (break -> Cancelled), so only
  the reader's EOF/no-input is treated as cancellation and other failures still surface.

- [test-coverage/high] TelemetryTests.AddWellKnownSensitiveStrings test was vacuous
  (asserted only IsTelemetryOn). Rewrite it to log Environment.MachineName through a
  ProbeEvent and assert the raw machine name is scrubbed and replaced with a <Token>,
  so it fails if the method is a no-op.

- [test-coverage/high] PromptConfirmationTaskTests "cancellation while waiting" fed an
  empty TestConsole, which throws InvalidOperationException immediately, so it passed
  via the EOF path and never exercised cancellation. Split into an honest EOF test
  (NoInputAvailable_TreatedAsCancelled) and a genuine cancellation test that blocks in
  ReadKeyAsync (new BlockingInputConsole) until the token fires, exercising the real
  OperationCanceledException path.

PromptConfirmationTask.cs coverage 96.4% -> 100%; Telemetry.cs 96.6%,
CommandInvokedEvent.cs 97.9%, tools/events 100% (all scope files >=95%).
Debug + Release (both projects) 0W/0E. 35 affected tests pass.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 16, 2026 05:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds focused tests to raise coverage for telemetry, confirmation prompts, and SDK tool wrappers.

Changes:

  • Adds telemetry and event serialization tests.
  • Adds comprehensive confirmation prompt tests and narrows EOF handling.
  • Adds MakeAppx and MakePri error-output tests.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
CommandInvokedEvent.cs Extracts testable context serialization.
PromptConfirmationTask.cs Narrows invalid-operation handling to input reads.
TelemetryTests.cs Tests telemetry emission and sanitization.
TelemetryEventsTests.cs Tests telemetry event models and ETW contracts.
PromptConfirmationTaskTests.cs Covers prompt input and cancellation paths.
MakePriToolTests.cs Tests MakePri error reporting.
MakeAppxToolTests.cs Tests MakeAppx error parsing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +144 to +145
Assert.IsFalse(detail!.Contains(Environment.MachineName, StringComparison.Ordinal),
"AddWellKnownSensitiveStrings must scrub the raw machine name from telemetry payloads; a no-op would leak it.");

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in d7bfed6. You are right that AddSensitiveString skips names of length <= 3, so a short computer name would make the machine-name assertion fail. The test now probes with the user-profile path (Environment.GetFolderPath(SpecialFolder.UserProfile)), which is always a real absolute path well over 3 chars, so it is registered regardless of host. It asserts the raw profile path is scrubbed (fails if the method is a no-op) and replaced with the <UserDirectory> placeholder. Host-independent now, same product lines covered.

Copilot auto-review on PR #650 noted the AddWellKnownSensitiveStrings test probed with
Environment.MachineName, but Telemetry.AddSensitiveString ignores names of length <= 3
(Telemetry.cs guards name.Length > 3). On a host whose computer name is 1-3 chars the
machine name would never be registered, so the "must be scrubbed" assertion would fail.

Probe with the user-profile path instead: it is always a real absolute path well over the
3-char minimum, so the value is always registered and the test is host-independent. Assert
the raw profile path is gone (fails if the method is a no-op) and that it was replaced with
the <UserDirectory> placeholder. Same product lines exercised; coverage unchanged.

Debug + Release (tests project) 0W/0E; TelemetryTests 5/5 pass.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@azchohfi
Alexandre Zollinger Chohfi (azchohfi) merged commit e5b0c7f into main Jul 16, 2026
20 checks passed
@azchohfi
Alexandre Zollinger Chohfi (azchohfi) deleted the azchohfi-tests-uia-mopup branch July 16, 2026 05:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants