Skip to content

Also fail on Pester block/container failures (not just failed tests) - #605

Merged
Nikola Metulev (nmetulev) merged 1 commit into
microsoft:mainfrom
nohwnd:fix/pester5-count-blocks-containers
Jul 7, 2026
Merged

Nikola Metulev (nmetulev) merged 1 commit into
microsoft:mainfrom
nohwnd:fix/pester5-count-blocks-containers

Conversation

@nohwnd

@nohwnd Jakub Jareš (nohwnd) commented Jul 7, 2026 •

Copy link
Copy Markdown
Member

⚠️ This PR was generated by an AI agent (GitHub Copilot CLI). Please review accordingly.

I've been seeing this commonly across Pester 5 build scripts and I'm fixing it where it looks like it could hide real failures.

In Pester 5 a run reports three independent failure counts:

  • FailedCount — failed tests (It)
  • FailedBlocksCount — failed BeforeAll / AfterAll
  • FailedContainersCount — files that error during discovery / fail to load

Gating on only FailedCount means a BeforeAll that throws, or a test file that fails to load, leaves FailedCount = 0 and the build passes green despite real failures. Pester itself derives the overall pass/fail from the sum of all three.

This includes all three counts in the gate, and reports each count (failed tests, blocks, containers) in the messages instead of only the failed test count.

While here I also set Run.PassThru = $true on the Pester configuration — without it Invoke-Pester returns nothing, so $pesterResult was $null and the gate never actually fired.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the build pipeline’s NuGet Pester gating logic in scripts/build-cli.ps1 so the build treats Pester “block” and “container” failures as real failures (not just failed It tests), aligning the script’s gate with Pester 5’s overall pass/fail semantics.

Changes:

  • Expand the failure gate from FailedCount to FailedCount + FailedBlocksCount + FailedContainersCount for NuGet Pester tests.
Comments suppressed due to low confidence (1)

scripts/build-cli.ps1:412

  • The new failure gate counts block/container failures, but the error/warning messages still only report FailedCount. When FailedCount is 0 but FailedBlocksCount/FailedContainersCount is >0, this will print misleading output like "0 failed" even though the gate tripped.
                        if (($pesterResult.FailedCount + $pesterResult.FailedBlocksCount + $pesterResult.FailedContainersCount) -gt 0) {
                            if ($FailOnTestFailure) {
                                Write-Error "Stopping build due to NuGet Pester test failures (FailOnTestFailure flag set): $($pesterResult.FailedCount) failed"
                            } else {
                                Write-Warning "NuGet Pester tests had $($pesterResult.FailedCount) failure(s) — continuing"

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@nohwnd
Jakub Jareš (nohwnd) force-pushed the fix/pester5-count-blocks-containers branch from 8ea831b to f77b553 Compare July 7, 2026 12:36
@nmetulev
Nikola Metulev (nmetulev) merged commit 7a0d9c1 into microsoft:main Jul 7, 2026
22 checks passed
Zach Teutsch (zateutsch) pushed a commit that referenced this pull request Jul 8, 2026
…#606)

## Problem

#605 fixed the NuGet Pester test gate in `scripts/build-cli.ps1` — it
added `$pesterConfig.Run.PassThru = $true` (so `Invoke-Pester` returns a
result object) and changed the failure condition to count failed tests +
blocks + containers. That made the gate *reachable* for the first time.

But the gate was still incomplete: when a failure is detected and
`$FailOnTestFailure` is set (default `$true`), it called `Write-Error`
but never called `exit 1`. Under PowerShell's default
`$ErrorActionPreference = 'Continue'` — which is exactly what GitHub
Actions' default `pwsh` shell uses, and what
`.github/workflows/build-package.yml` (`.\scripts\build-cli.ps1
-SkipDocs`) relies on — `Write-Error` is **non-terminating**. So the
script printed "Stopping build…" and then kept running and exited `0`.
The build did **not** actually fail on NuGet Pester failures under
GitHub Actions. (It only failed in Azure DevOps
`.pipelines/templates/build.yaml`, where `PowerShell@2` forces
`errorActionPreference=stop`.)

## Fix

Add `exit 1` immediately after the `Write-Error` in the NuGet Pester
gate, making it consistent with the two sibling gates in the same file
that already do this:

- the npm unit-tests gate (~line 246–247)
- the CLI unit-tests gate (~line 298–299)

This is a single-line change.

## Verification

- `scripts/build-cli.ps1` parses cleanly
(`[System.Management.Automation.Language.Parser]::ParseFile`).
- `git diff` confirms the only change is the added `exit 1` line.

Co-authored-by: Nikola Metulev <711864+nmetulev@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants