Skip to content

Security (composer) goes red on a transient packagist 502 — the outage tolerance only matches npm's wording #323

Open

Description

@rubenvdlinde

Summary

quality / Security (composer) fails the whole job when packagist.org's advisory endpoint is briefly unavailable, because the outage-tolerance branch only recognises npm's wording. A transient 502 is reported identically to a real vulnerability.

Evidence

scholiq development, push run 31337136985, job quality / Security (composer):

composer audit --format=plain --abandoned=report
##[error]The "https://packagist.org/api/security-advisories/" file could not be downloaded (HTTP/2 502 )
##[error]Process completed with exit code 100.

No advisory was reported. The job is red purely because packagist returned a 502. The previous push run on the same branch had this job green, and nothing in composer.lock changed between them.

Root cause

The workflow already anticipates this failure mode — for npm only. The tolerance test is:

if [ "$rc" -ne 0 ] && printf '%s' "$out" | grep -qiE \
  "audit endpoint returned an error|Invalid package tree|ECONNRESET|ETIMEDOUT|ENOTFOUND|being retired|Service Unavailable"; then

Composer's outage does not use any of those phrasings. It says:

The "https://packagist.org/api/security-advisories/" file could not be downloaded (HTTP/2 502 )

so the guard does not match, and exit 100 propagates.

Why it matters beyond one red job

Security (composer) failing is exactly the signal that should stop a merge. Once it can go red for reasons unrelated to the dependency tree, the honest reading of a red security job becomes "probably packagist again" — and the next real advisory lands in a check people have learned to skip. The cost is not the false red; it is the true red that gets waved through after it.

It also breaks the merge rule the fleet uses: comparing a PR's failing set against development's is only meaningful if a job's colour is a property of the code. A job that flips on upstream availability makes the baseline non-deterministic — a PR opened during an outage inherits a failure its diff did not cause.

Proposed fix

Extend the existing tolerance to composer's phrasing, keeping it narrow enough that a genuine advisory still fails:

if [ "$rc" -ne 0 ] && printf '%s' "$out" | grep -qiE \
  "audit endpoint returned an error|Invalid package tree|ECONNRESET|ETIMEDOUT|ENOTFOUND|being retired|Service Unavailable|could not be downloaded \(HTTP/[0-9.]+ (429|5[0-9][0-9])|packagist\.org.*could not be downloaded"; then

Restricting the HTTP branch to 429 and 5xx matters: a 404 on that endpoint would mean the advisory API moved, which should stay red rather than be tolerated.

Two things worth doing alongside:

  1. Emit a ::warning:: when the tolerance fires, naming the endpoint and status, so a tolerated outage is visible in the run rather than silently green. A tolerance nobody can see is indistinguishable from a check that stopped running.
  2. Retry once before tolerating. A 502 from packagist is usually seconds long; one retry converts most of these into a real verdict instead of a skipped one. Tolerating on the first failure gives up the measurement more often than necessary.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions