Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/tests/create-release-fixture.sh
Original file line number Diff line number Diff line change
Expand Up @@ -168,3 +168,4 @@ warning-unsupported fix: exercise unsupported diagnostic configuration 1.2.4
warning-tool-error fix: exercise diagnostic tool failure 1.2.4
CASES
echo "PASS: $cases actual release decisions; warning enabled=$warning_enabled; issue-hook suppression=$disabled; no credentials"
bash "$root/.github/tests/create-release-stable-channel-fixture.sh" "$workflow" "$config"
100 changes: 100 additions & 0 deletions .github/tests/create-release-stable-channel-fixture.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
#!/usr/bin/env bash
# Native stable/default-branch decisions; deliberately excludes GitHub publication.
set -euo pipefail
root="$(CDPATH='' cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
workflow="${1:-$root/.github/workflows/create-release.yaml}"
config="${2:-$root/.releaserc}"
work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
fail() { printf 'FAIL: %s\n' "$1" >&2; exit 1; }
# Require the shipped production stack, removing only its authenticated publisher.
jq -e '.branches == ["main"] and
([.plugins[] | if type == "array" then .[0] else . end] ==
["@semantic-release/commit-analyzer","@semantic-release/release-notes-generator","@semantic-release/github"])' \
"$config" >/dev/null || fail 'unsupported stable/default-branch release configuration'
jq '.plugins |= map(select((if type == "array" then .[0] else . end) != "@semantic-release/github"))' \
"$config" > "$work/config.json"
run="$(yq -r '.jobs.release.steps[] | select(.name == "🎉 Release") | .run' "$workflow")"
hooks="\${{ inputs.disable-issue-side-effects && '--success false --fail false' || '' }}"
dry="\${{ inputs.dry-run && '--dry-run' || '' }}"
[[ "$run" == "npx semantic-release@25.0.3 $hooks $dry" ]] || fail 'unsupported shipped release command'
run="${run//"$hooks"/}"
run="${run//"$dry"/--dry-run}"
read -r -a command_parts <<< "$run"
cache="$(npm config get cache)"
: > "$work/npm.conf"
: > "$work/npm-global.conf"
fixture_git() {
env -i PATH="$PATH" GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_COUNT=0 \
GIT_ALLOW_PROTOCOL=file git -c core.hooksPath=/dev/null -c init.templateDir= "$@"
}
decisions=0
run_decision() {
local expected=$1 before remote_before config_before observed_notes expected_notes observed_version
# Match the native reader's commit-note visibility, including annotated tags.
observed_notes="$(fixture_git -C "$repo" log --no-walk --format=%N '--notes=refs/notes/semantic-release*' refs/tags/v1.2.4)"
expected_notes=''
[[ "$notes" != matching ]] || expected_notes='{"channels":[null]}'
[[ "$observed_notes" == "$expected_notes" ]] || fail "$mode incorrect channel-note projection"
before="$(fixture_git -C "$repo" show-ref)"
remote_before="$(fixture_git --git-dir="$remote" show-ref)"
config_before="$(fixture_git -C "$repo" hash-object .releaserc)"
if ! (cd "$repo" && env -i PATH="$PATH" npm_config_cache="$cache" npm_config_offline=true \
npm_config_yes=true npm_config_userconfig="$work/npm.conf" npm_config_globalconfig="$work/npm-global.conf" \
GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_COUNT=0 GIT_ALLOW_PROTOCOL=file \
"${command_parts[@]}" --no-ci --repository-url "file://$remote") > "$work/output" 2>&1; then
cat "$work/output" >&2
fail "$mode release command failed"
fi
observed_version="$(sed -n 's/^.*The next release version is //p' "$work/output")"
if [[ "$expected" == none ]]; then
if [[ -n "$observed_version" ]] || ! grep -qF 'no new version is released' "$work/output"; then
fail "$mode expected no release"
fi
else
[[ "$observed_version" == "$expected" ]] || fail "$mode expected $expected"
fi
[[ "$(fixture_git -C "$repo" show-ref)" == "$before" ]] || fail "$mode changed local refs"
[[ "$(fixture_git --git-dir="$remote" show-ref)" == "$remote_before" ]] || fail "$mode changed remote refs"
[[ "$(fixture_git -C "$repo" rev-parse 'refs/tags/v1.2.4^{commit}')" == "$stranded" ]] || fail "$mode changed tag source"
[[ "$(fixture_git -C "$repo" hash-object .releaserc)" == "$config_before" &&
-z "$(fixture_git -C "$repo" status --porcelain --untracked-files=all)" ]] || fail "$mode changed consumer files"
decisions=$((decisions + 1))
printf 'PASS: %s -> %s; files and local/remote refs unchanged\n' "$mode" "$expected"
}
for tag_kind in lightweight annotated; do
for notes in absent matching; do
mode="$tag_kind-$notes"
repo="$work/$mode"
remote="$work/$mode.git"
fixture_git -c init.defaultBranch=main init --quiet --bare "$remote"
fixture_git -c init.defaultBranch=main init --quiet "$repo"
fixture_git -C "$repo" config user.name 'Release fixture'
fixture_git -C "$repo" config user.email 'fixture@example.invalid'
cp "$work/config.json" "$repo/.releaserc"
fixture_git -C "$repo" add .releaserc
fixture_git -C "$repo" -c commit.gpgsign=false commit --quiet -m 'chore: baseline'
fixture_git -C "$repo" -c tag.gpgsign=false tag v1.2.3
fixture_git -C "$repo" -c commit.gpgsign=false commit --quiet --allow-empty -m 'fix: stranded publication'
stranded="$(fixture_git -C "$repo" rev-parse HEAD)"
if [[ "$tag_kind" == annotated ]]; then
fixture_git -C "$repo" -c tag.gpgsign=false tag -a -m 'stranded stable release' v1.2.4
else
fixture_git -C "$repo" -c tag.gpgsign=false tag v1.2.4
fi
if [[ "$notes" == matching ]]; then
fixture_git -C "$repo" -c core.notesRef=refs/notes/semantic-release-v1.2.4 notes add -m '{"channels":[null]}' "$stranded"
fi
fixture_git -C "$repo" remote add origin "file://$remote"
fixture_git -C "$repo" push --quiet origin main --tags
if [[ "$notes" == matching ]]; then
fixture_git -C "$repo" push --quiet origin refs/notes/semantic-release-v1.2.4
fi
run_decision none
fixture_git -C "$repo" -c commit.gpgsign=false commit --quiet --allow-empty -m 'fix: subsequent change'
fixture_git -C "$repo" push --quiet origin main
run_decision 1.2.5
done
done
[[ "$decisions" == 8 ]] || fail 'incomplete stable-channel matrix'
echo 'PASS: eight native stable/default-branch decisions; GitHub publication, prereleases and multiple branches not exercised'
4 changes: 4 additions & 0 deletions .github/tests/test-create-release-fixture-controls.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ if ! env GIT_DIR="$work/foreign-git" GIT_WORK_TREE="$work/foreign-tree" GIT_INDE
exit 1
fi
[[ ! -e "$work/foreign-git" && ! -e "$work/foreign-tree" && ! -e "$work/foreign-index" ]]
grep -qxF 'PASS: eight native stable/default-branch decisions; GitHub publication, prereleases and multiple branches not exercised' "$work/output" || {
echo 'FAIL: the reusable offline entrypoint omitted stable-channel decisions' >&2
exit 1
}
echo 'PASS: ambient Git paths, templates and hooks cannot redirect fixture operations'
reject() {
local label="$1" workflow_path="$2" config_path="$3" diagnostic="$4"
Expand Down
10 changes: 8 additions & 2 deletions .github/tests/test-create-release-offline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ guard() {
any($c.jobs["ci-required-checks"].steps[]; (.env.JOB_RESULTS // "" |
contains("needs."+$name+".result")) and (.run // "" | contains("$JOB_RESULTS"))))) | not
then error("secret-free release calls must gate required CI")
elif (["test-create-release-offline.sh","test-create-release-fixture-controls.sh"] | all(. as $script |
elif (["test-create-release-offline.sh","test-create-release-fixture-controls.sh","test-create-release-stable-channels.sh"] | all(. as $script |
[$c.jobs["test-create-release-config"].steps[] | select(.run == "bash .github/tests/"+$script)] |
length == 1 and all(runnable))) | not
then error("offline boundary regressions must execute in CI")
Expand Down Expand Up @@ -126,6 +126,7 @@ lost result .ci.jobs["ci-required-checks"].steps |= map(if .env.JOB_RESULTS then
boundary skipped .ci.jobs["test-create-release-config"].steps |= map(if .run == "bash .github/tests/test-create-release-offline.sh" then .if="false" else . end) regressions must execute
invalid offline mode .workflow.jobs["offline-test"].steps |= map(select(.name != "🔎 Validate offline mode")) require dry-run
behavior controls skipped .ci.jobs["test-create-release-config"].steps |= map(select(.run != "bash .github/tests/test-create-release-fixture-controls.sh")) regressions must execute
channel controls skipped .ci.jobs["test-create-release-config"].steps |= map(select(.run != "bash .github/tests/test-create-release-stable-channels.sh")) regressions must execute
CASES
echo "PASS: release offline boundary rejects $count independent regressions"

Expand All @@ -136,6 +137,7 @@ mkdir "$work/bin" "$work/source"
mkdir -p "$work/source/catalogue/.github/workflows" "$work/source/catalogue/.github/tests"
cp "$root/.github/workflows/create-release.yaml" "$work/source/catalogue/.github/workflows/"
cp "$root/.github/tests/create-release-fixture.sh" "$work/source/catalogue/.github/tests/"
cp "$root/.github/tests/create-release-stable-channel-fixture.sh" "$work/source/catalogue/.github/tests/"
cp "$root/.releaserc" "$work/source/catalogue/"
tar -czf "$work/valid.tar.gz" -C "$work/source" catalogue
cat >"$work/bin/curl" <<'CURL'
Expand Down Expand Up @@ -184,7 +186,7 @@ for invalid in repository short-sha mutable-ref path-sha uppercase-sha; do
done
echo 'PASS: five invalid source inputs fail before download or extraction'

for transport in failure truncated missing-fixture valid; do
for transport in failure truncated missing-fixture missing-channel-fixture valid; do
rm -rf "$work/download/.devantler-tech-actions"
rm -f "$work/catalogue.tar.gz" "$work/request"
printf '%s\n' "$transport" >"$work/mode"
Expand All @@ -194,6 +196,9 @@ for transport in failure truncated missing-fixture valid; do
missing-fixture)
tar -czf "$work/payload" -C "$work/source" catalogue/.releaserc catalogue/.github/workflows
;;
missing-channel-fixture)
tar -czf "$work/payload" -C "$work/source" catalogue/.releaserc catalogue/.github/workflows catalogue/.github/tests/create-release-fixture.sh
;;
valid) cp "$work/valid.tar.gz" "$work/payload" ;;
esac
result=0
Expand All @@ -210,6 +215,7 @@ for transport in failure truncated missing-fixture valid; do
[[ "$result" == 0 ]]
cmp "$root/.releaserc" "$work/download/.devantler-tech-actions/.releaserc"
cmp "$root/.github/tests/create-release-fixture.sh" "$work/download/.devantler-tech-actions/.github/tests/create-release-fixture.sh"
cmp "$root/.github/tests/create-release-stable-channel-fixture.sh" "$work/download/.devantler-tech-actions/.github/tests/create-release-stable-channel-fixture.sh"
else
[[ "$result" != 0 ]] || {
echo "FAIL: $transport source retrieval was accepted" >&2
Expand Down
91 changes: 91 additions & 0 deletions .github/tests/test-create-release-stable-channels.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
#!/usr/bin/env bash
# Exercise native decisions, then prove their observation oracles reject regressions.
set -euo pipefail
root="$(git rev-parse --show-toplevel)"
fixture="$root/.github/tests/create-release-stable-channel-fixture.sh"
work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
[[ -f "$fixture" ]] || { echo 'FAIL: maintained stable-channel fixture is missing' >&2; exit 1; }
real_npx="$(command -v npx)"
mkdir -p "$work/bin" "$work/template/hooks"
printf '#!/bin/sh\nexit 99\n' > "$work/template/hooks/pre-commit"
chmod +x "$work/template/hooks/pre-commit"
if ! env GIT_DIR="$work/foreign-git" GIT_WORK_TREE="$work/foreign-tree" GIT_INDEX_FILE="$work/foreign-index" \
GIT_TEMPLATE_DIR="$work/template" GIT_CONFIG_PARAMETERS="'core.hooksPath'='$work/template/hooks'" \
bash "$fixture" > "$work/output" 2>&1; then
cat "$work/output" >&2
exit 1
fi
[[ ! -e "$work/foreign-git" && ! -e "$work/foreign-tree" && ! -e "$work/foreign-index" ]]
for kind in lightweight annotated; do
for notes in absent matching; do
for version in none 1.2.5; do
grep -qxF "PASS: $kind-$notes -> $version; files and local/remote refs unchanged" "$work/output"
done
done
done
cat "$work/output"
printf '%s\n' "$real_npx" > "$work/real-npx"
cat > "$work/bin/npx" <<'WRAPPER'
#!/usr/bin/env bash
set -euo pipefail
work="$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd -P)"
"$(cat "$work/real-npx")" "$@" > "$work/native-output" 2>&1
# Each mutation follows a real pinned dry-run, inside its disposable local repository.
case "$(cat "$work/fault")" in
initial-decision) sed 's/no new version is released/fixture removed the no-release observation/g' "$work/native-output"; exit 0 ;;
next-decision) sed 's/The next release version is 1.2.5/The next release version is 1.2.50/g' "$work/native-output"; exit 0 ;;
prerelease-decision) sed 's/The next release version is 1.2.5/The next release version is 1.2.5-beta.1/g' "$work/native-output"; exit 0 ;;
local-note) git -c user.name=Fixture -c user.email=fixture@example.invalid -c commit.gpgsign=false notes --ref=fixture-unexpected add -m '{}' HEAD ;;
remote-note)
previous=''
for argument in "$@"; do
if [[ "$previous" == --repository-url ]]; then remote="${argument#file://}"; fi
previous="$argument"
done
git --git-dir="$remote" -c user.name=Fixture -c user.email=fixture@example.invalid -c commit.gpgsign=false notes --ref=fixture-unexpected add -m '{}' v1.2.4
;;
tag-move) git update-ref refs/tags/v1.2.4 HEAD~1 ;;
tracked-file) printf '\nchanged\n' >> .releaserc ;;
untracked-file) printf 'changed\n' > unexpected-file ;;
*) exit 99 ;;
esac
cat "$work/native-output"
WRAPPER
chmod +x "$work/bin/npx"
controls=0
while IFS=$'\t' read -r fault diagnostic; do
printf '%s\n' "$fault" > "$work/fault"
if PATH="$work/bin:$PATH" bash "$fixture" > "$work/output" 2>&1; then
echo "FAIL: $fault was accepted" >&2
exit 1
fi
grep -qF "$diagnostic" "$work/output" || { cat "$work/output" >&2; exit 1; }
controls=$((controls + 1))
printf 'PASS: rejected %s after native release decision\n' "$fault"
done <<'CASES'
initial-decision expected no release
next-decision expected 1.2.5
prerelease-decision expected 1.2.5
local-note changed local refs
remote-note changed remote refs
tag-move changed local refs
tracked-file changed consumer files
untracked-file changed consumer files
CASES
[[ "$controls" == 8 ]]
mkdir -p "$work/mutated/.github/tests"
# Reproduce the annotated-tag setup error: the tag object has a note, but the
# commit projection used by semantic-release cannot see it.
original="$(cat "$fixture")"
needle="notes add -m '{\"channels\":[null]}' \"\$stranded\""
replacement="notes add -m '{\"channels\":[null]}' v1.2.4"
[[ "$original" == *"$needle"* ]] || { echo 'FAIL: note-target mutation did not apply' >&2; exit 1; }
printf '%s\n' "${original/"$needle"/"$replacement"}" > "$work/mutated/.github/tests/fixture.sh"
if bash "$work/mutated/.github/tests/fixture.sh" "$root/.github/workflows/create-release.yaml" "$root/.releaserc" > "$work/output" 2>&1; then
echo 'FAIL: annotated tag-object note was accepted as a matching commit note' >&2
exit 1
fi
grep -qF 'FAIL: annotated-matching incorrect channel-note projection' "$work/output" || { cat "$work/output" >&2; exit 1; }
echo 'PASS: native projection rejects an annotated tag-object note'
echo 'PASS: stable-channel observation controls reject incorrect decisions, tag/notes changes and consumer-file mutations'
3 changes: 3 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4228,6 +4228,9 @@ jobs:
- name: 🧪 Verify release-decision regressions fail
run: bash .github/tests/test-create-release-fixture-controls.sh

- name: 🧪 Verify stable channel decisions preserve existing tags and notes
run: bash .github/tests/test-create-release-stable-channels.sh

test-release-please-semantics:
needs: [select-ci-tests]
if: "${{ github.event_name != 'merge_group' && !startsWith(github.event.head_commit.message, 'chore(main): release ') && contains(fromJSON(needs.select-ci-tests.outputs.selected), 'test-release-please-semantics') }}"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/create-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,7 @@ jobs:
mkdir .devantler-tech-actions
tar --extract --gzip --file="$RUNNER_TEMP/catalogue.tar.gz" \
--strip-components=1 --directory=.devantler-tech-actions
for source_file in .releaserc .github/workflows/create-release.yaml .github/tests/create-release-fixture.sh; do
for source_file in .releaserc .github/workflows/create-release.yaml .github/tests/create-release-fixture.sh .github/tests/create-release-stable-channel-fixture.sh; do
[[ -f ".devantler-tech-actions/$source_file" && ! -L ".devantler-tech-actions/$source_file" ]] || {
echo '::error::incomplete catalogue source' >&2
exit 1
Expand Down
8 changes: 8 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,14 @@ executes real HTTP controls and the immutable-source and read-only credential bo
inside `test-create-release-config` in required CI. Never treat its diagnosis as completed recovery
or write authorization; #587 owns publication, legacy compatibility and protected consumer proof.

The credential-free release fixture also exercises stranded stable tags with absent or matching
default-channel notes, for both lightweight and annotated tags. It verifies no release at the tagged
commit, the next patch after a subsequent fix, and unchanged consumer files and local/remote refs.
Run `bash .github/tests/test-create-release-stable-channels.sh` for native decisions and negative
observation controls. The existing offline reusable callers run the fixture; required CI runs the
controls. This covers stable/default-branch decisions only, without the authenticated GitHub
publisher; publication, prereleases and multiple release branches need separate evidence.

```text
actions/<action-name>/ # One directory per composite action
├── action.yaml # Action metadata (name, description, inputs, steps)
Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,12 @@ Consumer rollout and the decision on removing this temporary flag are tracked in

For catalogue self-tests, set `offline-test: true` together with `dry-run: true` and omit the App secret. This independent job requests no repository permissions and retrieves its immutable public catalogue source anonymously. It evaluates the pinned release command and catalogue configuration against disposable local repositories, verifying patch, minor, major, and no-release decisions without changing files or refs. It also honors `warn-missing-breaking-bang`: a missing-pattern fixture emits exactly one warning when enabled, while intact, disabled, unsupported-config and tool-error cases continue without warning or release blockage. Both issue-hook settings are exercised; this does not test live notification delivery. Normal releases and consumer dry-run previews continue to use the consumer's own configuration and App credentials, including callers that declare `permissions: {}`.

These offline tests also cover existing stable tags with absent or matching default-channel notes,
using both lightweight and annotated tags. No new release is selected at the tagged commit; a
subsequent fix selects the next patch while preserving all files, tags and notes. This supports
leaving channel notes unchanged for stable/default-branch recovery. It does not prove GitHub
publication, prerelease channels or multiple release branches.

#### Usage

```yaml
Expand Down
Loading