Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions .github/workflows/size.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: size

# Stripped release sizes of the `usage` CLI and a mise-sized derived CLI, against the commit
# this pull request would merge onto. Fails when a gated binary grows by more than the
# threshold in `tasks/size.sh`, which says what is measured and why.
#
# A hosted runner, unlike perf-pr: size does not depend on the machine, so there is nothing
# for a dedicated one to hold still. The checkout is GitHub's merge of the pull request into
# its base, so the merge base is the base branch's tip and the difference is this change alone.
#
# Never add `continue-on-error`, raise the threshold, or drop a binary to get a pull request
# through — see "Performance Gate" in AGENTS.md.

on:
pull_request:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true

env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: "0"

jobs:
size:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
# `tasks/size.sh` builds the merge base from a worktree.
fetch-depth: 0
persist-credentials: false
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 # zizmor: ignore[cache-poisoning]
Comment thread
jdx marked this conversation as resolved.
with:
shared-key: size
# The report reaches the job summary whether or not the gate passed, and the exit
# status is raised afterwards, so a failure always arrives with the table explaining it.
- name: Compare binary sizes against the base
env:
# From the pull request payload: perf-pr found the generic `github.base_ref`
# naming main for a reopened stacked pull request.
BASE_REF: origin/${{ github.event.pull_request.base.ref }}
run: |
set +e
./tasks/size.sh "$BASE_REF" /tmp/size-report.md
status=$?
set -e
if [ -s /tmp/size-report.md ]; then
tee -a "$GITHUB_STEP_SUMMARY" </tmp/size-report.md
else
echo "The size comparison never produced a report." | tee -a "$GITHUB_STEP_SUMMARY"
fi
exit "$status"
7 changes: 6 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,12 @@ no raising `gate.pct` in `tak.toml`, no dropping a benchmark, and no
other check whose purpose is to tell a human something: don't neuter the check to
get a green square.

When a change genuinely costs instructions, measure it, say so in the pull request
The `size` workflow is the same kind of check for binary size: `tasks/size.sh` compares
stripped release builds of the `usage` CLI and a mise-sized derived CLI against the merge
base and fails above 1%. The same rules apply: never raise `GATE_PCT`, drop a binary, or
mark one ungated to get a pull request through.

When a change genuinely costs instructions or bytes, measure it, say so in the pull request
with the numbers, and **ask the user** how to proceed. Let them decide between
absorbing the cost, optimizing, or adjusting the gate themselves. This applies
generally: whenever the fix under consideration is more invasive than the problem —
Expand Down
6 changes: 6 additions & 0 deletions mise.toml
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,12 @@ run = [
dir = "{{config_root}}"
run = ["cargo build --release", "./tasks/perf-shadow.sh"]

# Stripped binary sizes against the merge base, gated at 1%: what the `size` workflow runs.
# Takes an optional base ref (default origin/main) and output path.
[tasks.size]
dir = "{{config_root}}"
run = "./tasks/size.sh"

# What binding a mise-sized command line costs a Go CLI. Reported, never gated — see the
# note at the top of the script, and `perf:shadow` for the Rust side's version of the same
# comparison.
Expand Down
112 changes: 112 additions & 0 deletions tasks/size.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
#!/usr/bin/env bash
# Binary size: this checkout against the commit it branched from, stripped.
#
# Size is a property of the build, not of the machine, so unlike instruction counts it needs
# no dedicated runner and no noise margin to be trusted: two builds of the same source with
# the same toolchain come out the same size. The 1% gate is for what an adopter pays, not
# for noise.
#
# Never loosen GATE_PCT, drop a binary, or mark one ungated to get a pull request through —
# see "Performance Gate" in AGENTS.md, which applies here the same way. A change that
# genuinely costs bytes says so with these numbers, and the maintainer decides.
#
# Usage: tasks/size.sh [BASE_REF] [OUT]
# BASE_REF what to branch-point against (default: origin/main)
# OUT where to write the markdown report (default: stdout)
# Exits 1 when a gated binary grew by more than GATE_PCT.
set -euo pipefail

GATE_PCT=1
base_ref=${1:-origin/main}
out=${2:-/dev/stdout}

base=$(git merge-base "$base_ref" HEAD)
tmp=$(mktemp -d)
trap 'git worktree remove --force "$tmp/src" 2>/dev/null || true; rm -rf "$tmp"' EXIT

# One target directory for both builds, so crates.io dependencies compile once. Workspace
# crates still build twice: cargo keys them by path, and the base lives in a worktree.
# Absolute, because the base builds from inside the worktree: a relative path would put its
# binaries under the worktree while `build` copies them from here.
CARGO_TARGET_DIR=${CARGO_TARGET_DIR:-target/size}
case $CARGO_TARGET_DIR in /*) ;; *) CARGO_TARGET_DIR=$PWD/$CARGO_TARGET_DIR ;; esac
export CARGO_TARGET_DIR

# `parse-usage` is mise's whole CLI through the derive, which is the size an adopter's binary
# pays for usage. `parse-none` is the same program without the parse, subtracted so the gate
# measures usage's share rather than std's. `parse-clap` is context and never gated.
BINS=(parse-none parse-usage parse-clap usage)

build() {
local src=$1 dest=$2
# Both checkouts link to the same paths. Removing them first means a build that failed to
# relink stops `cp` here, rather than handing one checkout's binary to the other.
for bin in "${BINS[@]}"; do rm -f "$CARGO_TARGET_DIR/release/$bin"; done
(cd "$src" && cargo build --release --locked -q -p gate -p usage-cli \
--bin parse-none --bin parse-usage --bin parse-clap --bin usage)
mkdir -p "$dest"
for bin in "${BINS[@]}"; do
cp "$CARGO_TARGET_DIR/release/$bin" "$dest/$bin"
strip "$dest/$bin"
done
}

bytes() { wc -c <"$1" | tr -d ' '; }

git worktree add -q --detach "$tmp/src" "$base"
build "$tmp/src" "$tmp/base"
build . "$tmp/head"
Comment thread
jdx marked this conversation as resolved.

# The shadow is a fixture that grows on purpose (see tak.toml): a pull request that
# regenerates it changes what `parse-usage` is, so its size is reported but not gated.
fixture_changed=false
if ! git diff --quiet "$base" -- benches/mise.usage.kdl benches/shadows/mise; then
fixture_changed=true
fi

failed=false
row() {
local label=$1 base_bytes=$2 head_bytes=$3 gated=$4
local delta=$((head_bytes - base_bytes)) pct verdict
pct=$(awk -v b="$base_bytes" -v d="$delta" 'BEGIN { printf "%+.2f", b ? 100 * d / b : 0 }')
verdict=reported
if [ "$gated" = true ]; then
verdict=ok
# Against the exact ratio, not the rounded `pct`: +1.004% prints as +1.00% and still fails.
if awk -v b="$base_bytes" -v d="$delta" -v g="$GATE_PCT" 'BEGIN { exit !(b > 0 && 100 * d > g * b) }'; then
verdict="**over ${GATE_PCT}%**"
failed=true
fi
fi
printf '| %s | %s | %s | %+'"'"'d | %s%% | %s |\n' \
"$label" "$(printf "%'d" "$base_bytes")" "$(printf "%'d" "$head_bytes")" "$delta" "$pct" "$verdict"
}

size_of() { bytes "$tmp/$1/$2"; }
share() { echo $(($(size_of "$1" parse-usage) - $(size_of "$1" parse-none))); }

shadow_gated=true
[ "$fixture_changed" = true ] && shadow_gated=false

report=$tmp/report.md
# The backticks below are markdown, not command substitution.
# shellcheck disable=SC2016
{
printf '### Binary size\n\n'
printf 'Stripped release builds, `%s` against the merge base `%s`. ' "$(git rev-parse --short HEAD)" "$(git rev-parse --short "$base")"
printf 'Gated rows fail above +%s%%.\n\n' "$GATE_PCT"
printf '| binary | base | head | change | %% | gate |\n'
printf '|---|---:|---:|---:|---:|---|\n'
row '`usage` CLI' "$(size_of base usage)" "$(size_of head usage)" true
row 'derived mise CLI, usage'"'"'s share' "$(share base)" "$(share head)" "$shadow_gated"
row 'derived mise CLI, whole binary' "$(size_of base parse-usage)" "$(size_of head parse-usage)" false
row 'clap mise CLI, whole binary' "$(size_of base parse-clap)" "$(size_of head parse-clap)" false
printf '\n'
printf '"usage'"'"'s share" is `parse-usage` minus `parse-none`: the same program with and without the parse.\n'
if [ "$fixture_changed" = true ]; then
printf '\nThis change regenerates the mise shadow, so the derived CLI is reported rather than gated: its size moved with the fixture.\n'
fi
} >"$report"

cat "$report" >"$out"
[ "$failed" = false ]
Loading