Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -690,6 +690,11 @@ jobs:
# covered by 'k8s/**'.
- 'scripts/guard-baseline-context-optin-scan-scope.sh'
- 'scripts/tests/test-guard-baseline-context-optin-scan-scope.sh'
# Both halves and the reviewed list, for the same reasons. The guard also
# reads the kubescape HelmRelease, covered by 'k8s/**'.
- 'scripts/guard-kubescape-scan-scope.sh'
- 'scripts/tests/test-guard-kubescape-scan-scope.sh'
- 'scripts/kubescape-unscanned-namespaces.tsv'
# Both halves, for the same reason as the guards above. The guard also
# reads the exception manifest itself, covered by 'k8s/**'.
- 'scripts/guard-pod-security-exception-documented.sh'
Expand Down Expand Up @@ -1845,6 +1850,18 @@ jobs:
bash scripts/guard-baseline-context-optin-scan-scope.sh
bash scripts/tests/test-guard-baseline-context-optin-scan-scope.sh

- name: 🗺️ Validate every unscanned namespace is reviewed
if: needs.changes.outputs.k8s == 'true'
# A namespace in the operator's excludeNamespaces produces no scan result, and the
# posture score cannot tell that apart from a clean namespace. So the exclusions must
# match scripts/kubescape-unscanned-namespaces.tsv exactly, each row with a reason,
# and a namespace can only leave scan coverage through a reviewed change (#3215).
# The base HelmRelease is the only place the field may be set: an overlay patch or a
# valuesFrom that sets it would narrow the deployed scope while this read the base.
run: |
bash scripts/guard-kubescape-scan-scope.sh
bash scripts/tests/test-guard-kubescape-scan-scope.sh

- name: 🔎 Validate every suppressed control is documented as failing open
if: needs.changes.outputs.k8s == 'true'
# A control added under spec.posture that the file's fail-open warning does
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ spec:
# them from the score alone. The same trap is documented for the `kubescape`
# namespace in cluster-security-exceptions/pod-security-mutations-unscoped.yaml.
#
# Each namespace here has a row saying why in scripts/kubescape-unscanned-namespaces.tsv,
# and CI fails when the two lists differ, so change both together (#3215).
#
# It was undeclared until #3923, so the scope could only be learned by reading
# a live ConfigMap. Two issues were planned against `kube-system` on the
# assumption it was scanned (#3239, #3919); it is not, and the exception #3919
Expand Down
210 changes: 210 additions & 0 deletions scripts/guard-kubescape-scan-scope.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
#!/usr/bin/env bash
#
# Fail when the namespaces Kubescape does not scan differ from the reviewed list in
# scripts/kubescape-unscanned-namespaces.tsv (#3215).
#
# A namespace in the operator's `excludeNamespaces` produces no scan result at all, and the
# posture score cannot tell that apart from a namespace with nothing wrong. So adding one
# silently narrows every posture claim the platform makes. This guard makes that change
# visible: the exclusion list and the reviewed list must match exactly, and every reviewed
# row must say why the namespace is not scanned.
#
# Both directions fail. A namespace excluded without a row is the silent narrowing this guard
# exists to catch. A row whose namespace is no longer excluded is a stale claim that a scanned
# namespace is unscanned, which would mislead the next reader in the opposite direction.
#
# FAIL CLOSED. Both inputs must be non-empty before they are compared: a moved yq path or an
# emptied list would otherwise compare two empty sets and pass forever.
#
# Exit codes:
# 0 the exclusion list and the reviewed list match, and every row has a reason
# 1 they differ, or a row has no reason; each problem is named
# 2 cannot check: bad usage, a missing or unparseable file, or an empty input

set -uo pipefail

die() {
printf 'guard-kubescape-scan-scope: %s\n' "$*" >&2
exit 2
}

[ "$#" -le 3 ] || die "usage: $0 [<kubescape-helm-release> <reviewed-list> [<k8s-root>]]"
command -v yq >/dev/null 2>&1 || die "yq is required but not installed"

repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" ||
die "could not resolve the repository root"

helm_release="${1:-$repo_root/k8s/bases/infrastructure/controllers/kubescape/helm-release.yaml}"
reviewed="${2:-$repo_root/scripts/kubescape-unscanned-namespaces.tsv}"
k8s_root="${3:-$repo_root/k8s}"

[ -f "$helm_release" ] || die "kubescape HelmRelease '$helm_release' not found"
[ -f "$reviewed" ] || die "reviewed list '$reviewed' not found"
[ -d "$k8s_root" ] || die "k8s tree '$k8s_root' not found"

# --- input 1: what the operator is told not to scan -------------------------------------
excluded_raw="$(yq -r '.spec.values.excludeNamespaces' "$helm_release" 2>/dev/null)" ||
die "could not read .spec.values.excludeNamespaces from '$helm_release' — unparseable YAML"
Comment thread
devantler marked this conversation as resolved.
Resolved
[ -n "$excluded_raw" ] && [ "$excluded_raw" != "null" ] ||
die "excludeNamespaces came back EMPTY in '$helm_release'; refusing to compare against an empty set"

excluded="$(printf '%s\n' "$excluded_raw" | tr ',' '\n' |
sed 's/^[[:space:]]*//; s/[[:space:]]*$//' | sed '/^$/d' | sort -u)"
[ -n "$excluded" ] ||
die "excludeNamespaces in '$helm_release' held no namespace names after splitting on commas"

# --- nothing else may set the scan scope --------------------------------------------------
# The comparison above reads the base HelmRelease, so it is only the deployed scope while
# nothing overrides that field. Provider overlays already patch this HelmRelease, and a
# patch or valuesFrom that sets excludeNamespaces would narrow the deployed scope while this
# guard kept reading the unchanged base. So the base is the one place the field may be set:
# any other manifest that sets it, and any valuesFrom on the kubescape HelmRelease, fails.
base_rel="bases/infrastructure/controllers/kubescape/helm-release.yaml"
# Every branch ends in `filename + ...`, never a bare string literal: yq evaluates a literal
# after `select` even when the select matched nothing, so `select(x) | "text"` always prints.
overrides_expr='
Comment thread
devantler marked this conversation as resolved.
Resolved
(select(tag == "!!map" and .kind == "HelmRelease" and .metadata.name == "kubescape")
| select(.spec.values.excludeNamespaces != null)
| filename + ": the kubescape HelmRelease sets spec.values.excludeNamespaces"),
(select(tag == "!!map" and .kind == "HelmRelease" and .metadata.name == "kubescape")
| select(.spec.valuesFrom != null)
| filename + ": the kubescape HelmRelease sets spec.valuesFrom"),
(select(tag == "!!seq") | .[]
| select(((.path // "") | tostring) | test("excludeNamespaces|valuesFrom"))
| filename + ": a JSON patch op targets " + .path),
(select(tag == "!!map" and .patches != null) | .patches[]
| select(((.patch // "") | tostring) | test("excludeNamespaces"))
| filename + ": an inline patch sets excludeNamespaces"),
(select(tag == "!!map" and .kind == "Kustomization" and .spec.patches != null) | .spec.patches[]
| select(((.patch // "") | tostring) | test("excludeNamespaces|valuesFrom"))
| filename + ": a Flux Kustomization patch touches the scan scope")'

valuesfrom_base="$(yq -r '.spec.valuesFrom // ""' "$helm_release" 2>/dev/null)" ||
die "could not read .spec.valuesFrom from '$helm_release'"
if [ -n "$valuesfrom_base" ]; then
printf 'guard-kubescape-scan-scope: %s uses spec.valuesFrom, which can override excludeNamespaces unseen; set values inline\n' \
"$helm_release" >&2
overrides_found=1
else
overrides_found=0
fi

manifests="$(cd "$k8s_root" && find . -type f \( -name '*.yaml' -o -name '*.yml' \) ! -path "./$base_rel" | sort)" ||
die "could not list manifests under '$k8s_root'"
[ -n "$manifests" ] || die "no manifests found under '$k8s_root'; refusing to report no overrides"

overrides="$(cd "$k8s_root" && printf '%s\n' "$manifests" | tr '\n' '\0' |
xargs -0 yq e "$overrides_expr" 2>/dev/null)" ||
die "could not parse every manifest under '$k8s_root' while looking for scan-scope overrides"
overrides="$(printf '%s\n' "$overrides" | sed '/^---$/d; /^$/d')"
if [ -n "$overrides" ]; then
while IFS= read -r hit; do
printf 'guard-kubescape-scan-scope: %s; set the scan scope only in %s\n' "${hit#./}" "$base_rel" >&2
done <<EOF
$overrides
EOF
overrides_found=1
fi

# --- the rendered provider releases must carry exactly the base scope ---------------------
# The checks above name the override spellings this repository uses, but Kustomize has more
# (replacements, components, generators), so the deployed scope is also checked directly:
# every provider layer is rendered and any kubescape HelmRelease in it must set the same
# excludeNamespaces as the base, with no valuesFrom. A provider that does not deploy
# Kubescape renders no release and is not compared; at least one provider must render one,
# or a moved layout would compare nothing and pass.
command -v kubectl >/dev/null 2>&1 || die "kubectl is required to render the provider layers"
rendered_releases=0
for layer in "$k8s_root"/providers/*/infrastructure/controllers; do
[ -f "$layer/kustomization.yaml" ] || continue
rendered="$(kubectl kustomize "$layer" 2>/dev/null)" ||
die "could not render '$layer'"
release="$(printf '%s\n' "$rendered" |
yq -o=json -I=0 'select(.kind == "HelmRelease" and .metadata.name == "kubescape") |
{"scope": (.spec.values.excludeNamespaces // ""), "valuesFrom": (.spec.valuesFrom != null)}' 2>/dev/null)" ||
die "could not read the kubescape HelmRelease rendered from '$layer'"
[ -n "$release" ] || continue
[ "$(printf '%s\n' "$release" | wc -l | tr -d ' ')" -eq 1 ] ||
die "'$layer' renders more than one kubescape HelmRelease"
rendered_releases=$((rendered_releases + 1))
rendered_scope="$(printf '%s' "$release" | yq -r '.scope')"
if [ "$rendered_scope" != "$excluded_raw" ]; then
printf 'guard-kubescape-scan-scope: %s renders excludeNamespaces "%s", not the base "%s"; set the scan scope only in %s\n' \
"${layer#"$k8s_root"/}" "$rendered_scope" "$excluded_raw" "$base_rel" >&2
overrides_found=1
fi
if [ "$(printf '%s' "$release" | yq -r '.valuesFrom')" = "true" ]; then
printf 'guard-kubescape-scan-scope: %s renders a kubescape HelmRelease with valuesFrom; set values inline\n' \
"${layer#"$k8s_root"/}" >&2
overrides_found=1
fi
done
[ "$rendered_releases" -gt 0 ] ||
die "no provider layer under '$k8s_root/providers' renders the kubescape HelmRelease; refusing to report no overrides"

# --- input 2: the reviewed rows ---------------------------------------------------------
failures=$overrides_found
listed=""
while IFS= read -r line || [ -n "$line" ]; do
case "$line" in '' | '#'*) continue ;; esac
ns="${line%%$'\t'*}"
if [ "$ns" = "$line" ]; then
reason=""
else
reason="${line#*$'\t'}"
fi
reason="$(printf '%s' "$reason" | sed 's/^[[:space:]]*//; s/[[:space:]]*$//')"
if [ -z "$ns" ]; then
printf 'guard-kubescape-scan-scope: a row in %s has an empty namespace\n' "$reviewed" >&2
failures=$((failures + 1))
continue
fi
if [ -z "$reason" ]; then
printf 'guard-kubescape-scan-scope: %s is listed without a reason; say why it is not scanned\n' "$ns" >&2
failures=$((failures + 1))
fi
listed="${listed}${ns}"$'\n'
done <"$reviewed"

listed_sorted="$(printf '%s' "$listed" | sed '/^$/d' | sort)"
[ -n "$listed_sorted" ] ||
die "no rows in '$reviewed'; refusing to compare against an empty reviewed list"

duplicates="$(printf '%s\n' "$listed_sorted" | uniq -d)"
if [ -n "$duplicates" ]; then
while IFS= read -r ns; do
printf 'guard-kubescape-scan-scope: %s is listed more than once in %s\n' "$ns" "$reviewed" >&2
failures=$((failures + 1))
done <<EOF
$duplicates
EOF
fi
listed_unique="$(printf '%s\n' "$listed_sorted" | sort -u)"

# --- compare, both directions -------------------------------------------------------------
unreviewed="$(comm -23 <(printf '%s\n' "$excluded") <(printf '%s\n' "$listed_unique"))"
stale="$(comm -13 <(printf '%s\n' "$excluded") <(printf '%s\n' "$listed_unique"))"

if [ -n "$unreviewed" ]; then
while IFS= read -r ns; do
printf 'guard-kubescape-scan-scope: %s is excluded from scanning but has no row in %s; add one saying why, or scan it\n' \
"$ns" "$reviewed" >&2
failures=$((failures + 1))
done <<EOF
$unreviewed
EOF
fi
if [ -n "$stale" ]; then
while IFS= read -r ns; do
printf 'guard-kubescape-scan-scope: %s is listed as unscanned but is not in excludeNamespaces; remove its row\n' \
"$ns" >&2
failures=$((failures + 1))
done <<EOF
$stale
EOF
fi

[ "$failures" -eq 0 ] || exit 1

printf 'guard-kubescape-scan-scope: OK — %d unscanned namespace(s), each reviewed with a reason\n' \
"$(printf '%s\n' "$excluded" | wc -l | tr -d ' ')"
20 changes: 20 additions & 0 deletions scripts/kubescape-unscanned-namespaces.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Reviewed list of namespaces the in-cluster Kubescape scan does NOT cover (#3215).
#
# Checked by scripts/guard-kubescape-scan-scope.sh against the operator's
# `.spec.values.excludeNamespaces` in
# k8s/bases/infrastructure/controllers/kubescape/helm-release.yaml. The two must hold exactly
# the same namespaces, so a namespace can only leave scan coverage through a change that also
# adds a row here, with a reason, for review.
#
# A namespace listed here produces no posture result at all. That is NOT the same as passing:
# no control is evaluated there, and the posture score reads the same as if it were clean.
#
# One tab-separated row per namespace:
# <namespace> <why it is not scanned>
kubescape The scanner's own namespace. Excluded by the chart default; its four operator Deployments are measured by neither the in-cluster scan nor the CI scan, which never renders the HelmRelease (#3215).
kube-system Control-plane and CNI workloads that Talos and KSail install. Excluded by the chart default, so exceptions and opt-ins here change no posture result (#3919, #3923).
kube-public Holds only the cluster-info ConfigMap and no workloads. Excluded by the chart default.
kube-node-lease Holds only node heartbeat Leases and no workloads. Excluded by the chart default.
kubeconfig Not present on this platform. Kept to match the chart default the cluster already runs.
gmp-system Google Managed Prometheus. Not present on this platform. Kept to match the chart default.
gmp-public Google Managed Prometheus. Not present on this platform. Kept to match the chart default.
Loading
Loading