Skip to content

feat: rotation and sync runner, If-Match write-back and expiresAt (stacked on #914) - #926

Merged
rubenvdlinde merged 13 commits into
developmentfrom
feat/apps-secret-sync-and-rotation-runner
Oct 2, 2026
Merged

rubenvdlinde merged 13 commits into
developmentfrom
feat/apps-secret-sync-and-rotation-runner

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Closes #778

Stacked on #914 (it needs sdk/go). Merge #914 first; this diff then shrinks to the server change and the runner.

Builds the OpenSpec change apps-secret-sync-and-rotation-runner and archives it. Parity rows apps-20 and apps-25 move to built.

What changed

Server (additive to the machine API):

  • PUT /api/v1/app/secrets/{id} honours If-Match. A stale strong ETag answers 412 with the current ETag and writes nothing. No header writes as before. A secret of another vault stays a 404, so the precondition is no existence oracle.
  • The envelope's secret block gains expiresAt (ISO 8601 or null). Discovery advertises conditionalWrite: true and expiresAt: true.

keepiq-runner (integrations/runner/, Go, on sdk/go):

  • runner.yaml names Keepiq secrets, never values; unknown keys are refused. Modes: run, run --once, check.
  • Rotation is prove-then-record: generate, journal both values encrypted to the application key, set at the target, log in with the new value, then write back with If-Match.
  • A failed login or a 412 sets the old value back. The next start finishes a rotation that was interrupted after the target change.
  • Connectors: postgres, mysql, exec. Rotations run on a cron schedule and/or ahead of expiresAt.
  • Sync polls updated_since and pushes by ETag with backoff, to AWS Secrets Manager, Azure Key Vault, GitHub Actions secrets (sealed box) or exec.
  • The log redacts every value the runner handled; the state directory holds ids and ETags only.
  • integrations-runner.yml: tests with PostgreSQL and MySQL services; on runner-v*, binaries with SHA256SUMS and a signed image.
  • SDK (on feat: client libraries for Go, Python and TypeScript, GitHub Action and GitLab CI template #914's branch): UpdateIfMatch, ExpiresAt, PublicKey/Decrypt; the keepiqtest stub honours If-Match.
  • Docs: docs/rotation-and-sync.md, integrations/runner/runner.example.yaml.

Verified here

  • PHPUnit: ApplicationSecretsControllerTest, DiscoveryControllerTest, MachineSecretEnvelopeServiceTest, MachineEnvelopeCliFixtureTest, SdkEncryptedVectorsTest green. The five If-Match tests and the expiresAt/discovery tests were red before the change.
  • go vet and go test ./... in integrations/runner, sdk/go and cli: green.
  • TestPostgres and TestMySQL ran against postgres:16-alpine and mysql:8.4: the new password logs in, the old one is refused.
  • TestCrashAfterTargetChangeIsCompletedOnNextStart: a child process exits right after the target change; the next start writes the journalled value back.
  • keepiq-runner check --config runner.example.yaml passes.

Owed

  • Live (task 4.3): on the dev instance, an application owns pg-app-password; the runner rotates it at a local PostgreSQL; keepiq ci fetch pg-app-password returns a value that logs in.
  • A LocalStack run for AWS, and real pushes to Azure and GitHub.
  • CI: the workflow dry run; the first runner-v* release (not done here, by design).

Design note: the write-back does not move expiresAt, so followExpiry rotates once per expiry window and the docs say to set a new expiry date in Keepiq. The If-Match check runs in the controller before the write; two writers racing inside the same request window are not serialised (the same as the rest of the write path).

Checks

Checks: pending, coordinator runs check.sh.

Inherited findings: phpcs warns "Class ApplicationSecretsController is missing @SPEC PHPDoc tag", the same on development.

🤖 Generated with Claude Code

PUT /api/v1/app/secrets/{id} honours If-Match: a stale strong ETag
answers 412 with the current ETag and writes nothing; no header writes
as before; another vault stays a 404. The envelope's secret block gains
expiresAt, and discovery advertises conditionalWrite and expiresAt.
The CLI fixture gains expiresAt: null; Newman asserts both additions.
(Completes WIP 857aed1.)

Refs #778
The Go library writes back with If-Match (ErrPreconditionFailed on
412), reads secret.expiresAt, and exposes the public key and Decrypt
for values a caller keeps itself. The keepiqtest stub honours If-Match.

Refs #778
…nations

integrations/runner is a Go module on sdk/go. runner.yaml names Keepiq
secrets only (unknown keys refused). Rotation is prove-then-record:
generate, journal both values encrypted to the application key, set at
the target, log in with the new value, then PUT with If-Match; a failed
login or a 412 sets the old value back. Start-up recovery completes a
journalled rotation whose new value logs in. Rotations run on a cron
schedule or ahead of expiresAt. Sync polls updated_since and pushes by
ETag with backoff to AWS Secrets Manager, Azure Key Vault, GitHub
Actions secrets (sealed box) or an exec hook. Logs redact every value
the runner handled. Tests follow.

Refs #778
The crash test runs the rotation in a child process that exits right
after the target accepted the new value; the next start completes the
write-back from the journal. Other tests: prove-then-record, set-back on
a failed login, 412 conflict restores the target and is not retried, a
journal entry that never reached the target is dropped, generator
classes, cron and expiry lead time. Each asserts no value in the log,
the state files or any request to Keepiq.

Refs #778
Sync pushes a changed secret once and an unchanged one never, and a
failed push backs off without blocking the others. Destinations are
tested against stubs of the Secrets Manager JSON protocol, Azure token
and Set Secret, and the GitHub secrets API, where the sealed box opens
only with the repository key. The postgres and mysql connectors were
run against postgres:16 and mysql:8.4 (KEEPIQ_TEST_POSTGRES /
KEEPIQ_TEST_MYSQL): the new password logs in, the old one does not.
run --once rotates and syncs; the daemon waits for the schedule or the
expiry lead time.

Refs #778
integrations-runner.yml tests the runner with PostgreSQL and MySQL
service containers on pull requests, and on runner-v* tags attaches
static binaries with SHA256SUMS and pushes a cosign-signed multi-arch
image ghcr.io/conductionnl/keepiq-runner. docs/rotation-and-sync.md
covers setup, one application per runner, every connector and
destination; runner.example.yaml passes keepiq-runner check.

Refs #778
Ticks every task with what was verified and what is owed (4.3 needs
the dev instance), syncs secret-store-api and the new
secret-rotation-runner spec, and sets parity rows apps-20 and apps-25
to built.

Closes #778
Comment thread .github/workflows/cli-release.yml Fixed
Comment thread sdk/js/src/client.ts Fixed
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/keepiq @ 86169aa

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ❌
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-manifest ✅
test-l10n ✅
format ✅
check-l10n-js ✅
check-schema-l10n ✅
composer ✅ ✅ 114/114
npm ✅ ✅ 660/660
app:check-code ⏭️
info.xml ✅
REUSE ✅
lockfile sync ✅
PHPUnit ❌
Newman ✅
Playwright ⏭️ deferred: E2E runs locally and on the promotion path only. This pull request targets development, so the suite is asked once per promotion into beta and main rather than once per push per open pull request. Run it locally with npx playwright test, or from the Actions tab on a branch with no open pull request into development.
Hydra gates ✅

Quality workflow — 2026-10-02 11:11 UTC

Download the full PDF report from the workflow artifacts.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/keepiq @ 117f36e

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ❌
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-manifest ✅
test-l10n ✅
format ✅
check-l10n-js ✅
check-schema-l10n ✅
composer ✅ ✅ 114/114
npm ✅ ✅ 660/660
app:check-code ⏭️
info.xml ✅
REUSE ✅
lockfile sync ✅
PHPUnit ❌
Newman ✅
Playwright ⏭️ deferred: E2E runs locally and on the promotion path only. This pull request targets development, so the suite is asked once per promotion into beta and main rather than once per push per open pull request. Run it locally with npx playwright test, or from the Actions tab on a branch with no open pull request into development.
Hydra gates ✅

Quality workflow — 2026-10-02 17:54 UTC

Download the full PDF report from the workflow artifacts.

…compares

ApplicationSecretsController reached phpmd's class complexity threshold with
the If-Match precondition. The comparison (any listed strong tag or *) moves
to MachineSecretEnvelopeService::ifMatchHolds(), next to etag(); the
controller keeps the 412 response. Unit test covers current, *, a list,
stale and weak tags.
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/keepiq @ 6df6177

Check PHP Vue Security License Tests
lint ⏭️
phpcs ⏭️
phpmd ⏭️
psalm ⏭️
phpstan ⏭️
phpmetrics ⏭️
eslint ⏭️
stylelint ⏭️
build ⏭️
composer ⏭️ ⏭️
npm ⏭️ ⏭️
app:check-code ⏭️
info.xml ⏭️
REUSE ⏭️
lockfile sync ⏭️
PHPUnit ❌
Newman ❌
Playwright ❌
Hydra gates ❌

Quality workflow — 2026-10-02 18:11 UTC

Download the full PDF report from the workflow artifacts.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/keepiq @ 5257312

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-manifest ✅
test-l10n ✅
format ✅
check-l10n-js ✅
check-schema-l10n ✅
composer ✅ ✅ 114/114
npm ✅ ✅ 660/660
app:check-code ⏭️
info.xml ✅
REUSE ✅
lockfile sync ✅
PHPUnit ❌
Newman ✅
Playwright ⏭️ deferred: E2E runs locally and on the promotion path only. This pull request targets development, so the suite is asked once per promotion into beta and main rather than once per push per open pull request. Run it locally with npx playwright test, or from the Actions tab on a branch with no open pull request into development.
Hydra gates ✅

Quality workflow — 2026-10-02 18:19 UTC

Download the full PDF report from the workflow artifacts.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/keepiq @ 942e29b

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-manifest ✅
test-l10n ✅
format ✅
check-l10n-js ✅
check-schema-l10n ✅
composer ✅ ✅ 114/114
npm ✅ ✅ 660/660
app:check-code ⏭️
info.xml ✅
REUSE ✅
lockfile sync ✅
PHPUnit ❌
Newman ✅
Playwright ⏭️ deferred: E2E runs locally and on the promotion path only. This pull request targets development, so the suite is asked once per promotion into beta and main rather than once per push per open pull request. Run it locally with npx playwright test, or from the Actions tab on a branch with no open pull request into development.
Hydra gates ✅

Quality workflow — 2026-10-02 19:04 UTC

Download the full PDF report from the workflow artifacts.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/keepiq @ 02b9fa1

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-manifest ✅
test-l10n ✅
format ✅
check-l10n-js ✅
check-schema-l10n ✅
composer ✅ ✅ 114/114
npm ✅ ✅ 660/660
app:check-code ⏭️
info.xml ✅
REUSE ✅
lockfile sync ✅
PHPUnit ❌
Newman ✅
Playwright ⏭️ deferred: E2E runs locally and on the promotion path only. This pull request targets development, so the suite is asked once per promotion into beta and main rather than once per push per open pull request. Run it locally with npx playwright test, or from the Actions tab on a branch with no open pull request into development.
Hydra gates ✅

Quality workflow — 2026-10-02 19:20 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde

Copy link
Copy Markdown
Contributor Author

Local checks: full premerge set on 486170b (PHPUnit 1724 OK, vitest 1357 of 1357, phpmd 0 after moving the If-Match comparison to MachineSecretEnvelopeService::ifMatchHolds() with a unit test, phpstan 0, phpcs 0, format, lint, l10n, gate-16 and gate-25 clean); Hydra gates 0 and every npm check 0; go vet and go test green for cli/, sdk/go and integrations/runner on Go 1.25.13. After merging #965 (eae81e3, which touches disjoint files): PHPUnit 1735 tests OK, phpmd 0, phpstan 0. Merged with a merge commit because #930 is stacked on it.

@rubenvdlinde
rubenvdlinde merged commit 2c34fb2 into development Oct 2, 2026
68 of 72 checks passed
rubenvdlinde added a commit that referenced this pull request Oct 2, 2026
feat: Terraform provider for Keepiq secrets, values kept out of plan and state (stacked on #926)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants