Skip to content

docs: explain snapshot artifacts and deployment diffs - #405

Open
dangrondahl wants to merge 1 commit into
mainfrom
docs/snapshot-deployment-diffs
Open

docs: explain snapshot artifacts and deployment diffs#405
dangrondahl wants to merge 1 commit into
mainfrom
docs/snapshot-deployment-diffs

Conversation

@dangrondahl

Copy link
Copy Markdown
Contributor

Refs #403

What

Adds a hand-authored guide, api-reference/snapshot_artifacts_and_deployment_diffs.md, explaining the deployment_diff object in the get snapshot response:

  • the two kinds of "diff" (kosli diff snapshots vs deployment_diff) and when to use which
  • how a snapshot records change (fingerprint identity, started/exited events)
  • what "previous" means: same flow, different fingerprint, same template_reference_name, most recent start before this one — and why it is not the previous snapshot and not matched on name
  • one answer per flow, illustrated with the four-flow web artifact in public cyber-dojo/aws-prod; the top-level field is a copy of flows[0]
  • when it is null, and which sub-fields can be null or ""
  • caveats: previous_running is about this snapshot, compliance state is evaluated live, diff_url is empty for unrecognized git hosts

Cross-links added from understand_kosli/glossary.md (new "Deployment Diff" term), understand_kosli/how_kosli_works.md (Monitoring runtime), and tutorials/querying_kosli.md (Compare snapshots).

Where

Reference ▸ API Reference ▸ Guides, alongside the OpenAPI-generated Endpoints group. This follows the precedent of client_reference/overview.md and output_and_verbosity.md, which are hand-authored pages inside the generated CLI Reference. It was not placed under Understand Kosli: every sibling there (what_is_kosli, risks, controls, how_kosli_works) is one abstraction level up, and a page about per-flow predecessor resolution would read as an outlier. Readers of this page arrive from get-snapshot, so it sits one click away.

scripts/audit_navigation.py flags Guides as a single-child group. That is advisory only (--check exits 0), and the group is structurally required: Mintlify needs a group container to mix a hand page with an openapi group.

Relationship to #403

kosli-dev/server#6852 is merged and adds descriptions to all nine deployment_diff fields in the OpenAPI schema. The API Reference pulls https://app.kosli.com/api/v2/openapi.json live, so those descriptions appear on get-snapshot automatically once the server deploys — no docs change needed for that half. This PR covers the narrative half (what "previous" means, when it is null). Close #403 once the deploy lands and the field descriptions render.

Verification

  • mint dev: the new page and a generated endpoint page both render (200); the sidebar shows both Guides and Endpoints.
  • mint broken-links: one failure, tutorials/working_with_controls.mdx/getting_started/service-accounts, pre-existing on main and unrelated to this change.
  • python3 -m pytest tests/: 44 passed (navigation integrity included).
  • Every example value (fingerprints, commits, diff_urls, flows[0] equality) verified against the live cyber-dojo/aws-prod snapshot 5352.

Add a hand-authored guide alongside the generated API reference that
explains how deployment_diff in the get snapshot response is computed:
what "previous" means, why there is one answer per flow, when it is
null, and the caveats on previous_running, compliance state and
diff_url. Cross-link from the glossary, how_kosli_works, and the
querying tutorial.

Refs #403
@dangrondahl
dangrondahl requested a review from a team as a code owner September 11, 2026 11:17
@mintlify

mintlify Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
kosli 🟢 Ready View Preview Sep 11, 2026, 11:18 AM

@claude

claude Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Claude encountered an error after 5m 18s —— View job


Doc review

  • Read the doc-review skill and repo conventions
  • Read each changed file at branch HEAD (0dccc50)
  • Checked navigation entry, tab/group placement, Diátaxis fit
  • Cross-checked prose against the generated client_reference/kosli_get_snapshot.md sample data
  • Verified every internal link target resolves
  • Posted findings

4 findings, all Improvement. No Critical. Details are inline; summarized here because each one is a cross-file claim.

# Where Finding
1 api-reference/snapshot_artifacts_and_deployment_diffs.md:106 #N / ~N / @{...} are the CLI's environment-name grammar, not API path segments. A reader following the page's only example tries $BASE/#42, # is read as a URL fragment, and the request silently returns the latest snapshot instead of 42.
2 same file, lines 45-52 and 92-104 The four-flow table is snapshot 5352 of cyber-dojo/aws-prod, but the page says "the latest snapshot" and See it yourself uses -1. cyber-dojo deploys continuously, so the next web deploy makes the walkthrough contradict the table. Name the snapshot; use its id in the curl.
3 same file, lines 84 and 90 previous_git_commit_url — the ninth sub-field — is never mentioned, so the field inventory reads complete while being one short. It also makes the diff_url: "" caveat overstate the gap: the predecessor's commit link already exists; only the compare URL is missing.
4 config/navigation.json:463-474 Moving openapi inside a group changes the API Reference item's shape. Endpoint URLs are probably stable, but ~14 in-repo links plus config/redirects.json:84's destination depend on /api-reference/<tag>/<operation>, and neither mint broken-links nor link-rot validates auto-generated OpenAPI pages. Verify a pre-existing endpoint URL on the preview, not just any endpoint page.

On placement (raised since a new page was added): Reference ▸ API Reference is the right call — the page is explanation-shaped but its subject is a response field, and the reader arrives from get-snapshot, so it sits one click from the thing it explains. The single-child Guides group is structurally required to mix a hand-authored page with the openapi group, and audit_navigation.py only flags it as advisory shape. No redirect is needed: nothing was moved, renamed, or deleted.

Verified, not re-reported: the nav entry exists (core rule 2); all six internal targets in the new page resolve to real files; every link is root-relative; the four-flow table matches the generated sample in client_reference/kosli_get_snapshot.md:1668-1810 value for value; the snapshot-creation claim on line 21 agrees with getting_started/environments.md:46; the glossary insert is alphabetical and matches the file's Title Case term convention.

Couldn't run locally (sandbox blocks network and these invocations): mint broken-links, pytest tests/, scripts/audit_navigation.py, and any request to the live API. Link checking was done by resolving targets against the filesystem instead, and the sample data was cross-checked against the committed generated page rather than the live snapshot. Finding 4 is the one item that genuinely needs the Mintlify preview.

Verdict: 4 Improvements, 0 Critical. Mergeable; findings 1 and 3 are the two that can mislead a reader, and both are small edits.
· branch docs/snapshot-deployment-diffs

curl -s "$BASE/-1" | jq -r '.artifacts[] | "\(.name) -> \(.deployment_diff.diff_url // "no diff")"'
```

`-1` is the latest snapshot. The other snapshot expressions (`#N`, `~N`, `@{...}`) are listed on the [get snapshot](/api-reference/snapshots/get-snapshot) page. For a private organization, authenticate with your [API token](/administration/authentication/api_authentication_methods).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improvement — the other snapshot expressions don't work in a URL the way this reads.

#N, ~N and @{...} are the CLI's ENVIRONMENT-NAME-OR-EXPRESSION grammar, where they attach to the environment name (prod#42, prod~5, prod@{2.hours.ago} — see client_reference/kosli_get_snapshot.md:13-28). Here the only example is a URL path segment ($BASE/-1), so a reader naturally tries $BASE/#42 — and # starts a fragment, so curl never sends it and the request silently returns the latest snapshot instead of snapshot 42. @{...} has the same encoding problem with {/}.

Recommend showing the API form explicitly rather than deferring to the CLI grammar — e.g. a plain numbered id ($BASE/5352) alongside -1, and if #N really is accepted on this path, say it must be percent-encoded as %23N.

Fix this →

Comment on lines +45 to +52
The public `cyber-dojo` organization shows this. Its `web` image is reported into four flows, and the latest `aws-prod` snapshot returns four different answers for the same running image:

| Flow | Previous fingerprint | `diff_url` compares |
|---|---|---|
| `web-ci` | `36ad0020` | `cyber-dojo/web` `cbe481c...236898f` |
| `production-promotion` | `36ad0020` | `cyber-dojo/aws-prod-co-promotion` `7494758...7494758` |
| `snyk-aws-beta-per-artifact` | `36ad0020` | `cyber-dojo/snyk-scanning` `ed3c81d...30111f1` |
| `snyk-aws-prod-per-artifact` | `29c69c2f` | `cyber-dojo/snyk-scanning` `00c4797...30111f1` |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improvement — the example is pinned to a snapshot the page never names.

The four rows are verifiable today: they match the generated sample in client_reference/kosli_get_snapshot.md:1668-1810 exactly (36ad0020 for web-ci, production-promotion and snyk-aws-beta-per-artifact; 29c69c2f for snyk-aws-prod-per-artifact; cbe481c...236898f, 7494758...7494758, ed3c81d...30111f1, 00c4797...30111f1).

The problem is the framing: "the latest aws-prod snapshot", plus $BASE/-1 in See it yourself (line 100). cyber-dojo deploys continuously, so the next web deploy makes -1 disagree with this table, and a reader who follows the instructions and gets four different fingerprints concludes the page is wrong rather than stale. The PR description says these came from snapshot 5352 — say so on the page, and use that id in the walkthrough so the numbers keep reproducing.

Suggested: "the aws-prod snapshot 5352 (the latest at the time of writing) returns four different answers", and $BASE/5352 in the first curl, keeping -1 for the "show me current state" example.

Fix this →


- **`previous_running` is about this snapshot, not history.** `false` means the predecessor is gone, the normal steady state after a deploy completes. `true` means both versions are present in this snapshot at once: a rolling deploy caught mid-flight, a canary, or a partially drained service.
- **`previous_artifact_compliance_state` is evaluated when you make the request**, not frozen at snapshot time. If someone attests evidence against the older artifact tomorrow, this field can change for a snapshot taken today. Compliance is a live judgment about an artifact, not a historical record of the deploy.
- **`diff_url` can be `""`.** It is built by matching the commit URL against GitHub, GitLab, Bitbucket, and Azure DevOps. Any other git host produces an empty string rather than a broken link. `previous_git_commit` and the artifact's own `git_commit` are still there, so you can build the link yourself.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improvement — previous_git_commit_url is missing, and it's the field this caveat should point at.

The response carries a ninth sub-field the page never mentions: previous_git_commit_url (54 occurrences in the generated sample, e.g. client_reference/kosli_get_snapshot.md:1675). It is a ready-made link to the predecessor's commit.

Two consequences:

  • This bullet tells the reader to fall back to previous_git_commit and the artifact's own git_commit and "build the link yourself" — when for the previous commit specifically they already have previous_git_commit_url. The genuinely missing piece when diff_url is "" is only the compare URL.
  • Line 84's nullability enumeration walks the sub-fields and skips this one, so the page reads as a complete field inventory while being one short.

Recommend naming previous_git_commit_url in the line 84 sentence, and in this bullet saying what is actually absent: the compare link, which you can construct from previous_git_commit and git_commit (or note whether previous_git_commit_url is also empty for an unrecognized host — worth checking, since it's reported rather than pattern-matched).

Fix this →

Comment thread config/navigation.json
Comment on lines +463 to +474
"groups": [
{
"group": "Guides",
"pages": [
"api-reference/snapshot_artifacts_and_deployment_diffs"
]
},
{
"group": "Endpoints",
"openapi": "https://app.kosli.com/api/v2/openapi.json"
}
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improvement (verify on the preview) — this changes the shape of the API Reference item that ~14 existing links depend on.

openapi moved from being the item's direct value to sitting inside a groups entry. The auto-generated endpoint URLs (/api-reference/<tag>/<operation>) should be unaffected, since Mintlify derives page paths from the item, not the group label — but nothing in CI will tell you if that assumption is wrong: mint broken-links does not resolve auto-generated OpenAPI pages, and the link-rot check is unreliable per CLAUDE.md.

If the prefix did change, these break silently:

  • understand_kosli/what_is_kosli.md:12/api-reference/actions/list-actions
  • getting_started/enforce_policies.md:127,143/api-reference/asserts/assert-artifact
  • tutorials/rotating_api_keys.mdx:99,127,148-150 and administration/authentication/api_key_rotation.md:23-25/api-reference/service-accounts/...
  • user/default_organization.md:25/api-reference/user/set-default-organization
  • changelog/index.mdx:1068
  • config/redirects.json:84 — whose destination is /api-reference/actions/list-actions, so a broken destination turns a working redirect into a dead end
  • this PR's own page, which links /api-reference/snapshots/get-snapshot four times

The PR notes "a generated endpoint page renders (200)". Worth confirming specifically that one of the pre-existing URLs above still resolves on the Mintlify preview, not just that some endpoint page exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: describe the artifact fields in the get snapshot response, starting with deployment_diff

1 participant