Add AWS EC2 NitroTPM attestation and guest OS support by h4x3rotab · Pull Request #753 · Dstack-TEE/dstack

h4x3rotab · 2026-07-06T05:39:02Z

Summary

Adds the complete AWS EC2 NitroTPM platform path under the account-admin-untrusted threat model. This PR now combines the former dstack#753 implementation with the Yocto image changes from Dstack-TEE/meta-dstack#79.

Attestation and key release

parse and verify AWS NitroTPM Attestation Documents against the AWS NitroTPM PKI;
replay PCR14 launch measurements and use PCR23 only for runtime telemetry;
bind boot PCRs, OS image hash, measured config/app identity, freshness, and recipient keys;
support AWS-aware KMS, verifier, RA-TLS, auth-simple, auth-eth, guest-agent, and SDK flows;
expose canonical AWS platform details without depending on AWS KMS.

Guest OS

add the dstack-aws Yocto kernel feature with built-in NVMe, ENA, UEFI/GPT, serial-console, and TPM CRB support;
add tpm2-tss to the dstack guest recipe build dependencies;
keep AWS image/release helpers under os/yocto/tools/aws/ and formal documentation under docs/.

Operations and documentation

add production verifier, reproducibility, image hardening, AMI promotion, and EC2 live-smoke documentation/tooling;
adapt release-manifest generation to the unified dstack monorepo (--repo) rather than separate dstack/meta-dstack checkouts.

Validation

cargo test -p dstack-attest aws_nitro_tpm
cargo test -p dstack-kms aws_nitro_tpm
cargo test -p dstack-verifier aws
cargo test -p dstack-guest-agent -p dstack-guest-agent-simulator -p ra-tls -p tpm-attest -p tpm-qvl
npm run test:run and npm run lint in dstack/kms/auth-simple
bash -n/shellcheck through prek run --all-files
reuse lint
the original branch also recorded a live EC2 smoke test against its promoted Attestable AMI

forge test --ffi was not rerun locally because Foundry is not installed in this environment; CI covers the Foundry suite.

kvinwang · 2026-07-14T04:37:16Z

Simplification pass (operator decisions)

Rebased onto master (includes #777 empty TPM key-provider id) and applied:

Decision	Implementation
Single event lane (TDX-like)	All dstack events → SHA384 PCR14; dropped PCR23 runtime split + `emit_launch` domain
Config not via UKI cmdline	Shared-disk `MrConfigV3` measured into PCR8; guest verifies PCR8 vs document
Unified `os_image_hash`	Optional `VmConfig.aws_measurement` (`sha256(sha256sum.txt)` + ref PCR4/7/12); legacy PCR-hash still works
Drop complexity	Removed GetAppKey v2 recipient encryption, GetQuote `attestation_options`, `aws-pcr14-fault-injection`
`check_kms`	Only `mrAggregated` (no `kms.composeHashes`); pin early/boot-mr-done style like bare TDX

Commit: 40620832 simplify AWS NitroTPM: single PCR14, PCR8 config, drop v2 complexity

Still follow-up as needed: Yocto UKI recipe stop embedding dstack.mr_config_id in cmdline; production docs/smoke; auth-simple README examples; optional SDK cleanup if any client exposed attestation_options.

kvinwang · 2026-07-14T04:42:12Z

Follow-up commit `ca7909f4`

Completed the remaining cleanup called out after the simplify pass:

UKI / Yocto tools: document that UKI cmdline must not embed per-app dstack.mr_config_id (shared-disk MrConfigV3 → PCR8)
auth-simple: KMS allowlist is mrAggregated only; examples, schema, and tests updated (app composeHashes unchanged)
SDKs: removed attestation_options / AWS NitroTPM options from Rust, Python, JS, Go, and sdk/curl/api.md
Docs: production verifier runbook + security-model PCR map; security-evaluation banner that historical cmdline/PCR23/recipient/composeHashes sections are superseded

Add AWS EC2 NitroTPM as a first-class attestation platform under the account-admin-untrusted threat model, integrated like SEV-SNP rather than with AWS-specific auth surface. Measurement model (aligned with bare TDX): - Boot: PCR4/7/12 for AMI/UKI identity - Config: shared-disk MrConfigV3 measured into PCR8 (no UKI cmdline pin) - Events: all dstack events extend SHA384 PCR14 (RTMR3 analogue; no PCR23 launch/runtime split) - os_image_hash: prefer unified sha256(sha256sum.txt) via VmConfig.aws_measurement + reference PCR bind; legacy PCR-hash fallback - KMS GetAppKey: RA-TLS only (api_version=1); no recipient encryption - GetQuote/Attest: report_data only (no platform attestation_options) - check_kms / auth-simple: pin early mrAggregated only (boot-mr-done) Includes Yocto AWS kernel/image support, verifier/KMS paths, SDKs, docs, and release/smoke helpers. Empty TPM key-provider id (already on master) keeps early MR precomputable.

Local TPM sealing on AwsEc2 now binds only SHA384 PCR4/7/12 (boot/OS identity). PCR23 is unused after the single PCR14 event-lane model and is resettable, so it must not gate sealed app keys.

Local NitroTPM key-provider sealing now uses sha384:4,7,8,12,14 — boot path (4/7/12), shared-disk MrConfigV3 (8), and the dstack event lane at seal time (14). Seal runs after PCR8 measure and key-provider events, so reboot unseal matches the same intermediate PCR14 state.

Wire unified os_image_hash + aws_measurement end-to-end: - dstack-mr aws-measurement-cbor and assemble measurement.aws.cbor - guest quote embeds sys-config vm_config when aws_measurement binds attested PCR4/7/12 (legacy PCR-hash fallback otherwise) - dstack-cloud: platform=aws, prepare sys-config with aws_measurement, full EC2 lifecycle (AMI import, shared snapshot, run/stop/start/logs) - remove standalone aws-ec2-import/smoke/attach scripts; docs point to dstack-cloud. Keep image audit + release-manifest helpers only.

UKI assemble always precomputes NitroTPM PCR4/7/12 into measurement.aws.cbor and lists it in sha256sum.txt so digest.txt is fixed for the release. Host nitro-tpm-pcr-compute is preferred; Docker AL2023 is fallback; missing both fails the build. dstack-cloud prepare only embeds the release material and refuses packages that lack a checksum-committed measurement.aws.cbor.

AwsOsImageMeasurement now holds sha256(PCR4||PCR7||PCR12) instead of the three raw SHA384 PCR values, matching the legacy image-hash composition. hex encoding uses serde_human_bytes (hex_bytes alias). Verifier and guest quote compare digests; dstack-mr still takes PCR4/7/12 hex and folds them.

Drop panic from cbor_to_vec and the awkward context string parameter. Callers attach context with .context("…") instead. to_cbor_vec, from_cbor_slice, measurement_hash, from_measurement, decode_*, and document verify now use anyhow::Result for a consistent error model.

The pre-key-release check added no security guarantee: key release for an unauthorized compose is already blocked by KMS via the PCR14 event lane, and key use under a mismatched config is still rejected by the full mr_config check in verify_app before any key is used or persisted. The early check only duplicated the static-field validation for AWS, diverging from the other platforms. Remove it and let AWS follow the same single post-key verification path as TDX and SEV-SNP.

On the AWS EC2 platform the attestation mode must be NitroTPM only. Exposing TDX or SEV-SNP devices on an AWS instance must not silently switch the attestation mode away from the AWS platform path.

reuse policy_tcb_fields when filling VerificationDetails so details.tcb_status matches boot_info.tcbStatus (notably AWS NitroTPM, which is normalized to "UpToDate" for the auth gate but surfaced as null at the top level). also picks up the pending rustfmt import reorder that broke CI.

the "legacy paths without aws_measurement use the value directly" path no longer exists; aws_measurement is mandatory on AWS.

the KMS key-release pipeline gates non-SNP quotes through CvmVerifier::verify_os_image_hash; pin down that the AWS path rejects a vm_config without aws_measurement, accepts a unified vm_config bound to the quoted PCRs, and rejects a boot_pcr_digest mismatch.

- audit-aws-ec2-image-hardening.sh: a supplied --rootfs-squashfs with no unsquashfs available is now a FAIL instead of WARN + exit 0 - generate-aws-ec2-release-manifest.sh: capture the audit exit status explicitly so a failing audit prints its output instead of silently aborting the manifest run under set -e

hop limit 2 lets processes behind the docker bridge fetch IMDSv2 tokens, exposing instance-role credentials to app containers when iam_instance_profile is set. dstack itself never talks to IMDS, so default metadata_hop_limit to 1 and leave 2 as an explicit opt-in for deployments whose containers legitimately need the instance role.

generate_tpm_app_keys now fails closed on platforms whose TPM is not part of the platform trust model; document that in the app-compose key_provider field description.

drop the unpinned docker amazonlinux:2023 + dnf fallback from assemble.sh: the computed PCRs feed measurement.aws.cbor -> sha256sum.txt -> os_image_hash, i.e. the measurement the KMS and verifiers enforce, so only a version-pinned operator-installed nitro-tpm-pcr-compute may produce it.

release images and their evidence (sha256sum.txt, digest.txt, measurement.aws.cbor, aws-pcrs.json) are produced by the unified os/build.sh flow now, so drop generate-aws-ec2-release-manifest.sh and the reproducibility-manifest doc written around it. rework the production verifier runbook around the build.sh release package (reproducible rebuild + pinned host tools for reference measurements) and note the removal in the dated security evaluation. the hardening audit script stays.

Trim the superseded security evaluation to the live threat model and property table, document the actual UKI archive and PCR sidecar layout, and remove stale recipient/fallback claims. Also simplify the certificate output and build fragments referenced by the updated runbook.

Drop the write-only AppInfo.attestation field and its backend, simulator, SDK, and documentation plumbing. GetQuoteResponse.attestation remains the platform-agnostic evidence field.

Parameterize GCP/AWS vm-config generation, reuse one shared-disk builder with required-file checks, and remove dead config serialization and save helpers. Correct all project-file diagnostics to app.json.

Restore the established string-error measurement API while using or_panic for invariant-only CBOR encoding. Consolidate TCB and key-release policy helpers, move AWS image-binding tests to the verifier, and remove redundant tests, dead accessors, and unrelated TPM/NSM diff noise.

Copilot

Pull request overview

Adds end-to-end AWS EC2 NitroTPM support across the monorepo (guest OS build/measurement, attestation generation, verifier/KMS policy normalization, SDK surface updates, and operational documentation) under the account-admin-untrusted threat model.

Changes:

Introduces AWS NitroTPM attestation collection/verification and AWS-specific OS-image binding (aws_measurement, boot PCR digest) across attest/verifier/KMS.
Extends Yocto guest OS/kernel configuration and image assembly to produce AWS measurement artifacts and hardening audit tooling.
Updates SDKs and docs to expose/use a platform-agnostic attestation field and to document AWS verification/operations.

Reviewed changes

Copilot reviewed 60 out of 62 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
sdk/python/src/dstack_sdk/dstack_client.py	Adds `attestation` field to quote response model.
sdk/js/src/index.ts	Adds `attestation` field to JS SDK quote response type.
sdk/go/dstack/client.go	Adds `attestation` field and decoding helper to Go SDK.
sdk/curl/api.md	Documents `attestation` in `/GetQuote` and cloud fields in `/Info`.
os/yocto/tools/README.md	Clarifies Yocto host tools scope and AWS tooling location.
os/yocto/tools/aws/audit-aws-ec2-image-hardening.sh	Adds AWS EC2 image hardening audit script.
os/yocto/layers/meta-dstack/recipes-kernel/linux/linux-yocto%.bbappend	Adds AWS kernel feature config sources and feature inclusion.
os/yocto/layers/meta-dstack/recipes-kernel/linux/files/dstack-aws.scc	Introduces Yocto kernel feature definition for AWS.
os/yocto/layers/meta-dstack/recipes-kernel/linux/files/dstack-aws.cfg	Adds kernel config needed for EC2 Nitro boot devices/console.
os/yocto/layers/meta-dstack/recipes-core/images/dstack-uki.bb	Documents AWS app/config binding expectations for UKI cmdline.
os/yocto/layers/meta-dstack/recipes-core/dstack-guest/dstack-guest.bb	Adds `tpm2-tss` build dependency for guest recipe.
os/image/README.md	Documents AWS measurement artifact and tool requirements.
os/image/assemble.sh	Generates and includes `measurement.aws.cbor` in image digest/tar flows.
dstack/vmm/src/app.rs	Initializes `aws_measurement` field in VM config assembly path.
dstack/verifier/src/verification.rs	Adds AWS NitroTPM OS-image verification and policy boot info construction.
dstack/verifier/src/types.rs	Adds `PolicyBootInfo` and includes it in verification details.
dstack/verifier/src/main.rs	Adds `--verify-cert` oneshot mode for RA-TLS cert verification.
dstack/verifier/src/lib.rs	Re-exports `PolicyBootInfo` and `policy_tcb_fields`.
dstack/verifier/README.md	Documents AWS mode, policy boot info, and `--verify-cert` workflow.
dstack/verifier/Cargo.toml	Gates binary build on `binary` feature (default-enabled).
dstack/tpm2/src/types.rs	Adds RSA EK template and configurable name algorithm for templates/NV.
dstack/tpm2/src/device.rs	Adds raw session command helpers and flexible auth-area builders.
dstack/tpm2/src/commands.rs	Adds raw execution, NV name read, auth NV read/write, capabilities helpers.
dstack/tpm-types/src/lib.rs	Adds SHA384 PCR selection helper.
dstack/tpm-qvl/src/lib.rs	Rejects AWS EC2 platform for GCP AK quote root retrieval.
dstack/tpm-attest/tests/tpm_quote_sample.README.md	Documents TPM quote parsing fixture semantics/provenance.
dstack/tpm-attest/src/lib.rs	Adds AWS NitroTPM PCR policy and single-PCR read helper.
dstack/ra-tls/src/attestation.rs	Adds RA-TLS cert verification APIs returning verified certificate evidence.
dstack/ra-tls/Cargo.toml	Adds tokio dev-dependency for async tests.
dstack/nsm-qvl/tests/nitro_attestation.README.md	Documents NSM attestation fixture capture/verification model.
dstack/nsm-qvl/src/lib.rs	Adds alias for NitroTPM PCR field name in attestation document.
dstack/kms/src/onboard_service.rs	Adds platform evidence mapping for AWS NitroTPM.
dstack/kms/src/main_service/upgrade_authority.rs	Reuses verifier’s canonical policy boot info and TCB normalization.
dstack/kms/src/main_service.rs	Adds AWS NitroTPM key-release gate and extensive AWS-specific tests.
dstack/kms/src/config.rs	Adds `aws_nitro_tpm_key_release` config flag.
dstack/kms/kms.toml	Documents and defaults AWS NitroTPM key-release gate to disabled.
dstack/kms/Cargo.toml	Adds `cc-eventlog` dev-dependency for AWS-related tests.
dstack/kms/auth-simple/README.md	Updates allowlisting guidance and adds AWS NitroTPM example.
dstack/kms/auth-simple/index.test.ts	Normalizes `attestationMode` casing and updates tests/messages.
dstack/kms/auth-simple/auth-config.example.json	Updates example KMS MR label to “early/boot-mr-done”.
dstack/dstack-util/src/system_setup/config_id_verifier.rs	Skips AWS PCR8 config-id cross-check (guest-measured).
dstack/dstack-util/src/system_setup.rs	Adds AWS instance binding and AWS-specific PCR measurements/policy.
dstack/dstack-util/src/main.rs	Allows emitting raw AWS NitroTPM attestation doc in quote command path.
dstack/dstack-util/src/host_api.rs	Skips host notify on AWS EC2 platform.
dstack/dstack-util/src/crypto.rs	Rejects all-zero X25519 shared secret during DH decrypt.
dstack/dstack-util/Cargo.toml	Adds `tpm2` dependency to support AWS NitroTPM operations.
dstack/dstack-types/src/lib.rs	Adds AWS platform detection and AWS OS image measurement document types.
dstack/dstack-mr/src/main.rs	Adds AWS measurement encoding/inspection support.
dstack/dstack-attest/src/v1.rs	Adds AWS NitroTPM platform evidence variant.
dstack/dstack-attest/src/lib.rs	Extends measured-event emission to TPM PCRs (sha256/sha384) and adds PCR8 measure helper.
dstack/dstack-attest/src/aws_nitro_tpm.rs	Implements NitroTPM vendor-command flow to obtain attestation documents.
dstack/dstack-attest/src/attestation.rs	Adds AWS NitroTPM mode, quote/report verification, and MR derivation from PCRs.
dstack/dstack-attest/Cargo.toml	Adds AWS NitroTPM dependencies under `quote` feature.
dstack/Cargo.lock	Locks new dependencies for AWS NitroTPM functionality.
docs/verification.md	Adds platform-specific verification guidance including AWS NitroTPM.
docs/tutorials/attestation-verification.md	Clarifies TDX-specific flow and documents platform differences incl. AWS.
docs/security/security-model.md	Updates security model and measurement semantics for AWS NitroTPM mode.
docs/security/cvm-boundaries.md	Documents TPM key provider constraints and AWS measurement lane semantics.
docs/quickstart.md	Extends quickstart to include AWS NitroTPM option and config example.
docs/aws-ec2-production-verifier-runbook.md	New runbook for AWS EC2 NitroTPM production verification and ops.
docs/aws-attested-instance-security-evaluation.md	New AWS NitroTPM threat-model security evaluation document.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

kvinwang · 2026-07-16T08:34:42Z

@copilot resolve the merge conflicts in this pull request

…tform-spec # Conflicts: # dstack/dstack-types/src/lib.rs

Copilot · 2026-07-16T08:38:16Z

@copilot resolve the merge conflicts in this pull request

Resolved. The only true conflict was in dstack/dstack-types/src/lib.rs — both branches added a #[cfg(test)] module at the end of the file. I kept both:

platform_tests from this branch (AWS EC2 DMI detection tests)
vm_config_num_nics_tests from master (NIC count serialization tests)

All other changes from master merged cleanly (tee-simulator, VMM networking, Cargo.lock, etc.). Merge commit: 6bd4e6d

h4x3rotab mentioned this pull request Jul 6, 2026

[codex] Add AWS EC2 Nitro image support Dstack-TEE/meta-dstack#79

Closed

github-advanced-security AI found potential problems Jul 6, 2026

View reviewed changes

Comment thread dstack/dstack-attest/src/aws_nitro_tpm.rs Fixed

h4x3rotab force-pushed the aws-attestation-platform-spec branch 6 times, most recently from d6fb1cd to 504e342 Compare July 8, 2026 03:48

h4x3rotab mentioned this pull request Jul 8, 2026

Tracking: AWS EC2 NitroTPM attestation platform support #758

Open

2 tasks

h4x3rotab force-pushed the aws-attestation-platform-spec branch from 504e342 to 420ea41 Compare July 8, 2026 22:32

kvinwang force-pushed the aws-attestation-platform-spec branch from 420ea41 to 41f9b7e Compare July 13, 2026 04:14

kvinwang changed the title ~~[codex] Add AWS NitroTPM attestation platform support~~ Add AWS EC2 NitroTPM attestation and guest OS support Jul 13, 2026

kvinwang force-pushed the aws-attestation-platform-spec branch from 41f9b7e to 41a4aa1 Compare July 13, 2026 04:16

github-advanced-security AI found potential problems Jul 13, 2026

View reviewed changes

Comment thread dstack/dstack-attest/src/aws_nitro_tpm.rs Fixed

kvinwang force-pushed the aws-attestation-platform-spec branch from fe08404 to 4062083 Compare July 14, 2026 04:37

kvinwang force-pushed the aws-attestation-platform-spec branch from ca7909f to c49b7cf Compare July 14, 2026 04:46

kvinwang force-pushed the aws-attestation-platform-spec branch from c49b7cf to d8f4871 Compare July 14, 2026 04:51

kvinwang added 7 commits July 13, 2026 22:22

fix: drop unused PCR23 from AWS TPM key-provider seal policy

67075d4

Local TPM sealing on AwsEc2 now binds only SHA384 PCR4/7/12 (boot/OS identity). PCR23 is unused after the single PCR14 event-lane model and is resettable, so it must not gate sealed app keys.

chore: update Cargo.lock for dstack-types anyhow dep

f6ed3c4

kvinwang force-pushed the aws-attestation-platform-spec branch from 4fe61c8 to f6ed3c4 Compare July 14, 2026 10:58

kvinwang added 2 commits July 14, 2026 08:16

fix: never fall back to TDX/SEV-SNP attestation on AWS EC2

a2fcb89

On the AWS EC2 platform the attestation mode must be NitroTPM only. Exposing TDX or SEV-SNP devices on an AWS instance must not silently switch the attestation mode away from the AWS platform path.

kvinwang added 12 commits July 15, 2026 00:33

docs(attest): drop stale aws_measurement legacy-fallback comment

1db7423

the "legacy paths without aws_measurement use the value directly" path no longer exists; aws_measurement is mandatory on AWS.

docs(security): note tpm key provider is gcp/aws-only

a230584

generate_tpm_app_keys now fails closed on platforms whose TPM is not part of the platform trust model; document that in the app-compose key_provider field description.

refactor(api): remove unused app info attestation mode

c7f02ba

Drop the write-only AppInfo.attestation field and its backend, simulator, SDK, and documentation plumbing. GetQuoteResponse.attestation remains the platform-agnostic evidence field.

refactor(cloud): share multi-cloud image preparation

f301ebf

Parameterize GCP/AWS vm-config generation, reuse one shared-disk builder with required-file checks, and remove dead config serialization and save helpers. Correct all project-file diagnostics to app.json.

kvinwang requested a review from Copilot July 15, 2026 09:35

Copilot started reviewing on behalf of kvinwang July 15, 2026 09:36 View session

Copilot AI reviewed Jul 15, 2026

View reviewed changes

Comment thread dstack/verifier/src/main.rs

kvinwang added 3 commits July 15, 2026 10:04

fix(os): harden AWS guest image release validation

77df428

feat(cloud): use EBS Direct for AWS snapshots

3729683

fix(os): drop init_on_free hardening that breaks GCP TDX boot

46fc3cf

kvinwang marked this pull request as ready for review July 16, 2026 05:50

kvinwang added 3 commits July 15, 2026 23:36

fix(os): restore empty /dev/vdb data-disk auto-initialization

128db22

Merge origin/master into aws-attestation-platform-spec

cbde61b

test(attest): serialize environment override tests

a363794

Copilot started work on behalf of kvinwang July 16, 2026 08:35 View session

Merge remote-tracking branch 'origin/master' into aws-attestation-pla…

6bd4e6d

…tform-spec # Conflicts: # dstack/dstack-types/src/lib.rs

Copilot finished work on behalf of kvinwang July 16, 2026 08:38

Copilot AI requested a review from kvinwang July 16, 2026 08:38

kvinwang merged commit 9ed633a into master Jul 16, 2026
15 checks passed

kvinwang deleted the aws-attestation-platform-spec branch July 20, 2026 14:05

Navigation Menu

Uh oh!

Add AWS EC2 NitroTPM attestation and guest OS support - #753

Add AWS EC2 NitroTPM attestation and guest OS support#753
kvinwang merged 42 commits into
masterfrom
aws-attestation-platform-spec

h4x3rotab commented Jul 6, 2026 •

edited by kvinwang

Loading

Uh oh!

Uh oh!

Uh oh!

kvinwang commented Jul 14, 2026

Uh oh!

kvinwang commented Jul 14, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

kvinwang commented Jul 16, 2026

Uh oh!

Copilot AI commented Jul 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

h4x3rotab commented Jul 6, 2026 • edited by kvinwang Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Attestation and key release

Guest OS

Operations and documentation

Validation

Uh oh!

Uh oh!

Uh oh!

kvinwang commented Jul 14, 2026

Simplification pass (operator decisions)

Uh oh!

kvinwang commented Jul 14, 2026

Follow-up commit ca7909f4

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

kvinwang commented Jul 16, 2026

Uh oh!

Copilot AI commented Jul 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

h4x3rotab commented Jul 6, 2026 •

edited by kvinwang

Loading

Follow-up commit `ca7909f4`