feat(s3): support S3 Metadata configuration on buckets - #38469
Open
yasomaru wants to merge 1 commit into
Open
Conversation
Adds `BucketMetadataConfiguration`, a mixin that configures S3 Metadata on a bucket, plus a `metadataConfiguration` property on `BucketProps` that applies it. S3 Metadata captures object-level changes as a journal table, and optionally the current state of the bucket as an inventory table and custom business context as an annotation table, all stored as Apache Iceberg tables. Closes aws#38468
aws-cdk-automation
previously requested changes
Aug 1, 2026
aws-cdk-automation
dismissed
their stale review
August 1, 2026 11:12
✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.
aws-cdk-automation
temporarily deployed
to
automation
August 1, 2026 11:15 — with
GitHub Actions
Inactive
aws-cdk-automation
temporarily deployed
to
automation
August 1, 2026 11:15 — with
GitHub Actions
Inactive
lpizzinidev
suggested changes
Aug 5, 2026
| * | ||
| * @see https://docs.aws.amazon.com/AmazonS3/latest/userguide/metadata-tables-overview.html | ||
| */ | ||
| export interface MetadataConfiguration { |
| /** | ||
| * Whether an S3 Metadata table configuration is enabled or disabled. | ||
| */ | ||
| export enum MetadataConfigurationState { |
Contributor
There was a problem hiding this comment.
Can we express this as a boolean attribute? Example.
Same for MetadataRecordExpiration below.
| if (!expirationEnabled && days !== undefined) { | ||
| throw new ValidationError(lit`JournalTableRecordExpirationDisabled`, "'recordExpirationAfter' can only be specified when 'recordExpiration' is ENABLED", construct); | ||
| } | ||
| if (days !== undefined && !Token.isUnresolved(days) && (days < 1 || days > 2147483647)) { |
| * | ||
| * The journal table records the changes that are made to the objects in the bucket. | ||
| */ | ||
| export interface JournalTableConfiguration { |
Contributor
There was a problem hiding this comment.
All the CFN definitions seem to allow passing a customer table (via ARN and name). Why don't we support the feature here? Docs
| }, | ||
| inventoryTable: { | ||
| configurationState: s3.MetadataConfigurationState.ENABLED, | ||
| encryption: s3.MetadataTableEncryption.kms(key), |
Contributor
There was a problem hiding this comment.
Can we test one with aes256?
aws-cdk-automation
temporarily deployed
to
automation
August 5, 2026 14:01 — with
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue # (if applicable)
Closes #38468.
Reason for this change
S3 Metadata captures the metadata of the objects in a bucket as queryable Apache Iceberg tables: a journal table of every change made to the objects, an inventory table of their current state, and an annotation table holding the custom business context that S3 Annotations attach to objects.
AWS::S3::Buckethas supportedMetadataConfigurationfor a while, but nothing in the L2 reaches it, so users have to drop to the escape hatch and hand-write the raw CloudFormation shape:That loses everything the L2 normally provides: no typed values for
Expiration/ConfigurationState/SseAlgorithm, noDurationfor the retention period, raw ARN strings instead ofkms.IKeyandiam.IRoleRef, and no synth-time validation.Description of changes
Per the feature placement decision, this feature is about the bucket, extends its own L1 properties, and carries logic beyond a prop passthrough, so it is implemented as a mixin first and exposed through an L2 prop for convenience:
s3.mixins.BucketMetadataConfigurationholds all the logic — validation and rendering toCfnBucket.metadataConfiguration. It is applicable to L1 and L2 buckets alike.BucketProps.metadataConfigurationapplies that mixin viathis.with(...).Design decisions:
MetadataTableEncryption.s3Managed()/.kms(key)) rather than separatesseAlgorithm+kmsKeyArnprops. The L1 pair is only valid in two combinations, and folding them into factory methods makes the invalid ones unrepresentable while accepting akms.IKeyinstead of an ARN string.Durationfor record expiration instead of a raw day count, guarded withToken.isUnresolved()so a tokenized duration (e.g. from aCfnParameter) is not rejected by the range check.iam.IRoleReffor the annotation table role, since only the ARN is needed.journalTableconfigures it rather than enabling it, andmetadataConfiguration: {}is a valid minimal configuration.roleis validated as required when the annotation table is enabled. CloudFormation documentsAnnotationTableConfiguration.Roleas optional, but S3 rejects the stack withInvalid request provided: Role is required when AnnotationTableConfiguration ConfigurationState is ENABLED. This was found by deploying the integration test. Because the input was accepted at synth but always failed at deploy, this is a fail-fast synth-time validation and does not need a feature flag.Alternatives considered and rejected:
BucketPropsonly, matching how the olderversionedandinventoriesprops render straight to the L1 in theBucketconstructor. Rejected because new features must be building blocks first, and a mixin also serves L1 users.MetadataTableConfigurationin the same PR. Rejected to keep one concern per PR: V1 stores its tables in a customer-managed table bucket, andITableBucketcurrently only exists in the alphaaws-s3tables-alphamodule, whichaws-cdk-libcannot depend on. Using the generatedITableBucketRefinstead is a separate design decision. This PR therefore addresses the V2 configuration from (aws-s3): support S3 Metadata (metadataConfiguration / metadataTableConfiguration) on the Bucket L2 #38468; V1 remains open.No breaking changes: every new property is optional and additive.
Describe any new or updated permissions being added
None. This change does not create or modify any IAM policy.
Two permission-related notes for users, both documented in the README rather than wired automatically:
kms:Decryptandkms:GenerateDataKeyfor themetadata.s3.amazonaws.comandmaintenance.s3tables.amazonaws.comservice principals. The construct does not modify the key policy, because the key is frequently shared and imported.metadata.s3.amazonaws.com.Description of how you validated changes
Unit tests — 15 new tests. In
bucket.test.ts: absent by default, the minimal{}configuration, record expiration, inventory and annotation tables, both encryption modes, and every validation failure (missing/superfluous retention, out-of-range retention viatest.each, annotation table without a role) plus a tokenizedDurationpassing the range check. Inmixins.test.ts: the mixin applied directly to aCfnBucket, its validation, andsupports()returning false for a non-S3 construct. The wholeaws-s3suite passes (363 tests).Integration test —
integ.bucket-metadata-configuration.tscovers a bucket with only a journal table and a bucket with all three tables, KMS encryption and a 7 day retention. It was deployed to us-east-1 and the assertions verified the configuration by callingGetBucketMetadataConfigurationagainst the live buckets.Deploying is what surfaced the required-role behaviour described above; the first deploy failed on it. Two notes on the test itself:
s3:GetBucketMetadataTableConfigurationgrant, because that is the IAM action name for both the V1 and V2 API operations and it does not match the V2 API name the assertion derives its grant from.AnnotationTableConfigurationResultis deliberately not asserted: the assertion provider's SDK did not return that key. The annotation table is still exercised, since S3 rejects the stack outright if its configuration is invalid.Other —
yarn buildforaws-cdk-libpasses, including awslint and eslint. The README examples compile under Rosetta.Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license