Skip to content

feat(s3): support S3 Metadata configuration on buckets - #38469

Open
yasomaru wants to merge 1 commit into
aws:mainfrom
yasomaru:feat/issue-38468-s3-metadata-configuration
Open

feat(s3): support S3 Metadata configuration on buckets#38469
yasomaru wants to merge 1 commit into
aws:mainfrom
yasomaru:feat/issue-38468-s3-metadata-configuration

Conversation

@yasomaru

@yasomaru yasomaru commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Issue # (if applicable)

Closes #38468.

Reason for this change

S3 Metadata captures the metadata of the objects in a bucket as queryable Apache Iceberg tables: a journal table of every change made to the objects, an inventory table of their current state, and an annotation table holding the custom business context that S3 Annotations attach to objects.

AWS::S3::Bucket has supported MetadataConfiguration for a while, but nothing in the L2 reaches it, so users have to drop to the escape hatch and hand-write the raw CloudFormation shape:

const cfnBucket = bucket.node.defaultChild as s3.CfnBucket;
cfnBucket.metadataConfiguration = {
  journalTableConfiguration: {
    recordExpiration: { expiration: 'ENABLED', days: 10 },
    encryptionConfiguration: { sseAlgorithm: 'aws:kms', kmsKeyArn: key.keyArn },
  },
};

That loses everything the L2 normally provides: no typed values for Expiration / ConfigurationState / SseAlgorithm, no Duration for the retention period, raw ARN strings instead of kms.IKey and iam.IRoleRef, and no synth-time validation.

Description of changes

Per the feature placement decision, this feature is about the bucket, extends its own L1 properties, and carries logic beyond a prop passthrough, so it is implemented as a mixin first and exposed through an L2 prop for convenience:

  • s3.mixins.BucketMetadataConfiguration holds all the logic — validation and rendering to CfnBucket.metadataConfiguration. It is applicable to L1 and L2 buckets alike.
  • BucketProps.metadataConfiguration applies that mixin via this.with(...).

Design decisions:

  • Encryption is an enum-like class (MetadataTableEncryption.s3Managed() / .kms(key)) rather than separate sseAlgorithm + kmsKeyArn props. The L1 pair is only valid in two combinations, and folding them into factory methods makes the invalid ones unrepresentable while accepting a kms.IKey instead of an ARN string.
  • Duration for record expiration instead of a raw day count, guarded with Token.isUnresolved() so a tokenized duration (e.g. from a CfnParameter) is not rejected by the range check.
  • iam.IRoleRef for the annotation table role, since only the ARN is needed.
  • A journal table is always rendered. S3 always creates one for a V2 configuration, so journalTable configures it rather than enabling it, and metadataConfiguration: {} is a valid minimal configuration.
  • role is validated as required when the annotation table is enabled. CloudFormation documents AnnotationTableConfiguration.Role as optional, but S3 rejects the stack with Invalid request provided: Role is required when AnnotationTableConfiguration ConfigurationState is ENABLED. This was found by deploying the integration test. Because the input was accepted at synth but always failed at deploy, this is a fail-fast synth-time validation and does not need a feature flag.

Alternatives considered and rejected:

  • Adding the property directly to BucketProps only, matching how the older versioned and inventories props render straight to the L1 in the Bucket constructor. Rejected because new features must be building blocks first, and a mixin also serves L1 users.
  • Modelling V1 MetadataTableConfiguration in the same PR. Rejected to keep one concern per PR: V1 stores its tables in a customer-managed table bucket, and ITableBucket currently only exists in the alpha aws-s3tables-alpha module, which aws-cdk-lib cannot depend on. Using the generated ITableBucketRef instead is a separate design decision. This PR therefore addresses the V2 configuration from (aws-s3): support S3 Metadata (metadataConfiguration / metadataTableConfiguration) on the Bucket L2 #38468; V1 remains open.

No breaking changes: every new property is optional and additive.

Describe any new or updated permissions being added

None. This change does not create or modify any IAM policy.

Two permission-related notes for users, both documented in the README rather than wired automatically:

  • When the metadata tables are encrypted with a customer managed key, that key's policy must allow kms:Decrypt and kms:GenerateDataKey for the metadata.s3.amazonaws.com and maintenance.s3tables.amazonaws.com service principals. The construct does not modify the key policy, because the key is frequently shared and imported.
  • Enabling the annotation table requires the caller to supply a role assumable by metadata.s3.amazonaws.com.

Description of how you validated changes

Unit tests — 15 new tests. In bucket.test.ts: absent by default, the minimal {} configuration, record expiration, inventory and annotation tables, both encryption modes, and every validation failure (missing/superfluous retention, out-of-range retention via test.each, annotation table without a role) plus a tokenized Duration passing the range check. In mixins.test.ts: the mixin applied directly to a CfnBucket, its validation, and supports() returning false for a non-S3 construct. The whole aws-s3 suite passes (363 tests).

Integration testinteg.bucket-metadata-configuration.ts covers a bucket with only a journal table and a bucket with all three tables, KMS encryption and a 7 day retention. It was deployed to us-east-1 and the assertions verified the configuration by calling GetBucketMetadataConfiguration against the live buckets.

Deploying is what surfaced the required-role behaviour described above; the first deploy failed on it. Two notes on the test itself:

  • The assertion needs an explicit s3:GetBucketMetadataTableConfiguration grant, because that is the IAM action name for both the V1 and V2 API operations and it does not match the V2 API name the assertion derives its grant from.
  • AnnotationTableConfigurationResult is deliberately not asserted: the assertion provider's SDK did not return that key. The annotation table is still exercised, since S3 rejects the stack outright if its configuration is invalid.

Otheryarn build for aws-cdk-lib passes, including awslint and eslint. The README examples compile under Rosetta.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

Adds `BucketMetadataConfiguration`, a mixin that configures S3 Metadata on a
bucket, plus a `metadataConfiguration` property on `BucketProps` that applies it.

S3 Metadata captures object-level changes as a journal table, and optionally the
current state of the bucket as an inventory table and custom business context as
an annotation table, all stored as Apache Iceberg tables.

Closes aws#38468
@github-actions github-actions Bot added p2 valued-contributor [Pilot] contributed between 6-12 PRs to the CDK labels Aug 1, 2026
@aws-cdk-automation
aws-cdk-automation requested a review from a team August 1, 2026 10:06

@aws-cdk-automation aws-cdk-automation left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This review is outdated)

@yasomaru yasomaru changed the title feat(s3): S3 Metadata configuration on buckets feat(s3): support S3 Metadata configuration on buckets Aug 1, 2026
@aws-cdk-automation
aws-cdk-automation dismissed their stale review August 1, 2026 11:12

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 1, 2026

@lpizzinidev lpizzinidev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments, ty!

*
* @see https://docs.aws.amazon.com/AmazonS3/latest/userguide/metadata-tables-overview.html
*/
export interface MetadataConfiguration {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we missing the Destination attribute?

/**
* Whether an S3 Metadata table configuration is enabled or disabled.
*/
export enum MetadataConfigurationState {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we express this as a boolean attribute? Example.
Same for MetadataRecordExpiration below.

if (!expirationEnabled && days !== undefined) {
throw new ValidationError(lit`JournalTableRecordExpirationDisabled`, "'recordExpirationAfter' can only be specified when 'recordExpiration' is ENABLED", construct);
}
if (days !== undefined && !Token.isUnresolved(days) && (days < 1 || days > 2147483647)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minimum seems to be 7, see docs.

*
* The journal table records the changes that are made to the objects in the bucket.
*/
export interface JournalTableConfiguration {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the CFN definitions seem to allow passing a customer table (via ARN and name). Why don't we support the feature here? Docs

},
inventoryTable: {
configurationState: s3.MetadataConfigurationState.ENABLED,
encryption: s3.MetadataTableEncryption.kms(key),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we test one with aes256?

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

p2 valued-contributor [Pilot] contributed between 6-12 PRs to the CDK

Projects

None yet

Development

Successfully merging this pull request may close these issues.

(aws-s3): support S3 Metadata (metadataConfiguration / metadataTableConfiguration) on the Bucket L2

3 participants