S3 BucketLifecycleConfigurationV2RuleArgs Expiration is not deleted when set to nil

[cooloncoolon]

Describe what happened

we are implementing S3 Lifecycle policy for our S3 buckets and noticed that all lifecycle policy configuration works as expected except "Expiration" when 'nil' is passed to this field. We are expecting that the Expiration configuration is removed from the rule configuration but the preview shows no objects to be changed. At the same time the "Expiration" works when creating or updating(ex. changing number of days).

we have the same configuration for the "NoncurrentVersionExpiration" but in this case the configuration block works as expected - created, updated and removed.

moreover the "Expiration" configuration is removed if there are another rule changes at the same time. but it is removed silently, preview does not shows any diff for the "Expiration" configuration.

Sample program

type LifecyclePolicyRulesArgs struct {
	RuleIdSuffix                                 string
	RuleStatus                                   string
	ObjectsFilterOptions                         ObjectsFilterOptionsArgs
	ObjectsIncompleteMultipartUploadAbortOptions *ObjectsIncompleteMultipartUploadAbortOptionsArgs
	ObjectsTransitionRulesOptions                ObjectsTransitionRulesOptionsArgs
	ObjectsExpirationRuleOptions                 *ObjectsExpirationRuleOptionsArgs
	NonCurrentVersionsTransitionRulesOptions     NonCurrentVersionsTransitionRulesOptionsArgs
	NonCurrentVersionsExpirationRuleOptions      *NonCurrentVersionsExpirationRuleOptionsArgs
}

// S3 Bucket Lifecycle Rules configuration - Objects expiration
type ObjectsExpirationRuleOptionsArgs struct {
	AfterDays                  int
	AfterDate                  string
	IsExpireObjectDeleteMarker bool
}

// S3 Bucket Lifecycle Rules configuration - Noncurrent Versions expiration
type NonCurrentVersionsExpirationRuleOptionsArgs struct {
	AfterDays          int
	RetainNonCurVerQty string
}

func (args *BucketArgs) createS3LifecycleConfig(ctx *pulumi.Context) error {
	
	// Convert S3 Lyfecycle Rules configuration to s3.BucketLifecycleRule format
	var s3LifecyclePolicyRules s3.BucketLifecycleConfigurationV2RuleArray
	for _, lifecycleRules := range args.LifecyclePolicyOptions.Rules {

		// Parse and convert S3 Objects Expiration Rules configuration
		var s3ObjectsExpirations *s3.BucketLifecycleConfigurationV2RuleExpirationArgs
		if lifecycleRules.ObjectsExpirationRuleOptions != nil {
			s3ObjectsExpirations = &s3.BucketLifecycleConfigurationV2RuleExpirationArgs{
				Days:                      pulumi.Int(lifecycleRules.ObjectsExpirationRuleOptions.AfterDays),
				ExpiredObjectDeleteMarker: pulumi.Bool(lifecycleRules.ObjectsExpirationRuleOptions.IsExpireObjectDeleteMarker),
			}
		}

		// Build S3 Lifecycle Policy Rule
		s3LifecyclePolicyRules = append(s3LifecyclePolicyRules, s3.BucketLifecycleConfigurationV2RuleArgs{
			Status:                         pulumi.String(lifecycleRules.RuleStatus),
			Id:                             pulumi.String("s3-lifecycle-rule-" + args.BucketName + "_" + lifecycleRules.RuleIdSuffix),
			Filter:                         s3ObjectsFilters,
			AbortIncompleteMultipartUpload: s3ObjectsIncompleteMultipartUploadAbort,
			Transitions:                    s3ObjectsTransitions,
			NoncurrentVersionTransitions:   s3NonCurVerTransitions,
			NoncurrentVersionExpiration:    s3NonCurVerExpirations,
			Expiration:                     s3ObjectsExpirations,
		})
	}
	// Create S3 Lifecycle Policy Rule
	_, err := s3.NewBucketLifecycleConfigurationV2(ctx, "s3-lifecycle-policy-"+args.BucketName, &s3.BucketLifecycleConfigurationV2Args{
		Bucket: args.internal.Bucket.ID(),
		Rules:  s3LifecyclePolicyRules,
	})

	if err != nil {
		return err
	}
	return nil
}

Log output

No response

Affected Resource(s)

s3.NewBucketLifecycleConfigurationV2 Expiration

Output of pulumi about

CLI
Version 3.132.0
Go Version go1.23.1
Go Compiler gc

Plugins
KIND NAME VERSION
resource aws 5.24.0
language go 3.132.0
resource kubernetes 3.30.2

Host
OS darwin
Version 14.6.1
Arch arm64

This project is written in go: executable='/opt/homebrew/bin/go' version='go version go1.23.1 darwin/arm64'

Dependencies:
NAME VERSION
github.com/lib/pq v1.10.9
github.com/pulumi/pulumi-aws/sdk/v5 v5.24.0
github.com/pulumi/pulumi-kubernetes/sdk/v3 v3.30.2
github.com/pulumi/pulumi/sdk/v3 v3.81.0
github.com/stretchr/testify v1.8.3

Additional context

lifecycle policy configuration declared as follows:

testing rule array

  lifecyclePolicyOptions:
    enabled: true
    rules:
    #rule №1
    #- ruleStatus: Enabled
      #ruleIdSuffix: "lifecycle_for_IncompleteMultipartUploadAbort"
      #objectsIncompleteMultipartUploadAbortOptions:
        #afterDays: 15
    #rule №2
    #- ruleStatus: Enabled
      #ruleIdSuffix: "lifecycle_for_ExpireObjectDeleteMarker"
      #objectsExpirationRuleOptions:
        #isExpireObjectDeleteMarker: true
    #rule №3
    #- ruleStatus: Enabled
      #ruleIdSuffix: "lifecycle_for_expire_tag_logs"
      #objectsFilterOptions:
        #tags:
          #rule: "log"
          #autoclean: "true"
      #objectsExpirationRuleOptions:
        #afterDays: 90
    #rule №4
    #- ruleStatus: Enabled
      #ruleIdSuffix: "lifecycle_for_transition"
      #objectsFilterOptions:
        #prefix: "log/"
        #sizeGreaterThan: 132000
        #sizeLessThan: 133000
      #objectsTransitionRulesOptions:
        #rules:
        #- afterDays: 30 
          #afterDate: "2025-01-01T00:00:00Z"
          #toStorageClass: "STANDARD_IA"
        #- afterDays: 60
          #afterDate: "2026-01-02T00:00:00Z"
          #toStorageClass: "GLACIER"
    #rule №5
    #- ruleStatus: Enabled
      #ruleIdSuffix: "lifecycle_for_noncurrentversions"
      #nonCurrentVersionsTransitionRulesOptions:
        #rules:
        #- afterDays: 30 
          #toStorageClass: "STANDARD_IA"
          #retainNonCurVerQtyAfterTransition: "7"
        #- afterDays: 60
          #toStorageClass: "GLACIER"
          #retainNonCurVerQtyAfterTransition: "7"
      #nonCurrentVersionsExpirationRuleOptions:
          #retainNonCurVerQty: "7"
          #afterDays: 90    
    #rule №6
    - ruleStatus: Enabled
      ruleIdSuffix: "lifecycle_for_transition"
      #objectsIncompleteMultipartUploadAbortOptions:
        #afterDays: 15
      #objectsFilterOptions:
        #prefix: "log/"
        #sizeGreaterThan: 132000
        #sizeLessThan: 133000
        #tags:
          #rule_1: "log_1"
          #autoclean_1: "true_1"
        #tag:
          #key: "log_1"
          #value: "true_1"
      objectsTransitionRulesOptions:
        rules:
        - afterDays: 30 
          #afterDate: "2025-01-02T00:00:00Z"
          toStorageClass: "STANDARD_IA"
        - afterDays: 60
          #afterDate: "2026-01-03T00:00:00Z"
          toStorageClass: "GLACIER"
      #objectsExpirationRuleOptions:
        #afterDays: 90
      #nonCurrentVersionsTransitionRulesOptions:
        #rules:
        #- afterDays: 30 
          #toStorageClass: "STANDARD_IA"
          #retainNonCurVerQtyAfterTransition: "7"
        #- afterDays: 60
          #toStorageClass: "GLACIER"
          #retainNonCurVerQtyAfterTransition: "7"
      #nonCurrentVersionsExpirationRuleOptions:
        #retainNonCurVerQty: "7"
        #afterDays: 90
      
     testing rule array

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[corymhall]

@cooloncoolon it looks like from the about info you are on a previous major version v5. Can you try upgrading to v6 and see if you still have the same issue?

[cooloncoolon]

hi @corymhall ,
ill give a try v6 and get back with the results