Cannot create S3 bucket inventory with SSE-S3 encryption

[danielwhite]

Version of AWS SDK for Go?

• 1.14.8
• v1.14.15-1-g0625b326

Version of Go (go version)?

$ go version
go version go1.10 darwin/amd64

What issue did you see?

The SDK produces a malformed request when configuring S3 bucket inventory where the destination is using SSE-S3 encryption.

Given the following s3.PutBucketInventoryConfigurationInput:

{
  Bucket: "example-inventory-bucket",
  Id: "SSE-S3 Encryption",
  InventoryConfiguration: {
    Destination: {
      S3BucketDestination: {
        Bucket: "aws:s3:::example-inventory-bucket",
        Encryption: {
          SSES3: {

          }
        },
        Format: "ORC"
      }
    },
    Id: "SSE-S3 Encryption",
    IncludedObjectVersions: "All",
    IsEnabled: true,
    Schedule: {
      Frequency: "Daily"
    }
  }
}

The operation produces the following error:

2018/06/29 02:33:43 DEBUG: Request s3/PutBucketInventoryConfiguration Details:
---[ REQUEST POST-SIGN ]-----------------------------
PUT /?id=SSE-S3%20Encryption&inventory= HTTP/1.1
[ ... ]
<InventoryConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Schedule><Frequency>Daily</Frequency></Schedule><Destination><S3BucketDestination><Bucket>aws:s3:::example-inventory-bucket</Bucket><Encryption></Encryption><Format>ORC</Format></S3BucketDestination></Destination><Id>SSE-S3 Encryption</Id><IncludedObjectVersions>All</IncludedObjectVersions><IsEnabled>true</IsEnabled></InventoryConfiguration>
-----------------------------------------------------
2018/06/29 02:33:49 DEBUG: Response s3/PutBucketInventoryConfiguration Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 400 Bad Request
[ ... ]
-----------------------------------------------------
2018/06/29 02:33:49 <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedXML</Code><Message>The XML you provided was not well-formed or did not validate against our published schema</Message><RequestId>8D2C362D0151D051</RequestId><HostId>vBEciPKXOl72I+Uj3A2Qhn8T56Clm5S4I9XhmskMAKuw1ayBhDYAHbFqHi7CizUORMa1mp2J+FU=</HostId></Error>

At first glance, the main offender seems to be a check in xmlutil.buildStruct to prevent marshaling of structs where no child fields have been marshaled.

aws-sdk-go/private/protocol/xml/xmlutil/build.go

Lines 156 to 158 in 0625b32

	if fieldAdded { // only append this child if we have one ore more valid members
	current.AddChild(child)
	}

Steps to reproduce

The following will produce the error described above:

package main

import (
	"flag"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/awserr"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
)

func main() {
	bucket := flag.String("bucket", "example-inventory-bucket", "Name of bucket to create")
	flag.Parse()

	sess := session.Must(session.NewSessionWithOptions(session.Options{
		SharedConfigState: session.SharedConfigEnable,
	}))
	cfg := aws.NewConfig().WithLogLevel(aws.LogDebugWithHTTPBody)
	client := s3.New(sess, cfg)

	// Set up a bucket to use as an S3 inventory source and destination.
	if _, err := client.CreateBucket(&s3.CreateBucketInput{
		Bucket: bucket,
		ACL:    aws.String(s3.BucketCannedACLPrivate),
	}); err != nil {
		if aerr, ok := err.(awserr.Error); ok && aerr.Code() == s3.ErrCodeBucketAlreadyOwnedByYou {
			// second run
		} else {
			panic(err)
		}
	}

	if err := client.WaitUntilBucketExists(&s3.HeadBucketInput{
		Bucket: bucket,
	}); err != nil {
		panic(err)
	}

	// Configure inventory to use SSE-S3 encryption when writing.
	input := &s3.PutBucketInventoryConfigurationInput{
		Bucket: bucket,
		Id:     aws.String("SSE-S3 Encryption"),
		InventoryConfiguration: &s3.InventoryConfiguration{
			Id:                     aws.String("SSE-S3 Encryption"),
			IsEnabled:              aws.Bool(true),
			IncludedObjectVersions: aws.String(s3.InventoryIncludedObjectVersionsAll),
			Schedule: &s3.InventorySchedule{
				Frequency: aws.String(s3.InventoryFrequencyDaily),
			},
			Destination: &s3.InventoryDestination{
				S3BucketDestination: &s3.InventoryS3BucketDestination{
					Format: aws.String(s3.InventoryFormatOrc),
					Bucket: aws.String("aws:s3:::" + *bucket),
					Encryption: &s3.InventoryEncryption{
						SSES3: &s3.SSES3{}, // This is not marshaled.
					},
				},
			},
		},
	}

	log.Printf("%#v", input)

	if _, err := client.PutBucketInventoryConfiguration(input); err != nil {
		panic(err)
	}
}

[jasdel]

Thanks for reporting this issue @danielwhite I think you're correct the SDK's XML marshaler is intentionally excluding elements from the XML document if that structure does not have nested values. I know the SDK's JSON marshaler are able to handle this case to marshaler empty objects.

PR #1998 adds additional tests from XML marshaling that we can use for validating this functionality. The SDK's xml marshaler is pretty fragile, additional tests need to be included with the fix for this.