MediaConvert CreateJob failing with required property error

[2rebi]

Describe the bug

Link for a similar case to this problem.

Expected Behavior

I want to write 0 value in MinSegmentLength field of the types.HlsGroupSettings

Current Behavior

operation error MediaConvert: CreateJob, https response error StatusCode: 400, RequestID: 8b703b69-cd87-459b-964e-354dedf42392, BadRequestException: /outputGroups/2/outputGroupSettings/hlsGroupSettings: minSegmentLength is a required property

Reproduction Steps

queueName := "queue-name"
roleName := "roleName"

destination := "s3://bucket-name/sample"
inputFile := "s3://bucket-name/file.mp4"

ctx := context.Background()
resp, err := mc.CreateJob(ctx, &mediaconvert.CreateJobInput{

	Queue: aws.String(queueName),
	Role:  aws.String(roleName),
	Settings: &types.JobSettings{
		OutputGroups: []types.OutputGroup{
			{
				Name: aws.String("Apple HLS"),
				OutputGroupSettings: &types.OutputGroupSettings{
					Type: types.OutputGroupTypeHlsGroupSettings,
					HlsGroupSettings: &types.HlsGroupSettings{
						SegmentLength:		5,
						SegmentLengthControl:	types.HlsSegmentLengthControlExact,
						Destination:		aws.String(destination),
						MinSegmentLength:	0, // =========== error point =============
						SegmentControl:		types.HlsSegmentControlSegmentedFiles,
					},
				},
				Outputs: []types.Output{
					{
						ContainerSettings: &types.ContainerSettings{
							Container: types.ContainerTypeM3u8,
						},
						VideoDescription: &types.VideoDescription{
							Height: 1280,
							Width:  720,
							CodecSettings: &types.VideoCodecSettings{
								Codec: types.VideoCodecH264,
								H264Settings: &types.H264Settings{
									InterlaceMode:			types.H264InterlaceModeProgressive,
									FramerateDenominator:		1001,
									MaxBitrate:			13000000,
									Bitrate:			6500000,
									FramerateControl:		types.H264FramerateControlSpecified,
									RateControlMode:		types.H264RateControlModeVbr,
									Telecine:			types.H264TelecineNone,
									FramerateNumerator:		30000,
									QualityTuningLevel:		types.H264QualityTuningLevelSinglePassHq,
									FramerateConversionAlgorithm:	types.H264FramerateConversionAlgorithmDuplicateDrop,
								},
							},
						},
						AudioDescriptions: []types.AudioDescription{
							{
								CodecSettings: &types.AudioCodecSettings{
									Codec: types.AudioCodecAac,
									AacSettings: &types.AacSettings{
										Bitrate:    96000,
										CodingMode: types.AacCodingModeCodingMode20,
										SampleRate: 48000,
									},
								},
							},
						},
						NameModifier: aws.String(".720p"),
					},
				},
			},
		},

		Inputs: []types.Input{
			{
				AudioSelectors: map[string]types.AudioSelector{
					"Audio Selector 1": {
						DefaultSelection: types.AudioDefaultSelectionDefault,
					},
				},
				TimecodeSource: types.InputTimecodeSourceZerobased,
				FileInput:      aws.String(inputFile),
			},
		},
		TimecodeConfig: &types.TimecodeConfig{
			Source: types.TimecodeSourceZerobased,
		},
	},
	AccelerationSettings: &types.AccelerationSettings{Mode: types.AccelerationModeDisabled},
	StatusUpdateInterval: types.StatusUpdateIntervalSeconds60,
	Priority:             0,
})

Possible Solution

No response

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

module awesomeProject8

go 1.17

require (
	github.com/aws/aws-sdk-go-v2 v1.16.11
	github.com/aws/aws-sdk-go-v2/credentials v1.12.11
	github.com/aws/aws-sdk-go-v2/service/s3 v1.27.3
)

require (
	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.3 // indirect
	github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.18 // indirect
	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.12 // indirect
	github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.7 // indirect
	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.3 // indirect
	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.11 // indirect
	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.10 // indirect
	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.13.10 // indirect
	github.com/aws/aws-sdk-go-v2/service/mediaconvert v1.25.7 // indirect
	github.com/aws/smithy-go v1.12.1 // indirect
	github.com/google/uuid v1.3.0 // indirect
)

Compiler and Version used

go version go1.17.11 darwin/arm64

Operating System and version

osx 12.4

[RanVaknin]

Hi @RebirthLee ,

Sorry for the super long response time. This took some time to investigate.
We are working on a fix and I will let you know as soon as this is fixed.

Thanks,
Ran

[crazygit]

any feedback now?

[thomas-johansson]

Hitting the same issue.

[ikovalyov]

Same issue

[RanVaknin]

Hi all,

Sorry for the delay. this issue fell off the map.

There are many issues with how the MediaConvert Service team annotated their model. Specifically around serialization of default value 0.
Because 0 is a default value of an integer, without the proper API model annotation, the Go SDK serializer would treat this as an empty value, causing this to not get sent in the request.

While we try to contact the service team for a fix, you can use the following workaround by adding middleware that edits the body raw body with the desired parameter:

myMiddleware := middleware.SerializeMiddlewareFunc("MyTestMiddleware",
		func(ctx context.Context, input middleware.SerializeInput, next middleware.SerializeHandler,
		) (
			output middleware.SerializeOutput, metadata middleware.Metadata, err error,
		) {
			req, ok := input.Request.(*smithyhttp.Request)
			if !ok {
				return output, metadata, fmt.Errorf("unexpected transport: %T", input.Request)
			}

			bodyStream := req.GetStream()
			buf := new(strings.Builder)
			_, err = io.Copy(buf, bodyStream)
			actualBodyString := buf.String()

			// the location after which we want to insert the parameter.
			substring := `hlsGroupSettings":{`
			index := strings.Index(actualBodyString, substring)

			newBodyString := actualBodyString[:index+len(substring)] + `"MinSegmentLength":0,` + actualBodyString[len(substring)+index:]
			newStream := strings.NewReader(newBodyString)

			req.ContentLength = int64(len(newBodyString))
			input.Request = req

			input.Request, err = req.SetStream(newStream)
			if err != nil {
				panic(err)
			}

			return next.HandleSerialize(ctx, input)
		})

	cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithEndpointResolverWithOptions(customResolver), config.WithClientLogMode(aws.LogRequestWithBody|aws.LogResponseWithBody))
	if err != nil {
		log.Fatalf("unable to load SDK config, %v", err)
	}

	client := mediaconvert.NewFromConfig(cfg, func(options *mediaconvert.Options) {
		options.APIOptions = append(options.APIOptions, func(stack *middleware.Stack) error {
			return stack.Serialize.Add(myMiddleware, middleware.After)
		})

	})

Will result in the following body to be written:

{
  "accelerationSettings": {
    "mode": "DISABLED"
  },
  "clientRequestToken": "04311a67-a7c0-41ed-a7a0-33adae795976",
  "queue": "queue-name",
  "role": "roleName",
  "settings": {
    "inputs": [
      {
        "audioSelectors": {
          "Audio Selector 1": {
            "defaultSelection": "DEFAULT"
          }
        },
        "fileInput": "s3://bucket-name/file.mp4",
        "timecodeSource": "ZEROBASED"
      }
    ],
    "outputGroups": [
      {
        "name": "Apple HLS",
        "outputGroupSettings": {
          "hlsGroupSettings": {
            "MinSegmentLength": 0, 
            "destination": "s3://bucket-name/sample",
            "segmentControl": "SEGMENTED_FILES",
            "segmentLength": 5,
            "segmentLengthControl": "EXACT"
          },
          "type": "HLS_GROUP_SETTINGS"
        },
        "outputs": [
          {
            "audioDescriptions": [
              {
                "codecSettings": {
                  "aacSettings": {
                    "bitrate": 96000,
                    "codingMode": "CODING_MODE_2_0",
                    "sampleRate": 48000
                  },
                  "codec": "AAC"
                }
              }
            ],
            "containerSettings": {
              "container": "M3U8"
            },
            "nameModifier": ".720p",
            "videoDescription": {
              "codecSettings": {
                "codec": "H_264",
                "h264Settings": {
                  "bitrate": 6500000,
                  "framerateControl": "SPECIFIED",
                  "framerateConversionAlgorithm": "DUPLICATE_DROP",
                  "framerateDenominator": 1001,
                  "framerateNumerator": 30000,
                  "interlaceMode": "PROGRESSIVE",
                  "maxBitrate": 13000000,
                  "qualityTuningLevel": "SINGLE_PASS_HQ",
                  "rateControlMode": "VBR",
                  "telecine": "NONE"
                }
              },
              "height": 1280,
              "width": 720
            }
          }
        ]
      }
    ],
    "timecodeConfig": {
      "source": "ZEROBASED"
    }
  },
  "statusUpdateInterval": "SECONDS_60"
}

Thanks,
Ran~

[RanVaknin]

I have opened a ticket with the service team to address this
P96572828

[RanVaknin]

This is now fixed 👍

[github-actions]

This issue is now closed.

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.