SAM CLI depends on old version of aws-sam-translator

[beck3905]

Description

aws-sam-cli lists aws-sam-translator==1.10.0 in the requirements/base.txt however the latest version of aws-sam-translator is 1.13.1. There are new features available in the latest version of aws-sam-translator that fail to build using the latest version of aws-sam-cli.

For example, I have an AWS::Serverless::API resource in my template and I am trying to use the GatewayResponses property. When running SAM build I get an error during template validation.

samtranslator.model.exceptions.InvalidDocumentException: [InvalidResourceException('IdentityApi', 'property GatewayResponses not defined for resource of type AWS::Serverless::Api')]

This is a different symptom/reasoning than issue #1266

Steps to reproduce

1. Create a SAM template with an AWS::Serverless::API resource with GatewayResponses property
2. Run sam build

Observed result

$ sam build --use-container --skip-pull-image --debug
2019-08-09 09:54:55 Using SAM Template at [REDACTED]/sam-app/template.yaml
2019-08-09 09:54:55 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-08-09 09:54:55 Changing event name from before-call.apigateway to before-call.api-gateway
2019-08-09 09:54:55 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-08-09 09:54:55 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-08-09 09:54:55 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-08-09 09:54:55 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-08-09 09:54:55 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-08-09 09:54:55 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-08-09 09:54:55 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-08-09 09:54:55 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-08-09 09:54:55 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-08-09 09:54:55 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-08-09 09:54:55 Changing event name from before-call.apigateway to before-call.api-gateway
2019-08-09 09:54:55 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-08-09 09:54:55 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-08-09 09:54:55 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-08-09 09:54:55 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-08-09 09:54:55 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-08-09 09:54:55 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-08-09 09:54:55 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-08-09 09:54:55 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-08-09 09:54:55 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-08-09 09:54:55 Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2019-08-09 09:54:55 'build' command is called
2019-08-09 09:54:55 Starting Build inside a container
2019-08-09 09:54:55 Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 169, 'exitReason': 'InvalidSamDocumentException', 'exitCode': 255, 'requestId': '6a02f71c-7537-4b30-a24b-0c8c64de9abe', 'installationId': '981b2aa9-b253-41ac-b17a-1a7aaa054df5', 'sessionId': '6df8d61c-43a4-4feb-8555-3d292affe3fb', 'executionEnvironment': 'CLI', 'pyversion': '3.7.3', 'samcliVersion': '0.19.0'}}]}
2019-08-09 09:54:55 Starting new HTTPS connection (1): aws-serverless-tools-telemetry.us-west-2.amazonaws.com:443
2019-08-09 09:54:56 HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Traceback (most recent call last):
  File "[REDACTED]/lib/python3.7/site-packages/samcli/lib/samlib/wrapper.py", line 67, in run_plugins
    parser.parse(template_copy, all_plugins)  # parse() will run all configured plugins
  File "[REDACTED]/lib/python3.7/site-packages/samcli/lib/samlib/wrapper.py", line 134, in parse
    raise InvalidDocumentException(document_errors)
samtranslator.model.exceptions.InvalidDocumentException: [InvalidResourceException('IdentityApi', 'property GatewayResponses not defined for resource of type AWS::Serverless::Api')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "[REDACTED]/bin/sam", line 10, in <module>
    sys.exit(cli())
  File "[REDACTED]/lib/python3.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "[REDACTED]/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "[REDACTED]/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "[REDACTED]/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "[REDACTED]/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "[REDACTED]/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args[1:], **kwargs)
  File "[REDACTED]/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "[REDACTED]/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 94, in wrapped
    raise exception  # pylint: disable=raising-bad-type
  File "[REDACTED]/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 65, in wrapped
    return_value = func(*args, **kwargs)
  File "[REDACTED]/lib/python3.7/site-packages/samcli/commands/build/command.py", line 105, in cli
    skip_pull_image, parameter_overrides, mode)  # pragma: no cover
  File "[REDACTED]/lib/python3.7/site-packages/samcli/commands/build/command.py", line 138, in do_cli
    mode=mode) as ctx:
  File "[REDACTED]/lib/python3.7/site-packages/samcli/commands/build/build_context.py", line 65, in __enter__
    self._function_provider = SamFunctionProvider(self._template_dict, self._parameter_overrides)
  File "[REDACTED]/lib/python3.7/site-packages/samcli/commands/local/lib/sam_function_provider.py", line 46, in __init__
    self.template_dict = SamBaseProvider.get_template(template_dict, parameter_overrides)
  File "[REDACTED]/lib/python3.7/site-packages/samcli/commands/local/lib/sam_base_provider.py", line 62, in get_template
    template_dict = SamTranslatorWrapper(template_dict).run_plugins()
  File "[REDACTED]/lib/python3.7/site-packages/samcli/lib/samlib/wrapper.py", line 70, in run_plugins
    functools.reduce(lambda message, error: message + ' ' + str(error), e.causes, str(e)))
samcli.commands.validate.lib.exceptions.InvalidSamDocumentException: [InvalidResourceException('IdentityApi', 'property GatewayResponses not defined for resource of type AWS::Serverless::Api')] ('IdentityApi', 'property GatewayResponses not defined for resource of type AWS::Serverless::Api')

Expected result

Successful build of my sam template.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Mac OS 10.14.6
2. sam --version: SAM CLI, version 0.19.0

Add --debug flag to command you are running

[sriram-mv]

Currently pinned to version 1.11.0, will update to the latest in next release.

[benkehoe]

Being this far behind is causing me problems. I install both aws-sam-cli and cfn-lint as dev dependencies on a per-project basis using pipenv. cfn-lint now depends on the SAM translator being at least 1.13, and since aws-sam-cli pins the version at 1.11, there's an unresolvable conflict. Here's a Pipfile demonstrating the problem. Does the SAM CLI really need to pin the version? I could install it with pipx, I guess, but I prefer to have my Python tools localized to my projects.

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]
aws-sam-cli = "*"
cfn-lint = "*"

[requires]
python_version = "3.6"

[jfuss]

During the 0.21.0 release, we ran into an issue with one of our installers and the upgraded aws-sam-translator. Since then, we have addressed the issue and are staging this for a release (#1412 was the upgrade PR).

[jfuss]

v0.22.0 upgraded to 1.14.0 of the SAM Translator.

Closing