Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Add sam publish command design doc #844

Merged
merged 23 commits into from
Jan 8, 2019
Merged
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
297 changes: 297 additions & 0 deletions designs/sam_publish_app_cmd.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,297 @@
.. contents:: **Table of Contents**
:depth: 2
:local:

``sam publish app`` command
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we ever expecting a different type to publish?

If not, perhaps the command structure should be sam app <publish|remove|foo|bar>

This would the main change I would implement for this doc. It would give a more structured feel and would pave the way to richer functionality in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As Jacob has commented, the command itself should be an action instead of an entity to remain consistent with other sam commands.

====================================

This is the design for a command to publish an application to `AWS Serverless Application Repository (SAR)`_ with a SAM
template. It can be used to create a new application w/ its first version, update existing application's metadata, and
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be something we could auto suggest, or implement flags for --bump-minor/patch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We thought about doing this like npm version, but this is out of scope for publish currently.

create new versions of the application.

.. _AWS Serverless Application Repository (SAR): https://aws.amazon.com/serverless/serverlessrepo/


What is the problem?
--------------------
To publish an app to AWS Serverless Application Repository, customers need to go through the following steps: first upload
the application code and SAM template to an Amazon S3 bucket, correctly set S3 bucket policy that grants the service read
permissions for artifacts uploaded to S3, then open the AWS Serverless Application Repository console and provide information
in a bunch of input boxes. If they use the AWS CLI, they need to pass all the information as parameters, and it's easy to make
a mistake while typing in the command line.


What will be changed?
---------------------
In this proposal, we will be providing a new command, ``sam publish app``, which takes a SAM template as input and publishes
paoptu023 marked this conversation as resolved.
Show resolved Hide resolved
an application to AWS Serverless Application Repository using applicaiton metadata specified in the template. Customers
need to provide application metadata information in the template, then ``sam package`` will handle uploading local files to S3,
and ``sam publish app`` will create the app in Serverless Application Repository.
paoptu023 marked this conversation as resolved.
Show resolved Hide resolved


Success criteria for the change
-------------------------------
#. Support all the following use cases:

* Create new application w/ its first version in SAR using ``sam publish app``
* Create new version of existing SAR application using ``sam publish app``
* Update application metadata of existing SAR application using ``sam publish app``

#. ``sam package`` command can upload local readme/license files to S3.


Out-of-Scope
------------
#. Manage application permissions while publishing the app.

#. Recursively publish nested apps in the template (SAR CreateApplication API doesn't support yet).
paoptu023 marked this conversation as resolved.
Show resolved Hide resolved

#. Run through CI/CD pipeline for the application before publishing.

#. Publish to other repositories besides SAR.

#. Recognize template changes and suggest version number.

#. Publish appication if ``AWS::ServerlessRepo::Application`` section is not found in the template's ``Metadata`` section.

.. _here: https://docs.aws.amazon.com/serverlessrepo/latest/devguide/access-control-resource-based.html#access-control-resource-based-example-multiple-permissions


User Experience Walkthrough
---------------------------

Assuming that customers have the following SAM template:

.. code-block:: yaml

Metadata:
AWS::ServerlessRepo::Application:
Name: my-app
Description: hello world
Author: user1
SpdxLicenseId: Apache-2.0
LicenseUrl: ./LICENSE.txt
ReadmeUrl: ./README.md
Labels: ['tests']
HomepageUrl: https://github.com/user1/my-app-project
SemanticVersion: 0.0.1
SourceCodeUrl: https://github.com/user1/my-app-project

Resources:
HelloWorldFunction:
Type: AWS::Lambda::Function
Properties:
...
CodeUri: ./source-code1
...

Build Lambda source code
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be something sam app publish could do out of the box. Detect the last build date or diff the prebuild app in $PROJECT_DIR/.aws-sam/build against the $PROJECT_DIR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of scope for this command currently. This should be handled when we address this in other commands. Currently no command 'auto builds'.

Run ``sam build -t template.yaml -b ./build -o built-template.yaml`` to build all functions in the template and output
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the future: sam app publish could trigger a build if there's none so one could have the following ideal workflow:

  • sam init
  • make changes to SAM and define a app locally
  • sam deploy and test whether that works
  • sam publish

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is improvement outside of this command scope and something we are thinking about wider (invoke, package, etc) doing this build if not there.

a SAM template that can be run through the package command.

Package built artifacts and local file references
Run ``sam package --template-file built-template.yaml --output-template-file packaged.yaml --s3-bucket my-bucket``
to upload code artifacts, readme and license files to S3 and generate the packaged template.

Create new application in SAR
Run ``sam publish app -t ./packaged.yaml`` to publish a new application named my-app in SAR with the first version
paoptu023 marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be great to have packaged.yaml as a default similarly to sam deploy does it today as it's less to remember and more straightforward from a CLI UX :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should cover this when doing the migration of package.

Note: sam deploy doesn't have a default value for the template Option.

created as 0.0.1. If no permission option is passed, the app will be created as private by default.
paoptu023 marked this conversation as resolved.
Show resolved Hide resolved

SAM CLI prints application created message, metadata used to create application and link to the console details page.

>>> sam publish app -t ./packaged.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For integration purposes, it would be cool if we printed a sample CFN resource that a user could copy & paste.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, please see comment above: #844 (comment)

Publish Succeeded
Created new application with the following metadata:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of giving metadata of the published app (e.g. sam app info instead) it'd be great to leave a AWS::Serverless::Application resource filled out with the information about this app newly published so one can go copy/paste into somewhere (slack, template, etc.).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion! This can be added in a follow-up iteration to output CFN resource of the published app.

{
"Name": "my-app",
"Description": "hello world",
"Author": "user1",
"SpdxLicenseId": "Apache-2.0",
"LicenseUrl": "s3://test/LICENSE.txt",
"ReadmeUrl": "s3://test/README.md",
"Labels": ['tests'],
"HomepageUrl": "https://github.com/user1/my-app-project",
"SemanticVersion": "0.0.1",
"SourceCodeUrl": "https://github.com/user1/my-app-project"
}
Click the link below to view your application in AWS console:
https://console.aws.amazon.com/serverlessrepo/home?region=<region>#/published-applications/<arn>

Create new version of an existing SAR application
Modify the existing template, change SemanticVersion to 0.0.2, and run ``sam publish app -t ./packaged.yaml`` again.

SAM CLI prints application metadata updated message and link to the console details page. If no permission option
is passed, the application's permission remains the same.

>>> sam publish app -t ./packaged.yaml
Publish Succeeded
The following metadata of application <id> has been updated:
{
"Author": "user1",
"Description": "description",
"ReadmeUrl": "s3://test/README.md",
...
"SemanticVersion": "0.0.2",
"SourceCodeUrl": "https://github.com/hello"
}
Click the link below to view your application in AWS console:
https://console.aws.amazon.com/serverlessrepo/home?region=<region>#/published-applications/<arn>

Update the metadata of an existing application without creating new version
Keep SemanticVersion unchanged, then modify metadata fields like Description or ReadmeUrl, and run
``sam publish app -t ./packaged.yaml``. SAM CLI prints application metadata updated message, values of the current
application metadata and link to the console details page.

>>> sam publish app -t ./packaged.yaml
Publish Succeeded
The following metadata of application <id> has been updated:
{
"Author": "qwang",
"Description": "description",
"ReadmeUrl": "s3://test/README.md"
...
}
Click the link below to view your application in AWS console:
https://console.aws.amazon.com/serverlessrepo/home?region=<region>#/published-applications/<arn>

Once the application is published, other developers in your team or your organization will be able to deploy it with a few
clicks. If the application is shared publicly, the whole community will be able to find it by visiting the AWS Serverless
Application Repository `public site`_.

.. _public site: https://serverlessrepo.aws.amazon.com/applications


Implementation
==============

CLI Changes
-----------
*Explain the changes to command line interface, including adding new commands, modifying arguments etc*

1. Add a new top-level command called ``sam publish app`` with the following help message.

.. code-block:: text

Usage: sam publish app [OPTIONS]

Use this command to publish a packaged AWS SAM template to the AWS
Serverless Application Repository to share within your team, across your
organization, or with the community at large.

This command expects the template's Metadata section to contain an
AWS::ServerlessRepo::Application section with application metadata
for publishing. For more details on this metadata section, see
https://docs.aws.amazon.com/serverlessrepo/latest/devguide/serverless-app-publishing-applications.html

Examples
--------
To publish an application
$ sam publish app -t packaged.yaml --region <region>
paoptu023 marked this conversation as resolved.
Show resolved Hide resolved

Options:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add the ability to share account IDs or chose to make the app completely public?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment above: #844 (comment)

-t, --template PATH AWS SAM template file [default: template.[yaml|yml]]
--profile TEXT Select a specific profile from your credential file to
get AWS credentials.
--region TEXT Set the AWS Region of the service (e.g. us-east-1).
--debug Turn on debug logging to print debug message generated
by SAM CLI.
--help Show this message and exit.

2. Update ``sam package`` (``aws cloudformation package``) command to support uploading locally referenced readme and
license files to S3.

Breaking Change
~~~~~~~~~~~~~~~
*Are there any breaking changes to CLI interface? Explain*

N/A

Design
------
*Explain how this feature will be implemented. Highlight the components of your implementation, relationships*
*between components, constraints, etc.*

SAM CLI will read the packaged SAM template and pass it as string to `aws-serverlessrepo-python <https://github.com/awslabs/aws-serverlessrepo-python>`_
library. The algorithm for ``sam publish app -t ./packaged.yaml`` looks like this:

.. code-block:: python

from serverlessrepo import publish_application

with open('./packaged.yaml', 'r') as f:
template = f.read()
result = publish_application(template)


``.samrc`` Changes
------------------
*Explain the new configuration entries, if any, you want to add to .samrc*

N/A

Security
--------

*Tip: How does this change impact security? Answer the following questions to help answer this question better:*

**What new dependencies (libraries/cli) does this change require?**

A new dependency `aws-serverlessrepo-python <https://github.com/awslabs/aws-serverlessrepo-python>`_ will be added to interact with SAR.

**What other Docker container images are you using?**

N/A

**Are you creating a new HTTP endpoint? If so explain how it will be created & used**

N/A

**Are you connecting to a remote API? If so explain how is this connection secured**

Will be connecting to boto3 serverlessrepo `create_application`_, `update_application`_, `create_application_version`_ APIs through
the `aws-serverlessrepo-python <https://github.com/awslabs/aws-serverlessrepo-python>`_ library. The connection is secured by requiring
AWS credentials and permissions for the target application.

.. _create_application : https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/serverlessrepo.html#ServerlessApplicationRepository.Client.create_application
.. _update_application : https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/serverlessrepo.html#ServerlessApplicationRepository.Client.update_application
.. _create_application_version: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/serverlessrepo.html#ServerlessApplicationRepository.Client.create_application_version


**Are you reading/writing to a temporary folder? If so, what is this used for and when do you clean up?**

N/A

**How do you validate new .samrc configuration?**

N/A

Documentation Changes
---------------------

#. Add "AWS::ServerlessRepo::Application" sepc in `Publishing Applications`_ guide and document how to use ``sam publish app``.

#. Add ``ReadmeUrl`` and ``LicenseUrl`` in `aws cloudformation package`_ documentation.

#. Add ``sam publish app`` in `AWS SAM CLI Command Reference`_, and explain the command, usage, examples, options.

#. Add a quick start guide "Publishing your application to AWS Serverless Application Repository" under SAM CLI `Get Started`_.

.. _Publishing Applications: https://docs.aws.amazon.com/serverlessrepo/latest/devguide/serverless-app-publishing-applications.html
.. _aws cloudformation package: https://docs.aws.amazon.com/cli/latest/reference/cloudformation/package.html
.. _AWS SAM CLI Command Reference: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-command-reference.html
.. _Get Started: https://github.com/awslabs/aws-sam-cli#get-started

Open Issues
-----------

N/A

Task Breakdown
--------------
- [x] Send a Pull Request with this design document
- [ ] Build the command line interface
- [ ] Build the underlying library
- [ ] Unit tests
- [ ] Integration tests
- [ ] Run all tests on Windows
- [ ] Update documentation