Impossible to install in Python3.5 on debian, even with pip

[rvolgers]

Description

It is not possible to install aws-sam-cli on Debian stable.

Steps to reproduce

$ pip3 install aws-sam-cli --user
Collecting aws-sam-cli
  Using cached https://files.pythonhosted.org/packages/c3/7c/79463add5772a98f044fab3024c27db61550bb1d9bbd06a715e3c3dc6b2a/aws_sam_cli-0.10.0-py3-none-any.whl
Collecting PyYAML~=3.12 (from aws-sam-cli)
  Downloading https://files.pythonhosted.org/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz (270kB)
    100% |████████████████████████████████| 276kB 2.7MB/s
Collecting serverlessrepo==0.1.5 (from aws-sam-cli)
  Could not find a version that satisfies the requirement serverlessrepo==0.1.5 (from aws-sam-cli) (from versions: 0.1.6, 0.1.7, 0.1.8)
No matching distribution found for serverlessrepo==0.1.5 (from aws-sam-cli)

Observed result

The above error message.

It turns out that aws-sam-cli dropped support for Python 3.5 in 0.11. So on Debian stable pip install aws-sam-cli will silently revert to trying to install 0.10, which for some reason isn't installable at all anymore.

This results in both a very obscure failure and the inability to use aws-sam-cli on a very common platform.

Expected result

Since Debian stable is a very common OS in general and a very common base OS for Docker images in particular I expected to be able to install aws-sam-cli.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

[rvolgers]

Okay it's a bit stranger. Support for 3.6+ was added here: #483

Before that all Python 3.x was supposed to be unsupported, so I don't quite understand why pip tries to install version 0.10.

It seems there was never a clear motivation for why only 3.6+ is supported. I am hopeful that this means support can be added without too much trouble.

[rvolgers]

These are the steps which I think would be required:

• Update the version requirement in setup.py
• Update the list of Python versions in tox.ini
• Update the list of Python versions in DEVELOPMENT_GUIDE.md
• Run tests according to the instructions in DEVELOPMENT_GUIDE.md and make sure they pass

I'm not sure if this is desired or not though.

[jfuss]

@rvolgers Thanks for the issue but Python3.5 has never been supported. The CLI has been installable in other versions but that was due to an issue in our setup.py (info can be found #966).

When we went to support Python3, we made the explicit decision to only support the newest Python3 version (3.6 at the time). In the ideal case, we will only need to support a single version of Python and customers never have to care what version (or even that it is written in Python). We are getting closer to this ideal world with our installers. For example, the MSI for windows we provide has Python packaged with it. This means windows users, can just install without needing to go download python or make sure they have the right version. For linux, we support Linux Brew and shortly will have a snap (not sure if we have it published yet but we did accept a contribution recently for snap support).

Installing with pip (even though it is not recommended), requires you to make sure you are installing in a supported version. Since this isn't the case, you should either install the CLI into a supported Python version with pip or use one of our installers.

Closing as Python3.5 is not supported.

[rvolgers]

Thanks for explaining your decision. For what it's worth I'm not a fan of this approach.

The aws-sam-cli tool is extremely highly privileged when used for deployments and depending on things like brew or snap or shipping huge binary dependencies that were built in unknown circumstances from uncertain sources means we now have to trust many more people and systems.

I realize it's still in beta and moving fast, but I was hoping to benefit from some of the ergonomic improvements around lambdas for some boring old software.

[jfuss]

@rvolgers You can still use pip just with supported versions, but problems that come about are then left to you (old setuptools, wheels, libraries conflicting with other projects you have installed, etc). Our goal with installers, are to make installations more reproducible and put the burden off customers.

The aws-sam-cli tool is extremely highly privileged when used for deployments and depending on things like brew or snap or shipping huge binary dependencies that were built in unknown circumstances from uncertain sources means we now have to trust many more people and systems.

Can you explain this further? Our brew and snap formulas are all open sourced, brew currently builds from source into a venv. Not sure what about our current approach makes you uneasy.

[wojtekk]

Simple installation on Linux is extremely important, without tools like brew - I'm trying to run SAM on Travis and this is really hard. I feel that this is the wrong approach :/

[rvolgers]

For what it's worth, I will no longer be following this issue since with the release of Debian Buster there is now a viable path forward for us that doesn't involve taking a dependency on another distribution channel like snap or brew.

I never answered the question why relying on more distribution channels makes me uneasy, because I feel like it is obvious and don't quite know where to begin explaining it. But here goes:

• I don't know anything about how they deal with transfers of control / maintainership
• I don't know anything about their social controls / vetting
• I don't know anything about their testing approach
• I don't know anything about their stability guarantees
• I don't know anything about who provides their bulk data hosting, and whether the packaging tools require them to be trusted
• I don't know anything about their packaging format and how signatures and hashes are validated
• I don't know anything about who provides, funds, and maintains their core infrastructure
• I don't know anything about their usage numbers and use cases. For example, they may get millions of downloads for e.g. OSX but only a few for Linux, which would put us into a much smaller niche of users than we are comfortable with.
• I am not yet familiar with their tooling and would have to take on the burden of adding it to our deployment scripts and becoming the maintainer for that integration.
• Our CI probably does not have integrated security update monitoring in place yet for it.

Especially in small teams you are asking someone to make a judgement call that all of this will be fine and will remain so for however long the dependency persists, even after the person who originally added it has moved on. The types and number of questions it raises is much greater than adding a dependency on a package in an already trusted distribution channel, something we also don't take lightly.