Tracking issue for AWS CLI v2

[jamesls]

Version 1.0.0 of the AWS CLI was tagged just over 5 years ago. In that time, a lot has changed. We've received tons of feedback from customers. We've learned what features customers enjoy, and what parts of the CLI we'd like to change. There are changes we've wanted to make, but needed to wait until the next major version of the AWS CLI due to either the backwards incompatible nature or the large scope of the change.

Well, we're here now. This issue is to track all the work for CLI v2. We wanted to do this work on GitHub right from the beginning. Even though we're in the early stages of development, community feedback will play an important role in its development.

• Any AWS CLI v2 related issue/PR will be labeled with v2.
• We'll also reference this issue for any AWS CLI v2 related PRs so everyone can see all the changes we're making.
• All the development work will be in the v2 branch.

If you have an idea for CLI v2, whether it's a new feature or a change we couldn't make in v1 of the AWS CLI, please feel free to open a GitHub issue and let us know. You can also go through any issue labeled as CLI v2 and +1 features that you'd like to see us implement.

Blog: https://aws.amazon.com/blogs/developer/aws-cli-v2-development/

[MarkRose]

Highest on our wish list would be to not require entering a 2fa code every hour when doing cross-account role-assumption despite the assumed role allowing for a longer session duration. Is there any chance #2748 can be looked at?

[jamesls]

Is there any chance #2748 can be looked at?

Yeah definitely. Reading over #2748 it looks like the main thing we needed was service side support for longer role durations, which has since happened (https://aws.amazon.com/about-aws/whats-new/2018/03/longer-role-sessions/). This should be possible now.

[MarkRose]

That's awesome to hear. That's the one complaint I frequently get after implementing a centralized auth account.

[tkelman]

Could the default setting on #2619 be flipped now in v2 so aws ecr get-login works with recent docker versions?

[dserodio]

You should get inspiration from awless, it feels like a CLI actually designed for humans :)

[AnthonyWC]

Would like to see more details on the actual/planned differences between version 1 and 2.

[007]

PLEASE fix #2507 since the only blocker was "backward compatibility"

[gregmarr]

@007 Most recent comment: "Hey everyone, just a quick update. While we can't remove this functionality in the current major version of the CLI, based on everyone's feedback, we plan on removing this for CLI v2: #3590" and it's already merged.

[jamesls]

@tkelman Yep, that's something we plan on addressing. There's also new functionality in docker that wasn't available when that command was originally created (IIRC) so I think it's worth looking at the docker credential helper protocol as well.

[adamchainz]

#3470 and #3569 would be backwards incompatible but better for scripts.

[joffotron]

Have you considered writing the tool in something like Go? Having a single, static binary you can deploy / install places, rather than having a dependency on the Python runtime would be amazing

[rnhurt]

Have you considered writing the tool in something like Go? Having a single, static binary you can deploy / install places, rather than having a dependency on the Python runtime would be amazing

Hahaha, I just suggested this in #3605. :)

[MarkRose]

Now that AWS finally supports U2F in the console, it would be wonderful if #3607 were included.

[jqtrde]

Maybe (definitely) a bit petty, but I'd love it if help became --help and --dryrun became --dry-run - which I think is how most other cli flags are structured 💁‍♂️

[adamchainz]

@jacquestardie yes! #303 is the original issue. Even after years of using AWS CLI, I still regularly try --help first.

[joguSD]

@joffotron This is something we've seriously considered, but a re-write in Go is probably out of scope for V2. In particular the high level S3 command and the related library s3transfer would also need to be re-written which greatly increases the scope of switching to another language. That being said, distribution formats and installation mechanisms are something we really want to improve upon in V2.

@jacquestardie I would also love to see this in V2, and it's definitely on our radar.

[graingert]

Have you considered writing the tool in something like Haskell? Having a single, static binary you can deploy / install places, rather than having a dependency on the Python runtime would be amazing

[lorengordon]

There's always pyinstaller. Can keep the code in python, and have a single binary that includes the python runtime.

[oxidizeddreams]

Support aliases (aside from just scripting them into .zshrc). Certainly following "some" direction from ElasticBeanstalk CLI, and awless. My apologies beforehand, if any of the following has been said/done/repeated.

aws ssh my-dev-node
aws list instances
aws list services # list all available/global services available based to a profile/region
aws list price-index # list current price indexes of AWS services in an easily ingestible manner
aws list instance-tiers # list all available instance-tiers, and show stats based on provided filters

Borrowing from how wonderful awless is:

aws switch <profile-name> <region> # to support multiple IAM credentials pairs/accounts
aws show <reference> # show resource dependencies, configurations, api_usage, credit_usage
aws sync # be able to sync infra+service data from environment/region/account with filters

• allows us to query against local data versus hammering the AWS api, and sync when stale

Not implemented anywhere, or not well:

aws graph <reference> # build a dependency graph based on referenced id

• graph an IAM user and their privileges/policies/roles and show a web of trust
• graph a resource and associated/related AWS assets
• graph a TAG and relationships

aws analyze <reference> # some form of being able to quickly and easily:

• analyze costs based on tags, resource_ids, generated dependency graphs (see above), etc
  -- to quickly predict and forecast small to large infrastructure deployments/changes in a given time period
• analyze objects like usage breakdown of an S3 bucket, or how many events generated through SNS in the last hour, or day

aws profile lock # ability to lock a "session" or profile of executing user

• helps operators control their profiles on shared host environments (jumpbox/bastion hosts) or those without proper auth/secrets management
• another user should not be able to invoke commands using someone else's profile with unlocking/unsealing the profile
• this idea was not thought out well

[rnhurt]

@oxidizeddreams, I really like the aws switch <profile> <region> idea. With the growing number of accounts most of us have it has become quite cumbersome to constantly switch between profiles.

I would like to add another solution. What about using a hidden file (.aws_profile) to dedicate operations in a particular directory tree to a particular account/region? This would work similar to .ruby_version and .ruby_gem. My CloudFormation templates are split up between accounts and this type of profile management would come in super handy and prevent me from applying a template to the wrong account by mistake.

[atkinsonm]

To expand on the point @rnhurt brought up, another challenge organizations face is using IAM roles on the command line compounded with multi-account. Many organizations I work with use AWS SSO or custom SAML IdPs to federate access to IAM roles. Using the command line or SDKs from local or on-prem machines to leverage roles typically requires setting up IAM users/keys with AssumeRole permissions. This creates a security risk in systems otherwise using temporary credentials exclusively.

AWS SSO partially resolves this with the interface that provides commands to copy/paste into a local terminal to assume the same role available in the console, but it can be prohibitive to require a customer to log into the console to get those instructions and can be difficult to successfully perform AssumeRole without them.

If there were some way to better support federation on the command line, I'd be all for it.

[lorengordon]

@melusom Have you tried using awsprocesscreds with the credential_process provider? I use that with Okta and it works great (it also supports ADFS). No more personal IAM users/keys!

[atkinsonm]

@lorengordon I'll have to check that out. Thanks for sharing!

[mattnewell]

So, what happened to the v2 idea? Is this dead?

[jamesls]

So, what happened to the v2 idea? Is this dead?

Not dead! 🎉 There's still a good amount of planning going on, and in terms of feature work, a lot of the work lately has been on installation mechanisms. Moving away from pip and making sure the installation process is solid has taken a good chunk of time. A lot of that work is internal as part of our release and build infrastructure so there's no much to show on github.

I do expect actual feature development to pick up over the next couple of months though. Thanks everyone for your patience on this.

[graingert]

Moving away from pip?

[PeterBengtson]

After spending a couple of days trying to get the aws2 cli and the credential-helper to play nice together on Mac, I'm wondering if this integration works at all in aws2? The git version is very recent, the OS X Keychain isn't involved in any way, everything is set up according to instructions, but I still get 'NoneType' object has no attribute 'secret_key' and questions about username and password.

[bdrx312]

Are there any plans for a stable plugin API (#2350) or some sort of event system as it exists in botocore?

We have written a plugin for the aws cli v1 to allow us to have a custom authentication mechanism via a custom credential provider. It would be a shame if aws2 cli doesn't also support a plugin system.

[kdaily]

Hello all! Since the AWS CLI is generally available, I'm closing this tracking issue. You can follow along on any existing AWS CLI v2 issues, bugs, or feature requests with the v2 label:

v2

Please feel free to open a new issue for anything that comes up with the AWS CLI v2. Thanks!

https://github.com/aws/aws-cli/issues/new/choose