Assets: upload to and grant permissions on prefix

[rix0rrr]

We need to give asset consumers permissions on all versions of
an asset, not just the latest version. Otherwise, we will never
be able to do rolling updates.

Also add caching on AWS client instances, so with multiple
asset uploads we don't have to construct a new S3 client for
every asset (incurring credential lookups for each one).

This fixes #484.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

[rix0rrr]

Practical changes:

The assets are now uploaded to something like:

s3://cdktoolkit-stagingbucket-i0tp1gw9qhfn/assets/cdk-dev-huijbersWindowsInstallationTestDirectory46EF3817/978e060951d0c6ad5319aee80a274a0a214db401526914729cbf8b604a105fbb.zip

And we give permissions on the prefix assets/cdk-dev-huijbersWindowsInstallationTestDirectory46EF3817/*

I'm not entirely sure I like the additional input Parameter to the template to specify the prefix. We might run out of Parameters quickly this way. A number of ideas to solve this:

• Require all assets to use the same bucket
• Encode multiple parameters into a single parameter string, split on a magic character like | or something.
• Mandate that the asset URL structure is ALWAYS the same, (has the same number of components), so we can split on / and append the *. Because CFN doesn't have a concept of slices, we must know exactly how many path components there will be in that case, though (we'll have to do join('/', [select(1, split('/', p)), select(2, split('/', p)), ..., '*']))

The last one seems very brittle, the first one lacks flexibility and helps only by 33%. So I'm leaning towards combining multiple parameter values into one formal CloudFormation Parameter, but I'm not sure how well that will play with CodePipelines, for example.

[eladb]

@rix0rrr I agree that the additional parameter is a going to be a problem very quickly. And I am concerned how this is going to work with CodePipelines. Will we use the pipeline bucket as the prefix?

I wonder if instead of using only a path separator, we could delimit using a sequence (i.e. @/@), so the S3 key will always look something like bla/bla/bla/prefix@/@suffix and then we can just Fn::Split with "@/@" as the delimiter to separate the prefix and the concrete object key. Still not sure how this will work with CP.

[eladb]

I keep wondering if we should just expose a uniqueId in the Construct level (and use for resource.logicalId). This keeps coming back and I think might be useful generally, and not only for CloudFormation resource identifiers.

[rix0rrr]

I agree, but their scopes are different, which complicates it a bit.

In a tree like this:

root
└── Stack
      └── Construct
             └── Construct

    ├─────── path ───────┤

         ├─ logical id ──┤

The .path is from the root to the construct, but the logical ID is only calculated off the nearest enclosing stack to the construct. If we didn't do that, all enclosing logical IDs would being with the stack name, which is not terrible for the IDs that they are, but they're also used to generate human-readable names, where that would be quite ugly (and in fact CFN sometimes--but not always!--prepends the stack name itself as well).

So we should probably still keep them separated, or have something like uniqueIdRelativeTo(ancestor)

[eladb]

We could allow constructs to opt-out of the human portion of name generation, so they will be "skipped" when producing the human portion (something like hideName = true).

[eladb]

We should implement #24

[rix0rrr]

This change is pending #518

[RomainMuller]

You realize (however unlikely that is), that || might actually be part of the object key?

[rix0rrr]

That is going to be true for any separator we pick. So yeah, I'm balancing aesthetics vs likelihood

[rix0rrr]

You do realize that git commit hashes may clash? :p