Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging for beta releases #6

Closed
eladb opened this issue May 31, 2018 · 8 comments
Closed

Packaging for beta releases #6

eladb opened this issue May 31, 2018 · 8 comments

Comments

@eladb
Copy link
Contributor

eladb commented May 31, 2018

Let's work backwards from how we want beta participants to use the CDK.

Installation

Theoretically, this step should only install the toolkit on the system, but since we still don't have the CDK libraries released into package managers, the installed bundle will also contain a local repository of all modules in all languages. The "magic" will happen in cdk init, which will take care of binding the newly created project to the local copy instead of the package manager (depending on the language).

  1. Download a zip from an authenticated URL (GitHub releases would be an idiomatic way to do it, but we can also publish to S3 and have users download using the CLI).
  2. Run a setup command which unzips the file and install it under ~/.cdk. Ideally we don't want any system-level installs (including jsii-runtime), but as a temporary workaround until we bundle the runtime with the java client we can do with a installing jsii-runtime to /usr/local/bin (requires sudo).
  3. Ask users to add ~/.cdk/bin to their PATH.

Creating Projects

During beta we expect all CDK projects to be created using cdk init.

Post-beta users will be able to create CDK projects in any way they want and just consume modules idiomatically through their package manager, but this technique should also work.

We should have two types of templates for each supported language:

  1. CDK App: defines a user Stack, an App and the build process will synthesize CloudFormation templates for all stacks defined in the app.
  2. CDK Library: exports a construct.

TypeScript

For example, cdk init typescript-app will initialize the current directory with a typescript CDK app project and then run npm install to bring in all the deps.

The directory will include:

  • package.json file with devDependencies which includes pointers to file:// tarballs installed udner ~/.cdk. The package.json file will include all CDK modules (aws-cdk-xxx). This will make it easier for users to get started, and will also solve the issue of dependency resolution without npm.
  • prepare and watch scripts use tsc (not jsii), since this is an app, also requires tsconfig.json.
  • cdk.json has { app: node index.js }
  • cdk script will run the toolkit cdk.
  • prepare script will also invoke cdk synth -o cdk-out or something like that. Which means that building the project will automatically synthesize the templates.

Java

cdk init java-app will initialize the current directory with a java maven project. The pom.xml file will use a system scope (or file:// repository, whichever makes more sense) to point to aws-cdk and jsii-java-runtime packages under ~/.cdk.

The maven project will also include a post-build step that will invoke synthesize and will produce templates to cdk-out (same name as the JS projects, to make it easier for CI/CD to be set up later).

@rix0rrr
Copy link
Contributor

rix0rrr commented May 31, 2018

Ideally we don't want any system-level installs (including jsii-runtime), but as a temporary workaround until we bundle the runtime with the java client we can do with a installing jsii-runtime to /usr/local/bin (requires sudo).

Why not install those to ~/.cdk/bin as well (instead of /usr/local/bin), if the user is going to add that directory to the path anyway?

Yes, you don't technically need the the rest of the CDK install for the Java version, but users won't care about that anyway. "I have the CDK installed" is what they'll think, and they'll have it, and it'll work.

@rix0rrr
Copy link
Contributor

rix0rrr commented May 31, 2018

Agreed on all of the rest. As one more note, I think I'd prefer <LANGUAGE>-<TYPE>, so:

typescript-app
java-app

For me (at least) that feels more natural.

@eladb
Copy link
Contributor Author

eladb commented May 31, 2018

Why not install those to ~/.cdk/bin as well (instead of /usr/local/bin), if the user is going to add that directory to the path anyway?

Yes, I'll try to avoid. The challenge is when you use an IDE, you need jsii-runtime to be available. The long-term solution is that we want to actually bundle the runtime as a .js resource into the client library and then only node.js will be an external dependency (which you need anyway for the toolkit). In the short term, I might be able to find some magical Maven incantation that will do the trick.

@RomainMuller
Copy link
Contributor

I'm not certain I like cdk init app-ts, because that looks messy. To a certain extend, I would rather have something similar to how swift package init --type=[empty|library|executable|system-module] is.

So the ergonomics I would like to have is more like:

cdk init --type=[app|library] --language=[ts|java|dotnet|ruby|...]

@rix0rrr
Copy link
Contributor

rix0rrr commented May 31, 2018

The challenge is when you use an IDE, you need jsii-runtime to be available.

Well yeah, but then all you have to do is start your IDE after updating the PATH. Although I got you, if you start your IDE from a GUI, where does the PATH even come from?

@eladb
Copy link
Contributor Author

eladb commented May 31, 2018

cdk init --type=[app|library] --language=[ts|java|dotnet|ruby|...]

Let's not over-engineer this at this point. I really want to keep cdk-init dead simple and stupid. There's a high abuse potential for templating engines and we should be very careful not to cross that line...

@rix0rrr
Copy link
Contributor

rix0rrr commented May 31, 2018

cdk init --type=[app|library] --language=[ts|java|dotnet|ruby|...]

I mean, maybe. But we won't have all combinations to start with.

Also, I prefer typescript to ts.

@RomainMuller
Copy link
Contributor

@rix0rrr I think we can also offer aliases ts for typescript; .net for dotnet, ...
And also, available languages could differ by type -- just a matter of giving out a proper error message.

@eladb The way I see this is very simple... --type is a category, --language is a sub-category. That is all. It would still work like it does now, except two-level folder structure instead of one...

@eladb eladb mentioned this issue May 31, 2018
eladb added a commit that referenced this issue Jun 3, 2018
Initial implementation for #6 - this change will result in `dist/` containing a bundle that contains:

 - `node_modules` - with all jsii and CDK modules
 - `bin` - symlink to __node_modules/.bin__
 - `repo/npm` - tarballs for all npm modules (CDK + jsii)
 - `repo/maven` - jars for jsii and CDK
 - `docs` - the rendered documentation website
@eladb eladb closed this as completed Jun 3, 2018
ncaq referenced this issue in ncaq/aws-cdk Oct 15, 2020
aws-lambda-nodejs have issue

* [npm install -g yarn · Issue #6 · nodejs/corepack](nodejs/corepack#6)
* [npm install yarn --global fails in docker container · Issue aws#8358 · yarnpkg/yarn](yarnpkg/yarn#8358)

From issue comment,

> For future reference, you can (should) pin your version rather than use whatever the latest is on npm (by using yarn@1.22.6, etc) - it's a good practice anyway regardless of the conditions, as you never know which bug could slip by us. You can also use the yarn-path setting to ensure that upgrades go through the appropriate review processes (including CI).
>
> <yarnpkg/yarn#8358 (comment)>

So we'll follow it.

And from issue comment,

> Fwiw we don't plan to add any more features to Yarn 1, as all of our resources have shifted to Yarn 2. The past few commits have been aimed toward making the transition a bit easier, in particular thanks to the Corepack initiative which we hope will make it easier to use Yarn (both 1 & 2) by removing the need to manually install them.
>
> <yarnpkg/yarn#8358 (comment)>

There's not much need to be concerned with the latest version of 1.
mergify bot pushed a commit that referenced this issue Oct 21, 2020
fixes #10881

# fix from sh to bash

`lib/bundlers.ts` actually uses not `sh` but `bash`,
`bash` is a good way to test.

# lock yarn version

aws-lambda-nodejs have issue

* [npm install -g yarn · Issue #6 · nodejs/corepack](nodejs/corepack#6)
* [npm install yarn --global fails in docker container · Issue #8358 · yarnpkg/yarn](yarnpkg/yarn#8358)

From issue comment,

> For future reference, you can (should) pin your version rather than use whatever the latest is on npm (by using yarn@1.22.6, etc) - it's a good practice anyway regardless of the conditions, as you never know which bug could slip by us. You can also use the yarn-path setting to ensure that upgrades go through the appropriate review processes (including CI).
>
> <yarnpkg/yarn#8358 (comment)>

So we'll follow it.

And from issue comment,

> Fwiw we don't plan to add any more features to Yarn 1, as all of our resources have shifted to Yarn 2. The past few commits have been aimed toward making the transition a bit easier, in particular thanks to the Corepack initiative which we hope will make it easier to use Yarn (both 1 & 2) by removing the need to manually install them.
>
> <yarnpkg/yarn#8358 (comment)>

There's not much need to be concerned with the latest version of 1.

# allow execute command for non root user

`amazon/aws-sam-cli-build-image-nodejs12.x` don't have user that index is 1000.
So create non root user.

`/` in `amazon/aws-sam-cli-build-image-nodejs12.x` permission is `700`.
change to allow execute command for non root user.
I really don't want to change around the permissions,
but I don't have a choice.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
smguggen referenced this issue in smguggen/aws-cdk Aug 24, 2021
# This is the 1st commit message:

Add Identity Pool construct

# This is the commit message #2:

Bug fixes

# This is the commit message #3:

Bug fixes

# This is the commit message #4:

Formatting

# This is the commit message #5:

Add construct methods

# This is the commit message #6:

Remove flat

# This is the commit message #7:

Fix issues
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants