Streamline Usage of Prebuilt Providers

README.md

@@ -61,7 +61,7 @@ Choose a language:
 ## Documentation
 
 * Install and run a quick start tutorial at [HashiCorp Learn](https://learn.hashicorp.com/terraform/cdktf/cdktf-install)
-* Using CDK for Terraform CLI for importing Terraform [modules and providers](./docs/working-with-cdk-for-terraform/importing-providers-and-modules.md).
+* Using CDK for Terraform CLI for importing Terraform [modules and providers](./docs/working-with-cdk-for-terraform/using-providers-and-modules.md).
 * Explore the CDK for Terraform [CLI](./docs/cli-commands.md).
 * Defining Terraform [outputs](./docs/working-with-cdk-for-terraform/terraform-outputs.md).
 * Using Terraform [remote backend](./docs/working-with-cdk-for-terraform/remote-backend.md).

docs/working-with-cdk-for-terraform/cdktf-json.md

@@ -0,0 +1,94 @@
+# cdktf.json Configuration File
+
+With the `cdktf.json` in your project root directory you can configure the behaviour of the Terraform CDK CLI:
+
+## Specification
+
+```ts
+export enum Language {
+  TYPESCRIPT = 'typescript',
+  PYTHON = 'python',
+  DOTNET = 'dotnet', // not yet supported
+  JAVA = 'java', // not yet supported
+}
+
+export interface Config {
+  readonly app?: string; // The command to run in order to synthesize the code to Terraform compatible JSON
+  readonly language?: Language; // Target language for building provider or module bindings. Currently supported: `typescript` or `python`
+  readonly output: string; // Default: 'cdktf.out'. Where the synthesized JSON should go. Also will be the working directory for Terraform operations
+  readonly codeMakerOutput: string; // Default: '.gen'. Path where generated provider bindings will be rendered to.
+  readonly terraformProviders?: string[]; // Terraform Providers to build
+  readonly terraformModules?: string[]; // Terraform Modules to build
+}
+```
+
+### Terraform Providers and Modules
+
+The [following specifications](https://www.terraform.io/docs/configuration/provider-requirements.html#requiring-providers) schema should be followed for providers and modules
+
+[source](https://www.terraform.io/docs/configuration/provider-requirements.html#source-addresses)@[version](https://www.terraform.io/docs/configuration/provider-requirements.html#version-constraints)
+
+#### For HashiCorp maintained providers
+
+Official HashiCorp [maintained providers](https://registry.terraform.io/browse/providers?tier=official) (e.g. `aws`, `google` or `azurerm`) can be specified in a short version like this: `"aws@~> 2.0"`
+
+#### 3rd Party Providers
+
+Community providers have to provide a fully qualified name, e.g. to define the Docker provider: `terraform-providers/docker@~> 2.0`
+
+#### Modules
+
+Similar to 3rd Party providers, the full registry namespace should be provided. Please note that only modules from the registry are supported at this point.
+
+## Examples
+
+### Minimal Configuration
+
+A minimal configuration would define `app` only. This is useful, when planning to use [prebuilt providers]() and therefore no provider or modules bindings should be generated.
+
+```json
+{
+  "app": "npm run --silent compile && node main.js"
+}
+```
+
+### Changing Code Output Directories
+
+This will synthesize JSON into `my-workdir` and all Terraform operations - such as `deploy` or `destroy` - will be performed in this directory.
+
+```json
+{
+  "app": "npm run --silent compile && node main.js",
+  "output": "my-workdir",
+}
+```
+
+### Building Providers
+
+This will synthesize JSON into `my-workdir` and all Terraform operations - such as `deploy` or `destroy` - will be performed in this directory. See more details about this [here](./using-providers-and-modules.md)
+
+```json
+{
+  "language": "typescript",
+  "app": "npm run --silent compile && node main.js",
+  "terraformProviders": [
+    "aws@~> 2.0"
+  ]
+}
+```
+
+### Building Providers in Custom Directory
+
+This will generate the `aws` provider bindings in the folder `./imports`.
+
+```json
+{
+  "language": "typescript",
+  "app": "npm run --silent compile && node main.js",
+  "terraformProviders": [
+    "aws@~> 2.0"
+  ],
+  "codeMakerOutput": "imports"
+}
+```
+

docs/working-with-cdk-for-terraform/importing-providers-and-modules.md → docs/working-with-cdk-for-terraform/using-providers-and-modules.md

@@ -1,4 +1,26 @@
-# Importing Providers and Modules
+# Using Providers and Modules
+
+## Prebuilt Providers
+
+To improve the general user experience around provider imports and to allow building further abstractions on top of the Terraform provider bindings, a few popular providers are offered as prebuilt packages. At the moment the following providers are built and published to NPM / PyPi on a regular basis automatically.
+
+- [AWS Provider](https://cdk.tf/provider/aws)
+- [Google Provider](https://cdk.tf/provider/google)
+- [Azure Provider](https://cdk.tf/provider/azurerm)
+- [Kubernetes Provider](https://cdk.tf/provider/kubernetes)
+- [Docker Provider](https://cdk.tf/provider/docker)
+- [Github Provider](https://cdk.tf/provider/github)
+- [Null Provider](https://cdk.tf/provider/null)
+
+Please check the [Terraform CDK Proivders](https://cdk.tf/provider) organisation as well for an up to date list. As these are are normal npm / pypi packages, they can be used as any other dependency.
+
+e.g. in Typescript / Node:
+
+```
+npm install -a @cdktf/provider-aws
+```
+
+## Build Providers and Modules
 
 CDK for Terraform allows you to import Terraform [providers](https://www.terraform.io/docs/providers/index.html) and [modules](https://www.terraform.io/docs/modules/index.html) to your project
 using this workflow.
@@ -31,7 +53,7 @@ new MyStack(app, 'hello-terraform');
 app.synth();
 ```
 
-The project also has the `cdktf.json` file that defines what providers and modules are being used by the project.
+The project also has the [cdktf.json](./cdktf-json.md) file that defines what providers and modules are being used by the project.
 
 ```bash
 vim cdktf.json

examples/typescript/aws-prebuilt/.gitignore

@@ -0,0 +1,4 @@
+.gen
+cdktf.out
+terraform.tfstate*
+!tsconfig.json

examples/typescript/aws-prebuilt/README.md

@@ -0,0 +1,15 @@
+# Typescript AWS with Prebuilt Provider
+
+A CDK for Terraform application in TypeScript using the prebuilt [AWS provider](https://cdk.tf/provider/aws).
+
+## Usage
+
+Install project dependencies:
+
+```shell
+yarn install
+```
+
+You can now edit the [main.ts](./main.ts) file if you want to modify any code.
+
+Perform a `cdktf diff` to get a plan of what would be deployed.

examples/typescript/aws-prebuilt/cdktf.json

@@ -0,0 +1,3 @@
+{
+  "app": "npm run --silent compile && node main.js"
+}

examples/typescript/aws-prebuilt/main.ts

@@ -0,0 +1,49 @@
+import { Construct } from 'constructs';
+import { App, TerraformStack, TerraformOutput } from 'cdktf';
+import { DataAwsRegion, AwsProvider, DynamodbTable, SnsTopic } from '@cdktf/provider-aws'
+
+export class HelloTerra extends TerraformStack {
+  constructor(scope: Construct, id: string) {
+    super(scope, id);
+
+    new AwsProvider(this, 'aws', {
+      region: 'eu-central-1'
+    })
+
+    const region = new DataAwsRegion(this, 'region')
+
+    const table = new DynamodbTable(this, 'Hello', {
+      name: `my-first-table-${region.name}`,
+      hashKey: 'temp',
+      attribute: [
+        { name: 'id', type: 'S' },
+      ],
+      billingMode: "PAY_PER_REQUEST"
+    });
+
+    table.addOverride('hash_key', 'id')
+    // table.addOverride('hash_key', 'foo')
+    table.addOverride('lifecycle', { create_before_destroy: true })
+
+    const topicCount = 1
+    const topics = [...Array(topicCount).keys()].map((i) => {
+      return new SnsTopic(this, `Topic${i}`, {
+        displayName: `my-first-sns-topic${i}`
+      });
+    })
+
+    new TerraformOutput(this, 'table_name', {
+      value: table.name
+    })
+
+    topics.forEach((topic, i) => {
+      new TerraformOutput(this, `sns_topic${i}`, {
+        value: topic.name
+      })
+    })
+  }
+}
+
+const app = new App();
+new HelloTerra(app, 'hello-terra');
+app.synth();

examples/typescript/aws-prebuilt/package.json

@@ -0,0 +1,21 @@
+{
+  "name": "@examples/typescript-aws-prebuilt",
+  "version": "0.0.0",
+  "main": "index.js",
+  "license": "MPL-2.0",
+  "scripts": {
+    "build": "tsc",
+    "compile": "tsc --pretty",
+    "synth": "cdktf synth"
+  },
+  "devDependencies": {
+    "@types/node": "^14.0.26",
+    "cdktf-cli": "0.0.0",
+    "typescript": "^3.9.7"
+  },
+  "dependencies": {
+    "@cdktf/provider-aws": "^0.0.19",
+    "cdktf": "0.0.0",
+    "constructs": "^3.0.0"
+  }
+}

examples/typescript/aws-prebuilt/tsconfig.json

@@ -0,0 +1,33 @@
+{
+  "compilerOptions": {
+    "alwaysStrict": true,
+    "charset": "utf8",
+    "declaration": true,
+    "experimentalDecorators": true,
+    "inlineSourceMap": true,
+    "inlineSources": true,
+    "lib": [
+      "es2018"
+    ],
+    "module": "CommonJS",
+    "noEmitOnError": true,
+    "noFallthroughCasesInSwitch": true,
+    "noImplicitAny": true,
+    "noImplicitReturns": true,
+    "noImplicitThis": true,
+    "noUnusedLocals": true,
+    "noUnusedParameters": true,
+    "resolveJsonModule": true,
+    "strict": true,
+    "strictNullChecks": true,
+    "strictPropertyInitialization": true,
+    "stripInternal": true,
+    "target": "ES2018"
+  },
+  "include": [
+    "**/*.ts"
+  ],
+  "exclude": [
+    "node_modules"
+  ]
+}

examples/typescript/aws-prebuilt/yarn.lock

@@ -0,0 +1,18 @@
+# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
+# yarn lockfile v1
+
+
+"@types/node@^13.1.1":
+  version "13.1.4"
+  resolved "https://registry.yarnpkg.com/@types/node/-/node-13.1.4.tgz#4cfd90175a200ee9b02bd6b1cd19bc349741607e"
+  integrity sha512-Lue/mlp2egZJoHXZr4LndxDAd7i/7SQYhV0EjWfb/a4/OZ6tuVwMCVPiwkU5nsEipxEf7hmkSU7Em5VQ8P5NGA==
+
+constructs@^2.0.1:
+  version "2.0.1"
+  resolved "https://registry.yarnpkg.com/constructs/-/constructs-2.0.1.tgz#274d6b8ce6a698813495c444466b0c745349ddb5"
+  integrity sha512-edR85YFGI9TBT9byAo5vAfI0PRi+jFGzinwN3RAJwKfv6Yc9x9kALYfoEmgotp95qT7/k/iUQWHrH9BMJeqpdg==
+
+typescript@^3.7.4:
+  version "3.7.4"
+  resolved "https://registry.yarnpkg.com/typescript/-/typescript-3.7.4.tgz#1743a5ec5fef6a1fa9f3e4708e33c81c73876c19"
+  integrity sha512-A25xv5XCtarLwXpcDNZzCGvW2D1S3/bACratYBx2sax8PefsFhlYmkQicKHvpYflFS8if4zne5zT5kpJ7pzuvw==

packages/cdktf-cli/bin/cmds/synth.ts

@@ -25,7 +25,7 @@ class Command implements yargs.CommandModule {
     const outdir = argv.output;
     const jsonOutput = argv.json;
 
-    if (!await fs.pathExists(config.codeMakerOutput)) {
+    if (config.checkCodeMakerOutput && !await fs.pathExists(config.codeMakerOutput)) {
       console.error(`ERROR: synthesis failed, run "cdktf get" to generate providers in ${config.codeMakerOutput}`);
       process.exit(1);
     }

packages/cdktf-cli/lib/config.ts

@@ -16,6 +16,7 @@ export interface Config {
   readonly codeMakerOutput: string;
   readonly terraformProviders?: string[];
   readonly terraformModules?: string[];
+  checkCodeMakerOutput?: boolean;
 }
 
 export function readConfigSync(): Config {
@@ -28,5 +29,7 @@ export function readConfigSync(): Config {
     };
   }
 
+  config.checkCodeMakerOutput = Array.isArray(config.terraformModules) || Array.isArray(config.terraformProviders)
+
   return config;
 }

packages/cdktf-cli/templates/python/help

@@ -9,7 +9,7 @@
 
   Synthesize:
     cdktf synth           Synthesize Terraform resources to cdktf.out/
-  
+
   Diff:
     cdktf diff            Perform a diff (terraform plan) for the given stack
 
@@ -19,4 +19,19 @@
   Destroy:
     cdktf destroy         Destroy the given stack
 
+  Use Prebuilt Providers:
+
+  You can add one or multiple of the prebuilt providers listed below:
+
+  npm install -a @cdktf/provider-aws
+  npm install -a @cdktf/provider-google
+  npm install -a @cdktf/provider-azurerm
+  npm install -a @cdktf/provider-docker
+  npm install -a @cdktf/provider-github
+  npm install -a @cdktf/provider-null
+
+  Check for an up to date list here https://cdk.tf/provider
+
+  Alternatively, you can also build the provider bindings yourself:
+
 ========================================================================================================

packages/cdktf-cli/templates/typescript-minimal/.hooks.sscaff.js

@@ -0,0 +1,48 @@
+const { execSync } = require('child_process');
+const { readFileSync, writeFileSync } = require('fs');
+
+const constructs_version = require('../../package.json').dependencies.constructs;
+
+exports.post = ctx => {
+  // Terraform Cloud configuration settings if the organization name and workspace is set.
+  if (ctx.OrganizationName != '') {
+    console.log(`\nGenerating Terraform Cloud configuration for '${ctx.OrganizationName}' organization and '${ctx.WorkspaceName}' workspace.....`)
+    terraformCloudConfig(ctx.$base, ctx.OrganizationName, ctx.WorkspaceName);
+  }
+
+  const npm_cdktf = ctx.npm_cdktf;
+  const npm_cdktf_cli = ctx.npm_cdktf_cli;
+
+  if (!npm_cdktf) { throw new Error(`missing context "npm_cdktf"`); }
+  if (!npm_cdktf_cli) { throw new Error(`missing context "npm_cdktf_cli"`); }
+
+  installDeps([npm_cdktf, `constructs@${constructs_version}`]);
+  installDeps([npm_cdktf_cli, '@types/node', 'typescript'], true);
+
+  console.log(readFileSync('./help', 'utf-8'));
+};
+
+function installDeps(deps, isDev) {
+  const devDep = isDev ? '-D' : '';
+  // make sure we're installing dev dependencies as well
+  const env = Object.assign({}, process.env)
+  env['NODE_ENV'] = 'development'
+
+  execSync(`npm install ${devDep} ${deps.join(' ')}`, { stdio: 'inherit', env });
+}
+
+function terraformCloudConfig(baseName, organizationName, workspaceName) {
+  template = readFileSync('./main.ts', 'utf-8');
+
+  result = template.replace(`import { App, TerraformStack } from 'cdktf';`, `import { App, TerraformStack, RemoteBackend } from 'cdktf';`);
+  result = result.replace(`new MyStack(app, '${baseName}');`, `const stack = new MyStack(app, '${baseName}');
+new RemoteBackend(stack, {
+  hostname: 'app.terraform.io',
+  organization: '${organizationName}',
+  workspaces: {
+    name: '${workspaceName}'
+  }
+});`);
+
+  writeFileSync('./main.ts', result, 'utf-8');
+}