feat(compliance): Compliance suite

gh-pages/content/specification/.pages.yml

@@ -6,3 +6,4 @@ nav:
  - The jsii kernel API: 3-kernel-api.md
  - 4-standard-compliance-suite.md
  - 5-new-language-intake.md
+ - 6-compliance-report.md

package.json

@@ -9,9 +9,10 @@
  "bump": "bash scripts/bump.sh",
  "dist-clean": "lerna run dist-clean --stream && rm -rf dist",
  "package": "bash scripts/package.sh",
- "test": "lerna run test --concurrency=1 --stream",
+ "test": "lerna run test --concurrency=1 --stream && yarn compliance",
  "test:integ": "lerna run test:integ --stream",
- "test:update": "lerna run test:update --concurrency=1 --stream"
+ "test:update": "lerna run test:update --concurrency=1 --stream",
+ "compliance": "tools/jsii-compliance/report.js"
  },
  "devDependencies": {
  "@jest/types": "^26.6.2",
@@ -27,6 +28,7 @@
  "jest-circus": "^26.6.3",
  "jest-config": "^26.6.3",
  "jest-expect-message": "^1.0.2",
+ "tablemark": "^2.0.0",
  "lerna": "^3.22.1",
  "prettier": "^2.2.1",
  "standard-version": "^9.1.0",

tools/jsii-compliance/README.md

@@ -0,0 +1,33 @@
+# JSII Compliance
+
+This directory contains scripts and resources to create and validate the JSII compliance suite.
+
+### Compliance Suite
+
+The compliance suite is defined as a collection of abstract test cases, that each language binding must implement individualy.
+Language specific exclusions may be added to each test separately, or to the entire suite.
+
+> See [compliance-suite.json](./compliance-suite.json).
+
+### Compliance Report
+
+Each language binding is responsible for creating a language specific report during the `test` phase of our build.
+
+These reports are then aggregated into a single multi-language compliance report. It is generated during build time,
+and should be checked into [source control](../../gh-pages/content/specification/6-compliance-report.md).
+
+> To directly generate the report, run `yarn compliance` from the top level direcotry.
+
+Note that if you add a test and don't re-generate the report, the build will fail:
+
+```console
+```
+
+The report contains the state of each test with respect to each language binding.
+Every test will have one of the following statuses:
+
+- ✅ - Test passes.
+- ❌ - Test is either missing or is not reporting success.
+- N/A - The test was excluded.
+
+Eventually, this report lands on our [docs site](https://aws.github.io/jsii/specification/6-compliance-report/).