JSII compliance suite baseline mechanisms

[MrArnoldPalmer]

Define a way to extract the list of test cases from the current java compliance suite for reuse in other languages. Consume this list of test cases from each language test suite and verify all cases are covered. Produce a compliance report for each language during repository build + test.

Ideally publish this report as part of the JSII docs site.

Implementation Plan

(1) Compliance SOT

Make sure we have a single source of truth for all required tests. We will bootstrap by extracting the current Java compliance tests.

To accomplish this we need the following validations:

• A test added to the compliance suite definition must be implemented in all language bindings.
• A test added directly to a language specific compliance suite, must be added to the generic compliance suite definition.

Exclusion mechanisms will be added as needed when we see the use-cases.

(2) Report Generation

On build, generate a compliance report for each language in a known location, and aggregate into a single report. The aggregated report is pushed into source control and used in the JSII docs site.

• Add an anti-temper check on that report.

(3) New Language Intake

Create a procedure for adding a compliance suite for a new language.

• Create a new package (e.g golang-runtime-test)
• Enforce package generates our standard report at build time.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.