Add conformance tests into docs #1417
Conversation
✅ Deploy Preview for fdc3 ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
@openfin-johans this might be a good docs PR for you to take a quick look at Johan. |
|
what's your thinking here, @kriswest? Why do we even want to do this? |
Do you mean why would we want to put the conformance test definitions into the docs? Over the last 6 months or so we've answered numerous clarifying questions to 3 teams implementing Desktop Agents. Of those at least 2, if not all 3 were referred to the conformance tests to clarify the intended behaviour in particular areas - they did not discover the tests on their own (because they are not in the documentation), but did find them useful. Two of those teams have since applied for and achieved conformance, by passing tests that implement the test definitions. Hence, the tests are useful additional detail; by adding them to the docs we make them easier to find and to scrutinize/comment on/suggest extensions and corrections. They are more formally part of the standard. We also want to start having PRs that alter the standard also address necessary changes or additions to the conformance test in the same PR - something we agreed to add to the checklist in the proposed PR template. Having the tests visible in the docs will make that easier. Finally, the tests should be better tied to specific standard versions than they are. This was a good opportunity to break from the current format which blends 1.2 and 2.0 tests (which gets preserved in the 2.0 and 2.1 versions) and to create a cleaner version for 2.2 onwards. I think that will help with your proposed rewrite of the implementation (and presumably definition of cases) in gherkin syntax for use with cucumber. |
resolves #1077
Adds the FDC3 conformance tests into the main documentation and docs for the 2.0 and 2.1 versions. Admonitions are included to indicate that 2.1 uses the 2.0 tests, while 2.2 tests are yet to be defined.
Pages updated:
This is a precursor to creating a version for 2.2 compliance (#1415). When creating that set, I believe we should withdraw the 1.2 tests from the docs pages (will exist as is in the version above) and focus on only the current test set. That has been applied to the 2.2 (/docs) versions already, ready for 2.2 tests to be added and labelled as new to that version.
I have not implemented the conformance test pages for 1.2 as they will require more editing to remove 2.0 tests. If that is needed in your opinion, let me know in a review.
N.B finos/FDC3-conformance-framework#264 should be merged after this to correct links in the confromance framework's readme to point to the new Overview page (replaces the README.md file)