Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add conformance tests into docs #1417

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
---
id: App-Channel-Tests
sidebar_label: App Channel Tests
title: App Channel Tests
hide_title: true
---

# App Channel Tests ![1.2](https://img.shields.io/badge/FDC3-1.2-green) ![2.0](https://img.shields.io/badge/FDC3-2.0-blue)

## Basic Broadcast
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
---
id: Basic-Tests
sidebar_label: Basic Tests
title: Basic Tests
hide_title: true
---

# Basic Tests ![1.2](https://img.shields.io/badge/FDC3-1.2-green) ![2.0](https://img.shields.io/badge/FDC3-2.0-blue)

_These are some basic sanity tests implemented in the FDC3 Conformance Framework. It is expected that Desktop Agent testers will run these first before commencing the much more thorough tests in section 2 onwards._
Expand Down Expand Up @@ -26,4 +33,3 @@ _These are some basic sanity tests implemented in the FDC3 Conformance Framework
- `BasicRI2`: The application should be able to raise an intent for some item of context by invoking:
- `fdc3.raiseIntentForContext(<context>)`
- A promise should be returned.

Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
---
id: Intents-Tests
sidebar_label: Intents Tests
title: Intents Tests
hide_title: true
---

# Intents Tests

## Version 1.2 Intents Tests ![1.2](https://img.shields.io/badge/FDC3-1.2-green)
Expand Down Expand Up @@ -35,8 +42,8 @@ Also we assume a fourth app **D** that is going to discover the intents in the o

| App | Step | Details |
|-----|----------------|---------------------------------------------------------------------------------------------------|
| D | 1. Raise | `fdc3.raiseIntent(‘sharedTestingIntent1’, {testContextY})`<br>starts app B. |
| B | 2. Gather Context | `fdc.addIntentListener(‘sharedTestingIntent1’)`<br>Receives testContextY, matching that sent by D |
| D | 1. Raise | `fdc3.raiseIntent(‘sharedTestingIntent1’, {testContextY})`<br />starts app B. |
| B | 2. Gather Context | `fdc.addIntentListener(‘sharedTestingIntent1’)`<br />Receives testContextY, matching that sent by D |

- `SingleResolve1`: Perform above test
- `TargetedResolve1`: Use `fdc3.raiseIntent(‘aTestingIntent’, {testContextX}, <A’s App Name>)` to start app A, otherwise, as above
Expand Down Expand Up @@ -117,8 +124,8 @@ Finally, please note that this is a larger set of apps than were required for 1.

| App | Step | Details |
|-------|----------------|---------------------------------------------------------------------------------------------------|
| Test | 1. Raise | `fdc3.raiseIntent("aTestingIntent", testContextX)`<br>starts app A. |
| A | 2. Receive Intent & Context | After starting up, A runs `fdc3.addIntentListener("aTestingIntent1")` to register its listener.<br>It then receives `testContextX`, matching that sent by Test |
| Test | 1. Raise | `fdc3.raiseIntent("aTestingIntent", testContextX)`<br />starts app A. |
| A | 2. Receive Intent & Context | After starting up, A runs `fdc3.addIntentListener("aTestingIntent1")` to register its listener.<br />It then receives `testContextX`, matching that sent by Test |
| Test | 3. IntentResolution | The `raiseIntent` call returns an `IntentResolution` Object with an `AppIdentifier` as the `source field` with App A's `appId` and `instanceId` set.** |

- `2.0-RaiseIntentSingleResolve`: Perform above test
Expand Down Expand Up @@ -163,8 +170,8 @@ Finally, please note that this is a larger set of apps than were required for 1.

| App | Step | Details |
|-----|----------------|---------------------------------------------------------------------------------------------------|
| Test | 1. Raise | `fdc3.raiseIntent("aTestingIntent", testContextX)`<br>starts app A. |
| A | 2. Receive Intent & Context | After starting up, A runs `fdc3.addIntentListener("aTestingIntent")` to register its listener.<br>It then receives `testContextX`, matching that sent by Test |
| Test | 1. Raise | `fdc3.raiseIntent("aTestingIntent", testContextX)`<br />starts app A. |
| A | 2. Receive Intent & Context | After starting up, A runs `fdc3.addIntentListener("aTestingIntent")` to register its listener.<br />It then receives `testContextX`, matching that sent by Test |
| Test | 3. IntentResolution | The `raiseIntent` call returns an `IntentResolution` Object with an `AppIdentifier` as the `source field` with App A's `appId` and `instanceId` set. |
| Test | 4. await results | Test should `await resolution.getResult()` on the `IntentResolution` object returned in the previous step. A promise should be returned quickly. |
| A | 5. return void | A should return `void` after a short delay (e.g. 5 seconds). |
Expand All @@ -178,8 +185,8 @@ Finally, please note that this is a larger set of apps than were required for 1.

| App | Step | Details |
|-----|----------------|---------------------------------------------------------------------------------------------------|
| Test | 1. Raise | `fdc3.raiseIntent("sharedTestingIntent1", testContextY)`<br>starts app **B**. |
| B | 2. Receive Intent & Context | After starting up, B runs `fdc3.addIntentListener("sharedTestingIntent1")` to register its listener.<br>It then receives `testContextY`, matching that sent by Test |
| Test | 1. Raise | `fdc3.raiseIntent("sharedTestingIntent1", testContextY)`<br />starts app **B**. |
| B | 2. Receive Intent & Context | After starting up, B runs `fdc3.addIntentListener("sharedTestingIntent1")` to register its listener.<br />It then receives `testContextY`, matching that sent by Test |
| Test | 3. IntentResolution | The `raiseIntent` call returns an `IntentResolution` Object with an `AppIdentifier` as the `source field` with App B's `appId` and `instanceId` set. |
| Test | 4. await results | Test should `await resolution.getResult()` on the `IntentResolution` object returned in the previous step. A promise should be returned quickly. |
| B | 5. return `testContextY` | B should return a `testContextY` instance after a short delay (e.g. 5 seconds). |
Expand All @@ -193,8 +200,8 @@ Finally, please note that this is a larger set of apps than were required for 1.

| App | Step | Details |
|-------|-----------------------|---------------------------------------------------------------------------------------------------|
| Test | 1. Raise Intent | Test raises an intent with `fdc3.raiseIntent("sharedTestingIntent2", testContextY, {appId: "<E's appId>"})`<br>starts app E. |
| E | 2. Receive Intent & Context | After starting up, E runs `fdc3.addIntentListener("sharedTestingIntent2")` to register its listener.<br>It them receives `testContextY`, matching that sent by Test |
| Test | 1. Raise Intent | Test raises an intent with `fdc3.raiseIntent("sharedTestingIntent2", testContextY, {appId: "<E's appId>"})`<br />starts app E. |
| E | 2. Receive Intent & Context | After starting up, E runs `fdc3.addIntentListener("sharedTestingIntent2")` to register its listener.<br />It them receives `testContextY`, matching that sent by Test |
| Test | 3. IntentResolution | The `raiseIntent` call returns an `IntentResolution` Object with an `AppIdentifier` as the `source field` with App E's `appId` and `instanceId` set. |
| Test | 4. await results | Test should `await resolution.getResult()` on the `IntentResolution` object returned in the previous step. A promise should be returned quickly. |
| E | 5. return Channel | E should retrieve a Channel object via `fdc3.getOrCreateChannel("someChannelName")` and return it immediately. |
Expand Down Expand Up @@ -227,11 +234,11 @@ Finally, please note that this is a larger set of apps than were required for 1.

| App | Step | Details |
|-------|-----------------|---------------------------------------------------------------------------------------------------|
| Test | 1. Raise intent | Test raises an intent with `fdc3.raiseIntent(‘"kTestingIntent", testContextX, {appId: "<K's appId>"})`<br>starts app K. |
| K | 2. Receive Intent & Context | After starting up, K runs `fdc3.addIntentListener("kTestingIntent")` to register its listener.<br>It them receives `testContextX`, matching that sent by Test |
| Test | 1. Raise intent | Test raises an intent with `fdc3.raiseIntent(‘"kTestingIntent", testContextX, {appId: "<K's appId>"})`<br />starts app K. |
| K | 2. Receive Intent & Context | After starting up, K runs `fdc3.addIntentListener("kTestingIntent")` to register its listener.<br />It them receives `testContextX`, matching that sent by Test |
| Test | 3. IntentResolution | The `raiseIntent` call returns an `IntentResolution` Object with an `AppIdentifier` as the `source field` with App K's `appId` and `instanceId` set. |
| Test | 4. await results | Test should `await resolution.getResult()` on the `IntentResolution` object returned in the previous step. A promise should be returned quickly. |
| K | 5. Create PrivateChannel and setup event listeners | K should create a `PrivateChannel` object via `const privChan = await fdc3.createPrivateChannel()`,<br>it should then add listeners for the 3 events offered + a context listener via:<br>- `const listener1 = await privChan.onAddContextListener(handler1);`<br>- `const listener2 = await privChan.onUnsubscribe(handler2);`<br>- `const listener3 = await privChan.onDisconnect(handler3);`<br>- `const listener4 = await privChan.addContextListener("testContextX", handler4)`<br>it should then return the `PrivateChannel`. |
| K | 5. Create PrivateChannel and setup event listeners | K should create a `PrivateChannel` object via `const privChan = await fdc3.createPrivateChannel()`,<br />it should then add listeners for the 3 events offered + a context listener via:<br />- `const listener1 = await privChan.onAddContextListener(handler1);`<br />- `const listener2 = await privChan.onUnsubscribe(handler2);`<br />- `const listener3 = await privChan.onDisconnect(handler3);`<br />- `const listener4 = await privChan.addContextListener("testContextX", handler4)`<br />it should then return the `PrivateChannel`. |
| Test | 6. receive PrivateChannel | The promise received by Test from `resolution.getResult()` should resolve to a `PrivateChannel` object. Confirm that the `type` of the Channel object is "private".
| Test | 7. addContextListener | Test should add a context listener to the PrivateChannel object via `const listener1 = privChan.addContextListener("testContextZ", handler)` |
| K | 8. Receive event & broadcast context | The `onAddContextListener` handler (`listener1`) added in step 5 should fire after Test adds its listener. Once it has, K should broadcast a short stream of `testContextZ` objects, with consecutive integer values in them (e.g. 1-5). |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
---
id: Metadata-Tests
sidebar_label: Metadata Tests
title: Metadata Tests
hide_title: true
---

# Metadata & Instance Test Cases
# Metadata & Instance Test Cases

You will need to pre-populate the AppDirectory with the following items:

| App | Required Metadata |
|-----|------------------------------------------|
| A | Generic AppD Record which contains at least the following fields:<br>- `name`<br>- `version`<br>- `title`<br>- `tooltip`<br>- `description`<br>- `icons` (`Array<Icon>`)<br>- `screenshots` (`Array<Image>`)<br>- `interop.intents.listensFor` (`aTestingIntent` with at least context type `testContextX`) |
| A | Generic AppD Record which contains at least the following fields:<br />- `name`<br />- `version`<br />- `title`<br />- `tooltip`<br />- `description`<br />- `icons` (`Array<Icon>`)<br />- `screenshots` (`Array<Image>`)<br />- `interop.intents.listensFor` (`aTestingIntent` with at least context type `testContextX`) |

## Retrieve `AppMetadata` ![2.0](https://img.shields.io/badge/FDC3-2.0-blue)

Expand All @@ -21,7 +27,7 @@ You will need to pre-populate the AppDirectory with the following items:
| App | Step | Details |
|-----|----------------|---------------------------------------------------------------------------------------------------|
| Test | 1.Open1 | Open a first instance of App A using <br/> `const appIdentifier1 = await fdc3.open({appId: "<A's appId>"})` <br/>and confirm that its `AppIdentifier` contains an `instanceId`. |
| Test | 2.Open2 |Open a second instance of App A using <br>`const appIdentifier2 = await fdc3.open({appId: "<A's appId>"})` <br/>and confirm that its `AppIdentifier` contains an `instanceId` and that its value differs from that returned for the first instance. |
| Test | 2.Open2 |Open a second instance of App A using <br />`const appIdentifier2 = await fdc3.open({appId: "<A's appId>"})` <br/>and confirm that its `AppIdentifier` contains an `instanceId` and that its value differs from that returned for the first instance. |
| Test | 3.getAppMetadata1 | Retrieve metadata for the first instance of the app with<br/> `const metadata1 = fdc3.getAppMetadata(appIdentifier1)` |
| Test | 4.Confirm1 | Compare the `AppMetadata` object to the expected definition for the fields provided above during setup and ensure that the metadata matches. |
| Test | 5.getAppMetadata2 | Retrieve metadata for the second instance of the app with <br/>`const metadata2 = fdc3.getAppMetadata(appIdentifier2)` |
Expand All @@ -34,7 +40,7 @@ You will need to pre-populate the AppDirectory with the following items:
| App | Step | Details |
|-----|----------------|---------------------------------------------------------------------------------------------------|
| Test | 1.Open1 | Open the first instance of App A using <br/> `const appIdentifier1 = await fdc3.open({appId: "<A's appId>"})` <br/>and confirm that its `AppIdentifier` contains an `instanceId`. |
| Test | 2.Open2 |Open a second instance of App A using <br>`const appIdentifier2 = await fdc3.open({appId: "<A's appId>"})` <br/>and confirm that its `AppIdentifier` contains an `instanceId` and that its value differs from that returned for the first instance. |
| Test | 2.Open2 |Open a second instance of App A using <br />`const appIdentifier2 = await fdc3.open({appId: "<A's appId>"})` <br/>and confirm that its `AppIdentifier` contains an `instanceId` and that its value differs from that returned for the first instance. |
| Test | 3.FindInstances | Retrieve details of open instances of app A with <br/> `let instances = await fdc3.findInstances({appId: "<A's appId>"})` <br/> confirm that both `appIdentifier1` and `appIdentifier2` are both present in the array. |
| Test | 4.RaiseIntent | Use `appIdentifier1` to raise an intent and target that instance, with<br/> `const resolution = fdc3.raiseIntent("aTestingIntent", {"type": "testContextX"}, appIdentifier1)` |
| Test | 5.Confirm1 | Check that `resolution.source` matches `appIdentifier1` |
Expand All @@ -47,7 +53,7 @@ You will need to pre-populate the AppDirectory with the following items:
| App | Step | Details |
|-----|----------------|---------------------------------------------------------------------------------------------------|
| Test | 1.getInfo |Retrieve the `ImplementationMetadata` for the DesktopAgent with <br/> - ![1.2](https://img.shields.io/badge/FDC3-1.2-green) `let implMetadata = fdc3.getInfo()` <br/> - ![2.0](https://img.shields.io/badge/FDC3-2.0-blue) `fdc3.getInfo().then((implMetadata) => { subsequent steps }` <br />**Note that the use of `then` is deliberate and intended to confirm that a promise returned (as this function switched from synchronous to asynchronous in 2.0)**|
| Test | 2.CheckVersion | Check that the `fdc3Version` variable is present and at or greater than: <br /> - ![1.2](https://img.shields.io/badge/FDC3-1.2-green) 1.2 <br /> - ![2.0](https://img.shields.io/badge/FDC3-2.0-blue) 2.0 <br />(which you can do with the [`versionIsAtLeast` function from FDC3's Methods.ts](https://github.com/finos/FDC3/blob/add64f8302c6dcdc8437cf0e245101e927b69ec2/src/api/Methods.ts#L207):<br>`const isFDC3v2 = versionIsAtLeast(implMetadata, "2.0")` |
| Test | 2.CheckVersion | Check that the `fdc3Version` variable is present and at or greater than: <br /> - ![1.2](https://img.shields.io/badge/FDC3-1.2-green) 1.2 <br /> - ![2.0](https://img.shields.io/badge/FDC3-2.0-blue) 2.0 <br />(which you can do with the [`versionIsAtLeast` function from FDC3's Methods.ts](https://github.com/finos/FDC3/blob/add64f8302c6dcdc8437cf0e245101e927b69ec2/src/api/Methods.ts#L207):<br />`const isFDC3v2 = versionIsAtLeast(implMetadata, "2.0")` |
| Test | 3.CheckProvider | Check that the `provider` variable is present and not an empty string |
| Test | 4.CheckFeatures | ![2.0](https://img.shields.io/badge/FDC3-2.0-blue) Check that the `optionalFeatures`, `optionalFeatures.OriginatingAppMetadata` and `optionalFeatures.UserChannelMembershipAPIs` variables are all present and that the latter two provide boolean values |

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,11 @@
# Open Tests
---
id: Open-Tests
sidebar_label: Open Tests
title: Open Tests
hide_title: true
---

# Open Tests

## A Opens B

Expand Down
29 changes: 29 additions & 0 deletions docs/api/conformance/Overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
id: Conformance-Overview
sidebar_label: Overview
title: FDC3 Conformance Tests
hide_title: true
---

# FDC3 Conformance Tests

This section contains test definitions that are used to test for conformance of a Desktop Agent API implementation with FDC3.

:::warning

Additions to the conformance tests for functionality introduced in FDC3 2.2 are still to be defined.

Further, as FDC3 2.1 does not introduce changes to the Desktop Agent API, the conformance test set for FDC3 2.0 remains current at this time. Please see the [FDC3 2.1 Changelog entry](https://github.com/finos/FDC3/blob/main/CHANGELOG.md#fdc3-standard-21---2023-09-13) for more details.

:::

You can find the implementation of these tests in the [FDC3 Conformance Framework](https://github.com/finos/FDC3-conformance-framework) project.

There are currently 6 sections to the tests. Where tests apply to a particular version of FDC3, this is labelled with icons in the header, like so: ![1.2](https://img.shields.io/badge/FDC3-1.2-green) ![2.0](https://img.shields.io/badge/FDC3-2.0-blue)

- [Basic Tests](Basic-Tests.md)
- [Open Tests](Open-Tests.md)
- [User Channel Tests](User-Channel-Tests.md)
- [App Channel Tests](App-Channel-Tests.md)
- [Metadata Tests](Metadata-Tests.md)
- [Intents Tests](Intents-Tests.md)
Loading