Test Context Definition
The Test Context describes the environment that Conformance Tests run in. It is managed by the Conformance Test Tool via the blue box on the top right corner of the window. Changes can be made to the Test Context by clicking on the blue box.
The Test Context looks like this when open for editing.
All elements of the Test Context are managed in the External Cache. In the documentation that follows the location in the External Cache of the relevant element is labeled as ExternalCache:element (ExternalCache:TestLogCache for example where TestLogCache is a directory in the External Cache). The External Cache itself is a file system directory where Toolkit data and configurations are managed.
The Environment manages the codes configurations and client certificates. The codes configuration (codes.xml) (ExternalCache:environment/environment-name/codes.xml) is a table of the acceptable codes configured into the Affinity Domain being tested. The default testkit, which contains templates for all test messages generated by toolkit, is configured against this codes configuration.
A client certificate, if present, contains a single certificate used by the toolkit test engine to initiate TLS connections. TLS connections received by toolkit (more specifically by simulators in toolkit) are managed by a certificate installed directly in Tomcat.
The testkits area of the environment (ExternalCache:environment/environment-name/testkits) optionally contains a copy of the default testkit re-configured for the specific environment's codes configuration. (The default testkit always aligns with the codes configuration in the environment 'default'.) More specifically, the testkits area contains a collection of testkits, each testkit being used by the Test Session of the same name. Said another way, the test kit found at ExternalCache:environment/environment-name/testkits/42 (if it exists) is used by Test Session 42. If ExternalCache:environment/environment-name/testkits/42 does not exist then ExternalCache:environment/environment-name/testkits/default is used. If that does not exist then the internal testkit is used.
Note that if toolkit is being used in conjunction with Gazelle for CAS testing, the Toolkit Test Session name aligns with the Gazelle Test Session name.
If a new codes configuration is created, the corresponding testkit can be generated automatically (which is good since it requires a lot of editing). The tool to generate this testkit is found on the Toolkit Configuration tab of toolkit.
Toolkit can manage multiple environments. There is always a 'current environment' selected. The default environment name is 'default' which is automatically installed when Toolkit is started. The default environment contains the codes configuration used at IHE Connectathons, no client certificates, and no testkits. It contains no testkits because the internal testkit is used.
The Test Session defines a testing session. More specifically it identifies a repository of test results located at ExternalCache:TestLogCache/test-session-name/. Within that directory is found:
- a directory containing the test results for each test that has been run named for the test ID
- a file site.txt which contains the system under test
- a directory orchestration which contains a directory named for each actor type tested in this test session. That directory holds a single file, orchestration.properties, which is used to maintain the state of the testing process for that actor.
Remember that a test session manages test results for a single system. In toolkit a system may contain only one instance of an actor type.
Note that there is a second way to create or select test sessions in the control at the top of the tool. This edits the same information. This control will be removed from the Conformance Tool in the future and should not be used. The same goes for the Environment selector at the top of the tool.
The system, or System Under Test, is a configuration of endpoints and ids that identify the system being tested. This system definition resides at ExternalCache:actors/system-name.xml.
Note: if Toolkit is being used in conjunction with Gazelle for CAS testing then the system in Toolkit maps to a system in Gazelle.
To archive the test results from a testing session we suggest ZIPing and saving the entire External Cache.