Developer Guide
A general introduction is found in this link and the toolkit configuration information is available here.
NIST HL7 FHIR Toolkit offers a test engine that can execute HL7 FHIR TestScripts, validates FHIR resources based on a FHIR implementation guide (IG) package, and provides a platform for conformance and interoperability testing of system under test (SUT) through a collection of tests and assertions. At the time of FHIR Toolkit development, the FHIR TestScript resource was not adequate for developing a testing tool where reuse of common TestScript parts across many test collections was critical in maintaining the tests. To make this possible, resource extensions were developed. Extensions are covered in detail here. In most cases, when you see TestScript below, it refers to TestScript with extensions as developed by NIST.
Screenshot of the FHIR Toolkit Web application home page 
The main NIST FHIR Toolkit user interface components you should be aware of are TestSession, Channel, Events, and TestCollections. Test Session - an area to work in giving isolation from other users. Similar to a Test Session in NIST XDS Toolkit. A test session in FHIR Toolkit includes channel configurations.
A channel through the Proxy. FHIR Toolkit is a proxy to the SUT FHIR Base address. All traffic is routed through the Proxy which provides logs of the messages and translation. There are two kinds of channels: FHIR - data passed without modification and MHD - translation is done between MHD and XDS formats. New channel types can be added in the Java source code as an implementation of the IBaseChannel Java interface. For display purposes and by toolkit naming convention, a channel Id is prefixed with a test session if it originates from an Included test session. Channel name part without the test session prefix is displayed for channels local the current test session. For every channel interaction such as a HTTP GET or a HTTP POST, an event is created to be viewed in the channel log.
List the events for a chosen Channel. Each message through the Proxy generates an event. Full details of an event can be displayed by selecting it.
All tests reside in a Test Collection. There are two types of Test Collections: client and server. Client tests are used to evaluate an SUT that initiates a transaction such as a IHE MHD IG Document Source actor. Server tests evaluate SUTs that accept transactions such as a Document Recipient actor. Each Test Collection targets a particular actor. A test collection property file (represented as a Java Properties text file format) exists to identify the type of test collection and other properties. The FhirIgName is another test collection property and this correlates to the Channel FHIR IG Version support property. Thus, when selecting a SUT channel to test, all test collections which match the same FhirIgName are displayed in the user interface.
Channel configuration as displayed in the FHIR Toolkit channel editor:
Channel configuration properties text file:
Please note, if the channel configuration FHIR IG Version Support property value is missing, all test collections are displayed by default and the channel FHIR IG validation automatically defaults to the first entry. For example, validation will be performed using MHD v3.x IG package. If a newer IG test collection or an unrelated test collection is run on MHD v3.x IG version configured channel, then it will display test failures since it validates using the default IG package. It is important to configure the channel and the TestCollection.properties file with the appropriate value. Edit FHIR Toolkit channel configuration, as shown in the screenshot above, and select the proper FHIR IG version, and this will only display test collections which target the same FHIR IG. For testing purposes, new channels for the same FHIR base address can be created with different names (you may use the Copy icon in the Edit channel configuration page) and set the IG version, this way it will not disturb the existing channel configuration and allows testing of other test collections (perhaps a newer version of IG test collection). A screenshot of TestCollection.properties is shown below.
TestCollection properties text file:
Add your own tests (TestScripts) to your local FhirTestCollections External Cache directory. Also, you can update the assertions.json file which has all the IG validation assertion references, this is useful for mapping complex assertions which are not covered through the IG validation package downloaded from the FHIR IG registry. Below is an example screenshot of a basic assertion reference displayed in NIST FHIR Toolkit. The URLs to the online-version of the IG profile are automatically built based on the assertions JSON file, but the IG requirements text excerpt is stored in the assertion file as a static reference. Typically, there is a separate test collection for each release of the IG package to isolate tests and assertion validation references.
Assertion ID reference display:
In this example, the Asbestos Assertion ID RM4_1 links to the TestScript assertion in the TestScript.
TestScript assertion (standard TestScript feature):
Test Definition source code location
A new FHIR Toolkit feature has been created to help test ad-hoc TestScript Fixtures, called UserSuppliedTestFixture, this is a special type of TestCollection which allows the user to submit HTTP POST requests of an ad-hoc FHIR resource bundle in a user editable text box and use the Inspector tool to perform IG package validation. (See below screenshot.) Since this is the first version the feature, IG validation requires you to manually click through links to open the submitted request and received response in the Inspector tool. The idea is to use this tool to test the response of a SUT with an editable-resource fixture and incorporate those working updates to the test fixture to be referenced by TestScript.
UserSuppliedTestFixture tool:
FHIR base validation feature is supported in the March 2023 Release and the following describes the internal workings of an example TestScript. In the following TestScript.xml file, source code location src/main/webapp/data/TestCollections/MHDv410_DocumentRecipient_minimal/1_Prerequisite_Single_Document_with_Binary/TestScript.xml.
There are two TestScript variables declared which are specifically related to the IG package validation, internalPackageSpecificIgValidation and channelBasedIgValidation. The first is only used for internal validation and second is only used for the validation performed by an external FHIR base (outside of NIST FHIR Toolkit).
The internal validation variable is coded to automatically pull the IG version from the TestCollection.properties and passes it down to the other TestScript modules imported by the main TestScript.
The mapping of the IG version to the actual TestScription validation is defined in src/main/webapp/data/TestCollections/Library/AggregateModule/StructureDefinitionValidation.xml. Notice internalPackageSpecificIgValidation is passed to the MHDPDBSuccess TestScript component call and the channelBasedIgValidation is not used at this time. This means the TestScript uses internal IG package validation not external source based channel validation. (Local IG packages are placed here in the source code: src/main/resources/npm-fhir-ig-package.)
To use an external validation server, you would have to pass channelBasedIgValidation as the last argument to MHDPDBSuccess TestScript module or component call instead of the internal-variable. The default is to use the validation channel as defined in TestCollection.properties, in this case, the property is defined as FhirValidationChannelId: default__default. (In other words, the FHIR base of the "default" session and "default" FHIR channel name). A new FHIR type channel with the appropriate base address for external validation can be created and addressed in the TestCollection property file. This will go through the ValidateFhirResourceRequest Java class.
NIST FHIR Toolkit introduced a TestScript Debugger feature to help run TestScript actions step-by-step to be able to view each step action and evaluate expressions against request and/or response in a separate pop-up style evaluation window. In other words, the feature provides an opportunity to set breakpoints, pause the TestScript execution, and run user provided FHIRPath expressions on the FHIR resources at hand. This is especially useful in debugging complex TestScript assertions written in HL7 FHIRPath language. The debugger is inactive in the normal release distribution configuration. The TestScript debugger requires a TLS connection between the FHIR Toolkit and the developer’s Web browser, additional security configuration is required on the developer machine.
Adding TestScript breakpoints on the left column enables the Debugging feature button:
Debugging the TestScript enables debugger controls (Resume, Step-over, Skip all breakpoints, Stop), runs TestScript until a breakpoint is hit, and pauses execution before the breakpoint is resumed:
Resumes execution until next breakpoint is hit
Steps over next debuggable-item in TestScript
Skips any remaining breakpoints and finishes run
Stops execution and exits debugger in the current state
Eval button is displayed for assertions:
Opening the Eval pop-up window:
Assertion FHIRPath Expression Editor:
A copy of the NIST FHIR Toolkit Web application is hosted by IHE USA FHIR Toolkit (iheusa.org). (This website availability is controlled by an external host, outside of NIST, and is not guaranteed to be always accessible.)