-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test case "Bad profile" for DocumentRecipient comprehensive test collection #228
Comments
Correct. |
In future, please post to the IHE MHD group first so that others might take notice of the problem as well. |
Thanks! The tests are running fine on my setup now (interestingly, the "Missing Profile" test case was working fine for me even before the fix, though).
Will do. Regards, Andreas |
Let me fully explain this bug fix and why the Missing Profile appeared to have passed on your side. The Bad Profile negative test (and the Missing Profile test) originally had incorrect comprehensive DocumentReference metadata, it was missing the DocumentReference content attachment creation element. When you had reported the issue, only the Bad Profile had correct metadata but incorrect expected assertion fail Id, but the Missing Profile test had both incorrect metadata and incorrect expected assertion Id. You can see this in the referenced commit 82a768c. As a requirement to Comprehensive DocumentReference metadata, XDS DocumentEntry creationTime is required in the Optionality Table: The Toolkit tests incorrectly set the expected assertion fail to
When you had run the Missing Profile test, the (incorrect) assertion test subject failed, and test passed (incorrect). This is what appeared to have happened with your Missing Profile test result. Once the required
Sorry for the confusion. |
Ah, now I see: the Missing Profile test was testing something different than advertised, but it was doing so consistently. Thanks for the explanation! |
Hi,
I have been been using the FHIR Toolkit to test my implementation of an MHD Document Recipient/Document Responder, and I have noticed a strange test test result for the "Bad Profile" test case in the "MHDv410 DocumentRecipient comprehensive" test collection.
The test sends an ITI-65 request containing the erroneous Bundle profile declaration "http://ihe.net/fhir/StructureDefinition/IHE_MHD", and, by its decription, expects the MHD server to respond with an HTTP 400 response (which my MHD server does). Looking into the test's script, I can see that it expects the following set of assertions to fail (expectedFailAssertionIdList): 'RM410_4'|'CMPDR_3_1'|'TR_1'.
However, the test as a whole fails for me. Inspecting the details of the test results, I can see the test failure resulting from the step "PDB Validations." Here, the following three substeps are marked as failing:
So, by the looks of it, the second of the three above mentioned assertion failures is causing the test to fail here. However, the test's setup description ("Bundle.meta.profile contains an invalid value") is in direct contradiction to what this assertion checks for (and neither is it even in the DocumentRecipient's power to change this profile declaration on the request side).
Am I correct in my conclusion that the above described test failure is just due to the setup of this specific test in the toolkit, and not indicative of any deficiencies in the system under test?
The text was updated successfully, but these errors were encountered: