Run wmda consensus test exercise 1 and 2 through Atlas

[zabeen]

Source data: https://pubmed.ncbi.nlm.nih.gov/27219013/

Exercise requires comparison of every possible patient and donor combination in order to generate match counts and antigen/allele match counts. Cannot run the test patients as search requests as Atlas will not allow 0/10 searches or even 1/10. Will instead use the batch scoring endpoint to generate the necessary counts, as scoring depends on the same logic as matching.

Can use AN Dev as the donor data will be submitted within the scoring request itself, and so no test donor data needs to be uploaded to the donor store.

[zabeen]

Results files have been generated with the required mismatch counts. Still to-do: write code to analyse the results.

[zabeen]

Testing Results - Exercise 1

Run on Atlas v1.6.0, AN DEV, master commit 3b265116, against v3.50.0 of IMGT/HLA nomenclature files

Report of discrepant results - generated using new functions added to Atlas.ManualTesting.

Summary:

• Of the 10 million patient donor pairs (PDPs) analysed, approx. 5000 had discrepant total mismatch counts.
• These PDPs were groups by patient-donor locus HLA typings, there were 769 unique pairs where the locus MM count differed between consensus and Atlas results.
• Manual inspection of the 769 cases showed that there were actually only three causes of all the discrepancies:
  1. B*51:XX vs. *53:XX, because B*51:42P includes *53:62
  2. A*02:XX vs. *74:XX, because A*02:65P includes *74:21
  3. B*39:XX vs. *38:XX, because expanded B*38:XX includes four alleles that have been renamed to B*39 alleles (see Should deleted alleles be used when matching/scoring XX codes? #967 for more info).

The final discrepant case (B*39:XX vs. *38:XX) has been referred to an Anthony Nolan clinical scientist to determine if such a match is clinically valid or not.

[zabeen]

Testing Results - Exercise 2

Run on Atlas v1.6.0, AN DEV, master commit e9e19afa, against v3.50.0 of IMGT/HLA nomenclature files

Report of discrepant results - generated using new functions added to Atlas.ManualTesting.

Total Mismatch Summary

• Of the 10 million patient donor pairs (PDPs) analysed, approx. 3.5K had discrepant total mismatch counts.
• Grouping by patient-donor locus HLA typings, there were 354 unique pairs where the locus MM count differed between consensus and Atlas results.
• Manual inspection of these cases showed that there were actually only four causes of all the discrepancies (see spreadsheet for details), and all involved matching serology to DNA typing.

Antigen Mismatch Summary

• Of the 10 million patient donor pairs (PDPs) analysed, approx. 12.2K had discrepant antigen mismatch counts.
• Grouping by patient-donor locus HLA typings, there were 1,861 unique pairs where the locus MM count differed between consensus and Atlas results.
• Manual inspection of these cases showed that there were actually only seven rel_dna_ser mappings that were the cause of all the discrepancies (see spreadsheet for details).

Overall Summary

• First runs of exercise 2 did highlight bugs in the antigen match calculator, which have now been resolved (Antigen match incorrectly labelled as mismatch #971, Inconsistent antigen match values during WMDA exercise set 2 #978).
• The results above are from the final run, where all discrepant cases can now be explained by "unexpected" serological assignments present in the version of the rel_dna_ser file referenced during testing.