Skip to content

Diagnostic Tests (DxTest) and the Diagnostic Tests Manager (DxManager)

Jump to bottom
Tim Hallett edited this page May 24, 2020 · 24 revisions

What is the Diagnostics Test Manager and when I use it?

During a Health System Interaction Event, you will often need to simulate the process of a clinician using a diagnostic test to determine some hidden status of the patient that is not readily apparent (i.e. a property in the sim.population.props data frame): e.g. their blood pressure, urine glucose concentration or even very complex things like the drug-sensitivity of bacterial infection). There is quite a lot to consider:

  • Which tests to use?
  • Which consumable codes do these correspond to?
  • What if the preferred test is not available?
  • Do the test report the true status of the person with perfect accuracy or is there any error?

It will also be the case that the results of many investigations will not be compiled and considered systematically in a 'diagnostic algorithm'. All of this could be very fiddly!

It therefore will often make sense to use the Diagnostic Tests Manager.

When not to use a test:

It is definition of a 'symptom' that they are self-evident to the patient and readily reported to or observed by any health officer with whom they come into contact. Therefore, a DxTest would not be applied to a symptom and instead that information can be gathered using the the has_what() methods of the SymptomManager.

Overview

Preparation

The following steps would usually be done at: initialise_simulation()

  • The principle is that there is some 'property' of the individual that is not "visible" to the clinician [unlike a symptom, which is] but which is, nevertheless, potentially subject to investigation.

  • A particular 'DxTest' is something that can be used to examine that property. This may or may not entail the use consumables and the reporting of the test may be subject to error. This information is stored in a helper class called a DxTest. Thus, for example:

    # Declare a DxTest
    my_test = DxTest(
        property='mi_status',               # it will return the value of the property 'mi_status
        cons_req_as_item_code=item_code,    # it requires and consumes a particular item
        sensitivity=0.95,                   # it has a 95% chance of returning a positive when the true value is positive
        specificity=0.80,                   # it has an 80% chance of returning a negative when the true value is negative
    )
  • The consumables that are required and consumed in the course of performing this DxTest can be:
  • Specified as single item_code: use cons_req_as_item_code=. This is an integer value for the consumables code.
  • Specified as consumable footprint: use cons_req_as_footprint=. The footprint must be formatted appropriately.
  • Nothing: do not specify cons_req_as_item_code or cons_req_as_footprint (or set both to None). For more information about consumables, item codes and footprints, see here
  • The DxTests are then registered with the Diagnostic Test Manager, which is a part of the HealthSystem Model: sim.modules['HealthSystem'].dx_manager. They are registered by providing a name for them in the argument that is passed. For example:
   dx_manager.register_dx_test(
                           name_of_my_test_1 = my_test_1,
                           name_of_my_other_test = my_other test
   )

   dx_manager.register_dx_test(
                           name_of_one_more_test = this_is_the_last_test
   )
  • DxTests can be assembled into a 'tuple of DxTests' - to represent that if one test is not available or returns an inconclusive result, a next test can be tried automatically. This is done by passing in a tuple of DxTest to the .register_dx_test() method instead of a single DxTest object. The tests are executed in order, with the next only being used if all previous tests have failed (the consumables were not available or a value of None is returned).

For example:

    # This test is the first choice
    test_for_mi_first_choice = DxTest(
        cons_req_as_footprint=cons_first_choice_test,
        property='mi_status'
    )

    # This test is the second choice (use only if first choice fails)
    test_for_mi_second_choice = DxTest(
        cons_req_as_footprint=cons_second_choice_test,
        property='mi_status'
    )

    # Register tuple of DxTests with DxManager:
    dx_manager.register_dx_test(test_for_mi_status=
                                                   (
                                                       test_for_mi_first_choice,
                                                       test_for_mi_second_choice,
                                                    ))

Execution

The following steps will usually be done during an individual-level HSI_Event (either within the code of the event itself or a helper function that is called in the course of the HSI_Event).

  • A DxTest or tuple of DxTest can be "used" by calling the run_dx_test() method of the DxManager. It is necessary to declare the HSI_Event of which this test forms a part, and the name(s) of DxTest(s) that should be run. For example:
    dx_manager.run_dx_test(
                    dx_tests_to_run='test_for_mi_status',
                    hsi_event=self                    # self will typically be the HSI_Event that is calling the DxManager
    )

    dx_manager.run_dx_test(
                    dx_tests_to_run=['test1', 'test2', 'test3'],
                    hsi_event=self                    
    )
  • Returns:

    • If more than one DxTest is run, then the return will be the a dict() of the form {test_name: test_result}.

    • If only one dx_test is requested, then the return will be the value of the test, or None if the test failed (consumable was not available). You can force the output to be in the form of a dict() by passing use_dict_for_single=True in run_dx_test():

scalar_value = dx_manager.run_dx_test(dx_test='my_test', hsi_event=self, use_dict_for_single=True)

  • If you would like to know which DxTests have been tried during a call to the dx_manager.run_dx_test then pass report_DxTest_tried=True. This give a second return of a dict() of the form {'DxTest_tried': Outcome} where outcome is a bool: True for a result was returned from it, False for no result was returned.

Different Types of Test

TODO: The following types of test are currently supported:

  • Direct read of a property The DxTest will read the value of a property and return it exactly as it is in the sim.population.props dataframe

  • Read of a bool property with classification errors If the property is dtype=bool, then a sensitivity and specificity parameter can be used. The result returned by the test will reflect this error.

  • Read of a continuous property with random errors If the property is dtype=float, then a measure_error_stdev parameter can be used. The result returned by the test will be of the form: test_value = true_value + error where error ~ N(0, measure_error_stdev)

  • Interpretation of a continuous property with random errors If the property is dtype=float but the value that the result of test is True/False depending on the whether the value exceeds a threshold, then the following formulation can be adopted. Note that the reading of the continuous property may have measurement error. If the value of the property is >= threshold then True is returned.

    df = sim.population.props
    df['AboveThreshold'] = 1.0
    df['BelowThreshold'] = -1.0

    my_test_on_above_threshold = DxTest(
        property='AboveThreshold',
        threshold = 0.0
    )

    my_test_on_below_threshold = DxTest(
        property='BelowThreshold',
        threshold=0.0
    )

    # Register DxTest with DxManager:
    dx_manager = DxManager()
    dx_manager.register_dx_test(my_test_on_above_threshold=my_test_on_above_threshold,
                                my_test_on_below_threshold=my_test_on_below_threshold
                                )

    # Run it on one person and confirm the result is as expected
    person_id = 0
    hsi_event.target = person_id
    assert True is dx_manager.run_dx_test(
        dx_tests_to_run='my_test_on_above_threshold',
        hsi_event=hsi_event
    )

    assert False is dx_manager.run_dx_test(
        dx_tests_to_run='my_test_on_below_threshold',
        hsi_event=hsi_event
    )
  • Categorial Properties A DxTest can be used to assess if a categorial property is equal to some particular category, by specifying a target_category thus:
    my_test = DxTest(
        property='CategoricalProperty',
        target_category='category_that_will_give_a_True_result_in_this_test'
    )

The sensitivity and specificity of the test in detecting that the categorial property is at the target_cateogory can be specified in the usual way:

    my_test_w_no_sens = DxTest(
        property='CategoricalProperty',
        target_category='level2',
        sensitivity=0.5,
        specificity=0.6
    )

Notes

  • A module must register any DxTest or tuple of DxTests that will be used by any of its associated HSI_Events.
  • The DxManager will not allow duplicates of Test to be registered and will issue a warning. Thus, two modules can safely declare the same test without conflict.
  • The RandomState used by the DxManager to evaluate tests is that of the HSI event that is passed to the DxManager.run_dx_test (which will typically be that of the 'parent' module).
  • The HSI_Event that is using the DxManager must be an individual level HSI
  • No checking is done on the validity of the consumable item code or the consumable footprint when creating the DxTest. The usual checking is done when the test is run.
  • See test file for further examples and usage: https://github.com/UCL/TLOmodel/blob/hallett-DiagnosisManager/tests/test_dxmanager.py

Further Features:

What else would be useful here? Let @tbhallett know!

TLO Model Wiki

Clone this wiki locally