QC for TMAs and Control Samples

[srivarra]

If you haven't already, please read through our contributing guidelines before opening your PR

What is the purpose of this PR?

Closes #277.
Closes #362.
Adds QC metric for TMAs, and for tissue wide-batch effects.

How did you implement your changes

Added helper functions for ranking TMAs and computing tissue QC metrics.
Added plotting functions for QC TMA metrics, and updated the batch effect plot functions.

Sample Figures from the TONIC Cohort

TMA QC Plot

Violin Plot, for CD*

In addition, other notable changes include adding a Random number generator Fixture in conftest.py, so for any test, we can just add that fixture as a function argument like below:

def test_function_requries_random_numbers(rng: np.random.Generator):
    my_cool_random_numbers = rng.random(size=(10,))
    ...

Adjusted some Pandas function calls to be Pandas 2.0 compatible (Notably you cannot pd.append DataFrames, you must pd.concat them). In the future we can take advantage of 2.0 features.

Dependency Modifications:

• Bumped scipy to >=1.10.1
• Added loguru as a dev dependency.

Remaining issues

• Run the QC TMA on the Tonic Cohort
• Ran into some issues w.r.t image_stitching.get_tiled_names where some FOVs would return

      {"tonsil_bottom": "fov-1-scan-1"}

  instead of say,

      {"RnCm": "fov-1-scan-1"}

  Is this expected behavior for some jsons?

Next Steps

• Need to create a notebook which encapsulates all of these QC metrics (design doc needed)
• Need to add QC metrics as a function of acquisition order (design doc needed)

[ngreenwald]

This looks great. Can you open a new issue and self-assign for adding a ranks for the TMA-wide QC tissue ranks?

Right now, the only way to see if a specific TMA has a control sample that is higher (or lower) than all the others is to look by eye at the color of each dot and see if a specific colored dot is most often at the top or bottom of the violin. Having a way to automatically check this would be helpful.

I think you can copy what you did in the first part, averaging the rank across different markers. It would be useful to compute that same ranking for the control tissues. So instead of creating a rank across the 50 FOVs in an TMA, it would be across the 3 or 4 different LN_Top samples.

[ngreenwald]

The visualization looks good, let me know once you've generated the plots for each TMA so we can confirm there are no spatial effects.

[camisowers]

Looks good! Are we adding in a notebook for this as well?

[srivarra]

@camisowers Yeah the notebook will be the final PR for these extra QCs.

[alex-l-kong]

Looks good, one comment regarding condensing a deeply nested loop.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[ngreenwald]

Functionality looks great, added a bunch of comments about simplifying the notebook.

[camisowers]

Code looks good to me!

[srivarra]

@ngreenwald How does the notebook look after these updates?

[ngreenwald]

Looks good, some more notebook simplifications

[ngreenwald]

A couple issues from first round that didn't get completely addressed. Make sure to glance over the full notebook before re-requesting review. Almost there!

[ngreenwald]

Looks good