Sample pre-ordering vs ordering under the hood

[vkartha]

Hello!! I actually wanted to test run this as part of trying something out in my own work, and I was specifically wondering looking at the newer implementation, how one can run the search separately using either the decreasing ranking of the input scoring or increasing ranking (testing for left-skew in either case). I think in the previous implementation we provided the desired ranking order for this reason, so was curious about the newer implementation in this regard.

I was wondering if we are now expected to pre-rank the input variable (sort(value,decreasing=TRUE/FALSE)) - based on the example run, the variable utils::data(TAZYAP_BRCA_ACTIVITY) doesn't seem to be pre-sorted.

meta_plot always plots the var in decreasing order, which is also why I was curious

Thanks, and very pleased to see this cleaned up so well!

[vkartha]

From a quick scan, it appears to be under ks_rowscore, line 96

# KS is a ranked-based method
  # So we need to sort input_score from highest to lowest values
  input_score <- sort(input_score, decreasing=TRUE)
  
  # Re-order the matrix based on the order of input_score
  FS <- FS[, names(input_score), drop=FALSE]  

I believe this will provide the same best meta-feature no matter how you rank and input the dataset then (appears so), since it's ranking it internally, and hence you can't test an alternative 'observed' ranking as is?

[tetomonti]

Hi Vinay, glad you’re testing it. Please, let us know if you see anything else worth addressing Re your question, if you want the increased ranking (rather than decreasing), couldn’t you simply provide the inverse of the input score? I.e., if topn_res <- CaDrA::candidate_search(FS, input_score,…) searches based on decreased ranking, topn_res <- CaDrA::candidate_search(FS, -input_score,…) will search based on increased ranking. Best, —ste

…

On Jan 14, 2024, at 12:29 AM, Vinay Kartha ***@***.***> wrote: From a quick scan, it appears to be under ks_rowscore, line 96 # KS is a ranked-based method # So we need to sort input_score from highest to lowest values input_score <- sort(input_score, decreasing=TRUE) # Re-order the matrix based on the order of input_score FS <- FS[, names(input_score), drop=FALSE] I believe this will provide the same best meta-feature no matter how you rank and input the dataset then (appears so), since it's ranking it internally, and hence you can't test an alternative ranking as is? — Reply to this email directly, view it on GitHub <#44 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACIAGH4SMIMMFSWP7UIIDKDYONUMRAVCNFSM6AAAAABBZZWAVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJQHA2TGNRSGY>. You are receiving this because you are subscribed to this thread.

[vkartha]

Hi @tetomonti , I think that was what I was mentioning - regardless of the input vector (order) you get the same meta-feature, and I think it's because of that ranking command embedded under ks_rowscore where it always re-orders in descending order of the input metric (see my last comment), so I am not sure you can test different rankings?

[tetomonti]

Yes, it reorders based on sort, but sort(input_score) and sort(-input_score) will be opposite, so you can input one or the other depending on what you want. Or am I misunderstanding your point/need?Sent from my iPhoneOn 14 Jan 2024, at 12:02, Vinay Kartha ***@***.***> wrote: Hi @tetomonti , I think that was what I was mentioning - regardless of the input vector (order) you get the same meta-feature, and I think it's because of that ranking command embedded under ks_rowscore where it always re-orders in descending order of the input metric (see my last comment), so I am not sure you can test different rankings? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[vkartha]

I see - so there's no way of telling it what order to use, and if you want to flip the sign of the search we have to artifically negate the values first? This would make sense given it always re-sorts the input vector in decreasing order under the hood, which I didn't realize it was doing initially.

Wouldn't it be better if we just assumed the input vector was already ordered as intended (i.e. don't re-order internally), that way the resulting score values and plots don't have to be negative (in the reverse case) / interpretation is easier?

[tetomonti]

CaDrA is supposed to work w/ scoring functions other than rank-based, e.g., Mutual Information or Correlation, in which case the actual input score is needed, not simply the ranking of the samples. —ste

…

On Jan 14, 2024, at 8:26 PM, Vinay Kartha ***@***.***> wrote: I see - so there's no way of telling it what order to use, and if you want to flip the sign of the search we have to artifically negate the values first? This would make sense given it always sorts in decreasing order under the hood, which I didn't realize it was doing initially. Wouldn't it be better if we just assumed the input vector was already ordered as intended (i.e. don't re-order internally), that way the resulting score values and plots don't have to be negative (in the reverse case) / interpretation is easier? — Reply to this email directly, view it on GitHub <#44 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACIAGH5NKX5JFK7FWEGKHR3YOSAV3AVCNFSM6AAAAABBZZWAVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJRGE3DENRWGI>. You are receiving this because you were mentioned.

[vkartha]

Makes sense, thanks @tetomonti. Still wondering though, why not (in the case of rank-based approaches) assume the input is pre-ranked so that values need not be altered at all? (similar to pre-ranked GSEA, etc.)? Basically skipping the re-ordering in desc order under the hood for ks, in which case it should also work just fine for non-rank based approaches as well.

Otherwise you have to go with this:

[tetomonti]

I don’t follow. If you pass -input_score (i.e., the inverse of the original input score), the highest values will still be on the left and the lowest on the right. How did you manage to generate the attached plot?

…

On Jan 15, 2024, at 12:26 PM, Vinay Kartha ***@***.***> wrote: Makes sense, thanks @tetomonti <https://github.com/tetomonti>. Still wondering though, why not (in the case of rank-based approaches) assume the input is pre-ranked so that values need not be altered at all? (similar to pre-ranked GSEA, etc.)? Basically skipping the re-ordering in desc order under the hood for ks. Otherwise you have to go with this: image.png (view on web) <https://github.com/montilab/CaDrA/assets/9615730/897d94b0-e646-43be-96da-da6c110d62ae> — Reply to this email directly, view it on GitHub <#44 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACIAGH3HJS77VR4HUAHCW2DYOVRCXAVCNFSM6AAAAABBZZWAVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJSGU2TIMZQGM>. You are receiving this because you were mentioned.

[vkartha]

In the plot above, the highest values are on the left, indeed, post-inversion (using -input_score means larger originally positive values are now more negative)? The point I was making is that you are then showing negative YAP/TAZ activities which don't really exist in the data (as that is what was passed to candidate_search), and metaplot will use whatever input scores were fed to candidate_search. Happy to chat offline if easier. All I did above was reverse the input score shown in the tutorial (yap/taz e.g.) and run the candidate search + metaplot commands. You can try it yourself, to confirm the output