Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need better handling of CV folds if any class contains one sample only for DA analyses #122

Closed
aljabadi opened this issue Feb 16, 2021 · 0 comments · Fixed by #227
Closed
Assignees
Labels
bug Something isn't working

Comments

@aljabadi
Copy link
Collaborator

We should either prohibit the condition or handle it more efficiently. stratified.subsampling is also unnecessarily lengthy.

data(liver.toxicity)
X <- liver.toxicity$gene
Y <- liver.toxicity$treatment$Dose.Group
# create a class with one sample only
Y[1] <- 'FOO'

liver.plsda <- plsda(X, Y, ncomp = 2)
liver.val <- perf(liver.plsda, validation = "Mfold", folds = 3)
#> Warning in MCVfold.spls(X, Y, multilevel = multilevel, validation = validation, : At least one class is not represented in one fold, which may unbalance the error rate.
#>   Consider a number of folds lower than the minimum in table(Y): 1
#> Error in dimnames(x) <- dn: length of 'dimnames' [2] not equal to array extent

data(nutrimouse)
Y = nutrimouse$diet
# create a class with one sample only
Y = as.character(Y)
Y[1] <- 'FOO'
data = list(gene = nutrimouse$gene, lipid = nutrimouse$lipid)
nutrimouse.sgccda <- block.splsda(X=data,
                                  Y = Y,
                                  keepX = list(gene=c(10,10), lipid=c(15,15)),
                                  ncomp = 2,
                                  scheme = "horst")

perf = perf(nutrimouse.sgccda)
#> Warning in repeat_cv_perf.diablo(nrep): At least one class is not represented in one fold, which may unbalance the error rate.
#>   Consider a number of folds lower than the minimum in table(Y): 1
#> Error in (function (..., deparse.level = 1) : number of columns of matrices must match (see arg 2)
perf
#> function(object, ...)
#>     UseMethod("perf")
#> <environment: namespace:mixOmics>

Created on 2021-02-16 by the reprex package (v1.0.0)

@aljabadi aljabadi added the bug Something isn't working label Feb 16, 2021
@Max-Bladen Max-Bladen self-assigned this Mar 14, 2022
@Max-Bladen Max-Bladen added the wip work-in-progress label Mar 16, 2022
Max-Bladen added a commit that referenced this issue Mar 21, 2022
Implemented a check within the `perf.mixo_splsda()` function which prevents the method running if any of the classes have a single sample associated with them.

Also added to checks (in new file: `test-perf.mixo_splsda.R`) which ensure basic functionality as well as whether the function prevents a class with one sample being used.
@Max-Bladen Max-Bladen linked a pull request Mar 21, 2022 that will close this issue
@Max-Bladen Max-Bladen removed the wip work-in-progress label Mar 21, 2022
@Max-Bladen Max-Bladen linked a pull request Aug 1, 2022 that will close this issue
@Max-Bladen Max-Bladen added rapid-review for PRs which will take minimal time to review and close ready-to-review for all PRs that are ready to be reviewed. including complex, larger commits labels Aug 2, 2022
Max-Bladen added a commit that referenced this issue Dec 13, 2022
fix: added specific check to `perf()` to notify user when there is a single sample associated with one of the classes in a DA context

test: included a basic test for function and to test functionality of new singple sample check
@Max-Bladen Max-Bladen removed rapid-review for PRs which will take minimal time to review and close ready-to-review for all PRs that are ready to be reviewed. including complex, larger commits labels Dec 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants