Skip to content

Commit

Permalink
Initial CRAN release
Browse files Browse the repository at this point in the history
  • Loading branch information
sebffischer authored Jan 14, 2025
2 parents 15c7771 + a541829 commit a8560e9
Show file tree
Hide file tree
Showing 4 changed files with 37 additions and 19 deletions.
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: mlr3inferr
Title: Inference on the Generalization Error
Version: 0.0.0.9000
Version: 0.1.0
Authors@R: c(
person("Sebastian", "Fischer", , "sebf.fischer@gmail.com", role = c("cre", "aut"),
comment = c(ORCID = "0000-0002-9609-3197")),
Expand Down Expand Up @@ -43,8 +43,8 @@ Collate:
'MeasureCIConZ.R'
'MeasureCICorT.R'
'MeasureCIHoldout.R'
'MeasureCINestedCV.R'
'MeasureCIWaldCV.R'
'MeasureCiNestedCV.R'
'ResamplingNestedCV.R'
'ResamplingPairedSubsampling.R'
'bibentries.R'
Expand Down
14 changes: 14 additions & 0 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,20 @@ content = content[, c("key", "label", "resamplings", "only pointwise loss")]
knitr::kable(content, format = "markdown", col.names = tools::toTitleCase(names(content)))
```

## Citing mlr3

If we use mlr3inferr, please cite our paper:

@misc{kuempelfischer2024ciforge,
title={Constructing Confidence Intervals for 'the' Generalization Error -- a Comprehensive Benchmark Study},
author={Hannah Schulz-Kümpel and Sebastian Fischer and Thomas Nagler and Anne-Laure Boulesteix and Bernd Bischl and Roman Hornung},
year={2024},
eprint={2409.18836},
archivePrefix={arXiv},
primaryClass={stat.ML},
url={https://arxiv.org/abs/2409.18836},
}


## Bugs, Questions, Feedback

Expand Down
36 changes: 20 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,23 +89,27 @@ Note that:

## Inference Methods

``` r
content = as.data.table(mlr3::mlr_measures, objects = TRUE)[startsWith(get("key"), "ci."),]
content$resamplings = map(content$object, function(x) paste0(gsub("Resampling", "", x$resamplings), collapse = ", "))
content[["only pointwise loss"]] = map_chr(content$object, function(object) {
if (get_private(object)$.requires_obs_loss) "yes" else "false"
})
content = content[, c("key", "label", "resamplings", "only pointwise loss")]
knitr::kable(content, format = "markdown", col.names = tools::toTitleCase(names(content)))
```

| Key | Label | Resamplings | Only Pointwise Loss |
|:------------|:------------------|:------------------|:--------------------|
| ci.con_z | Conservative-Z CI | PairedSubsampling | false |
| ci.cor_t | Corrected-T CI | Subsampling | false |
| ci.holdout | Holdout CI | Holdout | yes |
| Key | Label | Resamplings | Only Pointwise Loss |
|:-----------|:------------------|:------------------|:--------------------|
| ci.con_z | Conservative-Z CI | PairedSubsampling | false |
| ci.cor_t | Corrected-T CI | Subsampling | false |
| ci.holdout | Holdout CI | Holdout | yes |
| ci.ncv | Nested CV CI | NestedCV | yes |
| ci.wald_cv | Naive CV CI | CV, LOO | yes |
| ci.ncv | Nested CV CI | NestedCV | yes |

## Citing mlr3

If we use mlr3inferr, please cite our paper:

@misc{kuempelfischer2024ciforge,
title={Constructing Confidence Intervals for 'the' Generalization Error -- a Comprehensive Benchmark Study},
author={Hannah Schulz-Kümpel and Sebastian Fischer and Thomas Nagler and Anne-Laure Boulesteix and Bernd Bischl and Roman Hornung},
year={2024},
eprint={2409.18836},
archivePrefix={arXiv},
primaryClass={stat.ML},
url={https://arxiv.org/abs/2409.18836},
}

## Bugs, Questions, Feedback

Expand Down
2 changes: 1 addition & 1 deletion man/mlr_measures_ci_ncv.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit a8560e9

Please sign in to comment.