Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): Allow to flatten index in reports #1300

Merged
merged 12 commits into from
Feb 20, 2025

Conversation

glemaitre
Copy link
Member

closes #1271

Add a parameter to allow to get a flat index to ease indexing.

@glemaitre glemaitre marked this pull request as draft February 7, 2025 22:36
Copy link
Contributor

github-actions bot commented Feb 7, 2025

Documentation preview @ 7e58ffe

Copy link
Contributor

github-actions bot commented Feb 18, 2025

Coverage

Coverage Report for backend
FileStmtsMissCoverMissing
venv/lib/python3.12/site-packages/skore
   __init__.py150100% 
   __main__.py880%3–19
   _config.py260100% 
   exceptions.py440%4–23
venv/lib/python3.12/site-packages/skore/cli
   __init__.py550%3–8
   cli.py22220%3–70
   color_format.py49490%3–116
venv/lib/python3.12/site-packages/skore/persistence
   __init__.py00100% 
venv/lib/python3.12/site-packages/skore/persistence/item
   __init__.py56393%96–99
   altair_chart_item.py19191%14
   item.py22195%86
   matplotlib_figure_item.py36195%19
   media_item.py220100% 
   numpy_array_item.py27194%16
   pandas_dataframe_item.py29194%14
   pandas_series_item.py29194%14
   pickle_item.py220100% 
   pillow_image_item.py25193%15
   plotly_figure_item.py20192%14
   polars_dataframe_item.py27194%14
   polars_series_item.py22192%14
   primitive_item.py23291%13–15
   sklearn_base_estimator_item.py29194%15
   skrub_table_report_item.py10186%11
venv/lib/python3.12/site-packages/skore/persistence/repository
   __init__.py20100% 
   item_repository.py59591%15–16, 202–203, 226
venv/lib/python3.12/site-packages/skore/persistence/storage
   __init__.py40100% 
   abstract_storage.py220100% 
   disk_cache_storage.py33195%44
   in_memory_storage.py200100% 
venv/lib/python3.12/site-packages/skore/persistence/view
   __init__.py220%3–5
   view.py550%3–20
venv/lib/python3.12/site-packages/skore/project
   __init__.py30100% 
   _launch.py150199%278
   _open.py90100% 
   project.py88199%285
venv/lib/python3.12/site-packages/skore/sklearn
   __init__.py60100% 
   _base.py1401490%36, 48, 58, 91, 94, 147–156, 168–>173, 183–184
   find_ml_task.py49197%103–>111, 135
   types.py20100% 
venv/lib/python3.12/site-packages/skore/sklearn/_comparison
   __init__.py50100% 
   metrics_accessor.py110198%151, 152–>154
   report.py59099%240–>243
venv/lib/python3.12/site-packages/skore/sklearn/_cross_validation
   __init__.py50100% 
   metrics_accessor.py152099%132–>134, 134–>136
   report.py950100% 
venv/lib/python3.12/site-packages/skore/sklearn/_estimator
   __init__.py50100% 
   metrics_accessor.py2711094%159–168, 194–>203, 202, 243–>245, 271, 298–302, 317, 339, 351, 352–>354
   report.py120099%213–>219, 221–>223
   utils.py11110%1–19
venv/lib/python3.12/site-packages/skore/sklearn/_plot
   __init__.py40100% 
   precision_recall_curve.py119198%229–>246, 317
   prediction_error.py95198%159, 173–>176
   roc_curve.py1260100% 
   utils.py89593%23, 47–49, 53
venv/lib/python3.12/site-packages/skore/sklearn/train_test_split
   __init__.py00100% 
   train_test_split.py36294%16–17
venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning
   __init__.py80100% 
   high_class_imbalance_too_few_examples_warning.py17378%16–18, 80
   high_class_imbalance_warning.py18288%16–18
   random_state_unset_warning.py11187%15
   shuffle_true_warning.py9091%44–>exit
   stratify_is_set_warning.py11187%15
   time_based_column_warning.py22189%17, 69–>exit
   train_test_split_warning.py5180%21
venv/lib/python3.12/site-packages/skore/ui
   __init__.py00100% 
   app.py32320%3–83
   dependencies.py440%3–10
   project_routes.py41410%3–92
   serializers.py77770%3–194
   server.py17170%3–40
venv/lib/python3.12/site-packages/skore/utils
   __init__.py60100% 
   _accessor.py70100% 
   _environment.py26097%29–>34
   _index.py50100% 
   _logger.py21484%14–18
   _parallel.py38388%23–33, 124
   _patch.py11546%19–35
   _progress_bar.py310100% 
   _show_versions.py310100% 
TOTAL289135887% 

Tests Skipped Failures Errors Time
614 3 💤 0 ❌ 0 🔥 54.564s ⏱️

@glemaitre glemaitre marked this pull request as ready for review February 19, 2025 10:11
@glemaitre
Copy link
Member Author

I change a bit the rule to not have any upper case, spaces and hash characters and replace with lower case, underscore, and nothing, respectively.

It would be the type of naming that I would use if I want to index a dataframe. I documented this behaviour in the different docstrings.

thomass-dev and others added 6 commits February 20, 2025 13:29
probabl-ai#1286)

- [x] Rename to `ComparisonReport`
- [x] Rebase on top of probabl-ai#1239 and adapt
- [x] Raise if `report.metrics.accuracy(data_source="train")` is called
with at least one EstimatorReport that does not have training data
- [x] Test
- [x] Docstrings
    - [x] MetricsAccessor
- [x] Move index column "#0" in front of each metric
- [x] Pass report names in comparator
- [ ]  ~Update plots legend~ see probabl-ai#1309 
- The actual `RocCurveDisplay` needs a full refactor to be splitted by
use-case: estimator report, cross-validation report and finally
comparison report. In each of these use-cases, there is two scenarios
with binary classification and multi-class classification. Otherwise, it
will be unmaintainable.
- [ ] ~Investigate missing metrics in `report_metrics`~ **(deferred to
future PR)**
- The logic is split between `report_metrics` and `available_if`; it
should be merged (ideally everything in `available_if`?)
- [ ] ~Refactor to make `CrossValidationReport` depend on it~
**(deferred to future PR)**
- [x] ~Change EstimatorReport `repr`?~ Issue
probabl-ai#1293

Closes probabl-ai#1245 

Co-authored-by: Auguste <auguste@probabl.ai>
Co-authored-by: Sylvain Combettes <48064216+sylvaincom@users.noreply.github.com>
…obabl-ai#1322)

`project.clear(delete_project=True)` now deletes the entire project,
while `project.clear(delete_project=False)` (the default) removes every
item from the project.

Closes probabl-ai#1294
@auguste-probabl
Copy link
Contributor

Time to rebase ^^

Copy link
Contributor

@auguste-probabl auguste-probabl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@auguste-probabl auguste-probabl merged commit 314dde5 into probabl-ai:main Feb 20, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

enh: expose a parameter to return a flat/multiindex in report_metrics
3 participants