Remove the batch suffix from examples #1364

razdoburdin · 2023-07-14T16:42:14Z

Description

In continue of discussion started by @samir-nasibli in #1320 (comment) and #1360.

I prepared an update for tests/run_examples.py:

Now example names without suffix are supported.
The processing of examples without suffix equivalent to the examples with _batch suffix in the priv version.
List of skipping files is introduced.

Issue:
I found, that log_reg_model_builder.py return error. This file wasn't launching before. I have temporarily added it to the skipping list. We should fix this example or remove it.

napetrov · 2023-07-15T14:24:16Z

tests/run_examples.py

+skiped_files = ['__init__.py',
+                'spmd_utils.py',
+                'log_reg_model_builder.py',
+                'n_jobs.py',


but we would like to run those examples to validate that they are work

but we would like to run those examples to validate that they are work

Most of these files are not the examples, but some helper functions. For this reason we shouldn't run it as examples. The only exception is log_reg_model_builder.py. For some unknown reason this file didn't have any suffix, for this reason, it wasn't launching before at all. So we have two independent tasks here:

Remove _batch suffix for more clear naming (this PR).

Investigate the problem with log_reg_model_builder.py and prepare a fix for this problem (some other PR). After the problem will be fixed, we can remove this file from the list of skipping examples.

This plan looks reasonable from my point of view, what is your opinion?

@razdoburdin I think as an option you can move non-example, helpers scripts into utils module examples/utils.

razdoburdin · 2023-07-20T10:47:19Z

/intelci: run

…b.com/razdoburdin/scikit-learn-intelex into remove_the_batch_suffix_from_examples

razdoburdin · 2023-07-28T07:26:17Z

/intelci: run

razdoburdin · 2023-07-28T10:20:30Z

/intelci: run

razdoburdin · 2023-07-28T14:32:30Z

/intelci: run

razdoburdin · 2023-08-08T07:37:42Z

/intelci: run

razdoburdin · 2023-08-08T12:15:38Z

GPU CI job run is here.

ethanglaser · 2023-08-14T13:24:00Z

Should examples in daal4py/sycl also have the batch suffix removed?

razdoburdin · 2023-08-14T13:26:15Z

Should examples in daal4py/sycl also have the batch suffix removed?

I think yes, I suggest removing it in a separate PR.

ethanglaser · 2023-08-14T13:30:00Z

Should examples in daal4py/sycl also have the batch suffix removed?

I think yes, I suggest removing it in a separate PR.

I think it could be added to this one if we are already doing both daal4py and sklearnex examples in this one

ethanglaser · 2023-08-14T13:49:19Z

GPU CI job run is here.

For reference, these are the examples that run in this job that did not run previously:

daal4py/log_reg_model_builder.py
daal4py/model_builders_lightgbm.py
daal4py/model_builders_catboost.py
daal4py/model_builders_xgboost.py
daal4py/spmd_utils.py
daal4py/sycl/sklearn_sycl.py
sklearnex/patch_sklearn.py
sklearnex/verbose_mode.py
sklearnex/n_jobs.py

Of these, it seems like the model builders should be ran. We have a separate step in recipe testing for daal4py/sycl/sklearn_sycl.py so this should be excluded from run_examples file, as well as spmd_utils since its just helper functions. Are sklearnex/ njobs, patch_sklearn, and verbose_mode intended to be ran with run_examples.py (@samir-nasibli)?

samir-nasibli · 2023-08-25T15:33:48Z

/intelci: run

samir-nasibli · 2023-08-25T15:35:26Z

GPU CI job run is here.

samir-nasibli · 2023-08-25T15:38:04Z

GPU CI job run is here.

For reference, these are the examples that run in this job that did not run previously:

daal4py/log_reg_model_builder.py daal4py/model_builders_lightgbm.py daal4py/model_builders_catboost.py daal4py/model_builders_xgboost.py daal4py/spmd_utils.py daal4py/sycl/sklearn_sycl.py sklearnex/patch_sklearn.py sklearnex/verbose_mode.py sklearnex/n_jobs.py

Of these, it seems like the model builders should be ran. We have a separate step in recipe testing for daal4py/sycl/sklearn_sycl.py so this should be excluded from run_examples file, as well as spmd_utils since its just helper functions. Are sklearnex/ njobs, patch_sklearn, and verbose_mode intended to be ran with run_examples.py (@samir-nasibli)?

spmd_utils.py should be renamed or restructured. since this is not helper module.

napetrov · 2023-08-29T13:17:28Z

daal4py/log_reg_model_builder.py - we can technically drop this if there are problems there.

meanwhile lot of failures with - TypeError: create_pytest_switches() missing 1 required positional argument: 'preview'

ethanglaser · 2023-08-29T13:20:32Z

meanwhile lot of failures with - TypeError: create_pytest_switches() missing 1 required positional argument: 'preview'

This should be fixed by rebase (#1418)

napetrov · 2023-08-29T13:22:00Z

@mergify rebase

mergify · 2023-08-29T13:22:08Z

rebase

❌ Base branch update has failed

Git reported the following error:

Rebasing (1/17)
Auto-merging doc/daal4py/algorithms.rst
Auto-merging doc/daal4py/examples.rst
Auto-merging examples/daal4py/decision_tree_regression_traverse.py
Auto-merging tests/run_examples.py
CONFLICT (content): Merge conflict in tests/run_examples.py
error: could not apply 532dbf0... initial
hint: Resolve all conflicts manually, mark them as resolved with
hint: "git add/rm <conflicted_files>", then run "git rebase --continue".
hint: You can instead skip this commit: run "git rebase --skip".
hint: To abort and get back to the state before "git rebase", run "git rebase --abort".
Could not apply 532dbf0... initial

err-code: 0B313

samir-nasibli · 2023-08-29T13:32:15Z

/intelci: run

samir-nasibli · 2023-08-29T14:15:55Z

GPU CI job run is here.

ethanglaser · 2023-08-29T15:40:16Z

GPU CI job run is here.

For reference, these are the examples that run in this job that did not run previously:

daal4py/log_reg_model_builder.py daal4py/model_builders_lightgbm.py daal4py/model_builders_catboost.py daal4py/model_builders_xgboost.py daal4py/spmd_utils.py daal4py/sycl/sklearn_sycl.py sklearnex/patch_sklearn.py sklearnex/verbose_mode.py sklearnex/n_jobs.py

Of these, it seems like the model builders should be ran. We have a separate step in recipe testing for daal4py/sycl/sklearn_sycl.py so this should be excluded from run_examples file, as well as spmd_utils since its just helper functions. Are sklearnex/ njobs, patch_sklearn, and verbose_mode intended to be ran with run_examples.py (@samir-nasibli)?

daal4py/sycl/sklearn_sycl.py
sklearnex/patch_sklearn.py
sklearnex/verbose_mode.py
sklearnex/n_jobs.py
are all still running during run_examples step - I don't think this is desired behavior (at least these did not run previously)

…b.com/razdoburdin/scikit-learn-intelex into remove_the_batch_suffix_from_examples

razdoburdin · 2023-08-30T10:02:46Z

done

razdoburdin · 2023-08-30T10:05:17Z

GPU CI job run is here.

For reference, these are the examples that run in this job that did not run previously:
daal4py/log_reg_model_builder.py daal4py/model_builders_lightgbm.py daal4py/model_builders_catboost.py daal4py/model_builders_xgboost.py daal4py/spmd_utils.py daal4py/sycl/sklearn_sycl.py sklearnex/patch_sklearn.py sklearnex/verbose_mode.py sklearnex/n_jobs.py
Of these, it seems like the model builders should be ran. We have a separate step in recipe testing for daal4py/sycl/sklearn_sycl.py so this should be excluded from run_examples file, as well as spmd_utils since its just helper functions. Are sklearnex/ njobs, patch_sklearn, and verbose_mode intended to be ran with run_examples.py (@samir-nasibli)?

daal4py/sycl/sklearn_sycl.py sklearnex/patch_sklearn.py sklearnex/verbose_mode.py sklearnex/n_jobs.py are all still running during run_examples step - I don't think this is desired behavior (at least these did not run previously)

In the best of my understanding, they should be launched. Multiple examples weren't launching before, due to the improper naming.

napetrov · 2023-08-30T12:27:05Z

/intelci: run
set infra_branch=razdoburdin-patch-1

napetrov

ok with changes - assuming CI would be green

Dmitry Razdoburdin added 2 commits July 14, 2023 07:19

initial

532dbf0

add list of skiped files

25dcd9e

razdoburdin requested review from maria-Petrova, Alexsandruss, samir-nasibli, napetrov and aepanchi as code owners July 14, 2023 16:42

pep8

071a657

napetrov reviewed Jul 15, 2023

View reviewed changes

samir-nasibli mentioned this pull request Jul 18, 2023

MAINT: fixing mb examples names #1360

Closed

Dmitry Razdoburdin added 2 commits July 19, 2023 00:30

fixes for test_examples.py

a0895ef

fix for low_order_moms_dense

7f5fff1

move spmd_utils.py to utils dir

90ee603

razdoburdin requested a review from KulikovNikita as a code owner July 27, 2023 10:43

Merge branch 'master' into remove_the_batch_suffix_from_examples

240ced7

razdoburdin marked this pull request as draft July 27, 2023 10:58

Dmitry Razdoburdin added 8 commits July 27, 2023 05:52

isort

f277e4f

Merge branch 'remove_the_batch_suffix_from_examples' of https://githu…

0c4704d

…b.com/razdoburdin/scikit-learn-intelex into remove_the_batch_suffix_from_examples

more isort

2f037f6

black

8644612

fixes

a60e815

black

1e35587

fixes

d720443

black

f0bf934

fix

cd47d9c

Merge branch 'intel:master' into remove_the_batch_suffix_from_examples

a8ec489

razdoburdin marked this pull request as ready for review August 8, 2023 11:08

samir-nasibli mentioned this pull request Aug 8, 2023

TEST: added different dataframes testing in sklearnex own tests #1379

Merged

Dmitry Razdoburdin added 2 commits August 25, 2023 05:08

merge conflicts fix

2fe9e25

remove the suffix from sycl example

f580a59

Merge branch 'master' into remove_the_batch_suffix_from_examples

a744793

samir-nasibli requested a review from ethanglaser August 29, 2023 14:16

Dmitry Razdoburdin added 2 commits August 30, 2023 03:01

rename spmd_utils.py

1270c97

Merge branch 'remove_the_batch_suffix_from_examples' of https://githu…

27351cb

…b.com/razdoburdin/scikit-learn-intelex into remove_the_batch_suffix_from_examples

napetrov approved these changes Aug 30, 2023

View reviewed changes

ethanglaser approved these changes Aug 30, 2023

View reviewed changes

razdoburdin merged commit 37ada42 into intel:master Aug 30, 2023
16 of 17 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove the batch suffix from examples #1364

Remove the batch suffix from examples #1364

razdoburdin commented Jul 14, 2023

napetrov Jul 15, 2023

razdoburdin Jul 16, 2023

samir-nasibli Jul 19, 2023

razdoburdin commented Jul 20, 2023

razdoburdin commented Jul 28, 2023

razdoburdin commented Jul 28, 2023

razdoburdin commented Jul 28, 2023

razdoburdin commented Aug 8, 2023

razdoburdin commented Aug 8, 2023 •

edited by samir-nasibli

Loading

ethanglaser commented Aug 14, 2023

razdoburdin commented Aug 14, 2023

ethanglaser commented Aug 14, 2023

ethanglaser commented Aug 14, 2023 •

edited

Loading

samir-nasibli commented Aug 25, 2023

samir-nasibli commented Aug 25, 2023

samir-nasibli commented Aug 25, 2023

napetrov commented Aug 29, 2023

ethanglaser commented Aug 29, 2023

napetrov commented Aug 29, 2023

mergify bot commented Aug 29, 2023

samir-nasibli commented Aug 29, 2023

samir-nasibli commented Aug 29, 2023

ethanglaser commented Aug 29, 2023 •

edited

Loading

razdoburdin commented Aug 30, 2023

razdoburdin commented Aug 30, 2023

napetrov commented Aug 30, 2023

napetrov left a comment

Remove the batch suffix from examples #1364

Remove the batch suffix from examples #1364

Conversation

razdoburdin commented Jul 14, 2023

Description

napetrov Jul 15, 2023

Choose a reason for hiding this comment

razdoburdin Jul 16, 2023

Choose a reason for hiding this comment

samir-nasibli Jul 19, 2023

Choose a reason for hiding this comment

razdoburdin commented Jul 20, 2023

razdoburdin commented Jul 28, 2023

razdoburdin commented Jul 28, 2023

razdoburdin commented Jul 28, 2023

razdoburdin commented Aug 8, 2023

razdoburdin commented Aug 8, 2023 • edited by samir-nasibli Loading

ethanglaser commented Aug 14, 2023

razdoburdin commented Aug 14, 2023

ethanglaser commented Aug 14, 2023

ethanglaser commented Aug 14, 2023 • edited Loading

samir-nasibli commented Aug 25, 2023

samir-nasibli commented Aug 25, 2023

samir-nasibli commented Aug 25, 2023

napetrov commented Aug 29, 2023

ethanglaser commented Aug 29, 2023

napetrov commented Aug 29, 2023

mergify bot commented Aug 29, 2023

❌ Base branch update has failed

samir-nasibli commented Aug 29, 2023

samir-nasibli commented Aug 29, 2023

ethanglaser commented Aug 29, 2023 • edited Loading

razdoburdin commented Aug 30, 2023

razdoburdin commented Aug 30, 2023

napetrov commented Aug 30, 2023

napetrov left a comment

Choose a reason for hiding this comment

razdoburdin commented Aug 8, 2023 •

edited by samir-nasibli

Loading

ethanglaser commented Aug 14, 2023 •

edited

Loading

ethanglaser commented Aug 29, 2023 •

edited

Loading