Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Predict and Scoring Bug in sklearnex.svm.SVR and Train_test_split index bug #1000

Open
nilslacroix opened this issue Apr 24, 2022 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@nilslacroix
Copy link

Describe the bug
When I use train_test_split the indeces are shuffled sometimes, sometimes not. Also if you use the scoring method of an SVR estimator multiple times in a row with the same estimator and the same parameters, it can produce different scores.

To Reproduce
well you should be able to write a simple reproducer yourself, since this occurs frequently.

Expected behavior
No bugs.

Environment:

  • Windows 10
  • Anaconda
@nilslacroix nilslacroix added the bug Something isn't working label Apr 24, 2022
@nilslacroix
Copy link
Author

nilslacroix commented Apr 25, 2022

sklearnex.svm.SVR is also buggy. Using .predict() on the same X_test set in a pipeline with preprocessors and encoders yiels different predictions. Just do it like ~50 times in a row and you will see what I mean.

This is also reproducible with basic examples like:

https://intel.github.io/scikit-learn-intelex/samples/nusvr_medical_charges.html

Maybe this happens because I use AMD CPU or something.

@nilslacroix nilslacroix changed the title Train_test_split index bug, scoring bug in SVR Predict and Scoring Bug in sklearnex.svm.SVR and Train_test_split index bug Apr 25, 2022
@nilslacroix
Copy link
Author

I confirmed this bug happens (at least the scoring) when using AMD Hardware. Please at least implement a warning message, which checks the current CPU so people get a notice that this library will only work with intel specific hardware.

@PivovarA PivovarA self-assigned this May 23, 2022
@jamescporter
Copy link

any update on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants