Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: adding array-api-compat and enabling array api conformance tests #2079

Merged

Conversation

samir-nasibli
Copy link
Contributor

@samir-nasibli samir-nasibli commented Oct 2, 2024

Description

Adding array-api-compat and enabling array api conformance tests.

Done:

  • added array-api-compat to test env.
    • test statistics:
      • 172 Array API tests:
        • SKIPPED 142
          • 129(due to torch/array_api.cupy availability) - not required.
          • 6 due to scikit-learn-intelex functional support doesn't support attributes consistency for the inputs. Disabled now.
            • Expected that real Array API support for PCA or Ridge will resolve this. No ticket required
          • 3 train_test_split inconsistency for Array API inputs.
            • Ticket N8533
          • 1 Ridge regression. Array API functional support supported for all solvers. Not raising error for non-svd solvers.
            • Expected that real Array API support for Ridge will resolve this. No ticket required
          • 1 PCA. Array API functionally supported for all factorizations. power_iteration_normalizer=["LU", "QR"]
          • 2 PCA. InvalidParameterError: The 'M' parameter of randomized_svd must be an instance of 'numpy.ndarray' or a sparse matrix.
            • Ticket N8534
        • PASSED 27
# scikit-learn-intelex functional support doesn't support attributes consistency for the inputs. Disabled now.
SKIPPED decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='covariance_eigh')-check_array_api_input_and_values-array_api_strict-None-None]
SKIPPED decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='covariance_eigh',whiten=True)-check_array_api_input_and_values-array_api_strict-None-None]
SKIPPED linear_model/tests/test_ridge.py::test_ridge_array_api_compliance[Ridge(solver='svd')-check_array_api_input_and_values-array_api_strict-None-None]
SKIPPED decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='covariance_eigh')-check_array_api_get_precision-array_api_strict-None-None]
SKIPPED decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='covariance_eigh',whiten=True)-check_array_api_get_precision-array_api_strict-None-None]
SKIPPED linear_model/tests/test_ridge.py::test_ridge_array_api_compliance[Ridge(solver='svd')-check_array_api_attributes-array_api_strict-None-None]

# `test_array_api_train_test_split` inconsistency for Array API inputs. TODO cover by ticket.
SKIPPED model_selection/tests/test_split.py::test_array_api_train_test_split[True-None-array_api_strict-None-None]
SKIPPED model_selection/tests/test_split.py::test_array_api_train_test_split[True-stratify1-array_api_strict-None-None]
SKIPPED model_selection/tests/test_split.py::test_array_api_train_test_split[False-None-array_api_strict-None-None]

# Ridge regression. Array API functional support supported for all solvers. Not raising error for non-svd solvers.
SKIPPED linear_model/tests/test_ridge.py::test_array_api_error_and_warnings_for_solver_parameter[array_api_strict]

# PCA. Array API functionally supported for all factorizations. power_iteration_normalizer=["LU", "QR"]
SKIPPED decomposition/tests/test_pca.py::test_array_api_error_and_warnings_on_unsupported_params
# PCA. InvalidParameterError: The 'M' parameter of randomized_svd must be an instance of 'numpy.ndarray' or a sparse matrix.
SKIPPED decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_input_and_values-array_api_strict-None-None]
SKIPPED decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,power_iteration_normalizer='QR',random_state=0,svd_solver='randomized')-check_array_api_get_precision-array_api_strict-None-None]

Documentation PR (if needed): #1340 (for example)

Benchmarks PR (if needed): IntelPython/scikit-learn_bench#155 (for example)


Checklist to comply with before moving PR from draft:

PR completeness and readability

  • I have reviewed my changes thoroughly before submitting this pull request.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
  • Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
  • I have added a respective label(s) to PR if I have a permission for that.
  • I have resolved any merge conflicts that might occur with the base branch.

Testing

  • I have run it locally and tested the changes extensively.
  • All CI jobs are green or I have provided justification why they aren't.
  • I have extended testing suite if new functionality was introduced in this PR.

Performance

  • I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
  • I have provided justification why performance has changed or why changes are not expected.
  • I have provided justification why quality metrics have changed or why changes are not expected.
  • I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

added array-api-compat to test env
@samir-nasibli samir-nasibli added enhancement New feature or request Array API labels Oct 2, 2024
@samir-nasibli samir-nasibli changed the title ENH: array api dispatching ENH: adding array-api-compat and enabling array api conformance tests Oct 3, 2024
- decomposition/tests/test_pca.py::test_pca_array_api_compliance[PCA(n_components=2,svd_solver='covariance_eigh',whiten=True)-check_array_api_get_precision-array_api_strict-None-None]
- linear_model/tests/test_ridge.py::test_ridge_array_api_compliance[Ridge(solver='svd')-check_array_api_attributes-array_api_strict-None-None]
- linear_model/tests/test_ridge.py::test_ridge_array_api_compliance[Ridge(solver='svd')-check_array_api_input_and_values-array_api_strict-None-None]
# `train_test_split` inconsistency for Array API inputs.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be covered by ticket.

@samir-nasibli samir-nasibli marked this pull request as ready for review October 8, 2024 20:09
@samir-nasibli samir-nasibli requested review from icfaust and removed request for ahuber21, napetrov and homksei October 8, 2024 20:09
@samir-nasibli samir-nasibli force-pushed the enh/array_api_dispatching branch from ced43bf to 404e8c0 Compare October 8, 2024 22:40
@samir-nasibli samir-nasibli mentioned this pull request Oct 8, 2024
13 tasks
Copy link
Contributor

@icfaust icfaust left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assume a follow-up ticket will be generated for private CI checks.

@samir-nasibli samir-nasibli merged commit 1cc02b3 into uxlfoundation:main Oct 9, 2024
38 of 48 checks passed
@samir-nasibli samir-nasibli deleted the enh/array_api_dispatching branch October 9, 2024 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Array API enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants