Skip to content

Commit

Permalink
Backport gh-2166, gh-2057, gh-2211 (#2214)
Browse files Browse the repository at this point in the history
This PR backports of #2166, #2057 and #2211 from development branch to
`maintenance/0.16.x`.
  • Loading branch information
antonwolfy authored Dec 5, 2024
1 parent cc58db0 commit 309a418
Show file tree
Hide file tree
Showing 7 changed files with 79 additions and 96 deletions.
1 change: 0 additions & 1 deletion .github/workflows/build-sphinx.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/conda-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down Expand Up @@ -197,7 +196,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down Expand Up @@ -327,7 +325,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down Expand Up @@ -469,7 +466,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down Expand Up @@ -505,7 +501,6 @@ jobs:
steps:
- uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/generate_coverage.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,6 @@ jobs:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@a4260408e20b96e80095f42ff7f1a15b27dd94ca # v3.0.4
with:
miniforge-variant: Mambaforge
miniforge-version: latest
use-mamba: true
channels: conda-forge
Expand Down
16 changes: 15 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,23 @@
# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.16.1] - 12/XX/2024

This is a bug-fix release.

### Changed

* Changed to use `Miniforge` installer in GutHub actions [#2057](https://github.com/IntelPython/dpnp/pull/2057)
* Updated `README.md` to reflect current installation requirements and available options [#2166](https://github.com/IntelPython/dpnp/pull/2166)

### Fixed

* Resolved a compilation error when building with DPC++ 2025.1 compiler [#2211](https://github.com/IntelPython/dpnp/pull/2211)


## [0.16.0] - 10/14/2024

This release reaches an important milestone by making offloading fully asynchronous. Calls to `dpnp` submit tasks for execution to DPC++ runtime and return without waiting for execution of these tasks to finish. The sequential semantics a user comes to expect from execution of Python script is preserved though.
Expand Down
129 changes: 44 additions & 85 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,117 +6,76 @@
[![Build Sphinx](https://github.com/IntelPython/dpnp/workflows/Build%20Sphinx/badge.svg)](https://intelpython.github.io/dpnp)
[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/IntelPython/dpnp/badge)](https://securityscorecards.dev/viewer/?uri=github.com/IntelPython/dpnp)

<img align="left" src="https://spec.oneapi.io/oneapi-logo-white-scaled.jpg" alt="oneAPI logo" width="75"/>

# DPNP - Data Parallel Extension for NumPy*

Data Parallel Extension for NumPy* or `dpnp` is a Python library that
implements a subset of NumPy* that can be executed on any data parallel device.
The subset is a drop-in replacement of core NumPy* functions and numerical data types.

[API coverage summary](https://intelpython.github.io/dpnp/reference/comparison.html#summary)

[Full documentation](https://intelpython.github.io/dpnp/)

[DPNP C++ backend documentation](https://intelpython.github.io/dpnp/backend_doc/)
`Dpnp` is the core part of a larger family of [data-parallel Python libraries and tools](https://www.intel.com/content/www/us/en/developer/tools/oneapi/distribution-for-python.html)
to program on XPUs.

## Build from source:
Ensure you have the following prerequisite packages installed:

- `cython`
- `cmake >=3.21`
- `dpcpp_linux-64` or `dpcpp_win-64` (depending on your OS)
- `dpctl`
- `mkl-devel-dpcpp`
- `onedpl-devel`
- `ninja`
- `numpy >=1.19,<1.25a0`
- `python`
- `scikit-build`
- `setuptools`
- `sysroot_linux-64 >=2.28` (only on Linux OS)
- `tbb-devel`
# Installing

After these steps, `dpnp` can be built in debug mode as follows:
You can install the library using `conda`, `mamba` or [pip](https://pypi.org/project/dpnp/)
package managers. It is also available as part of the [Intel(R) Distribution for Python](https://www.intel.com/content/www/us/en/developer/tools/oneapi/distribution-for-python.html)
(IDP).

```bash
git clone https://github.com/IntelPython/dpnp
cd dpnp
python scripts/build_locally.py
```
## Intel(R) Distribution for Python

## Install Wheel Package via pip
Install DPNP
```cmd
python -m pip install --index-url https://software.repos.intel.com/python/pypi dpnp
```
You can find the most recent release of `dpnp` every quarter as part of the IDP
releases.

Set path to Performance Libraries in case of using venv or system Python:
```cmd
export LD_LIBRARY_PATH=<path_to_your_env>/lib
```
To get the library from the latest release, follow the instructions from
[Get Started With Intel® Distribution for Python](https://www.intel.com/content/www/us/en/developer/articles/technical/get-started-with-intel-distribution-for-python.html).

It is also required to set following environment variables:
```cmd
export OCL_ICD_FILENAMES_RESET=1
export OCL_ICD_FILENAMES=libintelocl.so
```
## Conda

## Run test
```bash
pytest
# or
pytest tests/test_matmul.py -s -v
# or
python -m unittest tests/test_mixins.py
```
To install `dpnp` from the Intel(R) conda channel, use the following command:

## Run numpy external test
```bash
. ./0.env.sh
python -m tests.third_party.numpy_ext
# or
python -m tests.third_party.numpy_ext core/tests/test_umath.py
# or
python -m tests.third_party.numpy_ext core/tests/test_umath.py::TestHypot::test_simple
conda install dpnp -c https://software.repos.intel.com/python/conda/ -c conda-forge
```

### Building documentation:
## Pip

The `dpnp` can be installed using `pip` obtaining wheel packages either from
PyPi or from Intel(R) channel. To install `dpnp` wheel package from Intel(R)
channel, run the following command:

```bash
Prerequisites:
$ conda install sphinx sphinx_rtd_theme
Building:
1. Install dpnp into your python environment
2. $ cd doc && make html
3. The documentation will be in doc/_build/html
python -m pip install --index-url https://software.repos.intel.com/python/pypi dpnp
```

## Packaging:
## Installing the bleeding edge

To try out the latest features, install `dpnp` using our development channel on
Anaconda cloud:

```bash
. ./0.env.sh
conda-build conda-recipe/
conda install dpnp -c dppy/label/dev -c https://software.repos.intel.com/python/conda/ -c conda-forge
```

## Run benchmark:
```bash
cd benchmarks/

asv run --python=python --bench <filename without .py>
# example:
asv run --python=python --bench bench_elementwise
# Building

# or
Refer to our [Documentation](https://intelpython.github.io/dpnp/quick_start_guide.html)
for more information on setting up a development environment and building `dpnp`
from the source.

asv run --python=python --bench <class>.<bench>
# example:
asv run --python=python --bench Elementwise.time_square

# add --quick option to run every case once but looks like first execution has additional overheads and takes a lot of time (need to be investigated)
```
# Running Tests

Tests are located in folder [dpnp/tests](dpnp/tests).

## Tests matrix:
| # |Name |OS |distributive|interpreter|python used from|SYCL queue manager|build commands set |forced environment |
|---|------------------------------------|-----|------------|-----------|:--------------:|:----------------:|------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|
|1 |Ubuntu 20.04 Python37 |Linux|Ubuntu 20.04|Python 3.7 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace pytest |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|2 |Ubuntu 20.04 Python38 |Linux|Ubuntu 20.04|Python 3.8 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace pytest |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|3 |Ubuntu 20.04 Python39 |Linux|Ubuntu 20.04|Python 3.9 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace pytest |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|4 |Ubuntu 20.04 External Tests Python37|Linux|Ubuntu 20.04|Python 3.7 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace python -m tests_external.numpy.runtests|cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|5 |Ubuntu 20.04 External Tests Python38|Linux|Ubuntu 20.04|Python 3.8 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace python -m tests_external.numpy.runtests|cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|6 |Ubuntu 20.04 External Tests Python39|Linux|Ubuntu 20.04|Python 3.9 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace python -m tests_external.numpy.runtests|cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|7 |Code style |Linux|Ubuntu 20.04|Python 3.8 | IntelOneAPI | local |python ./setup.py style |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis, conda-verify, pycodestyle, autopep8, black |
|8 |Valgrind |Linux|Ubuntu 20.04| | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis |
|9 |Code coverage |Linux|Ubuntu 20.04|Python 3.8 | IntelOneAPI | local |export DPNP_DEBUG=1 python setup.py clean python setup.py build_clib python setup.py build_ext --inplace |cmake-3.19.2, valgrind, pytest-valgrind, conda-build, pytest, hypothesis, conda-verify, pycodestyle, autopep8, pytest-cov|
To run the tests, use:
```bash
python -m pytest --pyargs dpnp
```
21 changes: 19 additions & 2 deletions dpnp/backend/kernels/dpnp_krnl_common.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,23 @@
#include "queue_sycl.hpp"
#include <dpnp_iface.hpp>

/**
* Version of SYCL DPC++ 2025.1 compiler where support of
* sycl::ext::oneapi::experimental::properties was added.
*/
#ifndef __SYCL_COMPILER_REDUCTION_PROPERTIES_SUPPORT
#define __SYCL_COMPILER_REDUCTION_PROPERTIES_SUPPORT 20241129
#endif

namespace mkl_blas = oneapi::mkl::blas;
namespace mkl_blas_cm = oneapi::mkl::blas::column_major;
namespace mkl_blas_rm = oneapi::mkl::blas::row_major;
namespace mkl_lapack = oneapi::mkl::lapack;

#if __SYCL_COMPILER_VERSION >= __SYCL_COMPILER_REDUCTION_PROPERTIES_SUPPORT
namespace syclex = sycl::ext::oneapi::experimental;
#endif

template <typename _KernelNameSpecialization1,
typename _KernelNameSpecialization2,
typename _KernelNameSpecialization3>
Expand Down Expand Up @@ -76,8 +88,13 @@ sycl::event dot(sycl::queue &queue,
cgh.parallel_for(
sycl::range<1>{size},
sycl::reduction(
result_out, std::plus<_DataType_output>(),
sycl::property::reduction::initialize_to_identity{}),
result_out, sycl::plus<_DataType_output>(),
#if __SYCL_COMPILER_VERSION >= __SYCL_COMPILER_REDUCTION_PROPERTIES_SUPPORT
syclex::properties(syclex::initialize_to_identity)
#else
sycl::property::reduction::initialize_to_identity {}
#endif
),
[=](sycl::id<1> idx, auto &sum) {
sum += static_cast<_DataType_output>(
input1_in[idx * input1_strides]) *
Expand Down
2 changes: 1 addition & 1 deletion tests/third_party/cupy/creation_tests/test_ranges.py
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ def test_linspace_float_overflow(self, xp):
dtype = cupy.default_float_type()
return xp.linspace(0.0, xp.finfo(dtype).max / 5, 10, dtype=dtype)

@testing.numpy_cupy_allclose()
@testing.numpy_cupy_allclose(rtol={numpy.float32: 1e-6, "default": 1e-7})
def test_linspace_float_underflow(self, xp):
# find minimum subnormal number
dtype = cupy.default_float_type()
Expand Down

0 comments on commit 309a418

Please sign in to comment.