Implements accumulation functions in dpctl.tensor #1602

ndgrigorian · 2024-03-23T21:33:38Z

This pull request proposes implementation of dpctl.tensor.cumulative_sum, dpctl.tensor.cumulative_prod, and dpctl.tensor.cumulative_logsumexp.

cumulative_sum is already part of the array API standard and is implemented as per the spec. cumulative_prod and cumulative_logsumexp are implemented with a similar API, including an include_initial keyword argument.

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
If this PR is a work in progress, are you opening the PR as a draft?

…t.cumulative_sum` The Python bindings for these functions are implemented in a new submodule `_tensor_accumulation_impl`

github-actions · 2024-03-23T22:10:55Z

Deleted rendered PR docs from intelpython.github.com/dpctl, latest should be updated shortly. 🤞

github-actions · 2024-03-23T22:16:33Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_154 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-23T22:20:31Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_154 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

This resolves hangs in unique functions

github-actions · 2024-03-25T19:36:02Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_156 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

coveralls · 2024-03-25T22:44:00Z

coverage: 87.967% (+0.08%) from 87.884%
when pulling 27eb063 on feature/tensor-accumulation
into 57495af on master.

… temporary are needed

github-actions · 2024-03-26T05:32:08Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_158 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

… trailing axis Fixes a bug where in some cases output axes were not being permuted

…ogsumexp

github-actions · 2024-03-26T19:00:47Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_161 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-26T19:45:53Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_163 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-26T22:49:18Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_165 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

…yword

github-actions · 2024-03-27T17:08:39Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_167 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-28T23:26:17Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_188 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-30T04:04:39Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_189 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-30T14:45:01Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_190 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

dpctl/tensor/_reduction.py

dpctl/tensor/libtensor/include/kernels/accumulators.hpp

github-actions · 2024-03-30T15:42:35Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_190 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-30T16:49:29Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_191 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

Indexers are made const, integral variables in kernels made const too Make two-offset instances const references to avoid copying. Gor rid of get_src_const_ptr unused methods in stack_t structs. Replaced auto with size_t as appropriate. Added const to make compiler analysis easier (and faster).

github-actions · 2024-03-31T16:10:17Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_193 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

…ting against closed form

github-actions · 2024-03-31T17:17:25Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_190 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

By returning data from `local_mem_acc` after the group barrier, if memory is later overwritten, a race condition follows, which was especially obvious on CPU Now the value is stored in variable before the barrier and then returned

…ase size of test_logcumsumexp_basic

github-actions · 2024-04-01T00:24:38Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_192 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

dpctl/tensor/libtensor/include/utils/sycl_utils.hpp

…ion (#1624) added comments explaining why barriers are needed

github-actions · 2024-04-01T08:19:13Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_193 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

Add empty new line after list item to make Sphinx happy.

Docstring edits for accumulation functions

oleksandr-pavlyk

LGMT! Thank you @ndgrigorian

github-actions · 2024-04-01T15:34:47Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_195 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

ndgrigorian added 2 commits March 21, 2024 16:50

Use shT instead of std::vector<py::ssize_t> in repeat

e45c68e

Add missing host task to host_tasks_list in _reduction.py

85ccecb

ndgrigorian requested a review from oleksandr-pavlyk March 23, 2024 21:33

ndgrigorian added 2 commits March 23, 2024 14:37

Implements dpt.cumulative_logsumexp, dpt.cumulative_prod, and `dp…

80a77e0

…t.cumulative_sum` The Python bindings for these functions are implemented in a new submodule `_tensor_accumulation_impl`

Adds the first tests for dpt.cumulative_sum

761ecd4

ndgrigorian force-pushed the feature/tensor-accumulation branch from 53e927d to 761ecd4 Compare March 23, 2024 21:37

Pass host task vector to accumulator kernel calls

346c4aa

This resolves hangs in unique functions

ndgrigorian added 2 commits March 25, 2024 21:43

Implements out keyword for accumulators

015079b

Fixes cumulative_logsumexp when both an intermediate input and result…

15ea9ae

… temporary are needed

ndgrigorian added 3 commits March 26, 2024 10:58

Only permute dims of allocated outputs if accumulated axis is not the…

1b752e1

… trailing axis Fixes a bug where in some cases output axes were not being permuted

Enable scalar inputs to accumulation functions

506a8eb

Adds test for scalar inputs to cumulative_sum

eaa4011

ndgrigorian marked this pull request as ready for review March 26, 2024 18:18

Adds docstrings for cumulative_sum. cumulative_prod, and cumulative_l…

7b3fd8f

…ogsumexp

Merge branch 'master' into feature/tensor-accumulation

dc9d37b

ndgrigorian added 2 commits March 26, 2024 13:14

Removed redundant dtype kind check in _default_accumulation_dtype

5da1e66

Reduce repetition of code allocation out array in _accumulate_common

bc3f5b1

Adds tests for accumulation function identities, include_initial ke…

f8e0a75

…yword

ndgrigorian added 2 commits March 27, 2024 10:16

Merge branch 'master' into feature/tensor-accumulation

216cc95

Adds more tests for cumulative_sum

e93e202

Reduce size of array in test_logcumsumexp_basic

d947ba8

oleksandr-pavlyk force-pushed the feature/tensor-accumulation branch from cc3d88f to b88f8ee Compare March 30, 2024 15:03

oleksandr-pavlyk reviewed Mar 30, 2024

View reviewed changes

dpctl/tensor/_reduction.py Show resolved Hide resolved

oleksandr-pavlyk reviewed Mar 30, 2024

View reviewed changes

dpctl/tensor/libtensor/include/kernels/accumulators.hpp Show resolved Hide resolved

oleksandr-pavlyk force-pushed the feature/tensor-accumulation branch from b88f8ee to 5fd506c Compare March 30, 2024 16:11

Add test for cumulative_logsumexp for geometric series summation, tes…

4bd02b4

…ting against closed form

oleksandr-pavlyk force-pushed the feature/tensor-accumulation branch from b1219af to 4bd02b4 Compare March 31, 2024 16:39

ndgrigorian added 2 commits March 31, 2024 16:08

Fix race condition in custom_inclusive_scan_over_group

1b3ade0

By returning data from `local_mem_acc` after the group barrier, if memory is later overwritten, a race condition follows, which was especially obvious on CPU Now the value is stored in variable before the barrier and then returned

Remove use of Numpy functions from test_tensor_accumulation and incre…

0d40493

…ase size of test_logcumsumexp_basic

oleksandr-pavlyk reviewed Apr 1, 2024

View reviewed changes

dpctl/tensor/libtensor/include/utils/sycl_utils.hpp Show resolved Hide resolved

oleksandr-pavlyk reviewed Apr 1, 2024

View reviewed changes

dpctl/tensor/libtensor/include/utils/sycl_utils.hpp Outdated Show resolved Hide resolved

Need barrier after call to custom inclusive scan to avoid race condit…

b1016bf

…ion (#1624) added comments explaining why barriers are needed

oleksandr-pavlyk added 2 commits April 1, 2024 06:43

Docstring edits

f4cd66a

Add empty new line after list item to make Sphinx happy.

Merge pull request #1625 from IntelPython/accumulators-doc-edits

27eb063

Docstring edits for accumulation functions

oleksandr-pavlyk approved these changes Apr 1, 2024

View reviewed changes

ndgrigorian enabled auto-merge (squash) April 1, 2024 16:48

ndgrigorian disabled auto-merge April 1, 2024 16:48

ndgrigorian merged commit 65bb9ef into master Apr 1, 2024

ndgrigorian deleted the feature/tensor-accumulation branch April 1, 2024 20:05

Implements accumulation functions in dpctl.tensor #1602

Implements accumulation functions in dpctl.tensor #1602

Uh oh!

Conversation

ndgrigorian commented Mar 23, 2024 • edited by oleksandr-pavlyk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 23, 2024

Uh oh!

github-actions bot commented Mar 23, 2024

Uh oh!

github-actions bot commented Mar 25, 2024

Uh oh!

coveralls commented Mar 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 26, 2024

Uh oh!

github-actions bot commented Mar 26, 2024

Uh oh!

github-actions bot commented Mar 26, 2024

Uh oh!

github-actions bot commented Mar 26, 2024

Uh oh!

github-actions bot commented Mar 27, 2024

Uh oh!

github-actions bot commented Mar 28, 2024

Uh oh!

github-actions bot commented Mar 30, 2024

Uh oh!

github-actions bot commented Mar 30, 2024

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 30, 2024

Uh oh!

github-actions bot commented Mar 30, 2024

Uh oh!

github-actions bot commented Mar 31, 2024

Uh oh!

github-actions bot commented Mar 31, 2024

Uh oh!

github-actions bot commented Apr 1, 2024

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 1, 2024

Uh oh!

oleksandr-pavlyk left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 1, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ndgrigorian commented Mar 23, 2024 •

edited by oleksandr-pavlyk

Loading

github-actions bot commented Mar 23, 2024 •

edited

Loading

coveralls commented Mar 25, 2024 •

edited

Loading