Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implements dpctl.tensor.logsumexp and dpctl.tensor.reduce_hypot #1446

Merged
merged 14 commits into from
Oct 26, 2023
Merged

Implements dpctl.tensor.logsumexp and dpctl.tensor.reduce_hypot #1446

merged 14 commits into from
Oct 26, 2023

Conversation

ndgrigorian
Copy link
Collaborator

@ndgrigorian ndgrigorian commented Oct 19, 2023 •
edited
Loading

This pull request proposes the additions of logsumexp and reduce_hypot into dpctl.

reduce_hypot can be used to find the Euclidean distance in N-dimensional space

In [1]: import dpctl.tensor as dpt, numpy as np

In [2]: x1 = dpt.full(10, 3, dtype="f4")

In [3]: x2 = dpt.ones(10, dtype="f4")

In [4]: d = dpt.reduce_hypot(x1 - x2)

In [5]: d
Out[5]: usm_ndarray(6.324554, dtype=float32)

logsumexp can be used to sum probabilities stored as the logarithm of the probability.

This pull request also slips in a change to disable the use of sycl::atomic_ref for arithmetic reductions (sum and prod) of inexact data types due to concerns with numerical stability. To better support this change, dedicated kernels for tree reductions over contiguous inputs have been implemented.

  • Have you provided a meaningful PR description?
  • Have you added a test, reproducer or referred to an issue with a reproducer?
  • Have you tested your changes locally for CPU and GPU devices?
  • Have you made sure that new changes do not introduce compiler warnings?
  • Have you checked performance impact of proposed changes?
  • If this PR is a work in progress, are you opening the PR as a draft?

@coveralls
Copy link
Collaborator

coveralls commented Oct 19, 2023 •
edited
Loading

Coverage Status

coverage: 85.748% (+0.01%) from 85.735% when pulling d88e78f on logsumexp-reduce-hypot-impl into 2eba93e on master.

@github-actions
Copy link

View rendered docs @ https://intelpython.github.io/dpctl/pulls/1446/index.html

@github-actions
Copy link

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_48 ran successfully.
Passed: 934
Failed: 66
Skipped: 119

@github-actions
Copy link

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_50 ran successfully.
Passed: 934
Failed: 66
Skipped: 119

@github-actions
Copy link

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_69 ran successfully.
Passed: 935
Failed: 65
Skipped: 119

@ndgrigorian ndgrigorian force-pushed the logsumexp-reduce-hypot-impl branch from c8683e3 to e95ac44 Compare October 24, 2023 05:16
@github-actions
Copy link

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_69 ran successfully.
Passed: 933
Failed: 67
Skipped: 119

@github-actions
Copy link

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_70 ran successfully.
Passed: 934
Failed: 66
Skipped: 119

@ndgrigorian ndgrigorian force-pushed the logsumexp-reduce-hypot-impl branch from 5e3a671 to fff36a1 Compare October 25, 2023 01:23
@github-actions
Copy link

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_71 ran successfully.
Passed: 934
Failed: 66
Skipped: 119

1 similar comment
@github-actions
Copy link

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_71 ran successfully.
Passed: 934
Failed: 66
Skipped: 119

@ndgrigorian
Copy link
Collaborator Author

@oleksandr-pavlyk
I've added the per-type per-function atomic support checks we've discussed.

@github-actions
Copy link

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_72 ran successfully.
Passed: 935
Failed: 65
Skipped: 119

ndgrigorian and others added 14 commits October 25, 2023 14:15
@ndgrigorian
Implements logsumexp and reduce_hypot
c2d04a1
@ndgrigorian
Implements dedicated kernels for temp reductions over axes 1 and 0 in…
7b4a835 
… contiguous arrays
@ndgrigorian
logsumexp and reduce_hypot no longer use atomics
aa749b4 
This change was made to improve the accuracy of these functions
@ndgrigorian
Adds tests for reduce_hypot and logsumexp
6b07501
@ndgrigorian
Arithmetic reductions no longer use atomics for inexact types
1247e9b 
This change is intended to improve the numerical stability of sum and prod
@ndgrigorian
Removed support of atomic reduction for max and min
9d99b71
@ndgrigorian
Adds new tests for reductions
10ede9a
@ndgrigorian
Split reductions into multiple source files
7d243d8
@ndgrigorian
Remove unneccessary imports of reduction init functions
4638025
@ndgrigorian
Added functions for querying reduction atomic support per type per fu…
f2babe8 
…nction
@ndgrigorian
Corrected min contig variant typo
4e2789d 
These variants were using ``sycl::maximum`` rather than ``sycl::minimum``
@ndgrigorian
Removes _tree_reduction_over_axis
4dd054f 
Use lambdas to ignore atomic-specific arguments to hypot and logsumexp dtype_supported functions
@oleksandr-pavlyk @ndgrigorian
Always use atomic implementation for min/max if available
448a7f1 
For add/multiplies reductions, use tree reduction for FP types, real and complex,
to get better round-off accumulation properties.
@ndgrigorian
logaddexp implementation moved to math_utils
d88e78f 
Reduces code repetition between logsumexp and logaddexp
@ndgrigorian ndgrigorian force-pushed the logsumexp-reduce-hypot-impl branch from 599fadb to d88e78f Compare October 25, 2023 21:25
@github-actions
Copy link

Array API standard conformance tests for dpctl=0.15.1dev0=py310ha25a700_77 ran successfully.
Passed: 935
Failed: 65
Skipped: 119

oleksandr-pavlyk
oleksandr-pavlyk approved these changes Oct 26, 2023
View reviewed changes
Copy link
Collaborator

@oleksandr-pavlyk oleksandr-pavlyk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you @ndgrigorian

@ndgrigorian ndgrigorian merged commit 03fd737 into master Oct 26, 2023
26 checks passed
@oleksandr-pavlyk oleksandr-pavlyk deleted the logsumexp-reduce-hypot-impl branch October 27, 2023 02:24
@antonwolfy antonwolfy mentioned this pull request Oct 30, 2023
Closed
@ndgrigorian ndgrigorian mentioned this pull request Nov 4, 2023
Merged
6 tasks
ndgrigorian added a commit that referenced this pull request Dec 1, 2023
@ndgrigorian
Removes reduction files made redundant by gh-1446
4060e1a
ndgrigorian added a commit that referenced this pull request Dec 4, 2023
@ndgrigorian
Removes reduction files made redundant by gh-1446 (#1480)
07dea79
@oleksandr-pavlyk oleksandr-pavlyk mentioned this pull request Jan 26, 2024
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@oleksandr-pavlyk oleksandr-pavlyk oleksandr-pavlyk approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ndgrigorian @coveralls @oleksandr-pavlyk