Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add distributed ranges as experimental feature. #1479

Open
wants to merge 99 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 91 commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
318f723
copied shp and common
lslusarczyk Mar 13, 2024
1ac29d0
added tests
lslusarczyk Mar 13, 2024
8557b72
some compilation fixes, cmake now seems to be complete
lslusarczyk Mar 13, 2024
440e673
cleanups
mateuszpn Mar 14, 2024
2f4897f
cleanups of include paths
mateuszpn Mar 18, 2024
9189849
cleanups of include paths and error messages
mateuszpn Mar 18, 2024
11fdbe7
Merge branch 'dist-ranges_cleanup' of github.com:mateuszpn/oneDPL int…
mateuszpn Mar 18, 2024
e371685
Distributed Ranges cleanup (#1448)
mateuszpn Mar 18, 2024
cd56589
shp namespace moved to experimental
mateuszpn Mar 21, 2024
9e97305
namespace update
mateuszpn Mar 21, 2024
d492476
update
mateuszpn Mar 21, 2024
477b204
update
mateuszpn Mar 21, 2024
0152437
CI updated for distributed-ranges
mateuszpn Mar 25, 2024
5e43e65
Merge branch 'oneapi-src:main' into dr_namespace
mateuszpn Apr 4, 2024
2f84319
[oneDPL][hetero] + missed synch between patterns,removed unnecessary …
MikeDvorskiy Apr 5, 2024
bc74b46
Merge branch 'oneapi-src:main' into dr_namespace
mateuszpn Apr 8, 2024
658e465
[oneDPL][sycl] + sycl::is_device_copyable specialization of the SYCL …
MikeDvorskiy Feb 1, 2024
8e943bb
[oneDPL][sycl] + sycl::is_device_copyable specialization fixes
MikeDvorskiy Feb 12, 2024
f31af79
[oneDPL][sycl] sycl::is_device_copyable specialization: + clang format
MikeDvorskiy Feb 13, 2024
35bb6bf
[oneDPL][sycl] sycl::is_device_copyable specialization: + a comment
MikeDvorskiy Feb 13, 2024
80cf60b
[oneDPL][sycl] + #include "sycl_traits.h"
MikeDvorskiy Feb 13, 2024
89c92c3
[oneDPL][sycl] + forward declaration for __early_exit_find_or
MikeDvorskiy Feb 14, 2024
60884a2
[oneDPL][sycl] + necessary includes
MikeDvorskiy Feb 14, 2024
ddaf6fe
Revert "[oneDPL][sycl] + necessary includes"
MikeDvorskiy Feb 14, 2024
797498d
[oneDPL][sycl] + necessary forward declarations
MikeDvorskiy Feb 15, 2024
03c69f1
[oneDPL][sycl] include place changed
MikeDvorskiy Feb 15, 2024
1047d4a
[oneDPL][sycl][dpcpp] #include "sycl_traits.h" //SYCL traits speciali…
MikeDvorskiy Feb 22, 2024
db83006
[oneDPL] removed _ONEDPL_DEVICE_COPYABLE(zip_forward_iterator) due to…
MikeDvorskiy Apr 8, 2024
5e98434
remove matrices and logs
mateuszpn Apr 8, 2024
c2e3911
Merge branch 'dr_namespace' of github.com:mateuszpn/oneDPL into dr_na…
mateuszpn Apr 8, 2024
9fd9ac0
Merge branch 'oneapi-src:main' into dr_namespace
mateuszpn Apr 8, 2024
efaf73b
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Apr 8, 2024
c968195
Merge remote-tracking branch 'remotes/mateusz/dr_namespace' into dist…
lslusarczyk Apr 8, 2024
976bee4
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Apr 9, 2024
8aa96ff
namespaces redesign for dist ranges in oneDPL (#1486)
lslusarczyk Apr 9, 2024
54eecda
removed source_location from distributed-ranges
lslusarczyk Apr 9, 2024
232bf8e
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Apr 9, 2024
b6f7c14
requiring fmt no longer for distributed-ranges tests (#1500)
lslusarczyk Apr 12, 2024
969cb75
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk May 4, 2024
e46716d
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk May 9, 2024
35683c2
Remove ranges-v3 from experimental::dr (#1575)
mateuszpn May 9, 2024
60beae2
some fixes in valid fmt versions
lslusarczyk May 13, 2024
6e17faa
Remove cxxopt from dr tests suite (#1573)
mateuszpn May 14, 2024
7251f36
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk May 17, 2024
78fdcfe
removed non-working new CI in oneDPL
lslusarczyk May 17, 2024
bdfcbe7
fixed cmake for distributed-ranges
lslusarczyk May 17, 2024
08a5273
fixed format
lslusarczyk May 17, 2024
093834d
Remove references to internal bug tracker. (#1600)
BenBrock May 28, 2024
21a321e
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Jun 5, 2024
e3c6b26
Rename `device_vector` -> `remote_vector` and `device_span` -> (#1612)
BenBrock Jun 5, 2024
f7c82cc
run dr shp-tests (#1591)
haichangsi Jun 7, 2024
7a0673b
update of dr::shp::sort() (#1614)
lslusarczyk Jun 25, 2024
8c89e36
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Jun 26, 2024
dcd1702
Jmiller comments applied (#1641)
lslusarczyk Jun 26, 2024
791d5c2
removed future (#1643)
lslusarczyk Jun 26, 2024
98b8034
Jiraids removed (#1645)
lslusarczyk Jun 27, 2024
00673f5
removed && from for_each (#1646)
lslusarczyk Jun 27, 2024
86f8da5
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Jun 28, 2024
e4c19f1
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Jul 4, 2024
5d46146
DR ci made onedpl compliant (#1663)
lslusarczyk Jul 9, 2024
9223c15
Add for_each_n as in distributed-ranges (#1684)
Xewar313 Jul 9, 2024
ee243fb
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Jul 9, 2024
c09ef4b
using CPU instead of HOST in Distributed Ranges CI (#1685)
lslusarczyk Jul 9, 2024
9944c46
changed copyright comments in DR to be oneDPL compliant (#1687)
lslusarczyk Jul 10, 2024
c90223d
Dan comments applied (#1688)
lslusarczyk Jul 10, 2024
ee47222
running DR tests together with oneDPL tests (#1691)
lslusarczyk Jul 11, 2024
5065839
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Jul 11, 2024
f9c9a99
Alexey comments applied (#1699)
lslusarczyk Jul 17, 2024
1236faf
renamed rng namespace to stdrng (#1703)
lslusarczyk Jul 17, 2024
e775386
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Jul 17, 2024
1d1367a
Modify `inclusive_scan` to preserve oneDPL futures. (#1701)
BenBrock Jul 17, 2024
4e3a8c6
exclusive scan and sort preserves futures (#1705)
lslusarczyk Jul 17, 2024
f7149f8
Conditional use of libfmt (#1698)
mateuszpn Jul 18, 2024
0fdff35
Remove `views::slice` and `index<...>` tuple. (#1716)
BenBrock Jul 22, 2024
8e3e60d
Remove extraneous function (#1717)
BenBrock Jul 22, 2024
1fd08e6
rename - 'shp' to 'sp' (#1711)
haichangsi Jul 22, 2024
88c5181
Remove no-op `event` class used for non-SYCL MP. (#1718)
BenBrock Jul 22, 2024
3830442
Add equal to distributed ranges (#1719)
Xewar313 Jul 22, 2024
798b1e9
fixed gcc14 support (#1721)
lslusarczyk Jul 22, 2024
648eb43
removed unused init, added finalize (#1722)
lslusarczyk Jul 22, 2024
962e671
Merge remote-tracking branch 'origin/main' into distributed-ranges
lslusarczyk Jul 22, 2024
8e531af
Relative paths in includes (#1728)
mateuszpn Jul 31, 2024
41e2228
include gueards instead of pragma once (#1731)
mateuszpn Jul 31, 2024
b033088
Redundant conditions removed (#1730)
mateuszpn Jul 31, 2024
177daaf
includes from onedpl <> changed to "" (#1744)
mateuszpn Aug 1, 2024
4c5fc67
simple renames in ranges shim (#1739)
mateuszpn Aug 1, 2024
b0cd793
remove default dtors (#1745)
mateuszpn Aug 1, 2024
c3bc4ff
Merge branch 'main' into distributed-ranges
lslusarczyk Aug 1, 2024
81fb32e
distributed_device_policy renamed to sycl_device_collection (#1746)
mateuszpn Aug 1, 2024
3ca7233
removed fixranges (#1750)
lslusarczyk Aug 1, 2024
e760ce0
concepts.hpp moved and included in dpl/distributed_ranges (#1748)
mateuszpn Aug 1, 2024
cc69962
removed unused is_owning_view (#1753)
lslusarczyk Aug 2, 2024
b3e15fd
__detail::enumerate merged with views::enumerate (#1738)
mateuszpn Aug 2, 2024
d6f21d3
disabled all tests except distributed-ranges to speed up applying com…
lslusarczyk Aug 2, 2024
917eeb7
Fixes in __detail namespace (#1729)
mateuszpn Aug 5, 2024
530380b
sp/zip_view.hpp -> sp/views/zip.hpp (#1754)
mateuszpn Aug 5, 2024
f3a1df5
applied Mike comments about foraech (#1758)
lslusarczyk Aug 5, 2024
3557043
Move the use of ranges-v3 to "standalone" DR (#1761)
akukanov Aug 7, 2024
68f59fe
Updates from Adam's comments (#1766)
BenBrock Aug 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .clang-format
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
BasedOnStyle: LLVM

Standard: c++17
Standard: c++20
julianmi marked this conversation as resolved.
Show resolved Hide resolved

IndentWidth: 4
ColumnLimit: 120
Expand Down
18 changes: 18 additions & 0 deletions .github/workflows/ci.yml
julianmi marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ on:
- release_oneDPL
- main
- 'release/**'
- distributed-ranges
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an intention to keep a distributed-ranges active development branch which is distinct from main?

If not, this probably should be removed before merge.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is indeed no such intention. This is a temporary line. Let's keep this comment unresolved until the branch is approved to be merged and I will remove this line that time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good.


permissions: read-all

Expand Down Expand Up @@ -170,6 +171,12 @@ jobs:
build_type: release
backend: serial
device_type: HOST
- os: ubuntu-24.04
danhoeflinger marked this conversation as resolved.
Show resolved Hide resolved
cxx_compiler: icpx
std: 23
build_type: release
backend: dpcpp
device_type: CPU
steps:
- uses: actions/checkout@v4
- name: Set up Intel APT repository
Expand All @@ -194,6 +201,10 @@ jobs:
if: (matrix.device_type == 'FPGA_EMU')
run: |
sudo apt-get install intel-oneapi-compiler-fpga -y
- name: Install libfmt and gtest needed by Distributed Ranges tests
if: (matrix.std == '23')
run: |
sudo apt-get install libgtest-dev libfmt-dev -y
akukanov marked this conversation as resolved.
Show resolved Hide resolved
- name: Install OpenMP dependencies
if: (matrix.os == 'ubuntu-latest' && matrix.cxx_compiler == 'clang++' && matrix.backend == 'omp')
run: |
Expand All @@ -212,6 +223,13 @@ jobs:
make_targets="build-onedpl-sycl_iterator-tests"
ctest_flags="-R sycl_iterator_.*\.pass"
echo "::warning::dpcpp backend is set. Compile and run only sycl_iterator tests"

if [[ "${{ matrix.std }}" == "23" ]]; then
make_targets="$make_targets sp-all-tests"
ctest_flags="$ctest_flags|sp_tests"
echo "::warning::23 standard is set. Compile and run distributed ranges tests"
fi

else
make_targets="build-onedpl-tests"
fi
Expand Down
2 changes: 2 additions & 0 deletions cmake/oneDPLWindowsIntelLLVMApply.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ if (INTELLLVM_MSVC_WIN_STDOPTION_FIX)
set(CMAKE_CXX17_EXTENSION_COMPILE_OPTION "-Qstd:c++17")
set(CMAKE_CXX20_STANDARD_COMPILE_OPTION "-Qstd:c++20")
set(CMAKE_CXX20_EXTENSION_COMPILE_OPTION "-Qstd:c++20")
set(CMAKE_CXX23_STANDARD_COMPILE_OPTION "-Qstd:c++23")
set(CMAKE_CXX23_EXTENSION_COMPILE_OPTION "-Qstd:c++23")
endif()

if (INTELLLVM_MSVC_WIN_LINKORDER_FIX)
Expand Down
4 changes: 3 additions & 1 deletion cmake/oneDPLWindowsIntelLLVMConfig.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,15 @@ else()
set(CMAKE_CXX_COMPILER_ID "IntelLLVM GNU-Like Workaround" CACHE STRING "Switch compiler identification" FORCE)
set(CMAKE_CXX_COMPILER_VERSION "${INTEL_LLVM_COMPILER_VERSION}" CACHE STRING "Switch compiler version" FORCE)
# Explicitly setting the c++17 standard compile option so that "target_compile_features()" check functions properly for this workaround
set(CMAKE_CXX_COMPILE_FEATURES cxx_std_14 cxx_std_17 cxx_std_20)
set(CMAKE_CXX_COMPILE_FEATURES cxx_std_14 cxx_std_17 cxx_std_20 cxx_std_23)
set(CMAKE_CXX14_COMPILE_FEATURES cxx_std_14)
set(CMAKE_CXX17_COMPILE_FEATURES cxx_std_17)
set(CMAKE_CXX20_COMPILE_FEATURES cxx_std_20)
set(CMAKE_CXX23_COMPILE_FEATURES cxx_std_23)
set(CMAKE_CXX14_STANDARD_COMPILE_OPTION "-std=c++14")
set(CMAKE_CXX17_STANDARD_COMPILE_OPTION "-std=c++17")
set(CMAKE_CXX20_STANDARD_COMPILE_OPTION "-std=c++20")
set(CMAKE_CXX23_STANDARD_COMPILE_OPTION "-std=c++23")
set(CMAKE_CXX_STANDARD_COMPUTED_DEFAULT 14)
set(INTELLLVM_WIN_STD_IGNORE_FIX TRUE)
message(WARNING "On Windows, ${CMAKE_CXX_COMPILER} is not supported by CMake (https://gitlab.kitware.com/cmake/cmake/-/issues/24314) at this time and may encounter issues. We recommend using CMAKE_CXX_COMPILER=icx on Windows.")
Expand Down
29 changes: 29 additions & 0 deletions include/oneapi/dpl/distributed_ranges
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
// -*- C++ -*-
//===----------------------------------------------------------------------===//
//
// Copyright (C) Intel Corporation
//
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// This file incorporates work covered by the following copyright and permission
// notice:
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
//
Comment on lines +8 to +13
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this part of the copyright header should not be used within DR, since it does not inherit nor use any LLVM work. It should only apply to the standard parallel algorithms in oneDPL.

//===----------------------------------------------------------------------===//

#ifndef _ONEDPL_DISTRIBUTED_RANGES
#define _ONEDPL_DISTRIBUTED_RANGES

#include "oneapi/dpl/internal/common_config.h"
#include "oneapi/dpl/pstl/onedpl_config.h"

#if __cplusplus > 202002L
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This macro doesn't work on Microsoft. Let's use _ONEDPL___cplusplus or whatever it's called

#include "oneapi/dpl/internal/distributed_ranges_impl/concepts.hpp"
#include "oneapi/dpl/internal/distributed_ranges_impl/sp.hpp"
#else
#error "C++23 required to use Distributed Ranges"
#endif

#endif // _ONEDPL_DISTRIBUTED_RANGES
84 changes: 84 additions & 0 deletions include/oneapi/dpl/internal/distributed_ranges_impl/concepts.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
// -*- C++ -*-
//===----------------------------------------------------------------------===//
//
// Copyright (C) Intel Corporation
//
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// This file incorporates work covered by the following copyright and permission
// notice:
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
//
//===----------------------------------------------------------------------===//

#ifndef _ONEDPL_DR_CONCEPTS_CONCEPTS_HPP
#define _ONEDPL_DR_CONCEPTS_CONCEPTS_HPP

#include "detail/ranges.hpp"

namespace oneapi::dpl::experimental::dr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The biggest question is why those concepts are public?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is good to make these concepts public. At the very least, it makes the documentation clearer, but also it allows someone to implement their own variations of distributed containers, etc. And it certainly facilitates possible discussions.

{

template <typename I>
concept remote_iterator = std::forward_iterator<I> && requires(I& iter)
{
ranges::rank(iter);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we potentially end up with the name conflict for ranges namespace name? My guess is we should not based on my knowledge of lookup rules but do we want to change this line and the lines below to dr::ranges just to be safe?

To be honest I don't have a strong preference

};

template <typename R>
concept remote_range = stdrng::forward_range<R> && requires(R& r)
{
ranges::rank(r);
};

template <typename R>
concept distributed_range = stdrng::forward_range<R> && requires(R& r)
{
ranges::segments(r);
};

template <typename I>
concept remote_contiguous_iterator = std::random_access_iterator<I> && requires(I& iter)
{
ranges::rank(iter);
{
ranges::local(iter)
} -> std::contiguous_iterator;
};

template <typename I>
concept distributed_iterator = std::forward_iterator<I> && requires(I& iter)
{
ranges::segments(iter);
};

template <typename R>
concept remote_contiguous_range = remote_range<R> && stdrng::random_access_range<R> && requires(R& r)
{
{
ranges::local(r)
} -> stdrng::contiguous_range;
};

template <typename R>
concept distributed_contiguous_range = distributed_range<R> && stdrng::random_access_range<R> && requires(R& r)
{
{
ranges::segments(r)
} -> stdrng::random_access_range;
} && remote_contiguous_range<stdrng::range_value_t<decltype(ranges::segments(std::declval<R>()))>>;

template <typename Iter>
concept distributed_contiguous_iterator = distributed_iterator<Iter> && std::random_access_iterator<Iter> &&
requires(Iter& iter)
{
{
ranges::segments(iter)
} -> stdrng::random_access_range;
} && remote_contiguous_range<stdrng::range_value_t<decltype(ranges::segments(std::declval<Iter>()))>>;

} // namespace oneapi::dpl::experimental::dr

#endif /* _ONEDPL_DR_CONCEPTS_CONCEPTS_HPP */
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
// -*- C++ -*-
//===----------------------------------------------------------------------===//
//
// Copyright (C) Intel Corporation
//
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
// This file incorporates work covered by the following copyright and permission
// notice:
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
//
//===----------------------------------------------------------------------===//

#ifndef _ONEDPL_DR_DETAIL_ENUMERATE_HPP
#define _ONEDPL_DR_DETAIL_ENUMERATE_HPP

#include "std_ranges_shim.hpp"

namespace oneapi::dpl::experimental::dr
{

namespace __detail
{

namespace
{

template <stdrng::range R>
struct range_size
{
using type = std::size_t;
};

template <stdrng::sized_range R>
struct range_size<R>
{
using type = stdrng::range_size_t<R>;
};

template <stdrng::range R>
using range_size_t = typename range_size<R>::type;

} // namespace

class enumerate_adapter_closure
{
public:
template <stdrng::viewable_range R>
auto
operator()(R&& r) const
{
using S = range_size_t<R>;
// NOTE: This line only necessary due to bug in range-v3 where views
// have non-weakly-incrementable size types. (Standard mandates
// size type must be weakly incrementable.)
using W = std::conditional_t<std::weakly_incrementable<S>, S, std::size_t>;
danhoeflinger marked this conversation as resolved.
Show resolved Hide resolved
if constexpr (stdrng::sized_range<R>)
{
return stdrng::views::zip(stdrng::views::iota(W{0}, W{stdrng::size(r)}), std::forward<R>(r));
}
else
{
return stdrng::views::zip(stdrng::views::iota(W{0}), std::forward<R>(r));
}
}

template <stdrng::viewable_range R>
friend auto
operator|(R&& r, const enumerate_adapter_closure& closure)
{
return closure(std::forward<R>(r));
}
};

class enumerate_fn_
{
public:
template <stdrng::viewable_range R>
constexpr auto
operator()(R&& r) const
{
return enumerate_adapter_closure{}(std::forward<R>(r));
}

inline auto
enumerate() const
{
return enumerate_adapter_closure{};
}
};

inline constexpr auto enumerate = enumerate_fn_{};
lslusarczyk marked this conversation as resolved.
Show resolved Hide resolved

} // namespace __detail

} // namespace oneapi::dpl::experimental::dr

#endif /* _ONEDPL_DR_DETAIL_ENUMERATE_HPP */
Loading
Loading