[SYCL] Implement new env var SYCL_DEVICE_FILTER (#2239)

* [SYCL] Implement new env var SYCL_DEVICE_TRIPLE

This new env var takes a list of triples {device_type, backend, device_num}
1. This list means SYCL_RT will only use those specified devices.
2. This list also limits related plugins to be loaded by SYCL RT.
This PR only implemented new env var and selective plugin loading (#2)

Signed-off-by: Byoungro So <byoungro.so@intel.com>

Author: bso-intel

sycl/doc/EnvironmentVariables.md

@@ -12,8 +12,9 @@ subject to change. Do not rely on these variables in production code.
 | Environment variable | Values | Description |
 | -------------------- | ------ | ----------- |
 | SYCL_PI_TRACE | Described [below](#sycl_pi_trace-options)  | Enable specified level of tracing for PI. |
-| SYCL_BE | PI_OPENCL, PI_LEVEL_ZERO, PI_CUDA | Force SYCL RT to consider only devices of the specified backend during the device selection. |
-| SYCL_DEVICE_TYPE | One of: CPU, GPU, ACC, HOST | Force SYCL to use the specified device type. If unset, default selection rules are applied. If set to any unlisted value, this control has no effect. If the requested device type is not found, a `cl::sycl::runtime_error` exception is thrown. If a non-default device selector is used, a device must satisfy both the selector and this control to be chosen. This control only has effect on devices created with a selector. |
+| SYCL_BE | PI_OPENCL, PI_LEVEL_ZERO, PI_CUDA | Force SYCL RT to consider only devices of the specified backend during the device selection. We are planning to deprecate SYCL_BE environment variable in the future. The specific grace period is not decided yet. Please use the new env var SYCL_DEVICE_FILTER instead. |
+| SYCL_DEVICE_TYPE | One of: CPU, GPU, ACC, HOST | Force SYCL to use the specified device type. If unset, default selection rules are applied. If set to any unlisted value, this control has no effect. If the requested device type is not found, a `cl::sycl::runtime_error` exception is thrown. If a non-default device selector is used, a device must satisfy both the selector and this control to be chosen. This control only has effect on devices created with a selector. We are planning to deprecate SYCL_DEVICE_TYPE environment variable in the future. The specific grace period is not decided yet. Please use the new env var SYCL_DEVICE_FILTER instead. |
+| SYCL_DEVICE_FILTER (tentative name) | {backend:device_type:device_num} | Limits the SYCL RT to use only a subset of the system's devices. Setting this environment variable affects all of the device query functions and all of the device selectors. The value of this environment variable is a comma separated list of filters, where each filter is a triple of the form "backend:device_type:device_num" (without the quotes). Each element of the triple is optional, but each filter must have at least one value. Possible values of "backend" are "host", "level_zero", "opencl", "cuda", or "\*". Possible values of "device_type" are "host", "cpu", "gpu", "acc", or "\*". Device_num is an integer that indexes the enumeration of devices from the sycl::platform::get_device() call, where the first device in that enumeration has index zero. Assuming a filter has all three elements of the triple, it selects only those devices that come from the given backend, have the specified device type, AND have the given device index. If more than one filter is specified, the RT is restricted to the union of devices selected by all filters. The RT always includes the "host" backend and the host device regardless of the filter because the SYCL language requires this device to always be present. Therefore, including "host" in the list of filters is allowed but is unnecessary. Note that the standard selectors like gpu_selector or cpu_selector will throw an exception if the filtered list of devices does not include a device that satisfies the selector. In particular, limiting the devices to only those supported by the "level_zero" backend will cause the cpu_selector to throw an exception since that backend does not support any CPU devices. This environment variable can be used to limit loading only specified plugins into the SYCL RT. |
 | SYCL_PROGRAM_COMPILE_OPTIONS | String of valid OpenCL compile options | Override compile options for all programs. |
 | SYCL_PROGRAM_LINK_OPTIONS | String of valid OpenCL link options | Override link options for all programs. |
 | SYCL_USE_KERNEL_SPV | Path to the SPIR-V binary | Load device image from the specified file. If runtime is unable to read the file, `cl::sycl::runtime_error` exception is thrown.|

sycl/include/CL/sycl/backend_types.hpp

@@ -18,23 +18,26 @@
 __SYCL_INLINE_NAMESPACE(cl) {
 namespace sycl {
 
-enum class backend : char { host, opencl, level_zero, cuda };
+enum class backend : char { host, opencl, level_zero, cuda, all };
 
 template <backend name, typename SYCLObjectT> struct interop;
 
 inline std::ostream &operator<<(std::ostream &Out, backend be) {
   switch (be) {
   case backend::host:
-    Out << std::string("host");
+    Out << "host";
     break;
   case backend::opencl:
-    Out << std::string("opencl");
+    Out << "opencl";
     break;
   case backend::level_zero:
-    Out << std::string("level_zero");
+    Out << "level_zero";
     break;
   case backend::cuda:
-    Out << std::string("cuda");
+    Out << "cuda";
+    break;
+  case backend::all:
+    Out << "all";
   }
   return Out;
 }

sycl/include/CL/sycl/detail/device_filter.hpp

@@ -0,0 +1,83 @@
+//==---------- device_filter.hpp - SYCL device filter descriptor -----------==//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#pragma once
+
+#include <CL/sycl/backend_types.hpp>
+#include <CL/sycl/detail/defines.hpp>
+#include <CL/sycl/info/info_desc.hpp>
+
+#include <iostream>
+#include <string>
+
+__SYCL_INLINE_NAMESPACE(cl) {
+namespace sycl {
+namespace detail {
+
+struct device_filter {
+  backend Backend = backend::all;
+  info::device_type DeviceType = info::device_type::all;
+  int DeviceNum = 0;
+  bool HasBackend = false;
+  bool HasDeviceType = false;
+  bool HasDeviceNum = false;
+  int MatchesSeen = 0;
+
+  device_filter(){};
+  device_filter(const std::string &FilterString);
+  friend std::ostream &operator<<(std::ostream &Out,
+                                  const device_filter &Filter);
+};
+
+class device_filter_list {
+  std::vector<device_filter> FilterList;
+
+public:
+  device_filter_list() {}
+  device_filter_list(const std::string &FilterString);
+  device_filter_list(device_filter &Filter);
+  void addFilter(device_filter &Filter);
+  std::vector<device_filter> &get() { return FilterList; }
+  friend std::ostream &operator<<(std::ostream &Out,
+                                  const device_filter_list &List);
+};
+
+inline std::ostream &operator<<(std::ostream &Out,
+                                const device_filter &Filter) {
+  Out << Filter.Backend << ":";
+  if (Filter.DeviceType == info::device_type::host) {
+    Out << "host";
+  } else if (Filter.DeviceType == info::device_type::cpu) {
+    Out << "cpu";
+  } else if (Filter.DeviceType == info::device_type::gpu) {
+    Out << "gpu";
+  } else if (Filter.DeviceType == info::device_type::accelerator) {
+    Out << "accelerator";
+  } else if (Filter.DeviceType == info::device_type::all) {
+    Out << "*";
+  } else {
+    Out << "unknown";
+  }
+  if (Filter.HasDeviceNum) {
+    Out << ":" << Filter.DeviceNum;
+  }
+  return Out;
+}
+
+inline std::ostream &operator<<(std::ostream &Out,
+                                const device_filter_list &List) {
+  for (const device_filter &Filter : List.FilterList) {
+    Out << Filter;
+    Out << ",";
+  }
+  return Out;
+}
+
+} // namespace detail
+} // namespace sycl
+} // __SYCL_INLINE_NAMESPACE(cl)

sycl/source/CMakeLists.txt

@@ -108,6 +108,7 @@ set(SYCL_SOURCES
     "detail/config.cpp"
     "detail/context_impl.cpp"
     "detail/device_binary_image.cpp"
+    "detail/device_filter.cpp"
     "detail/device_impl.cpp"
     "detail/error_handling/enqueue_kernel.cpp"
     "detail/event_impl.cpp"

sycl/source/detail/config.def

@@ -16,3 +16,4 @@ CONFIG(SYCL_DEVICE_ALLOWLIST, 1024, __SYCL_DEVICE_ALLOWLIST)
 CONFIG(SYCL_BE, 16, __SYCL_BE)
 CONFIG(SYCL_PI_TRACE, 16, __SYCL_PI_TRACE)
 CONFIG(SYCL_DEVICELIB_NO_FALLBACK, 1, __SYCL_DEVICELIB_NO_FALLBACK)
+CONFIG(SYCL_DEVICE_FILTER, 1024, __SYCL_DEVICE_FILTER)

sycl/source/detail/config.hpp

@@ -10,7 +10,9 @@
 
 #include <CL/sycl/backend_types.hpp>
 #include <CL/sycl/detail/defines.hpp>
+#include <CL/sycl/detail/device_filter.hpp>
 #include <CL/sycl/detail/pi.hpp>
+#include <CL/sycl/info/info_desc.hpp>
 
 #include <algorithm>
 #include <array>
@@ -163,6 +165,35 @@ template <> class SYCLConfig<SYCL_PI_TRACE> {
   }
 };
 
+template <> class SYCLConfig<SYCL_DEVICE_FILTER> {
+  using BaseT = SYCLConfigBase<SYCL_DEVICE_FILTER>;
+
+public:
+  static device_filter_list *get() {
+    static bool Initialized = false;
+    static device_filter_list *FilterList = nullptr;
+
+    // Configuration parameters are processed only once, like reading a string
+    // from environment and converting it into a typed object.
+    if (Initialized) {
+      return FilterList;
+    }
+
+    const char *ValStr = BaseT::getRawValue();
+    if (ValStr) {
+      static device_filter_list DFL{ValStr};
+      FilterList = &DFL;
+    }
+    // As mentioned above, configuration parameters are processed only once.
+    // If multiple threads are checking this env var at the same time,
+    // they will end up setting the configration to the same value.
+    // If other threads check after one thread already set configration,
+    // the threads will get the same value as the first thread.
+    Initialized = true;
+    return FilterList;
+  }
+};
+
 } // namespace detail
 } // namespace sycl
 } // __SYCL_INLINE_NAMESPACE(cl)

sycl/source/detail/device_filter.cpp

@@ -0,0 +1,122 @@
+//==------------------- device_filter.cpp ----------------------------------==//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include <CL/sycl/detail/device_filter.hpp>
+#include <CL/sycl/info/info_desc.hpp>
+#include <detail/config.hpp>
+#include <detail/device_impl.hpp>
+
+#include <cstring>
+
+__SYCL_INLINE_NAMESPACE(cl) {
+namespace sycl {
+namespace detail {
+
+device_filter::device_filter(const std::string &FilterString) {
+  const std::array<std::pair<std::string, info::device_type>, 5>
+      SyclDeviceTypeMap = {{{"host", info::device_type::host},
+                            {"cpu", info::device_type::cpu},
+                            {"gpu", info::device_type::gpu},
+                            {"acc", info::device_type::accelerator},
+                            {"*", info::device_type::all}}};
+  const std::array<std::pair<std::string, backend>, 5> SyclBeMap = {
+      {{"host", backend::host},
+       {"opencl", backend::opencl},
+       {"level_zero", backend::level_zero},
+       {"cuda", backend::cuda},
+       {"*", backend::all}}};
+
+  size_t Cursor = 0;
+  size_t ColonPos = 0;
+  auto findElement = [&](auto Element) {
+    size_t Found = FilterString.find(Element.first, Cursor);
+    if (Found == std::string::npos)
+      return false;
+    Cursor = Found;
+    return true;
+  };
+  auto selectElement = [&](auto It, auto Map, auto EltIfNotFound) {
+    if (It == Map.end())
+      return EltIfNotFound;
+    ColonPos = FilterString.find(":", Cursor);
+    if (ColonPos != std::string::npos)
+      Cursor = ColonPos + 1;
+    else
+      Cursor = Cursor + It->first.size();
+    return It->second;
+  };
+
+  // Handle the optional 1st field of the filter, backend
+  // Check if the first entry matches with a known backend type
+  auto It =
+      std::find_if(std::begin(SyclBeMap), std::end(SyclBeMap), findElement);
+  // If no match is found, set the backend type backend::all
+  // which actually means 'any backend' will be a match.
+  Backend = selectElement(It, SyclBeMap, backend::all);
+
+  // Handle the optional 2nd field of the filter - device type.
+  // Check if the 2nd entry matches with any known device type.
+  if (Cursor >= FilterString.size()) {
+    DeviceType = info::device_type::all;
+  } else {
+    auto Iter = std::find_if(std::begin(SyclDeviceTypeMap),
+                             std::end(SyclDeviceTypeMap), findElement);
+    // If no match is found, set device_type 'all',
+    // which actually means 'any device_type' will be a match.
+    DeviceType = selectElement(Iter, SyclDeviceTypeMap, info::device_type::all);
+  }
+
+  // Handle the optional 3rd field of the filter, device number
+  // Try to convert the remaining string to an integer.
+  // If succeessful, the converted integer is the desired device num.
+  if (Cursor < FilterString.size()) {
+    try {
+      DeviceNum = stoi(FilterString.substr(ColonPos + 1));
+      HasDeviceNum = true;
+    } catch (...) {
+      std::string Message =
+          std::string("Invalid device filter: ") + FilterString +
+          "\nPossible backend values are {host,opencl,level_zero,cuda,*}.\n"
+          "Possible device types are {host,cpu,gpu,acc,*}.\n"
+          "Device number should be an non-negative integer.\n";
+      throw cl::sycl::invalid_parameter_error(Message, PI_INVALID_VALUE);
+    }
+  }
+}
+
+device_filter_list::device_filter_list(const std::string &FilterStr) {
+  // First, change the string in all lowercase.
+  // This means we allow the user to use both uppercase and lowercase strings.
+  std::string FilterString = FilterStr;
+  std::transform(FilterString.begin(), FilterString.end(), FilterString.begin(),
+                 ::tolower);
+  // SYCL_DEVICE_FILTER can set multiple filters separated by commas.
+  // convert each filter triple string into an istance of device_filter class.
+  size_t Pos = 0;
+  while (Pos < FilterString.size()) {
+    size_t CommaPos = FilterString.find(",", Pos);
+    if (CommaPos == std::string::npos) {
+      CommaPos = FilterString.size();
+    }
+    std::string SubString = FilterString.substr(Pos, CommaPos - Pos);
+    FilterList.push_back(device_filter(SubString));
+    Pos = CommaPos + 1;
+  }
+}
+
+device_filter_list::device_filter_list(device_filter &Filter) {
+  FilterList.push_back(Filter);
+}
+
+void device_filter_list::addFilter(device_filter &Filter) {
+  FilterList.push_back(Filter);
+}
+
+} // namespace detail
+} // namespace sycl
+} // __SYCL_INLINE_NAMESPACE(cl)

sycl/source/detail/filter_selector_impl.cpp

@@ -62,13 +62,13 @@ filter create_filter(const std::string &Input) {
 
   for (const std::string &Token : Tokens) {
     if (Token == "cpu" && !Result.HasDeviceType) {
-      Result.DeviceType = PI_DEVICE_TYPE_CPU;
+      Result.DeviceType = info::device_type::cpu;
       Result.HasDeviceType = true;
     } else if (Token == "gpu" && !Result.HasDeviceType) {
-      Result.DeviceType = PI_DEVICE_TYPE_GPU;
+      Result.DeviceType = info::device_type::gpu;
       Result.HasDeviceType = true;
     } else if (Token == "accelerator" && !Result.HasDeviceType) {
-      Result.DeviceType = PI_DEVICE_TYPE_ACC;
+      Result.DeviceType = info::device_type::accelerator;
       Result.HasDeviceType = true;
     } else if (Token == "opencl" && !Result.HasBackend) {
       Result.Backend = backend::opencl;
@@ -134,8 +134,7 @@ int filter_selector_impl::operator()(const device &Dev) const {
       BackendOK = (BE == Filter.Backend);
     }
     if (Filter.HasDeviceType) {
-      RT::PiDeviceType DT =
-          sycl::detail::getSyclObjImpl(Dev)->get_device_type();
+      info::device_type DT = Dev.get_info<info::device::device_type>();
       DeviceTypeOK = (DT == Filter.DeviceType);
     }
     if (Filter.HasDeviceNum) {

sycl/source/detail/filter_selector_impl.hpp

@@ -8,6 +8,7 @@
 
 #pragma once
 
+#include <CL/sycl/detail/device_filter.hpp>
 #include <CL/sycl/device_selector.hpp>
 
 #include <vector>
@@ -21,15 +22,7 @@ class device;
 namespace ONEAPI {
 namespace detail {
 
-struct filter {
-  backend Backend = backend::host;
-  RT::PiDeviceType DeviceType = PI_DEVICE_TYPE_ALL;
-  int DeviceNum = 0;
-  bool HasBackend = false;
-  bool HasDeviceType = false;
-  bool HasDeviceNum = false;
-  int MatchesSeen = 0;
-};
+typedef struct sycl::detail::device_filter filter;
 
 class filter_selector_impl {
 public: