TBB oneapi compatibility #1600

jiapei100 · 2021-08-23T22:04:42Z

Description of Issue

Looks like USD is NOT compatible with the most recent Intel oneapi 2021.3.0 ?

[ 49%] Building CXX object pxr/base/work/CMakeFiles/work.dir/utils.cpp.o
cd ....../USD/build/pxr/base/work && /usr/local/bin/c++ -DBOOST_PYTHON_NO_PY_SIGNATURES -DGLX_GLXEXT_PROTOTYPES -DGL_GLEXT_PROTOTYPES -DMFB_ALT_PACKAGE_NAME=work -DMFB_PACKAGE_MODULE=Work -DMFB_PACKAGE_NAME=work -DPXR_BUILD_LOCATION=usd -DPXR_OPENVDB_SUPPORT_ENABLED -DPXR_PLUGIN_BUILD_LOCATION=../plugin/usd -DPXR_PTEX_SUPPORT_ENABLED -DPXR_PYTHON_ENABLED=1 -DPXR_PYTHON_MODULES_ENABLED=1 -DWORK_EXPORTS=1 -Dwork_EXPORTS -I....../USD/build/pxr/base/work -I....../USD/pxr/base/work -I....../USD/build/include -isystem /opt/intel/oneapi/tbb/latest/include -isystem /usr/include/python3.8 -Wall -pthread -Wno-deprecated -Wno-deprecated-declarations -Wno-unused-local-typedefs  -O3 -DNDEBUG -fPIC -std=c++14 -MD -MT pxr/base/work/CMakeFiles/work.dir/utils.cpp.o -MF CMakeFiles/work.dir/utils.cpp.o.d -o CMakeFiles/work.dir/utils.cpp.o -c ....../USD/pxr/base/work/utils.cpp
....../USD/pxr/base/work/threadLimits.cpp:65:13: error: ‘task_scheduler_init’ in namespace ‘tbb’ does not name a type
   65 | static tbb::task_scheduler_init *_tbbTaskSchedInit;
      |             ^~~~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/threadLimits.cpp: In function ‘unsigned int pxrInternal_v0_21__pxrReserved__::WorkGetPhysicalConcurrencyLimit()’:
....../USD/pxr/base/work/threadLimits.cpp:72:17: error: ‘tbb::task_scheduler_init’ has not been declared
   72 |     return tbb::task_scheduler_init::default_num_threads();
      |                 ^~~~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/threadLimits.cpp: In function ‘void pxrInternal_v0_21__pxrReserved__::Work_InitializeThreading()’:
....../USD/pxr/base/work/threadLimits.cpp:128:9: error: ‘_tbbTaskSchedInit’ was not declared in this scope
  128 |         _tbbTaskSchedInit = new tbb::task_scheduler_init(threadLimit);
      |         ^~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/threadLimits.cpp:128:33: error: expected type-specifier
  128 |         _tbbTaskSchedInit = new tbb::task_scheduler_init(threadLimit);
      |                                 ^~~
....../USD/pxr/base/work/threadLimits.cpp:119:14: warning: unused variable ‘threadLimit’ [-Wunused-variable]
  119 |     unsigned threadLimit =
      |              ^~~~~~~~~~~
....../USD/pxr/base/work/threadLimits.cpp: In function ‘void pxrInternal_v0_21__pxrReserved__::WorkSetConcurrencyLimit(unsigned int)’:
....../USD/pxr/base/work/threadLimits.cpp:167:9: error: ‘_tbbTaskSchedInit’ was not declared in this scope
  167 |     if (_tbbTaskSchedInit) {
      |         ^~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/threadLimits.cpp:171:33: error: expected type-specifier
  171 |         _tbbTaskSchedInit = new tbb::task_scheduler_init(threadLimit);
      |                                 ^~~
make[2]: *** [pxr/base/work/CMakeFiles/work.dir/build.make:194: pxr/base/work/CMakeFiles/work.dir/threadLimits.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from ....../USD/pxr/base/work/detachedTask.h:32,
                 from ....../USD/pxr/base/work/detachedTask.cpp:26:
....../USD/pxr/base/work/dispatcher.h:136:44: error: expected class-name before ‘{’ token
  136 |     struct _InvokerTask : public tbb::task {
      |                                            ^
....../USD/pxr/base/work/dispatcher.h:143:22: error: ‘task’ in namespace ‘tbb’ does not name a type
  143 |         virtual tbb::task* execute() {
      |                      ^~~~
In file included from /opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/enumerable_thread_specific.h:36,
                 from /opt/intel/oneapi/tbb/latest/include/tbb/enumerable_thread_specific.h:17,
                 from ....../USD/pxr/base/tf/diagnosticMgr.h:46,
                 from ....../USD/pxr/base/tf/errorMark.h:30,
                 from ....../USD/pxr/base/work/detachedTask.h:30,
                 from ....../USD/pxr/base/work/detachedTask.cpp:26:
/opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/task.h:26:11: note: ‘tbb::v1::task’ declared here
   26 | namespace task {
      |           ^~~~
In file included from ....../USD/pxr/base/work/detachedTask.h:32,
                 from ....../USD/pxr/base/work/detachedTask.cpp:26:
....../USD/pxr/base/work/dispatcher.h:171:5: error: ‘detail’ does not name a type
  171 |     detail::d1::task_group_context _context;
      |     ^~~~~~
....../USD/pxr/base/work/dispatcher.h:172:10: error: ‘empty_task’ in namespace ‘tbb’ does not name a type
  172 |     tbb::empty_task* _rootTask;
      |          ^~~~~~~~~~
In file included from ....../USD/pxr/base/work/dispatcher.cpp:26:
....../USD/pxr/base/work/dispatcher.h:136:44: error: expected class-name before ‘{’ token
  136 |     struct _InvokerTask : public tbb::task {
      |                                            ^
....../USD/pxr/base/work/dispatcher.h:143:22: error: ‘task’ in namespace ‘tbb’ does not name a type
  143 |         virtual tbb::task* execute() {
      |                      ^~~~
In file included from ....../USD/pxr/base/work/detachedTask.h:32,
                 from ....../USD/pxr/base/work/utils.h:31,
                 from ....../USD/pxr/base/work/utils.cpp:26:
....../USD/pxr/base/work/dispatcher.h:136:44: error: expected class-name before ‘{’ token
  136 |     struct _InvokerTask : public tbb::task {
      |                                            ^
In file included from /opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/enumerable_thread_specific.h:36,
                 from /opt/intel/oneapi/tbb/latest/include/tbb/enumerable_thread_specific.h:17,
                 from ....../USD/pxr/base/tf/diagnosticMgr.h:46,
                 from ....../USD/pxr/base/tf/errorMark.h:30,
                 from ....../USD/pxr/base/work/dispatcher.h:33,
                 from ....../USD/pxr/base/work/dispatcher.cpp:26:
/opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/task.h:26:11: note: ‘tbb::v1::task’ declared here
   26 | namespace task {
      |           ^~~~
....../USD/pxr/base/work/dispatcher.h:143:22: error: ‘task’ in namespace ‘tbb’ does not name a type
  143 |         virtual tbb::task* execute() {
      |                      ^~~~
In file included from /opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/enumerable_thread_specific.h:36,
                 from /opt/intel/oneapi/tbb/latest/include/tbb/enumerable_thread_specific.h:17,
                 from ....../USD/pxr/base/tf/diagnosticMgr.h:46,
                 from ....../USD/pxr/base/tf/errorMark.h:30,
                 from ....../USD/pxr/base/work/detachedTask.h:30,
                 from ....../USD/pxr/base/work/utils.h:31,
                 from ....../USD/pxr/base/work/utils.cpp:26:
/opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/task.h:26:11: note: ‘tbb::v1::task’ declared here
   26 | namespace task {
      |           ^~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘void pxrInternal_v0_21__pxrReserved__::WorkDispatcher::Run(Callable&&)’:
....../USD/pxr/base/work/dispatcher.h:102:9: error: ‘_rootTask’ was not declared in this scope
  102 |         _rootTask->spawn(_MakeInvokerTask(std::forward<Callable>(c)));
      |         ^~~~~~~~~
In file included from ....../USD/pxr/base/work/dispatcher.cpp:26:
....../USD/pxr/base/work/dispatcher.h:171:5: error: ‘detail’ does not name a type
  171 |     detail::d1::task_group_context _context;
      |     ^~~~~~
....../USD/pxr/base/work/dispatcher.h:172:10: error: ‘empty_task’ in namespace ‘tbb’ does not name a type
  172 |     tbb::empty_task* _rootTask;
      |          ^~~~~~~~~~
In file included from ....../USD/pxr/base/work/detachedTask.h:32,
                 from ....../USD/pxr/base/work/utils.h:31,
                 from ....../USD/pxr/base/work/utils.cpp:26:
....../USD/pxr/base/work/dispatcher.h:171:5: error: ‘detail’ does not name a type
  171 |     detail::d1::task_group_context _context;
      |     ^~~~~~
....../USD/pxr/base/work/dispatcher.h:172:10: error: ‘empty_task’ in namespace ‘tbb’ does not name a type
  172 |     tbb::empty_task* _rootTask;
      |          ^~~~~~~~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_InvokerTask<typename std::remove_reference<_Tp>::type>& pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_MakeInvokerTask(Fn&&)’:
....../USD/pxr/base/work/dispatcher.h:159:22: error: ‘_rootTask’ was not declared in this scope
  159 |         return *new( _rootTask->allocate_additional_child_of(*_rootTask) )
      |                      ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘void pxrInternal_v0_21__pxrReserved__::WorkDispatcher::Run(Callable&&)’:
....../USD/pxr/base/work/dispatcher.h:102:9: error: ‘_rootTask’ was not declared in this scope
  102 |         _rootTask->spawn(_MakeInvokerTask(std::forward<Callable>(c)));
      |         ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘void pxrInternal_v0_21__pxrReserved__::WorkDispatcher::Run(Callable&&)’:
....../USD/pxr/base/work/dispatcher.h:102:9: error: ‘_rootTask’ was not declared in this scope
  102 |         _rootTask->spawn(_MakeInvokerTask(std::forward<Callable>(c)));
      |         ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_InvokerTask<typename std::remove_reference<_Tp>::type>& pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_MakeInvokerTask(Fn&&)’:
....../USD/pxr/base/work/dispatcher.h:159:22: error: ‘_rootTask’ was not declared in this scope
  159 |         return *new( _rootTask->allocate_additional_child_of(*_rootTask) )
      |                      ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.h: In member function ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_InvokerTask<typename std::remove_reference<_Tp>::type>& pxrInternal_v0_21__pxrReserved__::WorkDispatcher::_MakeInvokerTask(Fn&&)’:
....../USD/pxr/base/work/dispatcher.h:159:22: error: ‘_rootTask’ was not declared in this scope
  159 |         return *new( _rootTask->allocate_additional_child_of(*_rootTask) )
      |                      ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp: In constructor ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher::WorkDispatcher()’:
....../USD/pxr/base/work/dispatcher.cpp:31:7: error: class ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher’ does not have any field named ‘_context’
   31 |     : _context(
      |       ^~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:32:14: error: ‘tbb::task_group_context’ has not been declared
   32 |         tbb::task_group_context::isolated,
      |              ^~~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:33:14: error: ‘tbb::task_group_context’ has not been declared
   33 |         tbb::task_group_context::concurrent_wait |
      |              ^~~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:34:14: error: ‘tbb::task_group_context’ has not been declared
   34 |         tbb::task_group_context::default_traits)
      |              ^~~~~~~~~~~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:39:5: error: ‘_rootTask’ was not declared in this scope
   39 |     _rootTask = new(tbb::task::allocate_root(_context)) tbb::empty_task;
      |     ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:39:32: error: ‘allocate_root’ is not a member of ‘tbb::v1::task’
   39 |     _rootTask = new(tbb::task::allocate_root(_context)) tbb::empty_task;
      |                                ^~~~~~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:39:46: error: ‘_context’ was not declared in this scope
   39 |     _rootTask = new(tbb::task::allocate_root(_context)) tbb::empty_task;
      |                                              ^~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:39:62: error: ‘empty_task’ in namespace ‘tbb’ does not name a type
   39 |     _rootTask = new(tbb::task::allocate_root(_context)) tbb::empty_task;
      |                                                              ^~~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp: In destructor ‘pxrInternal_v0_21__pxrReserved__::WorkDispatcher::~WorkDispatcher()’:
....../USD/pxr/base/work/dispatcher.cpp:46:16: error: ‘destroy’ is not a member of ‘tbb::v1::task’; did you mean ‘tbb::detail::d1::destroy’?
   46 |     tbb::task::destroy(*_rootTask);
      |                ^~~~~~~
In file included from /opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/enumerable_thread_specific.h:29,
                 from /opt/intel/oneapi/tbb/latest/include/tbb/enumerable_thread_specific.h:17,
                 from ....../USD/pxr/base/tf/diagnosticMgr.h:46,
                 from ....../USD/pxr/base/tf/errorMark.h:30,
                 from ....../USD/pxr/base/work/dispatcher.h:33,
                 from ....../USD/pxr/base/work/dispatcher.cpp:26:
/opt/intel/oneapi/tbb/2021.3.0/include/oneapi/tbb/profiling.h:68:63: note: ‘tbb::detail::d1::destroy’ declared here
   68 |     enum notify_type {prepare=0, cancel, acquired, releasing, destroy};
      |                                                               ^~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:46:25: error: ‘_rootTask’ was not declared in this scope
   46 |     tbb::task::destroy(*_rootTask);
      |                         ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp: In member function ‘void pxrInternal_v0_21__pxrReserved__::WorkDispatcher::Wait()’:
....../USD/pxr/base/work/dispatcher.cpp:52:5: error: ‘_rootTask’ was not declared in this scope
   52 |     _rootTask->wait_for_all();
      |     ^~~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp:54:9: error: ‘_context’ was not declared in this scope
   54 |     if (_context.is_group_execution_cancelled()) {
      |         ^~~~~~~~
....../USD/pxr/base/work/dispatcher.cpp: In member function ‘void pxrInternal_v0_21__pxrReserved__::WorkDispatcher::Cancel()’:
....../USD/pxr/base/work/dispatcher.cpp:68:5: error: ‘_context’ was not declared in this scope
   68 |     _context.cancel_group_execution();
      |     ^~~~~~~~
make[2]: *** [pxr/base/work/CMakeFiles/work.dir/build.make:208: pxr/base/work/CMakeFiles/work.dir/utils.cpp.o] Error 1
make[2]: *** [pxr/base/work/CMakeFiles/work.dir/build.make:124: pxr/base/work/CMakeFiles/work.dir/detachedTask.cpp.o] Error 1
make[2]: *** [pxr/base/work/CMakeFiles/work.dir/build.make:138: pxr/base/work/CMakeFiles/work.dir/dispatcher.cpp.o] Error 1
make[2]: Leaving directory '....../USD/build'
make[1]: *** [CMakeFiles/Makefile2:3458: pxr/base/work/CMakeFiles/work.dir/all] Error 2
make[1]: Leaving directory '....../USD/build'
make: *** [Makefile:149: all] Error 2

Steps to Reproduce

Just build USD directly

System Information (OS, Hardware)

Ubuntu 20.04
GPU Geforce 2080 Ti
Python 3.8.10

Package Versions

➜  build git:(release) ✗ pip show pyside2
Name: PySide2
Version: 5.15.2
Summary: Python bindings for the Qt cross-platform application and UI framework
Home-page: https://www.pyside.org
Author: Qt for Python Team
Author-email: pyside@qt-project.org
License: LGPL
Location: ~/.local/lib/python3.8/site-packages
Requires: shiboken2
Required-by:

Build Flags

spiffmon · 2021-08-23T23:17:17Z

That is correct, @jiapei100 . OneAPI is a major change for TBB, and it does not make sense for us to invest in moving to it until the entire VFX industry is, because we must all share the same TBB instance when USD is hosted in an application. That will be guided by the VFX reference platform, which does not yet have OneAPI on its roadmap.

jilliene · 2021-08-24T21:49:06Z

Filed as internal issue #USD-6857

@LilyWangLL

* [tbb] Update to 2021.3.0 * [tbb] Add Threads dependency * [pagmo2] Update to support new TBB * [openvdb] Update to 8.1.0 * [embree2] Remove from baseline -- it is no longer receiving support from upstream * [usd] Mark as unsupported * [usd] Disable USD in CI due to policy PixarAnimationStudios/OpenUSD#1600 * [openvdb] Bump port-version * Update version database * [pagmo2] Fix vcpkg.json * update version * wip update * versions * [embree2] deprecate, [openvdb,usd] resolve conflicts, [tbb] update * Added libxml port. * Remove port version as it's initial port. * Added baseline version * Support only for windows and static * Allowed building debug version * update versions * Update ports/libxpm/portfile.cmake Co-authored-by: Thomas1664 <46387399+Thomas1664@users.noreply.github.com> * Added new lines * Update ports/libxpm/vcpkg.json Co-authored-by: Thomas1664 <46387399+Thomas1664@users.noreply.github.com> * Update ports/libxpm/portfile.cmake Co-authored-by: Thomas1664 <46387399+Thomas1664@users.noreply.github.com> * Update ports/libxpm/vcpkg.json Co-authored-by: Thomas1664 <46387399+Thomas1664@users.noreply.github.com> * Regenerated versions * Update ports/libxpm/vcpkg.json Co-authored-by: JonLiu1993 <63675417+JonLiu1993@users.noreply.github.com> * Libxpm is taken from gitlab now. * Dropped changes against master in original repo for libxpm. * Dropped libxpm from baseline * Dropped version for libxpm * Update cpuinfo * Updated date of version * Update version database * Removed support for arm32 & uwp as library is not supporting it. * Version regenerated * Update ports/cpuinfo/vcpkg.json Co-authored-by: Alexander Neumann <30894796+Neumann-A@users.noreply.github.com> * Updated versions * Initial commit WIP * Fixed compilation of latest stable version * Dropped comments * Switch from version-string to version * Added versions * Fixed path of cmake files for other projects * Updated versions * Various modernization. * Fully mark embree2 as deprecated. * [pagmo2] Disable -Werror * [cctag] Fix finding and use of TBB. * [openvdb] Baseline the static versions because they exceed FFFFFFFF in size, and I'm not sure if that's fixable.... * [gazebo] Apply gazebosim/gazebo-classic#3174 patch to fix TBB. * Respond to failures in https://dev.azure.com/vcpkg/public/_build/results?buildId=76586 * [pagmo2] Add license. See https://github.com/esa/pagmo2/blob/19d774fbb6128124305225803f8c1ad9e2f95c8c/src/bfe.cpp#L5-L27 * [tbb] Add license. * [embree2] Add "license". * [tbb] Use vcpkg_cmake_config_fixup and fix version as requested by @LilyWangLL * [usd] Add usd is known broken message. * [embree2] Remove completely. * Fix version database. * Fix usd version database. * Fix wrong case on Linux. * Lowercase the tbb directory to get to their configs. * [cctag] minimize patches * [tbb] Apply supports expression fix suggested in #26284 (comment) * [usd] Add note about upstream issue. * [pagmo2] Minimize patch. Co-authored-by: Robert Schumacher <roschuma@microsoft.com> Co-authored-by: Jonliu1993 <13720414433@163.com> Co-authored-by: Mathis Logemann <mathisloge@gmail.com> Co-authored-by: Victor Romero <viromer@microsoft.com> Co-authored-by: Vladimír Aubrecht <vladimir.aubrecht@me.com> Co-authored-by: Thomas1664 <46387399+Thomas1664@users.noreply.github.com> Co-authored-by: JonLiu1993 <63675417+JonLiu1993@users.noreply.github.com> Co-authored-by: Alexander Neumann <30894796+Neumann-A@users.noreply.github.com>

traversaro · 2024-10-06T19:20:17Z

Hello everyone, this was fixed in #2466 and released in 24.08 ?

ras0219-msft added a commit to ras0219-msft/vcpkg that referenced this issue Sep 15, 2021

[usd] Disable USD in CI due to policy PixarAnimationStudios/OpenUSD#1600

ac412fa

ras0219-msft mentioned this issue Sep 9, 2022

[tbb, pagmo2] Update TBB to 2021.5 and update pagmo2 to 2.18.0 microsoft/vcpkg#26284

Merged

Danielmelody mentioned this issue Oct 1, 2022

[opensubdiv] Upgrade to 3.4.4 microsoft/vcpkg#26628

Merged

Cheney-W mentioned this issue Dec 7, 2022

[USD] update to 22.08 microsoft/vcpkg#26920

Closed

Cheney-W mentioned this issue Feb 8, 2023

[USD] update to 22.08 microsoft/vcpkg#26941

Closed

This was referenced Mar 2, 2023

Use non-deprecated TBB headers in pxr/base/work #2321

Merged

Use non-deprecated TBB headers in pxr/usd/usd/instanceCache.h #2322

Merged

wangcan26 mentioned this issue Nov 23, 2023

cmake adapting oneTBB: migrate tbb to oneTBB #2825

Open

7 tasks

jwnimmer-tri mentioned this issue Feb 10, 2024

Add OpenUSD to the build RobotLocomotion/drake#20898

Open

22 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TBB oneapi compatibility #1600

TBB oneapi compatibility #1600

jiapei100 commented Aug 23, 2021 •

edited

Loading

spiffmon commented Aug 23, 2021

jilliene commented Aug 24, 2021

traversaro commented Oct 6, 2024

TBB oneapi compatibility #1600

TBB oneapi compatibility #1600

Comments

jiapei100 commented Aug 23, 2021 • edited Loading

Description of Issue

Steps to Reproduce

System Information (OS, Hardware)

Package Versions

Build Flags

spiffmon commented Aug 23, 2021

jilliene commented Aug 24, 2021

traversaro commented Oct 6, 2024

jiapei100 commented Aug 23, 2021 •

edited

Loading