Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[v1.x] Port changes from master to make CPP package properly build when large tensor support enabled #20768

Merged
merged 1 commit into from
Dec 16, 2021

Conversation

josephevans
Copy link
Contributor

We've had some users report build failures when attempting to enable large tensor support. Without this change, there are 3 operators that cause issues when large tensor support is enabled and CPP package is built.

Running: OpWrapperGenerator.py
argument "begin" of operator "slice" has unknown type "tuple of <>, required"
argument "end" of operator "slice" has unknown type "tuple of <>, required"
argument "step" of operator "slice" has unknown type "tuple of <>, optional, default=[]"
argument "end" of operator "slice_axis" has unknown type ", required"
FAILED: tests/CMakeFiles/mxnet_unit_tests.dir/cpp/thread_safety/thread_safety_test.cc.o
/usr/bin/c++  -DDMLC_CORE_USE_CMAKE -DDMLC_LOG_FATAL_THROW=1 -DDMLC_LOG_STACK_TRACE_SIZE=0 -DDMLC_MODERN_THREAD_LOCAL=0 -DDMLC_USE_CXX11=1 -DMSHADOW_FORCE_STREAM -DMSHAD
OW_INT64_TENSOR_SIZE=1 -DMSHADOW_IN_CXX11 -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_CUDA=1 -DMSHADOW_USE_CUDNN -DMSHADOW_USE_F16C=0 -DMSHADOW_USE_MKL=0 -DMSHADOW_USE_SSE -DMXNET_USE_BLAS_OPEN=1 -DMXNET_USE_CPP_PACKAGE=1 -DMXNET_U
SE_CUDA=1 -DMXNET_USE_INTGEMM=1 -DMXNET_USE_LAPACK=1 -DMXNET_USE_NVTX=1 -DMXNET_USE_OPENCV=1 -DMXNET_USE_OPENMP=1 -DMXNET_USE_OPERATOR_TUNING=1 -DMXNET_USE_SIGNAL_HANDLER=1 -DNDEBUG=1 -DUSE_CUDNN -D__USE_XOPEN2K8 -I../inclu
de -I../src -I../3rdparty/tvm/nnvm/include -I../3rdparty/tvm/include -I../3rdparty/dmlc-core/include -I../3rdparty/dlpack/include -I../tests/cpp/include -I../tests/../cpp-package/include -I../3rdparty/mshadow -I3rdparty/dml
c-core/include -isystem ../3rdparty/googletest/googletest/include -
isystem ../3rdparty/googletest/googletest -Wall -Wno-unknown-pragmas -Wno-sign-compare -O3 -std=c++11 -fopenmp -O3 -DNDEBUG -fPIE   -msse3 -std=gnu++11 -MD -MT tests/CMakeFiles/mxnet_unit_tests.dir/cpp/thread_safety/thread_
safety_test.cc.o -MF tests/CMakeFiles/mxnet_unit_tests.dir/cpp/thread_safety/thread_safety_test.cc.o.d -o tests/CMakeFiles/mxnet_unit_tests.dir/cpp/thread_safety/thread_safety_test.cc.o -c ../tests/cpp/thread_safety/thread_safety_test.cc
In file included from ../tests/../cpp-package/include/mxnet-cpp/optimizer.hpp:38,
                 from ../tests/../cpp-package/include/mxnet-cpp/MxNetCpp.h:35,
                 from ../tests/cpp/thread_safety/thread_safety_test.cc:37:
../tests/../cpp-package/include/mxnet-cpp/op.h:7959:22: error: ‘begin’ has not been declared
                      begin,
                      ^~~~~
../tests/../cpp-package/include/mxnet-cpp/op.h:7960:22: error: ‘end’ has not been declared
                      end,
                      ^~~
../tests/../cpp-package/include/mxnet-cpp/op.h:7961:22: error: ‘step’ has not been declared
                      step = Shape()) {
                      ^~~~
../tests/../cpp-package/include/mxnet-cpp/op.h:7961:29: error: could not convert ‘mxnet::cpp::Shape()’ from ‘mxnet::cpp::Shape’ to ‘int’
                      step = Shape()) {
                             ^~~~~~~
../tests/../cpp-package/include/mxnet-cpp/op.h: In function ‘mxnet::cpp::Symbol mxnet::cpp::slice(const string&, mxnet::cpp::Symbol, int, int, int)’:
../tests/../cpp-package/include/mxnet-cpp/op.h:7963:31: error: ‘begin’ was not declared in this scope
            .SetParam("begin", begin)
                               ^~~~~
../tests/../cpp-package/include/mxnet-cpp/op.h:7963:31: note: suggested alternative:

@mxnet-bot
Copy link

Hey @josephevans , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [sanity, windows-cpu, centos-gpu, miscellaneous, edge, unix-cpu, unix-gpu, windows-gpu, clang, website, centos-cpu]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added the pr-awaiting-testing PR is reviewed and waiting CI build and test label Dec 3, 2021
@mseth10 mseth10 added pr-awaiting-review PR is waiting for code review and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 3, 2021
@josephevans
Copy link
Contributor Author

@barry-jin Do you mind reviewing this? Thanks.

Copy link
Contributor

@barry-jin barry-jin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you!

@josephevans josephevans merged commit 60cab18 into apache:v1.x Dec 16, 2021
josephevans added a commit to josephevans/mxnet that referenced this pull request Jan 24, 2022
josephevans added a commit that referenced this pull request Jan 25, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants