Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

USE_NVRTC -> ENABLE_CUDA_RTC to fix maven build. Add compile-guard to fusion. #16838

Merged
merged 5 commits into from
Nov 20, 2019

Conversation

DickJC123
Copy link
Contributor

This PR fixes the maven build issue #16765 reported by @zachgk and first seen with PR #15167. The fix involved-

  1. Renaming the build flag used by maven (namely USE_NVRTC) to the name used by the pip builds (ENABLE_CUDA_RTC). The Makefile currently looks at ENABLE_CUDA_RTC (not USE_NVRTC) and in response sets the compile flag MXNET_ENABLE_CUDA_RTC=1 and adds -lcuda -lnvrtc to the LDFLAGS. It was the absence of these ld flags when creating im2rec that caused the issue filed.

  2. Adding compile guards to the newly-introduced files ./src/operator/fusion/* in a form similar to those that protect ./src/common/rtc.cc. Now both cuda runtime-compilation facilities are included into the build (or not) based on MXNET_ENABLE_CUDA_RTC.

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
  • [ X] Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • [X ] Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at https://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • Feature1, tests, (and when applicable, API doc)
  • Feature2, tests, (and when applicable, API doc)

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

Copy link
Member

@ptrendx ptrendx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for fixing this :-)

@ptrendx ptrendx added the R1.6.0 label Nov 16, 2019
Copy link
Contributor

@marcoabreu marcoabreu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I'm thinking about backwards compatibility - should the change of the build flag be of concern?

@ptrendx
Copy link
Member

ptrendx commented Nov 18, 2019

LGTM, but I'm thinking about backwards compatibility - should the change of the build flag be of concern?

I would say it is just fixing the bug and bringing the Scala build to parity with other builds (especially since the value of the flag is not changed, so the intent to enable this was there, it's just that it was never actually doing anything).

@lanking520
Copy link
Member

Honestly speaking, there is no backend build difference between pip build and scala for now. we can also use the pip mk on Scala/Java build without problems. @zachgk let's track on the deprecation on the maven mkl files.

@ptrendx
Copy link
Member

ptrendx commented Nov 19, 2019

[03:39:03] /work/mxnet/src/executor/attach_op_execs_pass.cc:355: Neither FCompute nor FComputeEx registered _FusedOp - @DickJC123 I believe it is still not guarded properly (seems that the graph pass works but the actual .cu file that contains FCompute is not included).

@ptrendx ptrendx merged commit 307037e into apache:master Nov 20, 2019
ptrendx pushed a commit to ptrendx/mxnet that referenced this pull request Nov 20, 2019
… fusion. (apache#16838)

* Rename USE_NVRTC -> ENABLE_CUDA_RTC to fix maven build.  Compile-guard fusion framework.

* Fix fusion-not-supported warning.

* Fix compile guards

* Fix cmake build so -DMXNET_ENABLE_CUDA_RTC=1 is passed to nvcc

* Minimize side-effects of prev change
ptrendx added a commit that referenced this pull request Nov 22, 2019
* Add unoptimized symbol to executor for sharing (#16798)

* Add unoptimized symbol to executor for sharing

* Copy the symbol in Reshape

* Added test for multiple reshapes

* Mixed precison binary op backward (use in) for numpy (#16791)

* mixed precison binary op backward

* reduce unix cpu runtime

* USE_NVRTC -> ENABLE_CUDA_RTC to fix maven build.  Add compile-guard to fusion. (#16838)

* Rename USE_NVRTC -> ENABLE_CUDA_RTC to fix maven build.  Compile-guard fusion framework.

* Fix fusion-not-supported warning.

* Fix compile guards

* Fix cmake build so -DMXNET_ENABLE_CUDA_RTC=1 is passed to nvcc

* Minimize side-effects of prev change

* Fix InferAttr/InferShapeAttr not calling inference for all nodes in a graph (#16836)

* Fix the attribute inference omitting nodes

* Add test

* Cleaning

* Fix lint

* Fix TransposeShape

* Fix WhileLoopType

* Changing a/b test for fusion to a/(b+1) to increase numerical stability

* Revert "Mixed precison binary op backward (use in) for numpy (#16791)"

This reverts commit 8b58b78.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants