Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ready to be reviewed] Support Customized HCTR Repo in the unified containers #85

Merged
merged 35 commits into from
Jan 26, 2022

Conversation

zehuanw
Copy link
Contributor

@zehuanw zehuanw commented Dec 20, 2021

Design point:

  • New added arguments are fully compatible with all the existing ci scripts, the behavior will not be changed with the default values.
  • Support HugeCTR Development in containers by ARG HUGECTR_DEV_MODE
  • Support the configuration to _HUGECTR_REPO / _HUGECTR_BRANCH / _HUGECTR_BACKEND_BRANCH / _HUGECTR_BACKEND_REPO

Arguments supported:

HugeCTR will not be installed when HUGECTR_DEV_MODE=true
ARG HUGECTR_DEV_MODE=false

Arguments "_XXXX" are only valid when $HUGECTR_DEV_MODE==false
ARG _HUGECTR_BRANCH=master
ARG _HUGECTR_REPO="github.com/NVIDIA-Merlin/HugeCTR.git"
ARG _HUGECTR_BACKEND_BRANCH=main
ARG _HUGECTR_BACKEND_REPO="github.com/triton-inference-server/hugectr_backend"
ARG _CI_JOB_TOKEN=""

@zehuanw zehuanw changed the title [WIP][Internal Review] Support Customized HCTR Repo in the unified containers [Ready to be reviewed] Support Customized HCTR Repo in the unified containers Dec 31, 2021
@zehuanw zehuanw changed the title [Ready to be reviewed] Support Customized HCTR Repo in the unified containers [WIP] Support Customized HCTR Repo in the unified containers Dec 31, 2021
@zehuanw zehuanw changed the title [WIP] Support Customized HCTR Repo in the unified containers [Ready to be reviewed] Support Customized HCTR Repo in the unified containers Dec 31, 2021
@zehuanw zehuanw requested a review from benfred December 31, 2021 02:39
Copy link
Member

@benfred benfred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making these changes @zehuanw !

@albert17 can you verify that this works with the build before we merge?

@zehuanw
Copy link
Contributor Author

zehuanw commented Dec 31, 2021

We disabled "ARROW_ORC" temporally in this MR because of some related errors in the nightly unified container building. Per @shijieliu those errors also exist in the main branch on NVIDIA-Merlin and he has communicated with @albert17.

@albert17
Copy link
Contributor

@zehuanw Please resolve conflicts since containers have been updated to 22.01 release

@shijieliu
Copy link
Contributor

@albert17 @benfred conflicts have been resolved. Please varify

@albert17 albert17 merged commit 71f1266 into NVIDIA-Merlin:main Jan 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants