Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Document the change to three containers #379

Merged
merged 3 commits into from
Jun 14, 2022
Merged

Document the change to three containers #379

merged 3 commits into from
Jun 14, 2022

Conversation

mikemckiernan
Copy link
Member 

In 22.05 and earlier, the NGC catalog provided one
container for training and a separate container for
inference.

For example, we had a `merlin-tensorflow-training` and
a `merlin-tensorflow-inference` container.

For 22.06, the training and inference software is combined
in a single container.  For example, the previous
capabilities are now provided in a `merlin-tensorflow`
container.

@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #379 of commit 8c3b007b3a689ea6ac65ac88f183c7196250ee08, no merge conflicts.
Running as SYSTEM
Setting status of 8c3b007b3a689ea6ac65ac88f183c7196250ee08 to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/137/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_merlin
using credential systems-login
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/379/*:refs/remotes/origin/pr/379/* # timeout=10
 > git rev-parse 8c3b007b3a689ea6ac65ac88f183c7196250ee08^{commit} # timeout=10
Checking out Revision 8c3b007b3a689ea6ac65ac88f183c7196250ee08 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 8c3b007b3a689ea6ac65ac88f183c7196250ee08 # timeout=10
Commit message: "Document the change to three containers"
 > git rev-list --no-walk bf27e543d9ede1df8e1cccc9b988e69baebde5fd # timeout=10
[merlin_merlin] $ /bin/bash /tmp/jenkins2894999136608854017.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 1 item

tests/unit/test_version.py .                                             [100%]


============================== 1 passed in 0.02s ===============================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Merlin/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_merlin] $ /bin/bash /tmp/jenkins876462771301033349.sh

@github-actions
Copy link

github-actions bot commented Jun 8, 2022

Documentation preview

https://nvidia-merlin.github.io/Merlin/review/pr-379

@mikemckiernan mikemckiernan force-pushed the docs-three-containers branch from 8c3b007 to a7c04c7 Compare June 8, 2022 19:05
@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #379 of commit a7c04c77a1b3d7731481c6c70ca4626193b3f2af, no merge conflicts.
Running as SYSTEM
Setting status of a7c04c77a1b3d7731481c6c70ca4626193b3f2af to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/138/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_merlin
using credential systems-login
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/379/*:refs/remotes/origin/pr/379/* # timeout=10
 > git rev-parse a7c04c77a1b3d7731481c6c70ca4626193b3f2af^{commit} # timeout=10
Checking out Revision a7c04c77a1b3d7731481c6c70ca4626193b3f2af (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f a7c04c77a1b3d7731481c6c70ca4626193b3f2af # timeout=10
Commit message: "Document the change to three containers"
 > git rev-list --no-walk 8c3b007b3a689ea6ac65ac88f183c7196250ee08 # timeout=10
[merlin_merlin] $ /bin/bash /tmp/jenkins65541967710372797.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 1 item

tests/unit/test_version.py .                                             [100%]


============================== 1 passed in 0.01s ===============================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Merlin/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_merlin] $ /bin/bash /tmp/jenkins16913341486183434711.sh

@mikemckiernan mikemckiernan added the documentation Improvements or additions to documentation label Jun 8, 2022
@mikemckiernan mikemckiernan added this to the Merlin 22.06 milestone Jun 8, 2022
@mikemckiernan mikemckiernan linked an issue Jun 8, 2022 that may be closed by this pull request
[DOC] Update references docs based on latest 3 containers (Merlin - HCTR/TF/PYT) #377
Closed
@mikemckiernan mikemckiernan force-pushed the docs-three-containers branch from a7c04c7 to 4822e7d Compare June 8, 2022 19:13
@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #379 of commit 4822e7d9f47908e18b82cc9f0fbb0aba1374dfb7, no merge conflicts.
Running as SYSTEM
Setting status of 4822e7d9f47908e18b82cc9f0fbb0aba1374dfb7 to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/139/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_merlin
using credential systems-login
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/379/*:refs/remotes/origin/pr/379/* # timeout=10
 > git rev-parse 4822e7d9f47908e18b82cc9f0fbb0aba1374dfb7^{commit} # timeout=10
Checking out Revision 4822e7d9f47908e18b82cc9f0fbb0aba1374dfb7 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 4822e7d9f47908e18b82cc9f0fbb0aba1374dfb7 # timeout=10
Commit message: "Document the change to three containers"
 > git rev-list --no-walk a7c04c77a1b3d7731481c6c70ca4626193b3f2af # timeout=10
[merlin_merlin] $ /bin/bash /tmp/jenkins10107435402433577877.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 1 item

tests/unit/test_version.py .                                             [100%]


============================== 1 passed in 0.01s ===============================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Merlin/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_merlin] $ /bin/bash /tmp/jenkins17888572686708090836.sh

@mikemckiernan
Copy link
Member Author

Key pages for the initial change:

@mikemckiernan mikemckiernan added documentation Improvements or additions to documentation and removed documentation Improvements or additions to documentation labels Jun 11, 2022
@mikemckiernan
Merlin offers three containers
fcf3b0f 
In 22.05 and earlier, the NGC catalog provided one
container for training and a separate container for
inference.

For example, we had a `merlin-tensorflow-training` and
a `merlin-tensorflow-inference` container.

For 22.06, the training and inference software is combined
in a single container.  For example, the previous
capabilities are now provided in a `merlin-tensorflow`
container.
@mikemckiernan mikemckiernan force-pushed the docs-three-containers branch from 4822e7d to fcf3b0f Compare June 13, 2022 19:46
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #379 of commit fcf3b0fd7bb83f57a1472a782f4c055ba3571c59, no merge conflicts.
Running as SYSTEM
Setting status of fcf3b0fd7bb83f57a1472a782f4c055ba3571c59 to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/152/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_merlin
using credential systems-login
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/379/*:refs/remotes/origin/pr/379/* # timeout=10
 > git rev-parse fcf3b0fd7bb83f57a1472a782f4c055ba3571c59^{commit} # timeout=10
Checking out Revision fcf3b0fd7bb83f57a1472a782f4c055ba3571c59 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f fcf3b0fd7bb83f57a1472a782f4c055ba3571c59 # timeout=10
Commit message: "Merlin offers three containers"
 > git rev-list --no-walk 66a1c8905e90b179e64b31a64f925a5c2fb0b74b # timeout=10
[merlin_merlin] $ /bin/bash /tmp/jenkins8871939275433179244.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 1 item

tests/unit/test_version.py .                                             [100%]


============================== 1 passed in 0.01s ===============================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Merlin/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_merlin] $ /bin/bash /tmp/jenkins13071137232859341157.sh

@mikemckiernan mikemckiernan requested a review from benfred June 13, 2022 19:47
@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #379 of commit aa4be623eae58aba4b0b77e65a528b8b2e0dfa8f, no merge conflicts.
Running as SYSTEM
Setting status of aa4be623eae58aba4b0b77e65a528b8b2e0dfa8f to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/159/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_merlin
using credential systems-login
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/379/*:refs/remotes/origin/pr/379/* # timeout=10
 > git rev-parse aa4be623eae58aba4b0b77e65a528b8b2e0dfa8f^{commit} # timeout=10
Checking out Revision aa4be623eae58aba4b0b77e65a528b8b2e0dfa8f (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f aa4be623eae58aba4b0b77e65a528b8b2e0dfa8f # timeout=10
Commit message: "Merge branch 'main' into docs-three-containers"
 > git rev-list --no-walk 72b9d6bfbc238a0c86f3a0e62f90d19ce69acaf0 # timeout=10
[merlin_merlin] $ /bin/bash /tmp/jenkins10062345955461226745.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 1 item

tests/unit/test_version.py .                                             [100%]


============================== 1 passed in 0.02s ===============================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Merlin/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_merlin] $ /bin/bash /tmp/jenkins18148120820566823338.sh

@mikemckiernan mikemckiernan self-assigned this Jun 14, 2022
@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #379 of commit 46395a98cde304435798351d02ad275b5c5700fb, no merge conflicts.
Running as SYSTEM
Setting status of 46395a98cde304435798351d02ad275b5c5700fb to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/167/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_merlin
using credential systems-login
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/379/*:refs/remotes/origin/pr/379/* # timeout=10
 > git rev-parse 46395a98cde304435798351d02ad275b5c5700fb^{commit} # timeout=10
Checking out Revision 46395a98cde304435798351d02ad275b5c5700fb (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 46395a98cde304435798351d02ad275b5c5700fb # timeout=10
Commit message: "Merge branch 'main' into docs-three-containers"
 > git rev-list --no-walk 20bee6e588d4c626efec1395e5358f67cea3202e # timeout=10
[merlin_merlin] $ /bin/bash /tmp/jenkins1724266881402810119.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 2 items

tests/unit/test_version.py .                                             [ 50%]

tests/unit/examples/test_building_deploying_multi_stage_RecSys.py .      [100%]


========================= 2 passed in 87.62s (0:01:27) =========================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Merlin/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_merlin] $ /bin/bash /tmp/jenkins5696872862662885443.sh

@benfred benfred merged commit a0fa064 into NVIDIA-Merlin:main Jun 14, 2022
@mikemckiernan mikemckiernan deleted the docs-three-containers branch July 5, 2022 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@benfred benfred benfred approved these changes

Assignees

@mikemckiernan mikemckiernan

Labels
documentation Improvements or additions to documentation
Projects
None yet
Milestone
Merlin 22.06
Development

Successfully merging this pull request may close these issues.

[DOC] Update references docs based on latest 3 containers (Merlin - HCTR/TF/PYT)
4 participants
@mikemckiernan @nvidia-merlin-bot @benfred @rnyak