-
Notifications
You must be signed in to change notification settings - Fork 118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Arrow S3 support to the tensorflow-training
container
#217
Conversation
This makes it possible to export an ensemble config from the training container, since Feast requires Arrow to have S3 support.
This is related to #216. It looks like we can do it and still sneak in under the 15GB cap:
|
Click to view CI ResultsGitHub pull request #217 of commit 8828ae38cd80839610c038720de747c7baed6af3, no merge conflicts. Running as SYSTEM Setting status of 8828ae38cd80839610c038720de747c7baed6af3 to PENDING with url https://10.20.13.93:8080/job/merlin_merlin/42/console and message: 'Pending' Using context: Jenkins Building on master in workspace /var/jenkins_home/workspace/merlin_merlin using credential systems-login > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/NVIDIA-Merlin/Merlin # timeout=10 Fetching upstream changes from https://github.com/NVIDIA-Merlin/Merlin > git --version # timeout=10 using GIT_ASKPASS to set credentials login for merlin-systems > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Merlin +refs/pull/217/*:refs/remotes/origin/pr/217/* # timeout=10 > git rev-parse 8828ae38cd80839610c038720de747c7baed6af3^{commit} # timeout=10 Checking out Revision 8828ae38cd80839610c038720de747c7baed6af3 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 8828ae38cd80839610c038720de747c7baed6af3 # timeout=10 Commit message: "Add Arrow S3 support to the `tensorflow-training` container" > git rev-list --no-walk 01d7726bf777a303d96998a2457afdd5d7ec36e5 # timeout=10 [merlin_merlin] $ /bin/bash /tmp/jenkins6955755692178741795.sh ============================= test session starts ============================== platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0 rootdir: /var/jenkins_home/workspace/merlin_merlin/merlin plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0 collected 1 item |
Documentation preview |
This makes it possible to export an ensemble config from the training container, since Feast requires Arrow to have S3 support.