Enable HF OOB testcase #62
Conversation
docker/Dockerfile.hf_oob
Outdated
|
|
||
| ARG HF_TEST_REPO=$HF_TEST_REPO | ||
| ARG HF_TEST_BRANCH=$HF_TEST_BRANCH | ||
| ARG HF_TEST_COMMIT=$HF_TEST_COMMIT |
There was a problem hiding this comment.
Suggest to move those scripts from scripts/modelbench into a new folder scripts/hf_oob, how do you think?
There was a problem hiding this comment.
Let's change all hf_oob to hf_pipeline in files name and path to avoid potential confusion.
There was a problem hiding this comment.
Sure, the test is launched with new name
docker/Dockerfile.hf_oob
Outdated
| COPY --from=build /opt/conda /opt/conda | ||
| COPY --from=build /workspace/pytorch /workspace/pytorch | ||
| COPY --from=build /workspace/hf_testcase /workspace/hf_testcase | ||
| COPY --from=build /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /opt/conda/lib |
There was a problem hiding this comment.
Yes, this is a dependency from hf_testcase repo. It will only search $CONDA/lib path for libstdc++.so.6
|
Hi @chuanqi129 |
docker/Dockerfile.hf_pipeline
Outdated
|
|
||
| ARG TRANSFORMERS_VERSION=${TRANSFORMERS_VERSION} | ||
| RUN echo ${TRANSFORMERS_VERSION} && if [ "${TRANSFORMERS_VERSION}" = "default" ]; then \ | ||
| git clone https://github.com/huggingface/transformers.git; \ |
There was a problem hiding this comment.
L86 - L91 can be replaced by pip install --force-reinstall git+https://github.com/huggingface/transformers@${TRANSFORMERS_COMMIT}. Our regular test #70 also can use this method
| pip install dist/*.whl; \ | ||
| cd ../; \ | ||
| export TRANSFORMERS_COMMIT=`cat /workspace/pytorch/.ci/docker/ci_commit_pins/huggingface.txt`; \ | ||
| pip install --force-reinstall git+https://github.com/huggingface/transformers@${TRANSFORMERS_COMMIT}; \ |
There was a problem hiding this comment.
let's also align this method in bisect search, and then test this PR with a bisect search test
There was a problem hiding this comment.
No description provided.