[tests] add transformers & diffusers integration tests
#962
Conversation
| python -m pip install --upgrade pip | ||
| python -m pip install .[test] | ||
| cd .. && git clone https://github.com/huggingface/transformers.git && cd transformers/ | ||
| python -m pip install ".[dev]" |
There was a problem hiding this comment.
it would be great to print the transformers version (commit sha) here - it will make your life easier :-)
There was a problem hiding this comment.
Added it together with printing env variables
There was a problem hiding this comment.
I think Yih-Dar meant the SHA of the transformers branch, not of PEFT. It is contained in the git log but I'm not sure that it's the best way to print the hash. It's an interactive command with user input, not sure how GH actions handle those. How about git rev-parse HEAD?
|
|
||
| jobs: | ||
| run_transformers_integration_tests: | ||
| runs-on: ubuntu-latest |
There was a problem hiding this comment.
is this powerful enough to run those peft tests?
|
The documentation is not available anymore as the PR was closed or merged. |
| run: | | ||
| python -m pip install --upgrade pip | ||
| python -m pip install .[test] | ||
| cd .. && git clone https://github.com/huggingface/transformers.git && cd transformers/ |
There was a problem hiding this comment.
I wonder if we need an option for transformers main vs latest release too?
There was a problem hiding this comment.
Makes sense yes, sounds great !
| pytorch_nightly: | ||
| description: 'Whether to test integration tests (diffusers + transformers)' | ||
| required: false | ||
| default: false |
There was a problem hiding this comment.
This parameter is not being used, is it? Also, description and parameter name don't fit each other.
There was a problem hiding this comment.
Correct, I have simply removed it here
| python -m pip install --upgrade pip | ||
| python -m pip install .[test] | ||
| cd .. && git clone https://github.com/huggingface/transformers.git && cd transformers/ | ||
| python -m pip install ".[dev]" |
There was a problem hiding this comment.
I think Yih-Dar meant the SHA of the transformers branch, not of PEFT. It is contained in the git log but I'm not sure that it's the best way to print the hash. It's an interactive command with user input, not sure how GH actions handle those. How about git rev-parse HEAD?
| - name: Test transformers integration | ||
| run: | | ||
| RUN_SLOW=1 pytest tests/peft_integration/test_peft_integration.py | ||
| run_transformers_integration_tests_release: |
There was a problem hiding this comment.
How is it controlled that this uses the release, not the latest main? I couldn't find that in the config.
Also, there is a lot of duplication with the previous task, depending on how they differ, it might be better to turn this into a test matrix.
There was a problem hiding this comment.
How is it controlled that this uses the release, not the latest main? I couldn't find that in the config.
for run_transformers_integration_tests_release I skip the install from main part of the workflow
There was a problem hiding this comment.
Ah I see, thx. So that should be pretty easy to integrate into a test matrix, WDYT?
There was a problem hiding this comment.
I think so yes, let me give it a try
|
Should be now ready ! Thanks a lot for your suggestions. For diffusers I have set |
Adds a workflow to manually run PEFT integration of transformers and diffusers tests to make sure new PRs will not break those integrations
cc @BenjaminBossan @pacman100