feat: add medium e2e CI job for each PR #551
Conversation
This CI job runs the "SDK" of training in conjunction with `ilab data generate`. The goal here is to test convergence on each PR in a timely manner Signed-off-by: Charlie Doern <cdoern@redhat.com>
| {"Key": "GitHubPR", "Value": "${{ github.event.number }}"} | ||
| ] | ||
|
|
||
| e2e-large-test: |
There was a problem hiding this comment.
these are saying large runner, but I think you're starting a medium runner.
There was a problem hiding this comment.
I'm not against merging this as a useful test because it's much faster than the large e2e that we use in the upstream ilab repo. However, I think it'd be nicer if we called this an integration-e2e rather than an sdk test. We can isolate the SDK testing slightly differently s.t. we're testing against useful upstream data rather than tightly coupling our testing vs. the SDG output.
|
|
||
| - name: Update instructlab-training library | ||
| run: | | ||
| export CUDA_HOME="/usr/local/cuda" |
There was a problem hiding this comment.
please use ./scripts/install-ilab-with-cuda.sh from ilab repo. It should give you the necessary environment.
| . venv/bin/activate | ||
| ls scripts | ||
| ls ./ | ||
| ./scripts/test-sdk.sh |
There was a problem hiding this comment.
Except this line, is there any significant difference between the current e2e job and this sdk one? We may want to consolidate it under reusable action, see #563 (I don't insist you adopt it since it's not merged yet, but please put a comment so that we revisit.)
| @@ -0,0 +1,157 @@ | |||
| chat: | |||
There was a problem hiding this comment.
can we trim this file to what was actually modified from the defaults / needed for sdk tests?
| pull_request_target: | ||
| branches: | ||
| - "main" | ||
| schedule: |
There was a problem hiding this comment.
do we need the schedule? (it was not mentioned in the PR description)
This CI job runs the "SDK" of training in conjunction with
ilab data generate. The goal here is to test convergence on each PR in a timely manner