-
Notifications
You must be signed in to change notification settings - Fork 119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: Integration Test for all OS's #1537
Conversation
CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅ |
@flank-it |
Integration tests were triggered at 2021-01-27 14:21:55, you can track progress here |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it does not work good. IT failed, but no message was posted
@piotradamczyk5 if i am correct this needs to be merged into master to actually be executable for the IT tests ? |
runs-on: ${{ matrix.os }} | ||
strategy: | ||
matrix: | ||
os: [ ubuntu-latest, windows-latest, macos-latest ] | ||
outputs: | ||
job_status: ${{ job.status }} | ||
build-scan-url: ${{ steps.run-it.outputs.build-scan-url }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure how to handle this build-scan-url
, because previously we expect 1 link, but now we will have 3 of them
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@piotradamczyk5 hmmm, maybe we should chat about it and attempt another solution. I personally feel that the IT should run on all platforms for stability. A requirement of flank is to provide support for all 3 platforms so it would be great.
We can however approach it differently.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure we could figure out the best solution, we should involve @pawelpasterz because he is a father of integration tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@piotradamczyk5 hmmm, maybe we should chat about it and attempt another solution. I personally feel that the IT should run on all platforms for stability. A requirement of flank is to provide support for all 3 platforms so it would be great.
We can however approach it differently.
IMO that would be great!
Modifying integration processResult
script won't be difficult. Question is how to pass 3 build scan URLs?
As I see it there are a couple of action items here:
- add more OS ✅
- figure out how to pass 3 URLs to the another jobs
- change
integration processResult
to handle 3 URLs (and 3 results actually) - change
post-comment_with_results
to handle 3 urls
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering how job is behaving with os matrix? Is it run sequentially/in parallel with different os each? How about outputs
- are they somehow aggregated? Maybe 2) from my previous comment won't be so difficult. We could set URL, for each os, with suffix, like:
echo "::set-output name=scan-url-${{ matrix.os }}::${{ steps.run-it.outputs.build-scan-url }}"
and then set explicit job output (example):
(...)
outputs:
build-scan-url-macos: ${{ steps.run-it.outputs.build-scan-url-macos }}
build-scan-url-ubuntu: ${{ steps.run-it.outputs.build-scan-url-ubuntu }}
(etc...)
Then fetching those data will be easy.
Of course, it's my guess and it should be verified
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Anyway, we should decide if we want to do it within this PR scope? IMO enabling new os brings value itself so we can handle additional logic in separated issue/pr.
I think we could create a separate job, running for ubuntu and windows. It will spin in parallel to the existing macOS job and will not interfere with the existing 'processing result logic`. Simultaneously let's create a new issue for updating logic and handle multiple build scan URLs. I would be happy to work on it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
captured here: #1539
AFAIK it is true for newly created workflows. If you just modify an existing one you should be able to launch 'new' version from the branch |
@flank-it |
Integration tests were triggered at 2021-01-28 08:48:42, you can track progress here |
@pawelpasterz @piotradamczyk5 as far as i can see it seems like it does need to be merged into master for changes to apply. |
if you listen for comment trigger, you must merge it first |
Didn't know, thanks! |
@piotradamczyk5 @pawelpasterz Shall i try fix the ubuntu workflow apart of this branch before we merge. it appears to be broken again :/ |
Fixes #1527
Test Plan
Checklist