Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

preserving PRT result as part of github comment & PRT pass/fail label #13979

Merged
merged 1 commit into from
Feb 12, 2024

Conversation

omkarkhatavkar
Copy link

@omkarkhatavkar omkarkhatavkar commented Feb 2, 2024

Problem Statement

Currently, the PRT result gets clean after a new commit and there is no way for PR to know what happened with the result. PRT passed/failed labels also need to be added automatically this helps in reviewing the PR quickly and merging them

Solution

Add the dispatch workflow that takes care of this thing from the Jenkins CI update the GitHub comment with all details and apply the filter.

Testing

I was able to test this on the my fork of robottelo here https://github.com/omkarkhatavkar/robottelo/actions/runs/7755989443/job/21152444058

@omkarkhatavkar omkarkhatavkar requested a review from a team as a code owner February 2, 2024 13:11
@omkarkhatavkar omkarkhatavkar added CherryPick PR needs CherryPick to previous branches 6.12.z Introduced in or relating directly to Satellite 6.12 6.13.z Introduced in or relating directly to Satellite 6.13 6.14.z Introduced in or relating directly to Satellite 6.14 6.15.z Introduced in or relating directly to Satellite 6.15 labels Feb 2, 2024
Copy link
Contributor

@synkd synkd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a very useful addition to the Robottelo GH actions. I think it's a great idea to preserve the history of PRT requests in a PR's comments. However, I do have a few questions about your test results in omkarkhatavkar#158:

  • In that PR, the build number for each run is reported as 10. Is this due to repeatedly retriggering the same check from the same PRT run?
  • There's some inconsistency in the reported build status, with a mix of SUCCESS, FAILURE, and UNSTABLE reported for a build with a failed test. It looks like your most recent testing has converged on this result being reported as UNSTABLE, which I think is fine. But would it be possible to see results from a build showing a status of SUCCESS when all tests have passed?
  • Which user account(s) will post the results comment to PRs when this is merged into Robottelo?

@omkarkhatavkar
Copy link
Author

omkarkhatavkar commented Feb 5, 2024

This is a very useful addition to the Robottelo GH actions. I think it's a great idea to preserve the history of PRT requests in a PR's comments. However, I do have a few questions about your test results in omkarkhatavkar#158:

Thanks for seeing value in this PR (Contribution)

  • In that PR, the build number for each run is reported as 10. Is this due to repeatedly retriggering the same check from the same PRT run?

I was testing this that's why it would have repeatedly shown build number 10, In real case it will be the build number coming from the Jenkins PRT run.

  • There's some inconsistency in the reported build status, with a mix of SUCCESS, FAILURE, and UNSTABLE reported for a build with a failed test. It looks like your most recent testing has converged on this result being reported as UNSTABLE, which I think is fine. But would it be possible to see results from a build showing a status of SUCCESS when all tests have passed?

I was just testing some dummy data, But the thumb rule is that if the test passes without any issues and the build is a Success it will present the label as PRT Passed, else PRT Failed. Adding more color code will look a bit better

  • Which user account(s) will post the results comment to PRs when this is merged into Robottelo?

The result will be posted by the token which I'm using here which is SatelliteQE user.

@omkarkhatavkar
Copy link
Author

omkarkhatavkar#158 (comment) This one the last comment by me. I think this looks much better as well

Copy link
Member

@jyejare jyejare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks Great! Some nitpick comments.

.github/workflows/prt_result.yml Show resolved Hide resolved
.github/workflows/prt_result.yml Show resolved Hide resolved
@omkarkhatavkar omkarkhatavkar force-pushed the prt_result_label branch 4 times, most recently from 37bec23 to 588f745 Compare February 6, 2024 13:56
@jyejare
Copy link
Member

jyejare commented Feb 7, 2024

I have an ACK.

@omkarkhatavkar
Copy link
Author

@jyejare Can we merge this beast ?

Copy link
Member

@jyejare jyejare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK pending comment !

.github/workflows/prt_result.yml Show resolved Hide resolved
@omkarkhatavkar
Copy link
Author

@jyejare Could you please create labels for "PRT-Passed" and "PRT-Failed"? Please choose some nice colours to represent them.

@jyejare
Copy link
Member

jyejare commented Feb 9, 2024

@omkarkhatavkar Added the labels for both! Now we should be good to merge if no complaints !!

@jyejare jyejare merged commit 2d0fed6 into SatelliteQE:master Feb 12, 2024
6 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 12, 2024
…#13979)

preserving the PRT result as part of github comment & PRT pass/fail label

(cherry picked from commit 2d0fed6)
github-actions bot pushed a commit that referenced this pull request Feb 12, 2024
…#13979)

preserving the PRT result as part of github comment & PRT pass/fail label

(cherry picked from commit 2d0fed6)
github-actions bot pushed a commit that referenced this pull request Feb 12, 2024
…#13979)

preserving the PRT result as part of github comment & PRT pass/fail label

(cherry picked from commit 2d0fed6)
omkarkhatavkar added a commit to omkarkhatavkar/robottelo that referenced this pull request Feb 13, 2024
…SatelliteQE#13979)

preserving the PRT result as part of github comment & PRT pass/fail label
Gauravtalreja1 pushed a commit that referenced this pull request Feb 13, 2024
…/fail label… (#14057)

preserving PRT result as part of github comment & PRT pass/fail label (#13979)

preserving the PRT result as part of github comment & PRT pass/fail label
Gauravtalreja1 pushed a commit that referenced this pull request Feb 13, 2024
…ail label (#14039)

preserving PRT result as part of github comment & PRT pass/fail label (#13979)

preserving the PRT result as part of github comment & PRT pass/fail label

(cherry picked from commit 2d0fed6)

Co-authored-by: Omkar Khatavkar <okhatavkar007@gmail.com>
Gauravtalreja1 pushed a commit that referenced this pull request Feb 13, 2024
…ail label (#14038)

preserving PRT result as part of github comment & PRT pass/fail label (#13979)

preserving the PRT result as part of github comment & PRT pass/fail label

(cherry picked from commit 2d0fed6)

Co-authored-by: Omkar Khatavkar <okhatavkar007@gmail.com>
Gauravtalreja1 pushed a commit that referenced this pull request Feb 13, 2024
…ail label (#14037)

preserving PRT result as part of github comment & PRT pass/fail label (#13979)

preserving the PRT result as part of github comment & PRT pass/fail label

(cherry picked from commit 2d0fed6)

Co-authored-by: Omkar Khatavkar <okhatavkar007@gmail.com>
shweta83 pushed a commit to shweta83/robottelo that referenced this pull request Apr 10, 2024
…SatelliteQE#13979)

preserving the PRT result as part of github comment & PRT pass/fail label
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.12.z Introduced in or relating directly to Satellite 6.12 6.13.z Introduced in or relating directly to Satellite 6.13 6.14.z Introduced in or relating directly to Satellite 6.14 6.15.z Introduced in or relating directly to Satellite 6.15 CherryPick PR needs CherryPick to previous branches
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants