test(scorecard): add custom target discovery to recording scorecard test #8
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mwangggg
force-pushed
the
698-rec-scorecard-cont
branch
2 times, most recently
from
7d137a5 to
701c7d9
Compare
|
/build_test |
|
oh I guess I can't do that here.. |
|
Looks reasonable to me. |
|
This has been passing much more reliably for me than with the built-in discovery. Although about 1/10th of the time I'll get the following EOF: This seems a bit similar to the error we encountered with the built-in discovery. There are some things we can try on the Golang client side, like: https://stackoverflow.com/a/34474535. I'm not sure if that's what we're seeing though. Given that the tests seem more reliable now, I think we should get the tests merged and investigate further. |
mwangggg
force-pushed
the
698-rec-scorecard-cont
branch
from
701c7d9 to
3d445c8
Compare
mwangggg
force-pushed
the
698-rec-scorecard-cont
branch
from
3d445c8 to
8732859
Compare
tthvo
reviewed
Feb 28, 2024
tthvo
reviewed
Feb 28, 2024
There was a problem hiding this comment.
This seems a bit similar to the error we encountered with the built-in discovery. There are some things we can try on the Golang client side, like: https://stackoverflow.com/a/34474535. I'm not sure if that's what we're seeing though. Given that the tests seem more reliable now, I think we should get the tests merged and investigate further.
Hey @ebaron Looks like some rate limit issue (not sure though)?. Maybe another option can be a retry when EOF is encountered?
Something like a wait loop with a check for errors.Is(err, io.EOF)?
That sounds like a good idea! |
|
Would it also be possible to dump out the response status code and headers when there's a failure? That could also help a lot with troubleshooting - it could tell us what the server thinks went wrong, or if it is some kind of rate-limiting behaviour then we should be able to tell that the response actually came from the Service/Ingress/etc., too. |
Sure good idea! The test currently emits a status code and response body on non-200 code. I can also log out headers too. |
tthvo
approved these changes
Feb 29, 2024
tthvo
pushed a commit
that referenced
this pull request
Mar 27, 2024
* Merge pull request #8 from ebaron/scorecard-methods test(scorecard): use methods for more easily passing data * update bundle image
tthvo
added a commit
that referenced
this pull request
Apr 24, 2024
* feat(discovery): options to configure discovery port names and numbers (backport cryostatio#715) (cryostatio#725) * feat(discovery): options to configure discovery port names and numbers (cryostatio#715) Signed-off-by: Thuan Vo <thuan.votann@gmail.com> (cherry picked from commit a552021) * resolve conflict --------- Co-authored-by: Thuan Vo <thuan.votann@gmail.com> Co-authored-by: Andrew Azores <aazores@redhat.com> * Deploy cryostat 3.0 * Remove extraneous file * test adjustments * feat(discovery): options to configure discovery port names and numbers (cryostatio#715) Signed-off-by: Thuan Vo <thuan.votann@gmail.com> * Fix typo in environment variable breaking reconciler test, fix missing SecurityContext * Fix conflict with cluster cryostat removal * ci(gh): add comment when /build_test is finished (cryostatio#745) * add scorecard test/suite selection (cryostatio#746) * test(scorecard): scorecard tests for recording management (cryostatio#698) * test(scorecard): scorecard tests for recording management Signed-off-by: Thuan Vo <thuan.votann@gmail.com> * fixup(scorecard): fix cr cleanup func * test(scorecard): registry recording test to suite * chore(scorecard): reorganize client def * chore(scorecard): clean up common setup func * chore(bundle): regenerate bundle with scorecard tag * chore(bundle): correct image tag in bundle * fix(bundle): add missing scorecard test config patch * feat(scorecard): scaffold cryostat API client * chore(scorecard): clean up API client * test(scorecard): implement recording scorecard test * fixup(scorecard): correctly add scorecard test via hack templates * fix(client): ignore unverified tls certs and base64 oauth token * chore(bundle): split cryostat tests to separate stage * fix(scorecard): extend default transport instead of overwriting * chore(scorecard): refactor client to support multi-part * fixup(client): fix request verb * fix(client): fix recording create form format * fix(scorecard): create stored credentials for target JVM * fix(scorecard): fix 502 status error * chore(scorecard): simplify client def * chore(scorecard): fetch recordings to ensure action is correctly performed * test(scorecard): test generating report for a recording * chore(scorecard): clean up * test(scorecard): list archives in tests * ci(scorecard): reconfigure ingress for kind * ci(k8s): correct cluster name * test(scorecard): use role instead of clusterrole for oauth rules * test(scorecard): parse health response for additional checks * chore(scorecard): add missing newline in logs * chore(scorecard): check status code before parsing body in health check * test(scorecard): add custom target discovery to recording scorecard test * add EOF wait and resp headers * add resp headers * chore(client): configure all clients to send safe requests * fix(clients): add missing content-type header * fix(scorecard): add missing test name in help message * chore(client): create new http requests when retrying * chore(bundle): update scorecard image tags --------- Signed-off-by: Thuan Vo <thuan.votann@gmail.com> Co-authored-by: Ming Yu Wang <90855268+mwangggg@users.noreply.github.com> Co-authored-by: Ming Wang <miwan@redhat.com> * test(scorecard): scorecard test for Cryostat CR configuration changes (cryostatio#739) * CR config scorecard * reformat * reviews * add kubectl license * test(scorecard): scorecard test for report generator (cryostatio#753) * deploy reports sidecar * report scorecard test * update * rebase fix * query health * fix(build-ci): fix scorecard image tag returned as null (cryostatio#760) Signed-off-by: Thuan Vo <thuan.votann@gmail.com> Co-authored-by: Elliott Baron <ebaron@redhat.com> * test(scorecard): add container logs to scorecard results (cryostatio#758) * test(scorecard): add container logs to scorecard results * build(bundle): regenerate bundle with new scorecard tags * chore(scorecard): refactor to remove duplicate codes * add permission to publish comment when ci fails (cryostatio#769) Co-authored-by: Elliott Baron <ebaron@redhat.com> * Update NewCoreContainer and associated tests * build(go): update Golang to 1.21 (cryostatio#777) * test(scorecard): logWorkloadEvent for cryostat-recording errors (cryostatio#759) * logWorkLoadEvent for cryostat-recording errors * reviews * tr.LogChannel --------- Co-authored-by: Elliott Baron <ebaron@redhat.com> * test(scorecard): fix rebasing skipped commit (cryostatio#780) * Merge pull request #8 from ebaron/scorecard-methods test(scorecard): use methods for more easily passing data * update bundle image * Review fixes * generate storage key, create expected Secret * fixup! generate storage key, create expected Secret * database secret handling corrections * combine database connection password and encryption key into one secret * correct storage secret key/access key * update datasource port number to not conflict with storage * precreate eventtemplates bucket * remove storage volume parameter overrides * use HTTP for Cryostat probe even when TLS is enabled - TLS will be done via auth proxy later * correct environment variable names for proxy awareness * Fix remaining merge conflict * Fix makefile * config cleanup and test fixup --------- Signed-off-by: Thuan Vo <thuan.votann@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Thuan Vo <thuan.votann@gmail.com> Co-authored-by: Andrew Azores <aazores@redhat.com> Co-authored-by: Ming Yu Wang <90855268+mwangggg@users.noreply.github.com> Co-authored-by: Ming Wang <miwan@redhat.com> Co-authored-by: Elliott Baron <ebaron@redhat.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Welcome to Cryostat! 👋
Before contributing, make sure you have:
mainbranch[chore, ci, docs, feat, fix, test]git commit -S -m "YOUR_COMMIT_MESSAGE"Related to: cryostatio#504