Revamping rekor e2e - part 1 of N #1089
Conversation
There was a problem hiding this comment.
Hello! As Bob mentioned, I recently updated the e2e tests to create code coverage reports in #1071. I've commented what needs to change below to support code coverage. Once util.go and e2e-test.sh are updated, all of the moving of tests going forward should be fine as long as they're using all the same util run functions.
| echo | ||
| echo "running tests" | ||
| REKORTMPDIR="$(mktemp -d -t rekor_test.XXXXXX)" | ||
| cp $dir/rekor-cli $REKORTMPDIR/rekor-cli |
There was a problem hiding this comment.
Is there a reason you're copying the cli instead of calling it with ../ in util.go?
Codecov Report
@@ Coverage Diff @@
## main #1089 +/- ##
===========================================
+ Coverage 48.89% 64.13% +15.23%
===========================================
Files 82 82
Lines 7620 7620
===========================================
+ Hits 3726 4887 +1161
+ Misses 3194 2109 -1085
+ Partials 700 624 -76
Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
- Refactor e2e tests to move it to specific folders. Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
naveensrinivasan
force-pushed
the
naveen/refactor-e2e
branch
from
8b50a00
to
6eabd04
Compare
Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
Thanks |
|
@sigstore/rekor-codeowners A friendly reminder! |
naveensrinivasan
force-pushed
the
naveen/refactor-e2e
branch
from
557bdc0
to
d4bb45e
Compare
Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
naveensrinivasan
force-pushed
the
naveen/refactor-e2e
branch
from
d4bb45e
to
7dfa43b
Compare
| echo "generating code coverage" | ||
| curl -X GET 0.0.0.0:2345/kill | ||
| sleep 5 |
There was a problem hiding this comment.
Something we need to consider, is that part of the e2e tests have dependencies on specific entries; the Rekor instance is launched without specifying a particular TreeID (for these tests), and therefore as you break this apart you will need to think through which entries a test needs in order to be successful.
There was a problem hiding this comment.
Since we kill the instance to gather code coverage here, the entries are lost unless in the future we change these tests to start Rekor targeted at a specific treeID.
There was a problem hiding this comment.
So should we populate the rekor with specific entries before the tests? How is it working for the existing e2e? Is this a missing feature at present?
There was a problem hiding this comment.
At the moment, this is more of an FYI rather than something that's directly actionable. Just something for you to be aware of as you continue moving things over, is that unless we change the test harness to be more shard aware, you may run into issues around tests that are dependent on one another. We should probably make those explicit (with wrapping and use of t.Run) whenever we see them.
There was a problem hiding this comment.
OK, Thanks. What else is required to get this merged in?
There was a problem hiding this comment.
tests need to pass (I just re-triggered them).
Signed-off-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
Summary
Refactoring e2e tests.
There will be multiple PRs to move the e2e to specific folders. Some code is duplicated up until all the e2e code is refactored.
#1075
Release Note
NONE
Documentation
NONE