refactor(test): dockerize tests and run sync in detached mode #3459
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Use a more ENV configurable Dockerfile - Remove cloudbuild dependency - Use compute optimized machine types - Use SSD instead of normal hard drives - Move Sentry endpoint to secrets - Use a single yml for auto & manual deploy - Migrate to Google Artifact Registry
- Use a more ENV configurable Dockerfile - Remove cloudbuild dependency - Use compute optimized machine types - Use SSD instead of normal hard drives - Move Sentry endpoint to secrets - Use a single yml for auto & manual deploy - Migrate to Google Artifact Registry
Also substitute defaults with parameters sent through the workflow_dispatch
Caching from the latest image is one of the main reasons to add this extra tag. Before this commit, the inline cache was not being used.
The inline cache exporter only supports `min` cache mode. To enable `max` cache mode, push the image and the cache separately by using the registry cache exporter. This also allows for smaller release images.
We're leveraging the registry to cache the actions, instead of using the 10GB limits from Github Actions cache storage
Use interactive shells for manual and test deployments. This allow greater flexibility if troubleshooting is needed inside the machines
Instead of using a VM to SSH into in to build and test. Build in GHA (to have the logs available), run the workspace tests in GHA, and just run the sync tests in GCP Use a cintainer VM with zebra's image directly on it, and pass the needed parameters to run the Sync past mandatory checkpoint.
Compiling might be slow because different steps are compiling the same code 2-4 times because of the variations
This comment was marked as resolved.
This comment was marked as resolved.
gustavovalverde
force-pushed
the
docker-test-refactor
branch
from
3b19c6f to
6ff20c0
Compare
Using the SHA from the base, creates confusion and it's not accurate with the SHA being shown and used on GitHub. We have to keep both as manual runs with `workflow_dispatch` does not have a PR SHA
There's no impact in this workflow when a change is done in the dockerfile
… commands afterwards in a different step. As GHA TTY is not working as expected, and workarounds does not play nicely with `gcloud compute ssh` actions/runner#241 (comment) we decided to get the container name from the logs, log directly to the container and run the cargo command from there.
gustavovalverde
force-pushed
the
docker-test-refactor
branch
from
06260aa to
ce3a269
Compare
teor2345
reviewed
Feb 14, 2022
|
@gustavovalverde it looks like RAM is very low on the regenerate-state-disks VM. This might be happening because the disk performance is very slow. (Or the disk could be slow due to VM swapping.) Do you want to try a Here are the important parts of the failed logs: Zebra starts downloading some new blocks, and checkpoints them: It takes 4 minutes to write 700 blocks to disk: Zebra's Peer Set isn't getting requests, so it seems like it might be the disk that's slow: Still writing blocks, 8 minutes to write 1200 blocks: The network part of the syncer got too far ahead, so it waited for a while: And then the process got killed for out of memory. Happy to jump on a call or chat if that would help diagnose things. We're so close! 🤔 |
This allows to follow logs in Github Actions terminal, while the GCP container is still running. Just delete the instance when following the logs ends successfully or fails
gustavovalverde
force-pushed
the
docker-test-refactor
branch
from
d8304e2 to
5b55949
Compare
gustavovalverde
changed the title
gustavovalverde
changed the title
This allows to follow logs in Github Actions terminal, while the GCP container is still running. Just delete the instance when following the logs ends successfully or fails
teor2345
approved these changes
Feb 15, 2022
23 tasks
26 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
*patch.ymlthat were not needed were causing a bunch of noise on GitHub dashboard and PRsFixes: #2411 #3344
Partially: #1592 #2653 #2995 #3447
Solution
Review
@teor2345 @conradoplg @dconnolly
Follow Up Work
.buildand.testDockerfiles