feat: batch and use a WaitGroup for loadCerts #4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This will make it easier to see the impact of performance fixes on these endpoints and help debug slow vault pki exporter runs
This prevents `x509_expired_cert_count` from creeping up despite certs not continuing to expire
There's no reason to reset the gauges and this hurts the exporter by clearing all metrics between runs leading to empty scrapes and stale time series
This is meant to increase the efficiency of loadCerts by batching them and using a WaitGroup to further utilize goRoutines with these lookups. It's a ton of API requests especially per certificate so I hope this will speed things up.
Adds a lot more checks to protect against bad or incomplete requests
this makes the loadCert batch size configuration via the `--batch-size-percent` variable with a default value of 1%. I found that to be pretty good on my large Vault installation
And some other cleanup
updates go to 1.21.0 and all dependecies
no need to push docker images on every PR
wbh1
reviewed
Aug 25, 2023
so we don't have to pass around the reference
Co-authored-by: Will Hegedus <will@wbhegedus.me>
…porter into fix/metric_expiry_
wbh1
approved these changes
Aug 25, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a large batch of changes, but most of them help the main goal, which was to significantly help performance of
loadCertson large Vault installations (180,000 certificates) through concurrency, fan-out, and batching.I saw anywhere from 7x to 30x performance improvements, especially on remote exporter installations.
I did include some technically unrelated fixes but think they're fine to bunch together.
There are mostly atomic commits but significant changes include:
x509_load_certs_duration_seconds_bucketandx509_watch_certs_duration_seconds_bucketto have a better understanding of performance--batch-size-percentfloat CLI argument (default 1%)