Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the way we're generating certificates on our test deployments #1396

Closed
svyatonik opened this issue May 16, 2022 · 4 comments · Fixed by #1554
Closed

Fix the way we're generating certificates on our test deployments #1396

svyatonik opened this issue May 16, 2022 · 4 comments · Fixed by #1554
Assignees
Labels
A-chores Something that has to be done, as part of regular maintenance P-Devops

Comments

@svyatonik
Copy link
Contributor

Right now we're hitting 5 certs/week limit (https://letsencrypt.org/docs/rate-limits/) when generating certificates for our test deployments nodes:

Create new order error. Le_OrderFinalize not found. {
  "type": "urn:ietf:params:acme:error:rateLimited",
  "detail": "Error creating new order :: too many certificates (5) already issued for this exact set of domains in the last 168 hours: ui.brucke.link: see https://letsencrypt.org/docs/rate-limits/",
  "status": 429
}

At first glance, possible solutions are:

  1. actually store generated certificates (proper way imo);
  2. do restarts less often (e.g. every other day).
@svyatonik svyatonik added A-chores Something that has to be done, as part of regular maintenance P-Devops labels May 16, 2022
@svyatonik svyatonik added this to the When Thou Have Spare Hour milestone May 17, 2022
@serban300 serban300 self-assigned this Aug 22, 2022
@serban300
Copy link
Collaborator

According to this and this issues we might need to add a volume:

- letsencrypt-acme:/etc/acme.sh

both for the nginx-proxy and the nginx-proxy-le containers in order to persist the SSL certificates.

@serban300
Copy link
Collaborator

Merged #1554 that should fix this. However keeping this opened for a while until the current rate-limiting window expires and we can check that the issue is actually fixed.

@serban300 serban300 reopened this Aug 22, 2022
@serban300
Copy link
Collaborator

The certificates seem to be downloaded in the persisted volume:

[Wed Aug 24 12:03:35 UTC 2022] Your cert is in  /etc/acme.sh/admin@parity.io/grafana.millau.brucke.link/grafana.millau.brucke.link.cer 
[Wed Aug 24 12:03:35 UTC 2022] Your cert key is in  /etc/acme.sh/admin@parity.io/grafana.millau.brucke.link/grafana.millau.brucke.link.key 
[Wed Aug 24 12:03:35 UTC 2022] The intermediate CA cert is in  /etc/acme.sh/admin@parity.io/grafana.millau.brucke.link/ca.cer 
[Wed Aug 24 12:03:35 UTC 2022] And the full chain certs is there:  /etc/acme.sh/admin@parity.io/grafana.millau.brucke.link/fullchain.cer 
[Wed Aug 24 12:03:35 UTC 2022] Installing cert to:/etc/nginx/certs/grafana.millau.brucke.link/cert.pem
[Wed Aug 24 12:03:36 UTC 2022] Installing CA to:/etc/nginx/certs/grafana.millau.brucke.link/chain.pem
[Wed Aug 24 12:03:36 UTC 2022] Installing key to:/etc/nginx/certs/grafana.millau.brucke.link/key.pem
[Wed Aug 24 12:03:36 UTC 2022] Installing full chain to:/etc/nginx/certs/grafana.millau.brucke.link/fullchain.pem

Waiting a couple more days to make sure that the issue doesn't reproduce.

@serban300
Copy link
Collaborator

The issue doesn't seem to reproduce and from the logs it seems that the certificates are persisted and aren't renewed on every restart:

Creating/renewal grafana.millau.brucke.link certificates... (grafana.millau.brucke.link grafana.rialto.brucke.link)
[Mon Aug 29 05:01:52 UTC 2022] Domains not changed.
[Mon Aug 29 05:01:52 UTC 2022] Skip, Next renewal time is: Sun Oct 23 12:03:35 UTC 2022
[Mon Aug 29 05:01:52 UTC 2022] Add '--force' to force to renew.

Closing the issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-chores Something that has to be done, as part of regular maintenance P-Devops
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants