Confconsole Let's Encrypt - possible edge (corner?) case issue - "add-water" not being killed

[JedMeister]

[Update]: This issue has been closed by the release of Confconsole v1.1.1 Confconsole v1.1.2. It's not yet available from our repos and still requires some specific steps to use on a v15.x server (although better than instructions published previously). Please see the release notes for full step by step setup and further info.

---

JP just noted in the forums that he had an issue getting a new certificate recently. The output was:

 root@lamp /usr/bin# /usr/lib/confconsole/plugins.d/Lets_Encrypt/dehydrated-wrapper
[2019-09-29 22:22:48] dehydrated-wrapper: INFO: started
[2019-09-29 22:22:48] dehydrated-wrapper: INFO: No process found listening on port 80; continuing
[2019-09-29 22:22:48] dehydrated-wrapper: INFO: running dehydrated
/etc/dehydrated/confconsole.hook.sh: line 33: kill: (2591) - No such process
cat: /var/run/add-water/pid: No such file or directory
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
rm: cannot remove '/var/run/add-water/pid': No such file or directory
[2019-09-29 22:22:54] dehydrated-wrapper: FATAL: dehydrated exited with a non-zero exit code.
[2019-09-29 22:22:54] dehydrated-wrapper: WARNING: Python is still listening on port 80
[2019-09-29 22:22:54] dehydrated-wrapper: INFO: attempting to kill add-water server
[2019-09-29 22:22:54] dehydrated-wrapper: WARNING: Something went wrong, restoring original cert & key.
[2019-09-29 22:22:54] dehydrated-wrapper: INFO: starting stunnel4
[2019-09-29 22:22:54] dehydrated-wrapper: WARNING: Check today's previous log entries for details of error.

I suspect that there is an edge (corner?) case bug here with the way that the dehydrated wrapper handles things. More info in the thread.

TBH, I'm not 100% sure of the nature of the potential issue, nor the best way to reduce the likelihood of it causing others issues, but we should have a closer look.

[JedMeister]

Current work-in-progress is in my personal GitHub. A brief overview of what I've done (and what remains to be done) has been posted on our forums.

[JedMeister]

Ryan has just reported a possible (short term) workaround to this issue via support.

I haven't double checked, but he notes the following (paraphrased by me):

If you do the update (as noted in #1359 or Igor's forum post) and run the wrapper script with just a single domain configured, then re-add the additional domains and re-run the wrapper script, that seems to work.

If you test that out and have any feedback, please report back and let us know how you go.

It's also worth noting that once renewed, the certificates are valid for 90 days. So even if the above workaround doesn't work for you, it may be worth manually retrying. Although you'll need to be careful that you don't hit the Let's Encrypt rate limit (it may then block you for up to a week!).

I am hoping to have a proper fix (improving the reliability of add-water; likely via inclusion of the add-water-ctl script that I have developed) ASAP. But I still haven't yet had a chance to circle back to update the hook script to make use of it. It shouldn't take too much more (the bulk of the work has been done). If anyone can help out with that, I would be eternally grateful! 😄

[OnGle]

Alright, gonna have a geez at this one right now

[OnGle]

Issue should now be closed, see referenced fix for details on the issue and the fix

[JedMeister]

This issue is closed by the release of Confconsole v1.1.1. It's not yet available from our repos and still requires some specific steps to use on a v15.x server (although better than instructions published previously). Please see the release notes for full step by step setup and further info.