Option to preseed accounts/subdomains upon startup

[znerol]

The registration workflow is quite cumbersome in environments which are heavily automated using configuration management tools. Note that in such environments ACME clients are typically kept stateless as well. Hence self-registration upon first use is not an option.

In order to bootstrap a machine from zero to acme-dns including the necessary CNAME records, the following steps must be encoded in the configuration management tool.

1. install basesystem, acme-dns, configuration (easy)
2. for each domain:
   1. register an account on acme-dns
   2. parse response and deploy CNAME on the DNS server (trouble)
   3. deploy registration result on machines running ACME client
   4. record account as state in configuration management system (trouble)

Especially the last point is cumbersome since keeping state in configuration management leads to brittle deployments. Also temporal dependencies between steps (i.e., tasks depending on output of previous steps) should be avoided.

The workflow would be much less troublesome if previously generated accounts and subdomains could be injected into the database upon start of acme-dns. With that feature the configuration management workflow could be broken up into independent and stateless tasks:

• install basesystem, acme-dns, configuration, preseed database (easy)
• for each domain deploy CNAME on the DNS server if changed (easy)
• for each domain deploy registration json file to machines running ACME client (easy)

I propose a simple json file with the same structure as the records table (passwords already hashed) as the database seeding source. The preseeding code reads the file and inserts records in the database upon database creation. Preseeding is skipped if the database already exists. The path to the preseeding file can be specified in a new config.cfg option in the database section.

Generating the preseeding file and the registration json is the responsibility of the configuration management tool and thus is out of scope.

Related to #110

[webprofusion-chrisc]

You could script the preseeding step as SQL, avoiding any changes to acme-dns? JSON file is fine but obviously requires a mechanism to be built.

I'm also looking at a similar/realated idea, as I'm interested in a way to capture and consolidate current configuration (so that acme-dns servers can be destroyed and recreated, possibly per account), as once a CNAME is created (which could be a manual step) you ideally don't want to have to change it again unless you have a working DNS API for the domain.

[znerol]

You could script the preseeding step as SQL, avoiding any changes to acme-dns?

I do not consider the db to be public API. Also it is not enough to create the accounts/subdomains, the tx records nedd to be created at the same time.

[joohoi]

Thanks for opening the issue! I certainly see the need for this feature.

The medium term development plan I have is to replace the currently used SQLite database implementation with a pure Go storage solution. While the primary incentive for this is to get rid of the only C dependency that the projects has, it'd probably make sense to implement the database pre-priming functionality at the same time (as large parts of the DB code will have to be refactored as well).

Meanwhile the most prominent and stable (albeit suboptimal) solution to the issue would probably be spinning up acme-dns on arbitruary host, registering the needed accounts, and storing the resulting DB file that will be deployed in the future.

Another solution that was brought up by @webprofusion-chrisc would be to actually create support script (probably within the acme-dns project itself!) to initialize the database, and create preprimed accounts before starting up the actual acme-dns process.

I do not consider the db to be public API. Also it is not enough to create the accounts/subdomains, the tx records nedd to be created at the same time.

This isn't actually true, and the DB code around it definitely isn't the best (my apologies). What gets done in the code where you linked it up, the DB rows for TXT record placeholders get created. That's done because we want to have exactly two TXT records per subdomain to be able to handle use cases where certificate gets requested for both: *.example.com and example.com, as the same TXT record will be used to validate both of the requests.

Fortunately this means that scripting solution is actually viable one.

[znerol]

Interesting. Looking through the project I realize that the db currently has two responsibilities:

1. Store long-living accounts/subdomains. The API allows to add new records, no way to modify/remove existing ones.
2. Maintain short-living state. The API allows to add/remove records if the caller supplies credentials matching one of the account/subdomain objects.

Unless there are any changes to the API planned, it looks like one could get away with a simple key-value store for the accounts/subdomains (subdomain is key). The actual DNS records could be maintained in-memory. Those are few short-living objects, there is no need to persist them on disk at all.

What gets done in the code where you linked it up, the DB rows for TXT record placeholders get created.

Sure, that's what I meant. The important thing is that acme-dns will fall on its face if those records do not exist.

[cpu]

Another solution that was brought up by @webprofusion-chrisc would be to actually create support script (probably within the acme-dns project itself!) to initialize the database, and create preprimed accounts before starting up the actual acme-dns process.

fwiw I included a small binary for a similar purpose in the goacmedns library: https://github.com/cpu/goacmedns#pre-registration

[znerol]

fwiw I included a small binary for a similar purpose in the goacmedns library:

Yes, I'm using that. Thanks a lot for sharing.

[znerol]

Testing the waters here. Very simple first PR: #162

[znerol]

Another very simple PR on the way to simple in-memory DNS txt records : #170

[jvanasco]

FWIW, a simple hack that I used to handle similar situations in an automated environment :

• the automated environment maintains a large list of UUIDs which will be used as subdomains
• when ready for acme-dns:
  • /register to get credentials
  • manually edit the database to swap the acme-dns uuid with the pre-generated uuid; save credentials to application
  • /update and continue

This is not part of the Public API and works because of some design implementations in this package - however it has been very useful.

Nothing is registered/entered into acme-dns until it is actually needed -- so it minimizes orphan accounts - which was our core need. We have "infinitely scalable" storage on our applications so we can pre-generate and allocate infinite UUIDs there, but the acme-dns server has limited storage.

In terms of this feature-request, having a support script or admin API endpoint that can 'import' a user+password+domain+record(optional) would be very useful as that would allow for the acme-dns server to be stateless and/or reset daily. if a record is lost, we can just import it from our applications.