Chicken/Egg problem setting up new server/domain

I’m trying to set up a new server for my domain. This time I’m documenting the server setup by using Ansible to do it automatically. I got a lot of it done. What remains is letsencrypt.

I found some playbook for setting up letsencrypt and I’m using that. It eventually comes down to the following failure:

$ /usr/local/share/letsencrypt/env/bin/letsencrypt -n --agree-tos --text -d defaria.com --email andrew@defaria.com --expand -a webroot --webroot-path /var/www certonly
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for defaria.com
Using the webroot path /var/www for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. defaria.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://defaria.com/.well-known/acme-challenge/NGXkno8wIMAXZzWzX7UAkT3ah9XdVwL01pa_9QbOooY: "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p"

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: defaria.com
   Type:   unauthorized
   Detail: Invalid response from
   http://defaria.com/.well-known/acme-challenge/NGXkno8wIMAXZzWzX7UAkT3ah9XdVwL01pa_9QbOooY:
   "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
   <html><head>
   <title>404 Not Found</title>
   </head><body>
   <h1>Not Found</h1>
   <p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

As I understand it, this webroot authentication is trying to determine that you have control over the domain by having the client side create a file in a .well-known directory in webroot of the server then the external side validates that and if OK can issue the certificate.

The problem is that my domain, defaria.com, already exists on another server. I’m setting up a new server that will replace the current defaria.com server. As we can see above it tries to go to http://defaria.com/.well-known/acme-challenge/NGXkno8wIMAXZzWzX7UAkT3ah9XdVwL01pa_9QbOooY but cannot find it because defaria.com resolves to my current server but not my newly setup server. I thought I might try using the IP address so that it would go to the new server but letsencrypt doesn’t like it when you use an IP address.

So then is this a chicken and egg problem? Is it so that I cannot use letsencrypt on the localhost unless I have a valid domain that resolves in DNS to the public IP address of the server I’m setting up?!? I would think that setting up tests like this without having to go through the full domain registration process would be common.

Thanks in advance…

There are several Let's Encrypt clients that have a "remote webroot" concept where you can ask the client to upload the files to the other server in this case (e.g. via scp or sftp). It is also possible to set up an HTTP redirect for only the directory /.well-known/acme-challenge which is used in the HTTP-01 validation process; the certificate authority follows these redirects when checking your authorization to get the certificate, and so if machine A sends an HTTP redirect to machine B for /.well-known/acme-challenge, machine A is delegating to machine B the ability to get certificates on its behalf.

But otherwise, yes, if you use the HTTP-01 or TLS-SNI-01 validation methods, you generally need to run the client on a machine that the subject DNS names in question all already point to. There is still another method called DNS-01 which does not require the DNS names to point anywhere in particular, but requires you to be able to create TXT records inside the DNS zone. This is useful when trying to get certificates for names that have not been delegated, that are used in a split-horizon DNS configuration, or that point to a different server than the machine you're running Let's Encrypt on.

Where are these remote webroot clients? I mean I have LetsEncrypt set up on my current server and I could copy the files over I guess (/opt/letsencrypt and /etc/letsencrypt) but would that work? Maybe only after I moved the domain name from its current server to the new server.

As for redirecting /.well-known/acme-challenge, where exactly would I redirect it to? I mean I don’t think I can easily redirect it to my home laptop on DHCP and I don’t have another server machine to redirect it to, 'cept perhaps the current defaria.com but that doesn’t seem to resilient. I mean I want to be able to set up a new server from scratch using Ansible and not have a dependency of the old server.

The one thing I have not done yet is DNS. I usually let the ISP do that. Why should I?

Finally, I would have thought I would have gotten an email if a response was posted here but I didn’t get any email notification. Must I poll this group?

Your existing certificates (if you got them with Certbot or something) aren't relevant to getssl and you would probably just start over with a new account and new certificates if you started using it.

If you're not using a remote webroot method, it would need to redirect to wherever the client is being run at the moment. If that doesn't have a stable domain name, you could use a dynamic DNS service or something. You only have a dependency on the old server if the domain name is pointing at the old server, because the point of the verification process is to confirm that you actually control the specific domain names for which you're requesting certificates.

The DNS-01 validation method works with a number of DNS providers that have APIs to let you update your zone file. Several clients support several of these APIs, so you can still have the DNS hosted by someone else, but post records in your DNS zone as requested by the CA.

I think normally you do get notifications from the forum, and I'm afraid I don't know what's responsible if you haven't been.

Setting up Dynamic DNS seems like overkill and wouldn’t work anyway AFAICT because there can’t be one defaria.com with two IP addresses. IOW I do have a dependency in that I need the old server to remain up until the new server is ready to take over. As such it seems to make sense to leave the letsencrypt Ansible role to be non-automatic and something that is done only after defaria.com’s IP address has been moved over to the new domain. Luckily TLS is not essential for me but I like to keep my web site encrypted.

As for notifications what is the from address? I ask because I’m actively blocking email from @discourseemail.com.

The dynamic DNS name doesn't have to be the same as the name that the certificate will be issued for, if there is an HTTP redirect in place from http://existingserver.example.com/.well-known/acme-challenge/ to http://temporaryname.dyn.example.net/.well-known/acme-challenge/. This will allow the temporaryname server to request and receive a fresh certificate for existingserver.example.com without updating DNS and while leaving the existing certificate in place and valid.

You do unless you use one of these methods that I've described, including the redirect method above, or using the DNS-01 method, or copying the private key and certificate from one server to the other will avoid this. These are the potential ways to avoid outages. Otherwise, switching DNS records will always result in an outage.

It comes from letsencrypt at discoursemail.com.

Thanks for the explanation but honestly this is just my little website so outages aren’t really a problem. IOW I can run unencrypted with no problems for a while until I get letsencrypt setup properly. If that’s a “create new OpenStack server, run Ansible playbook to set up new instance, do DNS swap then, sometime later, login and do a getssl (<- thanks for that pointer to that - I’m evaluate that now)” it really wouldn’t matter that much if my little server was running unencrypted for even a few days.

Yes, it was discoursemail.com. I opened that up and got the notification of your response.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.