404 Not Found - urn:acme:error:unauthorized

Hello. I tried creating a new cert for my domain using “letsencrypt certonly --apache -d lestech.space --dry-run” and I got the following:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Starting new HTTPS connection (1): acme-staging.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for lestech.space
Waiting for verification…
Cleaning up challenges
Failed authorization procedure. lestech.space (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://lestech.space/.well-known/acme-challenge/bijzivxbAGQk0y3fC1oWJpj7gugYLZSqbxwDEuxwegA: "

404 Not Found

404 Not Found


"

IMPORTANT NOTES:

Please help. This is on Ubuntu 16.04 server, and am running Apache.

Thanks,
Themeus

Hi @themius,

This is because the IPv4 and IPv6 versions of your web site, while they both exist, return different content. You’re advertising an AAAA record with an IPv6 address, but it doesn’t point to the same web content. This is the version that the Let’s Encrypt CA is checking and receiving the 404 error from.

Hi @schoen,

How do I have LE respond to the IPv4 address without necessarily disabling IPv6?
Am sure that I will need IPv6 soon when standards change.

There is no way to request this from the CA. It will always connect on IPv6 if an AAAA record is advertised. You should instead change your web server configuration so that the IPv6 and IPv4 versions of the site are both supported with the same content, or remove the AAAA record, or use the DNS-01 challenge method instead of HTTP-01 (so that an inbound connection to your server isn't required).

  1. Do I remove AAAA advertising by removing the entry from my DNS records?
  2. How do I perform the DNS-01 method? Can I also use this method for renewals in my crontab?

Yes, that's right.

Certbot's support for DNS-01 authentication is not great compared to some other clients.

In every case, DNS-01 authentication requires an API or script to update your DNS records with your DNS provider, in order to perform unattended issuance or renewal. If you installed Certbot from an OS package, you may be able to install another OS package that has Certbot DNS plugins that are compatible with certain DNS providers. If you installed Certbot in another way, there isn't a super-convenient way of getting access to these plugins.

You can also write a script that performs the update and tell Certbot where to find that script.

If you can't update your DNS records programmatically in this way, then Certbot won't be able to renew from crontab (or otherwise via certbot renew) a certificate obtained via the DNS-01 challenge.

An example of a Let’s Encrypt client with more extensive native DNS provider integration is acme.sh:

If your IPv4 and IPv6 addresses do point at the same physical server, there’s probably a small Apache configuration change that you can make so that the existing web service works correctly in IPv6, and then the way you’re using Certbot already (with the HTTP-01 challenge) should start to work.

Hmm. Am confused here. The IPv6 address does not match when I go to ifconfig.
Ifconfig has 2 IPv6 addresses.
One has a Scope:Link at the end and another has a Scope:Global.

The IPV6 one in my DNS looks like the Scope:Global IP, but only up to the 4th group of characters, that is 1111:2222:3333:4444. I moved this domain from another server on the same host. Strangely, the IPv6 in the DNS of a server on the former server also only matches up to the 4th group. This is confusing. I mean, IPv6.

Indeed, when I connected to it in IPv6 I got an nginx/1.10.3 server and when I connected in IPv4 I got Apache/2.4.18, so perhaps they are not the same machine after all.

I tried to load a URL http://lestech.space/foobarbaz in IPv6, so you can see if now you happen to have any server error logs on this machine that refer to that (nonexistent) file.

Hmm. Could you private message me the IPv6 address that you get? I want to look it up in my DNS and see how I can correct this. The Nginx server is the former server…lol. The new one has Apache.

Amusingly, you could then probably get the certificate on the old server! But the best course of action should be to update the DNS record.

It looks like updating IPv6 is my way forward. Just trying to figure out what IPv6 address you get from the domain. Are you able to direct message me that info?

Sorry. I see your DM.

Niiiiiice. I created a new fake DNS zone tied to the correct server then copied its IPv6 address to my domain. I will try again later and then update this thread.

@schoen. How long will it take for the newer IPv6 address to show up? Can you try accessing that domain via IPv6 and let me know, via DM, what address you get?

By the way, the foobarbaz access attempt came into the correct server. Am guessing you accessed via IPv4.

Let's Encrypt does minimal DNS caching. It will start working as soon as your authoritative DNS servers are updated.

Linode... is more complicated. The authoritative DNS servers update every quarter hour, and do some amount of DNS caching.

If you updated the DNS at 21:59 UTC or so, before your last post, it probably mostly started working almost immediately, at the 22:00 update.

If you updated a few minutes after 22:00, it will start working shortly after 22:15.

At the moment, the A and AAAA records are pointing at the same server, so I guess you did update them in time.

Thanks @mnordhoff. Could you please sharw with me, via DM, how you tested my A and AAAA records? I have lots more domains coming up on these servers and would like to be able to troubleshoot as amazingly as you and @schoen have today.

I just did some DNS queries, using the command line, "dig" and a recursive DNS server that didn't have your domain cached.

(To be more precise, I used a terrible Python program that ran dig, 'cause I like how it displays the information.)

There's not much reason to discuss this by DM. Domain names, Let's Encrypt certificates, and IP addresses (at least IPv4 addresses) are very public information. They're often scanned and archived in various places, though many of them aren't freely available to the public.

To distinguish the IPv6 behavior from the IPv4 behavior, I ran curl -v -4 and curl -v -6 from the command line with your URL on a computer that has an IPv6 connection.