Certbot renew errors with "unexpected error: Failed authorization procedure

I’ve searched this problem, found tons of matches, tried a lot of solutions, and can’t figure it out.

host ------------------- DigitalOcean
os ---------------------- Ubuntu 18.04.1 LTS
root access ---------- yes

web server ---------- Apache/2.4.29 (Ubuntu)
certbot version ----- certbot 0.31.0

domain --------------- thamesandkosmos.com
command ------------ certbot renew

(and also certbot certonly -d thamesandkosmos.com -d www.thamesandkosmos.com, certbot certonly --webroot -w /var/www/html)

output:

Waiting for verification...
Cleaning up challenges
Attempting to renew cert (thamesandkosmos.com-0001) from /etc/letsencrypt/renewal/thamesandkosmos.com-0001.conf produced an unexpected error: Failed authorization procedure. thamesandkosmos.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://thamesandkosmos.com/.well-known/acme-challenge/xbw3120HlsMvM5gK3Xgy7Zv0AkrDhKAJ9-lzI74T_10 [2604:a880:400:d1::956:9001]: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>". Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/sap.thamesandkosmos.com/fullchain.pem (failure)
  /etc/letsencrypt/live/thamesandkosmos.com-0001/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/sap.thamesandkosmos.com/fullchain.pem (failure)
  /etc/letsencrypt/live/thamesandkosmos.com-0001/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: thamesandkosmos.com
   Type:   unauthorized
   Detail: Invalid response from
   http://thamesandkosmos.com/.well-known/acme-challenge/xbw3120HlsMvM5gK3Xgy7Zv0AkrDhKAJ9-lzI74T_10
   [2604:a880:400:d1::956:9001]: "<html>\r\n<head><title>404 Not
   Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404
   Not Found</h1></center>\r\n<hr><center>"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.
2 Likes

Hi @JonathanA

your configuration is buggy, see the check of your domain, ~~1,5 hours old - https://check-your-website.server-daten.de/?q=thamesandkosmos.com

Your non-www has ipv4 and ipv6:

Host Type IP-Address is auth. ∑ Queries ∑ Timeout
thamesandkosmos.com A 178.128.135.3 New York/United States (US) - DigitalOcean No Hostname found yes 1 0
AAAA 2604:a880:400:d1::956:9001 North Bergen/New Jersey/United States (US) - DigitalOcean, LLC yes
www.thamesandkosmos.com A 178.128.135.3 New York/United States (US) - DigitalOcean No Hostname found yes 1 0
AAAA yes

But your non-www ipv4 - /.well-known/acme-challenge/random-filename has a http status 500 + Apache:

Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. Please contact the server administrator at webmaster@localhost to inform them of the time this error occurred, and the actions you performed just before this error. More information about this error may be available in the server error log. Apache/2.4.29 (Ubuntu) Server at thamesandkosmos.com Port 80

Your ipv6 - oh, there is a nginx:

Visible Content: 404 Not Found nginx

Checking your domain Letsencrypt prefers ipv6, so that can’t work.

May be

  • remove the ipv6 and
  • fix the ipv4 http status 500
3 Likes

Thank you for your reply. This is very strange, there shouldn’t be any ipv6 at all… There’s one A record that points to a floating IP, and one CNAME which points to the A record:

Exceedingly strange that nginx is coming up as it isn’t even installed on the server!

I’m not quite sure where to go from here… I’ve created a .well-known/acme-challenge directory in the webroot as suggested in some other issues and set permissions to 755, created an index file within it, and the route can be visited without issue, so I’m not sure what’s going on there.

I don’t normally deal with this kind of issue so thank you for bearing with me in my lack of knowledge.

edit: not sure it makes a difference, but certbot renew has been working for months, and this only just came up recently… no change in any server/DNS configs as far as I’m aware.

Please: All of your dns entries are public, that’s how the DNS works.

D:\temp>nslookup -type=MX thamesandkosmos.com.

thamesandkosmos.com MX preference = 10, mail exchanger = alt3.aspmx.l.google.com
thamesandkosmos.com MX preference = 5, mail exchanger = alt1.aspmx.l.google.com
thamesandkosmos.com MX preference = 1, mail exchanger = aspmx.l.google.com
thamesandkosmos.com MX preference = 5, mail exchanger = alt2.aspmx.l.google.com
thamesandkosmos.com MX preference = 10, mail exchanger = alt4.aspmx.l.google.com

How should a client find your mail server if these informations would be hidden or secret?

You have an AAAA entry, so that may be the wrong place you look.

Yep, completely wrong, read parts of your check result:

Fatal: Inconsistency between delegation and zone. The set of NS records served by the authoritative name servers must match those proposed for the delegation in the parent zone.: 

ns1045.ui-dns.biz (217.160.81.45): 

Delegation: ns-us.1and1-dns.com, ns-us.1and1-dns.de, ns-us.1and1-dns.org, 
ns-us.1and1-dns.us, Zone: ns1045.ui-dns.biz, ns1045.ui-dns.com, 
ns1045.ui-dns.de, ns1045.ui-dns.org. 

Name Servers defined in Delegation, missing in Zone: ns-us.1and1-dns.com, ns-us.1and1-dns.de, ns-us.1and1-dns.org, ns-us.1and1-dns.us.Name Servers defined in Zone, missing in Delegation: ns1045.ui-dns.biz, ns1045.ui-dns.com, ns1045.ui-dns.de, ns1045.ui-dns.org.

You have 1and1 - name servers, not digitalocean. So these entries aren’t used (or you have changed these).

3 Likes

Ahhh, I believe that you’ve found the issue… Removing the IPV6 record (on the correct registrar) has solved it. Thanks very much, sorry to bother with such a silly issue! I’m curious about why this issue might’ve popped up recently when it’s worked before, but wouldn’t be surprised if I didn’t understand the explanation, haha.

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.