Invalid Response (unauthorized)


#1

My domain is:

I ran this command:
certbot renew --dry-run

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/dev.cumo.de.conf


Cert not due for renewal, but simulating renewal for dry run

Plugins selected: Authenticator nginx, Installer nginx

Renewing an existing certificate

Performing the following challenges:

http-01 challenge for dev.cumo.de

Waiting for verification…

Cleaning up challenges


new certificate deployed with reload of nginx server; fullchain is

/etc/letsencrypt/live/dev.cumo.de/fullchain.pem



Processing /etc/letsencrypt/renewal/cumo.de.conf


Cert is due for renewal, auto-renewing…

Plugins selected: Authenticator nginx, Installer nginx

Renewing an existing certificate

Performing the following challenges:

http-01 challenge for cumo.de

Waiting for verification…

Cleaning up challenges

Attempting to renew cert (cumo.de) from /etc/letsencrypt/renewal/cumo.de.conf produced an unexpected error: Failed authorization procedure. cumo.de (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://cumo.de/.well-known/acme-challenge/nqzYEgyTBNbq83JfG3GC71Lff5PohZI2GWo6svs62_E: “<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=“white”>\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>”. Skipping.


Processing /etc/letsencrypt/renewal/www.cumo.de.conf


Cert is due for renewal, auto-renewing…

Plugins selected: Authenticator nginx, Installer nginx

Renewing an existing certificate

Performing the following challenges:

http-01 challenge for www.cumo.de

Waiting for verification…

Cleaning up challenges

Attempting to renew cert (www.cumo.de) from /etc/letsencrypt/renewal/www.cumo.de.conf produced an unexpected error: Failed authorization procedure. www.cumo.de (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.cumo.de/.well-known/acme-challenge/f91zeDEDqRNXtJiReHbg2c7-Ky2SGLclV2j-gEqo6oY: “<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=“white”>\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>”. Skipping.

The following certs could not be renewed:

/etc/letsencrypt/live/cumo.de/fullchain.pem (failure)

/etc/letsencrypt/live/www.cumo.de/fullchain.pem (failure)


** DRY RUN: simulating ‘certbot renew’ close to cert expiry

** (The test certificates below have not been saved.)

The following certs were successfully renewed:

/etc/letsencrypt/live/dev.cumo.de/fullchain.pem (success)

The following certs could not be renewed:

/etc/letsencrypt/live/cumo.de/fullchain.pem (failure)

/etc/letsencrypt/live/www.cumo.de/fullchain.pem (failure)

** DRY RUN: simulating ‘certbot renew’ close to cert expiry

** (The test certificates above have not been saved.)


2 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:

  • The following errors were reported by the server:

Domain: cumo.de

Type: unauthorized

Detail: Invalid response from

http://cumo.de/.well-known/acme-challenge/nqzYEgyTBNbq83JfG3GC71Lff5PohZI2GWo6svs62_E:

"<html>\r\n<head><title>404 Not Found</title></head>\r\n<body

bgcolor=“white”>\r\n<center><h1>404 Not

Found</h1></center>\r\n<hr><center>"

To fix these errors, please make sure that your domain name was

entered correctly and the DNS A/AAAA record(s) for that domain

contain(s) the right IP address.

  • The following errors were reported by the server:

Domain: www.cumo.de

Type: unauthorized

Detail: Invalid response from

http://www.cumo.de/.well-known/acme-challenge/f91zeDEDqRNXtJiReHbg2c7-Ky2SGLclV2j-gEqo6oY:

"<html>\r\n<head><title>404 Not Found</title></head>\r\n<body

bgcolor=“white”>\r\n<center><h1>404 Not

Found</h1></center>\r\n<hr><center>"

To fix these errors, please make sure that your domain name was

entered correctly and the DNS A/AAAA record(s) for that domain

contain(s) the right IP address.

My web server is (include version):
Nginx 1.10.3

The operating system my web server runs on is (include version):
Debian 9

My hosting provider, if applicable, is:
Hetzner

I can login to a root shell on my machine (yes or no, or I don’t know):
Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):
No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):
certbot 0.28.0


#2

Hi @welaunch

there is http + non-www checked. But checking your domain ( https://check-your-website.server-daten.de/?q=cumo.de ) you have 2 ipv4 and 2 ipv6 addresses, with non-www and www. In the end, every check of

/.well-known/acme-challenge/unknown-file

Domainname Http-Status redirect Sec. G
http://cumo.de/
104.24.108.87 301 https://cumo.de/ 0.030 A
http://cumo.de/
104.24.109.87 301 https://cumo.de/ 0.020 A
http://cumo.de/
2606:4700:30::6818:6c57 301 https://cumo.de/ 0.176 A
http://cumo.de/
2606:4700:30::6818:6d57 301 https://cumo.de/ 0.016 A
http://www.cumo.de/
104.24.108.87 301 https://www.cumo.de/ 0.017 A
http://www.cumo.de/
104.24.109.87 301 https://www.cumo.de/ 0.017 A
http://www.cumo.de/
2606:4700:30::6818:6c57 301 https://www.cumo.de/ 0.020 A
http://www.cumo.de/
2606:4700:30::6818:6d57 301 https://www.cumo.de/ 0.014 A
https://cumo.de/
104.24.108.87 301 https://www.cumo.de/ 1.297 B
https://cumo.de/
104.24.109.87 301 https://www.cumo.de/ 1.097 B
https://cumo.de/
2606:4700:30::6818:6c57 301 https://www.cumo.de/ 1.087 B
https://cumo.de/
2606:4700:30::6818:6d57 301 https://www.cumo.de/ 1.093 B
https://www.cumo.de/
104.24.108.87 200 1.294 A
https://www.cumo.de/
104.24.109.87 200 1.137 A
https://www.cumo.de/
2606:4700:30::6818:6c57 200 1.126 A
https://www.cumo.de/
2606:4700:30::6818:6d57 200 1.144 A
http://cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
104.24.108.87 301 https://cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.016 A
http://cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
104.24.109.87 301 https://cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.016 A
http://cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
2606:4700:30::6818:6c57 301 https://cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.014 A
http://cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
2606:4700:30::6818:6d57 301 https://cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.016 A
http://www.cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
104.24.108.87 301 https://www.cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.016 A
http://www.cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
104.24.109.87 301 https://www.cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.010 A
http://www.cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
2606:4700:30::6818:6c57 301 https://www.cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.013 A
http://www.cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
2606:4700:30::6818:6d57 301 https://www.cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.014 A
https://cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 301 https://www.cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 1.147 A
https://www.cumo.de/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 404 5.140 A
Not Found

ends in your https + www version. This is ok, but not that, what Letsencrypt has seen.

Can you find your webroot? Then create the two directories

/.well-known/acme-challenge

there a file (file name 1234), so you can load this file via

http://cumo.de/.well-known/acme-challenge/1234

Then use

certbot run -a webroot -i nginx -w PathToYourWebroot -d cumo.de -d www.cumo.de

to create the certificate.


#3

Another possibly relevant factor: dev.cumo.de dry-run succeeds because it is not obscured by Cloudflare, whereas the two domains that are using Cloudflare (bare domain and www) have the issue.

One way to exclude this may be to temporarily disable Cloudflare and see if validation succeeds. You shouldn’t have to do it for long, since Let’s Encrypt does not do any DNS caching, you can just disable it for one minute.


#4

Indeed that has helped - thanks! But how can I rely on auto renewal if I have to manually deactivate cloudflare all the time?


#5

Well, we only disabled it to confirm that it is Cloudflare causing the issue.

Let’s Encrypt renewal can definitely work successfully through Cloudflare - many users do exactly that.

You just need to identify what it is about your Cloudflare setup that causes the request to get muddled up when it arrives at your server.

Unfortunately that’s a bit opaque to everybody on the forum, since it can depend on things like your Page Rules and any other special config you have in nginx. For example, I notice that your server just drops the connection if you make a request to the bare IP address without any Host header. This wouldn’t cause this exact problem, but it’s not a standard nginx behavior, so who knows what else is lurking in there.

One suggestion I have is that Certbot’s nginx authenticator modifies your port 80 server with the challenge response directly. If Cloudflare’s CDN talks directly to your server over port 443, it would bypass the virtualhost with the challenge response.

In this case, using @JuergenAuer’s suggestion of using the --webroot authenticator may make more sense, as it ensures that the challenge response would be available over both HTTP and HTTPS virtual hosts.


#6

Do you offer paid support? I have all access, just need the auto renewal fixed :confused:


#7

Before jumping to extremes, you should really try the webroot alternative. Do you know the webroot directory for your www.cumo.de domain? You could try:

certbot renew --cert-name cumo.de -a webroot -w /path/to/webroot --dry-run
certbot renew --cert-name www.cumo.de -a webroot -w /path/to/webroot --dry-run

#8

Sure this has worked: certbot renew --cert-name www.cumo.de -a webroot -w /sites/www.cumo.de/public --dry-run

Output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log


Processing /etc/letsencrypt/renewal/www.cumo.de.conf


Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator webroot, Installer nginx
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for www.cumo.de
Using the webroot path /sites/www.cumo.de/public for all unmatched domains.
Waiting for verification…
Cleaning up challenges


new certificate deployed with reload of nginx server; fullchain is
/etc/letsencrypt/live/www.cumo.de/fullchain.pem



** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/www.cumo.de/fullchain.pem (success)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)

Is this now saved for all other renewals?

By the way this is a very fast and good community … never saw that before!


#9

You’ll need to renew each certificate this way without --dry-run for it to be saved to your renewal parameters (which are located in /etc/letsencrypt/renewal/).


#10

Okay so just execute this: certbot renew --cert-name www.cumo.de -a webroot -w /sites/www.cumo.de/public

But i get a “Cert not yet due for renewal”.


#11

I believe that you can force it to save the new settings by using --force-renewal , but only do this once per certificate or you might face trouble with rate limits …

(This is not good advice, I just don’t know what the better advice is).