How to solve The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80?

I ran this command:
sudo certbot certonly --standalone --agree-tos --no-eff-email --staple-ocsp --preferred-challenges http -m ****@quantox.com -d outline.quantox.dev

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for outline.quantox.dev

Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
Domain: outline.quantox.dev
Type: connection
Detail: 3.124.126.216: Fetching http://outline.quantox.dev/.well-known/acme-challenge/r5NRbIlEzwrQN7TZQ6a4fxQxmUCuarasPI9GcETkvsI: Error getting validation data

Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet.

My web server is (include version):

The operating system my web server runs on is (include version):
Ubuntu 20.04
My hosting provider, if applicable, is:
Cloudflare
I can login to a root shell on my machine (yes or no, or I don't know):
yes
I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 1.21.0

It looks like you changed your Cloudflare config since you posted. Your Cloudflare now redirects HTTP requests for .well-known/acme-challenge to HTTPS. But, the standalone authenticator for certbot only listens on port 80 (http). Trying an HTTPS request of any kind gets a 502 error from your nginx server (a proxy statement is likely failing).

You might want to try using a Cloudflare Origin CA certificate for your origin server. This is setup in your Cloudflare config and does not require you to run certbot. This Origin CA cert allows HTTPS requests between your origin server and the Cloudflare Edge.

3 Likes

You could also get very creative and...

  • Request HTTP via --standalone
  • Allow CloudFlare to redirect that to HTTPS
  • Accept the HTTPS challenge requests
  • Proxy them to http://localhost/

[this would require the HTTPS server continue to run while HTTP service is stopped]
[which is likely done easiest by using only one web server (for HTTPS)]
[OR two web servers - one only for HTTP and another for HTTPS, then stopping HTTP for certbot]

If that sounds a bit extreme...
Well, it is!

You could also take a very much more simple path and just use the --webroot challenge to have HTTP and HTTPS route challenge requests to the same folder [making both reach their expected location].

2 Likes

By the way, @Milenko, the original error affecting the IP address beginning in 3.124 is typically a router or firewall error where Let's Encrypt is not able to make an inbound connection to the server because a firewall prevents the connection (in a moderately unusual way).

2 Likes

I read all comments,and made a lot of changes.
docker-compose run --rm certbot certonly --webroot --webroot-path /var/www/certbot/ -d outline.quantox.dev

I got error
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: outline.quantox.dev
Type: unauthorized
Detail: 2606:4700:3033::6815:1e1e: Invalid response from https://outline.quantox.dev/.well-known/acme-challenge/FTkqKNqkig0HljqYqvdZn7eSSJbBEhNyS_t8Yas0P7s: 521

That 521 error is from the Cloudflare CDN. You can see it by going to

https://outline.quantox.dev/

That shows that Cloudflare can't reach your site at all. Reaching your site would be a prerequisite for completing the challenge this way.

Cloudflare's basic documentation on debugging this is at

https://support.cloudflare.com/hc/en-us/articles/115003011431-Error-521#521error

One thing they note:

If you have Cloudflare proxying your site, you need to have your origin server already listening on HTTPS on port 443 (serving the site's actual content there). In addition, if you don't already have a certificate, you need to not have "SSL/TLS mode set to Full or Full (Strict)", or else you have to use Cloudflare's origin CA.

Having your site behind Cloudflare may be useful for many reasons, but it somewhat complicates obtaining and using a Let's Encrypt certificate, and also complicates debugging (since we can't directly see the underlying error that prevented communications with your server!).

2 Likes