Temporary challenge files not available anymore

Help
1

Since tonight 00:01 my autorenewal for certs isn't working anymore. It seems that the acme-challenge is not available because it errors out with a 404.

My domain is: mail.tillewolle.de

I ran this command: sudo certbot renew --dry-run

It produced this output:
Certbot failed to authenticate some domains (authenticator: webroot).
The Certificate Authority reported these problems:
Domain: mail.tillewolle.de
Type: unauthorized
Detail: 82.165.178.102: Invalid response from http://mail.tillewolle.de/.well-known/acme-challenge/iFPqFsmeiq6YITrQRDUE83uARkTomyhkrp7PHvVxkR8: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

sudo tail -500 /var/log/letsencrypt.log

12:32:06,682:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
File
"/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line
106, in handle_authorizations
self._poll_authorizations(authzrs, max_retries, best_effort)
File
"/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line
206, in _poll_authorizations
raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2025-03-20 12:32:06,682:DEBUG:certbot._internal.error_handler:Calling
registered functions
2025-03-20 12:32:06,682:INFO:certbot._internal.auth_handler:Cleaning up
challenges
2025-03-20 12:32:06,682:DEBUG:certbot._internal.plugins.webroot:Removing
/var/www/html/.well-known/acme-challenge/KarhG_m6z_Q3HPy96gSEmwsdm6dZqKn8C-mQSUR6ii4
2025-03-20 12:32:06,683:DEBUG:certbot._internal.plugins.webroot:All
challenges cleaned up
2025-03-20 12:32:06,683:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
File "/usr/bin/certbot", line 33, in
sys.exit(load_entry_point('certbot==2.1.0', 'console_scripts',
'certbot')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/certbot/main.py", line 19, in main
return internal_main.main(cli_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line
1736, in main
return config.func(config, plugins)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line
1590, in certonly
lineage = _get_and_save_cert(le_client, config, domains, certname,
lineage)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/certbot/_internal/main.py", line
126, in _get_and_save_cert
renewal.renew_cert(config, domains, le_client, lineage)
File "/usr/lib/python3/dist-packages/certbot/_internal/renewal.py",
line 395, in renew_cert
new_cert, new_chain, new_key, _ =
le_client.obtain_certificate(domains, new_key)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/certbot/_internal/client.py",
line 428, in obtain_certificate
orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/certbot/_internal/client.py",
line 496, in _get_order_and_authorizations
authzr = self.auth_handler.handle_authorizations(orderr,
self.config, best_effort)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line
106, in handle_authorizations
self._poll_authorizations(authzrs, max_retries, best_effort)
File
"/usr/lib/python3/dist-packages/certbot/_internal/auth_handler.py", line
206, in _poll_authorizations
raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed

My web server is (include version):

nginx 1.22.1

The operating system my web server runs on is (include version):

debian 12

I can login to a root shell on my machine (yes or no, or I don't know):

yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

certbot 2.1.0

2

Welcome @certifiedfreak

Would you show the nginx server block for this mail domain?

Need to verify the root folder matches what you previously used to get the certificate.

2 Likes
3

Yeah of course. But what do you mean with server block?

How am I able to get this info?

4

It is the "server" definition in your nginx configuration (examples here).

We can see your entire active nginx config with this command. Upper case T is essential

sudo nginx -T

For now I just wanted to see the server "block" (or section) which has server_name for mail.tillewolle.de

NOTE:
For more info on how nginx config works see nginx docs here: Beginner’s Guide

2 Likes
5

Don't remember setting that up... it worked for years till this week. Checked my notes from setting this up and it seemed that I configured it with this command:

sudo certbot certonly --webroot --agree-tos --email you@example.com -d mail.your-domain.com -w /var/www/html/

replaced with my stuff... I think that it configured nginx with the iRedMail setup script.

My config:

# configuration file /etc/nginx/sites-enabled/00-default.conf:
#
# Note: This file must be loaded before other virtual host config files,
#
# HTTP
server {
    # Listen on ipv4
    listen 80;
    listen [::]:80;

    server_name _;

    # Allow ACME challenge to be served over HTTP (don't redirect to HTTPS).
    location ~* ^/.well-known/acme-challenge/ {
        root /opt/www/well_known;
        try_files $uri =404;
        allow all;
    }

    # Redirect all insecure http requests to https.
    location / {
        return 301 https://$host$request_uri;
    }
}
6

Yes, your original webroot path (-w) was /var/www/html as visible in the log

If you do not have a dedicated server block for the mail name then nginx will process requests for it using its default server block. In that you show the root path as /opt/www/well-known

Which fails because now your Certbot renewal profile thinks the webroot path is /var/www/html but nginx does not

You have several options for fixing this.

The simplest is to change the root folder in the above default server block to be /var/www/html. Whether this is appropriate for other uses you may have for this default block I cannot say. That change would at least allow the cert renewal to succeed

Better would be to make a NEW server block just like that default one but using
server_name mail.tillewolle.de; instead of _
Also change this new server block root folder to /var/www/html to match your Certbot renewal profile.

Another option is to modify your Certbot renewal profile to use the folder your default server block uses. Just reissue that command you showed but use the /opt/www/well_known for the -w path.

There are many ways to manage servers so choose the best method for your case.

I think making a new dedicated server block for this name is the best. Relying on the default server block often leads to problems like this. I leave the default server block to handle only invalid requests (like bots, scanners, and such) and never for expected valid requests.

1 Like
7

Changed it to:

 HTTP
server {
    # Listen on ipv4
    listen 80;
    listen [::]:80;

    server_name mail.tillewolle.de;

    # Allow ACME challenge to be served over HTTP (don't redirect to HTTPS).
    location ~* ^/.well-known/acme-challenge/ {
    #   root /opt/www/well_known;
        root /opt/well_known    
        try_files $uri =404;
        allow all;
    }

    # Redirect all insecure http requests to https.

But still no change: error message looks like:

sudo certbot -v renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/mail.tillewolle.de.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Certificate is due for renewal, auto-renewing...
Plugins selected: Authenticator webroot, Installer None
Simulating renewal of an existing certificate for mail.tillewolle.de
Performing the following challenges:
http-01 challenge for mail.tillewolle.de
Using the webroot path /var/www/html for all unmatched domains.
Waiting for verification...
Challenge failed for domain mail.tillewolle.de
http-01 challenge for mail.tillewolle.de

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: mail.tillewolle.de
  Type:   unauthorized
  Detail: 82.165.178.102: Invalid response from http://mail.tillewolle.de/.well-known/acme-challenge/hJ-iLRuzQUIbnETXFlIIh0bd_3owoJHJy-CqY9_RaNA: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Cleaning up challenges
Failed to renew certificate mail.tillewolle.de with error: Some challenges have failed.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All simulated renewals failed. The following certificates could not be renewed:
  /etc/letsencrypt/live/mail.tillewolle.de/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
8

that was not one of my suggestions plus is missing semi-colon at end

Try: root /var/www/html;

2 Likes
9

That seemed to work! No errors anymore!

Thank you so much!

3 Likes