Certbot renewal is unable to read acme-challenge on port 443 since 1 month (use to work before)

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: pnaf.fr

I ran this command: certbot certonly --webroot --rsa-key-size 4096 -w /Users/Shared/Sites/www.pnaf.fr/httpdocs -d pnaf.fr -d macmini.pnaf.fr -d mail.pnaf.fr -d smtp.pnaf.fr -d www.pnaf.fr --email webmaster@pnaf.fr

It produced this output:

certbot certonly --webroot --rsa-key-size 4096 -w /Users/Shared/Sites/www.pnaf.fr/httpdocs -d pnaf.fr -d macmini.pnaf.fr -d mail.pnaf.fr -d smtp.pnaf.fr -d www.pnaf.fr --email webmaster@pnaf.fr

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: macmini.pnaf.fr
Type: connection
Detail: 82.65.30.178: Fetching https://www.pnaf.fr/.well-known/acme-challenge/n37JxqDyC2XIZNY9rjIR_kruP1jGkm1QN5HHKOQJ_ac: Error getting validation data

Domain: mail.pnaf.fr
Type: connection
Detail: 82.65.30.178: Fetching https://www.pnaf.fr/.well-known/acme-challenge/uNn3gQFydIb3AV5b9ELIGW4Pb37fegrMiGLkl6V1M4c: Error getting validation data

Domain: smtp.pnaf.fr
Type: connection
Detail: 82.65.30.178: Fetching https://www.pnaf.fr/.well-known/acme-challenge/vkpt-NhhNsS94dPTnN7cf-MIsgv2-46m3gtdgav-MGA: Error getting validation data

Domain: www.pnaf.fr
Type: connection
Detail: 82.65.30.178: Fetching https://www.pnaf.fr/.well-known/acme-challenge/2f47AYdPL1j4scHbs149lFRdLF7iMKr49yvoXBCePOY: Error getting validation data

Domain: pnaf.fr
Type: connection
Detail: 82.65.30.178: Fetching https://www.pnaf.fr/.well-known/acme-challenge/28cA48B91_pxwu6_3yaHbVno1R6PLbpYKQEvplSJ6zU: Error getting validation data

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version): Apache/2.4.33 (Unix)

The operating system my web server runs on is (include version): macOS 15.6.1

My hosting provider, if applicable, is: myself

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 5.0.0

Comment:
My renewal was working like a charm for several years and I didn't change anything on my MacOS X Server recently. But since 1 month, renewal is not working anymore. It seems that files written in /.well-known/acme-challenge/ directory are not readable by letsencrypt if accessed via https(443:). It works if I'm using http(80:). Any ideas? Thanks.

2

You should use http :slight_smile:

This happens when http and https use different webroots / htdocs directories (it's usually a good thing)

2 Likes
3

Agree. They also should not redirect the incoming HTTP challenge request to HTTPS

Maybe a new redirect is what is causing the failure

2 Likes
4

OKay. I'm going to use http.

But as nothing has changed on my server side, I don't understand why check via https requests were working for many years and stopped since my last renewals.

For instance, you can try https://www.pnaf.fr/.well-known/acme-challenge/test.txt file. You will see that this file is reachable like any files generated during certbot works. I have checked that all certbot file are correctly generated and recorded in this directory during renew process. And '-w /Users/Shared/Sites/www.pnaf.net/httpdocs' is the right place were https://www.pnaf.fr/.well-known/acme-challenge/ sits.

Strange isn't it? Thanks.

5

Something unexpected might be going on in your Apache config. Some filtering, some reverse proxying... but it works.

1 Like
6

Yes, it is. I can see the error using Let's Debug test site:

The "Error getting validation data" is an invalid response from your domain. Do you have any kind of firewall or similar that would respond differently based on the IP of the requester?

That error indicates a problem from the Let's Encrypt primary server which is based in the US. I don't have any problem reaching your test file from my own US based test server so more likely an IP based firewall/routing rather than general geographic area.

You got three certs yesterday. Were those all with your vServer disabled? You should add --dry-run to your Certbot command to avoid running into rate limits when testing.

image
image1674×834 86.2 KB

2 Likes
7

Yes, sorry about running into rate limits. But I've done several test yesterday as I needed to get SSL certificates on 4 different domains (not only pnaf.fr). Those 4 domains were ending shortly (in 2 weeks) and I was worry about everything could stop working. That's why I didn't put '--dry-run' in my command.

I don't have any specific rules about external IPs. All external traffic arriving on my internet box at http(:80) and https(:443) is forwarded to the macos server hosting those 4 domains (I didn't change anything on the server side for a long time). I checked that https://www.pnaf.fr/.well-known/acme-challenge/28cA48B91_pxwu6_3yaHbVno1R6PLbpYKQEvplSJ6zU was generated and not empty when issuing the certbot command. Letsencrypt primary server seems unable to read that file via https(:443), but has no trouble on http(:80), which I can't explain why.

Thanks for your help.

8

If you can validate using port 80 that is best anyway.

Ideally you'd process the original inbound request from LE in your port 80 VirtualHost and not redirect it.

If you show us your current port 80 VirtualHost we can suggest how to do that.

2 Likes
10

Showing you my VirtualHost is not so easy. I use MacOS X Server.app to manage all my domains. I have a graphic interface to do so. Any VirtualHost changes will be overwritten by Apple's Server.app program periodically. I don't have many option in Server.app, I can redirect any http(:80) traffic to secured mode (:443), but I can't set specific options such as RewriteCond something. So forget about this solution.

I use to have an deamon to automatically issue certbot command each early Monday mornings and try to see if some domain needed to be 're-SSL-ised'.

Now, at renewal time, I will have stop manually each https domain's redirections on my Server.app. Issue the certbot command on Terminal.app and put back again secured redirection on each domains. Sound's much complicated, but I have no other solution right?

Thanks.

11

I was wondering if the problem is not with may pnaf.fr.conf file:

renew_before_expiry = 30 days

version = 2.5.0
archive_dir = /etc/letsencrypt/archive/pnaf.fr
cert = /etc/letsencrypt/live/pnaf.fr/cert.pem
privkey = /etc/letsencrypt/live/pnaf.fr/privkey.pem
chain = /etc/letsencrypt/live/pnaf.fr/chain.pem
fullchain = /etc/letsencrypt/live/pnaf.fr/fullchain.pem

Options used in the renewal process

[renewalparams]
account = ***************
rsa_key_size = 4096
authenticator = webroot
webroot_path = /Users/Shared/Sites/www.pnaf.fr/httpdocs,
server = https://acme-v02.api.letsencrypt.org/directory
key_type = ecdsa
[[webroot_map]]

Hope this could help. Thanks!

12

No, I don't think so. A problem with the --webroot directory shows as a "404 Not Found" error not the error you get. Your error is more closely related to comms configs.

It is tricky because we cannot reproduce that error from other origins.

Can you view your Apache access and/or error logs? The error log might show the reason for the error you get.

1 Like
13

That's for certbot and it's default enough.

Check what DocumentRoot you get in the port 80 virtualhost, even if you can't edit it manually.

1 Like
14

What about trying this? It might not work with your WEB DAV but might be easy fix if does

certbot certonly --dry-run --apache --rsa-key-size 4096 -d pnaf.fr

The --dry-run will not interfere with your existing certs. Using just one domain is fine for this test

2 Likes
15

Issuing that command produces
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Could not find ssl_module; not disabling session tickets.
Simulating a certificate request for pnaf.fr
Unable to find a virtual host listening on port 80 which is currently needed for Certbot to prove to the CA that you control your domain. Please add a virtual host for port 80.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

So, Letsencrypt verification process says it is mandatory to deal with http(:80) website. This behaviour has change I get in the last 2 months :thinking:

16

Here is my VirtualHost generated by Server.app

<VirtualHost 127.0.0.1:34580>
ServerName http://www.pnaf.fr:80
ServerAdmin admin@example.com
DocumentRoot "/Users/Shared/Sites/www.pnaf.fr/httpdocs"
DirectoryIndex index.html
CustomLog /var/log/apache2/access_log "%h %l %u %t "%r" %>s %b"
<IfModule mod_ssl.c>
SSLEngine Off
SSLCipherSuite "HIGH:MEDIUM:!MD5:!RC4:!3DES"
SSLProtocol -all +TLSv1.2
SSLProxyProtocol -all +TLSv1.2
</IfModule>
<IfModule mod_dav.c>
DAVLockDB "/var/run/davlocks/.davlock100"
DAVMinTimeout 600
</IfModule>
<IfModule mod_mem_cache.c>
CacheEnable mem /
MCacheSize 4096
</IfModule>
<IfModule mod_secure_transport.c>
MSTEngine Off
MSTCipherSuite HIGH, MEDIUM
MSTProtocolRange TLSv1.2 TLSv1.2
MSTProxyEngine On
MSTProxyProtocolRange TLSv1.2 TLSv1.2
</IfModule>
<Directory "/Users/Shared/Sites/www.pnaf.fr/httpdocs">
Options All -Indexes -ExecCGI -Includes +MultiViews
AllowOverride None
<IfModule mod_dav.c>
DAV Off
</IfModule>
<IfDefine !WEBSERVICE_ON>
Require all denied
ErrorDocument 403 /customerror/websitesoff403.html
</IfDefine>
</Directory>
<IfModule mod_proxy_balancer.c>
<Proxy "balancer://balancer-group">
</Proxy>
</IfModule>
<IfModule mod_alias.c>
Alias /collaboration "/usr/share/collaboration"
Alias /icons/ "/usr/share/httpd/icons/"
Alias /error/ "/usr/share/httpd/error/"
Alias /collaboration "/usr/share/collaboration"
Alias /icons/ "/usr/share/httpd/icons/"
Alias /error/ "/usr/share/httpd/error/"
</IfModule>
LogLevel warn
ServerAlias pnaf.fr macmini.pnaf.fr smtp.pnaf.fr mail.pnaf.fr
RewriteEngine On
RewriteCond %{HTTP_HOST} !(^localhost|^127.0.0.1|^::1)
RewriteCond %{REQUEST_URI} !^/netboot/ [NC]
RewriteRule .* https://www.pnaf.fr%{REQUEST_URI} [R]
</VirtualHost>

Unfortunately I can't anything there. I will be overwritten by Server.app
:confused:

17

No, the requirements for an HTTP Challenge have not changed. The --webroot option you use is for an HTTP Challenge. The LE Server sends an HTTP request (port 80) to your domain to prove control. We can tell from your error message that you redirect that request to HTTPS which apparently was working at one time.

Why that now fails is the puzzle.

The error when switching to the --apache option is very different. It also uses HTTP Challenge but will setup your port 80 VirtualHost to reply without redirect. But, it only works with standard Apache VirtualHost syntax. Some WEB DAV configs only have their extended settings outside of a VirtualHost so --apache can still work for those. But, yours is not one of them now that we can see what it looks like.

In short, just ignore the result from the --apache test.

2 Likes
18

Does that show any of the inbound HTTP request from Let's Encrypt server?

Is there a way to have your system setup an error log? These are often setup in the base Apache config before the VirtualHost. Maybe there is one setup there.

That is a very unusual combination. You set a specific IP and port on the VHost making an IP-Based VirtualHost but use a different port for the ServerName. I can't imagine how that is supposed to work. The "http://" scheme for a ServerName is allowed but is only rarely needed. That needs someone more experienced with Apache than me to make sense of it.

Are you able to show the VirtualHost for port 443?

2 Likes
19

I can imagine something interesting is happening with the firewall. This is an OSX server, I assume they work kinda like appliances and the only reason they don't handle acme automatically is that Apple got out of the server game a while ago.

These are the same, it should work. Only reason I can imagine for that is that the request never reaches that virtualhost, and that explains why it's performing a redirect without any directive redirecting anything in the virtualhost you posted.

$ curl -IL http://www.pnaf.fr/.well-known/acme-challenge/test.txt
HTTP/1.1 302 Found
Date: Fri, 19 Sep 2025 19:11:15 GMT
Server: Apache
Location: https://www.pnaf.fr/.well-known/acme-challenge/test.txt
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 200 OK
Date: Fri, 19 Sep 2025 19:11:15 GMT
Server: Apache
Last-Modified: Thu, 18 Sep 2025 18:10:18 GMT
ETag: "5-63f1744015680"
Accept-Ranges: bytes
Content-Length: 5
MS-Author-Via: DAV
Content-Type: text/plain
1 Like
20

Sure, but, as I understand Apache the VirtualHost statement listing a specific IP and port means that this VHost is ONLY a candidate for that IP:port combination.

If so, with the ServerName setting a port contrary to that how would Apache ever choose this VHost? Unless I suppose the request is inbound on port 34580 and Apache chooses it anyway if it is the only one (the default).

These rewrites shouldn't ever activate since the VHost statement is IP based only for 127.0.0.1 and the Conds preclude redirecting those. Am I reading that right?

Yet, they seem like they do since every error in first post was for https://www.pnaf.fr

Frankly, this Apache config seems especially convoluted.

That said, I don't see anything in it that would fail from LE primary server and yet succeed elsewhere. So, the problem lies in the gear or comms config on their premise that is in front of Apache.

2 Likes
21

I don't know, I didn't notice those. But %{HTTP_HOST} is whatever SNI the browser picks, and proxies don't usually touch it (but they can).

2 Likes