Automatically try replacing http with https in ACME challenge?

I have LE certs set up on several domains and they are set to auto-renew via an HTTP ACME challenge. This recently stopped working though as it appears an overzealous application firewall somewhere in our organization is blocking HTTP GETs including the /.well-known/acme-challenge/ string on port 80:

% curl 'http://mydomain/.well-known/acme-challenge/foo'
curl: (56) Recv failure: Connection reset by peer

Any other string (e.g. http://mydomain/.well-known/acme-challenge2/foo) works fine and returns an HTTP 301 redirect to the equivalent https: URL (the domain also has HSTS enabled). Also a direct request of https://mydomain/.well-known/acme-challenge/foo works without issues.

My proposal: the ACME protocol allows for a http->https redirect of course. Would you consider automatically trying an http->https substitution if the original ACME challenge fails in this way (perhaps only if HSTS is set up)?

Of course I would prefer if our organization did not interfere with HTTP traffic in this fashion, but we are large enough that the IT folks I've spoken to aren't even aware of this firewall or how it might be reconfigured!

1 Like

This sounds like you may have a Palo Alto brand firewall. They recently changed a default setting. See here:

8 Likes

The challenge using port 443 is called tls-alpn-01. That's the challenge that will try port 443 the first time. The http-01 challenge will always start on port 80 and can only change protocols (and thus ports) using redirects.

Changing the http-01 challenge to retry on an entire protocol (and thus port) is a major change and I'm afraid has a very slim change of ever being reality, even if LE would think it's a good idea.

An alternative due to blocking firewalls could always be the dns-01 challenge, but that has its own drawbacks.

Also, while I sympathise with your troubles (I'm sure a large company with ignorant/incompetent IT department(s) is very frustrating), I don't think that a firewall vendor wreaking havoc on the ACME protocol by setting the default to DENY is a good reason to even propose to change the http-01 challenge.

7 Likes

Just to add some clarity to the above comment by @Osiris: there once was a challenge TLS-SNI-01 that functioned just lot like http-01. It was removed several years ago because of a security vulnerability. Due to implementation details in most (all?) major web servers and shared hosting providers, it is impossible to authoritatively prove control of a domain with a secret file – like the http-01 and tls-sni-01 – rely on. The tls-alpn-01 challenge introduced a new protocol designed to avoid the vulnerability in tls-sni-01, but most web servers do not support this new protocol yet.

5 Likes

And they won't, because tls-alpn-01 is literally another protocol over TLS on port 443. RFC 8737 - Automated Certificate Management Environment (ACME) TLS Application-Layer Protocol Negotiation (ALPN) Challenge Extension

(443 TCP, eh, http3 wasn't a thing yet)

2 Likes

@9peppe, they could still implement it alongside HTTP within the same software application—I think Caddy can natively do the TLS-ALPN challenge in addition to being a web server.

4 Likes

Well, Apache does support TLS-ALPN via mod_md. It just takes a while for packages to filter down to all LTS distributions.

You can apt install apache2 && a2enmod md ssl on Debian Bullseye (maybe even earlier) and it will work.

In my opinion it is slightly more complicated in terms of configuration than using an external ACME client but impressive nonetheless.

nginx also has the infrastructure to take advantage of ALPN, but the module needs to be written.

5 Likes

IMHO: based on their earlier decisions, mission statement, and statements of employees - if there were a way to securely offer HTTPS validation outside of tls-alpn-01 that is available to a wider audience, LetsEncrypt would definitely pursue it.

There are several OpenResty modules that bring this into the Nginx ecosystem.

Under "vanilla" nginx, you can proxy only the ALPN requests to something like dehydrated.

3 Likes

Yes, they can. It's a bit out of scope for a webserver.

The one advantage of tls-alpn-01 is avoid interfering with http routing or the webroot. So you can host websites for arbitrary clients, and nothing they can do will break validation.

It makes sense for a big provider. But for a single user with complete control over machine and website... not really.

4 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.