I have a chicken-or-the-egg situation on my server. I am using acme_tiny.py and lighttpd to serve up the challenge token files. I want to run with port 80 disabled (it is actually used for a different http-to-stream proxy), and serve up token files on port 443 https. This works fine for any SAN subdomain that I have already obtained an earlier certificate for. But it doesn't work for any SAN subdomain the first time, creating a chicken-or-the egg situation or death by recursion. I must first have a certificate before I can get a certificate?
Is there a way in the protocol to specify I want only to perform https challenge responses and have it work for a subdomain I don't yet have a certificate for.
If you redirect all HTTP to HTTPS, then the HTTPS must use a certificate to converse.
[but it can be any certificate - even a self-signed cert OR any valid cert from any other FQDN]
"Our implementation of the HTTP-01 challenge follows redirects, up to 10 redirects deep. It only accepts redirects to “http:” or “https:”, and only to ports 80 or 443. It does not accept redirects to IP addresses. When redirected to an HTTPS URL, it does not validate certificates (since this challenge is intended to bootstrap valid certificates, it may encounter self-signed or expired certificates along the way)."
My issue is not with a redirect, and I don't care if it validates certificates. My issue is that I have another service on port 80.
For any certificate for a domain which is a renewal, the challenge will first try https on port 443 and succeeds because that is where my web server lives.
For any SAN on a (sub)-domain which is not a renewal, the challenge will first try http on port 80. If that fails, even with with a rejected connection (for example if I first shut down my port 80 service), the verification fails and it will not then try 443.
This is puzzling. Either: 1) try 443 first all the time - there is no reason why port 80 should ever be the first stop any more. Or 2) If port 80 fails on a rejected connection, move on to 443 and try that.
No, that is not correct. An http challenge always starts with an HTTP request to port 80. You may redirect that to https port 443. Always starts on port 80 for both new certs and renewals
Either Letsencrypt starts with 443 on these SANs, or it is starting with 80 but automatically proceeding to 443 without failing the challenge for domains which already have certificates.
I have one VPS with SANs that have not changed in three years, and it has not had port 80 enabled in that time. It has successfully renewed its certificates a dozen times with no port 80.
This IS the way Letsencrypt is working.
What I am saying is that I want the same behaviour for new SANs as existing ones. This is not an unreasonable request, since it is clearly already established behaviour.
What kind of challenge do they use? Because TLS-ALPN starts with HTTPS on port 443. An HTTP Challenge starts on port 80. These are ACME industry standard requirements.
What ACME Client are you using and what is the command format? Perhaps knowing more will help us resolve this riddle.
Also please provide these other domain name(s) you say work without having port 80 open. Perhaps you don't notice something that we will. You haven't given us much to work with. Your va1der.net has port 80 open, for example.
# acme_tiny.py --help
usage: acme_tiny.py [-h] --account-key ACCOUNT_KEY --csr CSR --acme-dir ACME_DIR [--quiet] [--disable-check] [--directory-url DIRECTORY_URL] [--ca CA] [--contact [CONTACT ...]]
This script automates the process of getting a signed TLS certificate from Let's Encrypt using
the ACME protocol. It will need to be run on your server and have access to your private
account key, so PLEASE READ THROUGH IT! It's only ~200 lines, so it won't take long.
Example Usage:
python acme_tiny.py --account-key ./account.key --csr ./domain.csr --acme-dir /usr/share/nginx/html/.well-known/acme-challenge/ > signed_chain.crt
Example Crontab Renewal (once per month):
0 0 1 * * python /path/to/acme_tiny.py --account-key /path/to/account.key --csr /path/to/domain.csr --acme-dir /usr/share/nginx/html/.well-known/acme-challenge/ > /path/to/signed_chain.crt 2>> /var/log/acme_tiny.log
options:
-h, --help show this help message and exit
--account-key ACCOUNT_KEY
path to your Let's Encrypt account private key
--csr CSR path to your certificate signing request
--acme-dir ACME_DIR path to the .well-known/acme-challenge/ directory
--quiet suppress output except for errors
--disable-check disable checking if the challenge file is hosted correctly before telling the CA
--directory-url DIRECTORY_URL
certificate authority directory url, default is Let's Encrypt
--ca CA DEPRECATED! USE --directory-url INSTEAD!
--contact [CONTACT ...]
Contact details (e.g. mailto:aaa@bbb.com) for your account-key
I have a rather extensive bash script to execute acme_tiny.py without root privs and handle the resulting certificates.
I employ port 80 for ALLCALL protocol. Technically it's http but it doesn't serve up any files. You can try and point a browser to it - no matter what you request you'll get a 418 "I'm a teapot" response.
I just checked artoo.va1der.net. It has a short and only recent cert history. Once the first is issued Let's Encrypt caches a successful challenge for 30 days. So, renewals will occur without any challenge at all for 30 days. It won't even look at port 443
The domains I gave are the ones I'm writing about.. the ones I want to get work properly with just 443 because I am adding SAN almost daily.
Main (established) server is va1der.ca, mail.va1der.ca, www.va1der.ca, productrevue.ca (plus mail, www, etc), mccullochcentre.ca, mccullochcenter.ca. That has run for years.
This is getting bogged down in irrelevancies though. Main point is getting letsencrypt to stop failing out on a port 80 rejection and only fail out on port 80 AND port 443 rejection on all domains and in all circumstances. What will this take?
ANotWorking
Error
va1der.ca has an A (IPv4) record (107.6.27.142) but a request to this address over port 80 did not succeed. Your web server must have at least one working IPv4 or IPv6 address.
Get "http://va1der.ca/.well-known/acme-challenge/letsdebug-test": dial tcp 107.6.27.142:80: connect: connection refused
Trace:
@0ms: Making a request to http://va1der.ca/.well-known/acme-challenge/letsdebug-test (using initial IP 107.6.27.142)
@0ms: Dialing 107.6.27.142
@113ms: Experienced error: dial tcp 107.6.27.142:80: connect: connection refused
IssueFromLetsEncrypt
Error
A test authorization for va1der.ca to the Let's Encrypt staging service has revealed issues that may prevent any certificate for this domain being issued.
107.6.27.142: Fetching http://va1der.ca/.well-known/acme-challenge/Gh1k0JQw6f763xOwu7xL0G-HQ3Dt8w-x9f4gS20GYq0: Connection refused
Can we please focus on the actual issue? Why is this hard? Port 80 isn't a thing. I use it for something more important. LE will already accept port 443, why can we not get it to do so at all times? Please answer THAT rather than other snide side issues.
This request MUST be sent to TCP port 80 on the HTTP server.
We can be redirected to port 443 from there, but the initial request MUST be made to port 80. We don't have a choice in the matter. I'm sorry it makes your system difficult. As the others have said, I recommend either switching to TLS-ALPN-01 (which happens entirely over port 443) or to DNS-01 (which doesn't require an open port at all).
For years port 80's sole purpose has been to forward browsers that started with http over to https. Now browsers don't even do that any more. There is no purpose for http on port 80 outside listening on localhost for ssh-forwarded connections to local web applications like phpmyadmin that you don't want to expose to the wider internet.
So port 80 has no purpose.
From the page you referenced:
So the 443 test uses software no one outside Google and other enterprise users have. So.... for the vast majority of the internet it's useless. Which you very well know.
In the wider scheme, what is troubling is:
The internet moves to https on 443, LetsEncrypt response = port 80 requirement
The internet moves to QUIC on UDP, LetsEncrypt response = port 443 TCP TLS requirement
Letsencrypt could move TODAY to lead with https on 443 for http-01. That makes sense.
