I tried to renew certs with certbot renew --dry-run
I also try to enable/disable virtual hosts on server, but without any success.
What does the requested numbers mean?
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/cloud.htl-mechatronik.at.conf produced an unexpected error
Failed authorization procedure. www.htl-mechatronik.at (tls-sni-01)
urn:acme:error:unauthorized
The client lacks sufficient authorization
Incorrect validation certificate for tls-sni-01 challenge.
Requested 24ac87df34fb2eb2fabc7f3bd27b673d.47948f55252001dab0287038e316794a.acme.invalid from 188.20.185.182:443.
Received 2 certificate(s), first certificate had names "www.htl-mechatronik.at", cloud.htl-mechatronik.at (tls-sni-01):
urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Incorrect validation certificate for tls-sni-01 challenge.
Requested 69ae1c7bc1a5f2fe4ac446a2898c4e9c.593606ab9b55487bbe24a125cd54e5ee.acme.invalid from 188.20.185.182:443.
Received 2 certificate(s), first certificate had names "www.htl-mechatronik.at". Skipping.
Domain: www.htl-mechatronik.at
Type: unauthorized
Detail: Incorrect validation certificate for tls-sni-01 challenge.
Requested
24ac87df34fb2eb2fabc7f3bd27b673d.47948f55252001dab0287038e316794a.acme.invalid
from 188.20.185.182:443. Received 2 certificate(s), first
certificate had names “www.htl-mechatronik.at”
Domain: cloud.htl-mechatronik.at
Type: unauthorized
Detail: Incorrect validation certificate for tls-sni-01 challenge.
Requested
69ae1c7bc1a5f2fe4ac446a2898c4e9c.593606ab9b55487bbe24a125cd54e5ee.acme.invalid
from 188.20.185.182:443. Received 2 certificate(s), first
certificate had names “www.htl-mechatronik.at”
this could probably be the reason. certbot is not finding your private key? Make sure all you keys and csr file are in the folder certbot is expecting them to be?
To a lesser extent, I'm not sure if this would also mean running that command with sudo. Not sure this would make sense in this context. Just mentioning.
I have started certbot as root on the host where the apache-server is running.
The private keys are in /etc/letsencrypt/keys, the csr files are in /etc/letsencrypt/csr.
The log file /var/log/letsencrypt/letsencrypt.log shows a lot of information, but I cannot see any hint for the problem cause.
Is there any information in detail available for the procedure in the program? It could help to check step by step. As I know now, the apache server configuration is modified on the fly to response something special for validiation…
So, in this log, what you see is Certbot trying to perform a challenge called tls-sni-01. In this challenge Let’s Encrypt talks to the HTTPS server with the name you want a certificate for, and it asks for a nonsensical virtual host name ending in “.invalid” using a feature called SNI (which is host virtual hosting works for TLS).
It does this because an ordinary web server won’t have such a host, and, as happened here for you, it’ll just send back a certificate for a host it does have, failing the challenge.
Probably this challenge worked for you when you originally obtained the certificate because you did not have HTTPS enabled at all (probably you planned to switch it on once you’d got a certificate). So that time Certbot was able to take the roll of web server, it answered the HTTPS connection from Let’s Encrypt and handed over acceptable proof.
But now your real web server is running with HTTPS, and it has no idea about any of these shenanigans, so that can’t work as before.
You have a few ways forward, I will list only the two most likely to be appropriate:
Look at switching to the http-01 challenge, aka “webroot” in which Let’s Encrypt essentially looks for a file on your web server with a special name, placed there by Certbot. This is suitable if you have a “normal” web server where it’s possible to put files somewhere (often named the “webroot”) and they’re presented by the web server software.
Temporarily stop the real HTTPS server every time you perform renewal, this might be OK for small personal or hobby sites where it’s OK for the site to be down for a few minutes every month, it’s obviously not OK for a business or anything that’s supposed to be accessible all the time.
Finally: Note that “dry run” means not for real, it comes from the idea of a practice fire drill in which everything is done except they don’t fill the hoses with water and drench everything, it’s dry. A dry run is useful to check how things work but will not actually renew your certificates.
tls-sni-01 works by creating a fake virtual host that only works over Server Name Indication, separate from any existing HTTP or HTTPS virtual hosts on the server. It will work just fine regardless of whether or not you have TLS enabled on the server. The certbot authors wouldn’t have made certbot renew in this way if they didn’t think it would work.
Please paste the whole output from your certbot invocation, including the full command you ran. The step before your output begins where certbot deploys the fake virtual host is what is failing. You could also add -vvv to your command to get more detail for certbot engineers to debug with.
Thank you for response. It’s not a problem to switch off server for a short time.
So I have stopped the server, removed all virtual hosts from apache server configuration, deleted all log files and started certbot renew --dry-run -vvv. Same result, but I got some little more information (option -vvv).
The following certs were successfully renewed:
/etc/letsencrypt/live/test.htl-mechatronik.at/fullchain.pem (success)
/etc/letsencrypt/live/app.htl-mechatronik.at/fullchain.pem (success)
The following certs could not be renewed:
/etc/letsencrypt/live/cloud.htl-mechatronik.at/fullchain.pem (failure)
That’s interesting. The currently active cert is for Subject: CN=app.htl-mechatronik.at.
Other server names are listened in the X509v3 Subject Alternative Name: field
For test.htl-mechatronik.at I created another dedicated certificate with this subject name (months ago), because recreation of the existing cert with an additional server name did not work.
It’ s interesting that the logs show success for two names, and failure for cloud.htl-mechatronik.at. What’s about the other names?
Do you think it would be good idea to create completely new separate certs from scratch, dedicated for only one server name?
I have now checked /etc/letsencrypt/live and /etc/letsencrypt/renewal and there is indeed a subdirectory cloud.htl-mechatronik.at which should not be there.
I removed these directory and run certbot again.
Now I get only warnings, which seems not to be a serious problem, but renewing succeeds.
Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/test.htl-mechatronik.at/fullchain.pem (success)
/etc/letsencrypt/live/app.htl-mechatronik.at/fullchain.pem (success)
** DRY RUN: simulating 'certbot renew' close to cert expiry
** (The test certificates above have not been saved.)
...
Hook command "/bin/run-parts /etc/letsencrypt/post-hook.d/" returned error code 1
Error output from run-parts:
run-parts: failed to open directory /etc/letsencrypt/post-hook.d/: No such file or directory
Removing the directory /etc/letsencrypt/live/cloud.htl-mechatronik.at looks good at first moment, but it was not the solution, because the directory app.htl-mechatronik.at does not contain the cert with the alternative server names.
I have found out that there is a name mismatch, probably caused by the certbot program itself when trying to add a new server name.
The original cert was generated for cloud.htl-mechatronik.at. Then I add the name app.htl-mechatronik.at and this probably causes the failure (it happens months ago). The new cert was created for subject app.htl-mechatronik.at but stored in subdirectory /etc/letsencrypt/live/cloud.htl-mechatronik.at. From this moment certbot creates errors.
I have now renamed /etc/letsencrypt/live/cloud.htl-mechatronik.at and /etc/letsencrypt/renewal/cloud.htl-mechatronik.at and all it’s content files to app.htl-mechatronik.at manually, and now it works.
I’m glad you got it working, but I don’t think that your solution directly relates to the origin of the problem.
The names in /etc/letsencrypt/live are arbitrary (their content does not have to match any domain name that is part of a certificate, and isn’t assumed to), and are based on the first name requested for a certificate when it was initially obtained. When the certificate is renewed or modified by adding or removing names, the certificate name in /etc/letsencrypt/live is never changed, and should never have to be changed.
The error that you saw reflects that Certbot was somehow unable to reconfigure your Apache server appropriately in order to prove your control of the domain names. I guess there are many possible reasons for that. It’s possible that it would have worked if you had upgraded to a newer version of Certbot, or that there is some still-unknown bug that makes Certbot think it’s made appropriate changes to your Apache configuration file when it hasn’t. This is very possible; if you have further problems along these lines when obtaining or renewing certificates in the future, you could help us by posting log files from /var/log/letsencrypt and Apache configuration files from /etc/apache2/sites-available. Then we could better understand what Certbot tried to do, and why it didn’t work.
(The ideal case would be that, if Certbot couldn’t reconfigure Apache in this way, it would understand and explain why, e.g., a lack of appropriate permissions, or an unusual Apache configuration, or something. So in any case if it didn’t do that, that’s a bug, although possibly one that’s been fixed in a subsequent release of Certbot.)
