TLS-SNI-01 Certbot Fix: Doesn't Fit Patterns

Help
1

I received a second Action required: Let’s Encrypt certificate renewals email yesterday: Your Let’s Encrypt client used ACME TLS-SNI-01 domain validation…

I have shell access to this Ubuntu 16.04 Apache 2.4.18 VPS. I began to follow the 3-step procedure posted by author bmw,

Step 1 (Ensure certbot or certbot-auto version >= 28)

jk@bird:~$ certbot --version
certbot 0.26.1
jk@bird:~$ certbot-auto --version
certbot-auto: command not found
jk@bird:~$ whereis certbot-auto
certbot-auto:

Conclusion: I have certbot, not certbot-auto, and this certbot needs to be updated. So I updated it using apt-get and friends. Looks good now:

jk@bird:~$ certbot --version
certbot 0.28.0

Step 2 (Update .conf files)

The long one-liner command in bmw’s post apparently invokes sed to apply a patch to all files in /etc/letsencrypt/renewal/, replacing any qualified occurrences of the string tls-sni-01 with http-01. Makes sense. But before applying this, I thought I’d have a look at the single file in that directory. I found this contents:

# renew_before_expiry = 30 days
version = 0.26.1
archive_dir = /etc/letsencrypt/archive/sheepsystems.com
cert = /etc/letsencrypt/live/sheepsystems.com/cert.pem
privkey = /etc/letsencrypt/live/sheepsystems.com/privkey.pem
chain = /etc/letsencrypt/live/sheepsystems.com/chain.pem
fullchain = /etc/letsencrypt/live/sheepsystems.com/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = 20a12c3be1cdb0a0aa032a4d0a2feb70
server = https://acme-v02.api.letsencrypt.org/directory
authenticator = apache
installer = apache

Hmmmm, there are no occurrences of string tls=sni-01. So that sed command will have no effect. Another surprise: There is not even any mention of validation method.

That version = 0.26.1 is apparently a reference to my old certbot installation. To see if maybe running the new certbot 0.28.0 would update that .conf file, I decided to skip up to…

Step 3 (See if it worked)

I ran

sudo certbot renew

both with and without --dry-run. But the .conf file remained the same. It still contains version = 0.26.1.

One final surprise. In both cases, the results indicated that http-01 was used, and congratulated me on success. Snippets:

. . . 
Performing the following challenges:
http-01 challenge for sheepsystems.com
http-01 challenge for www.sheepsystems.com
Waiting for verification...
. . .
Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/sheepsystems.com/fullchain.pem (success)

With all of these surprises, I feel that I somehow got into the wrong universe. May someone please tell me where I am?

2

The reason for updating is that since 0.28 the apache plugin will default to the http-01 challenge. So even when the sed command didn't do anything in your specific situation, by upgrading alone you've managed to fix the problem.

But other people might have a challenge hardcoded in their renewal configuration files, where sed would have been required.

Luckily, running the sed command on already "fixed" or not-needed-to-fix configs won't do any harm.

1 Like
3

Thank you, Osiris. I shall presume that the version = 0.26.1 key/value remaining in my .conf file was put there for debugging and has no significance to the running of certbot.

4

That entry should indicate which was the last version used to renew that cert.
NOTE: --dry-run will NOT actually renew a cert (so that option will not make any changes at all)

Did you get a "Congratulations!..." without the --dry-run ?
If only "Skipping..."/"Not ready for renewal", then you could try a --force-renewal on a specific cert and see if
A. Will it update with the new client and using http?
B. Will it update the renewal.conf file?

5

Hello, rg, and thank you for the additional questions.

OK, upon retesting I find that:

  • Without --dry-run, result says Cert not yet due for renewal, and I do not get Congratulations. Maybe I was wrong about this in my original post.
  • With --force-renewal
    • (A) It says that it used tls-sni-01 challenges, eeek!!
    • (B) It does update the version key in the .conf file to 0.28.0.

So you have explained everything except, eeek now we have this new indication that, when forced to do the real thing, my certbot is still using tls-sni-01. Here is the complete transcript:

jk@bird:/etc/letsencrypt/renewal$ sudo certbot renew --force-renewal
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/sheepsystems.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Plugins selected: Authenticator apache, Installer apache
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for sheepsystems.com
tls-sni-01 challenge for www.sheepsystems.com
TLS-SNI-01 is deprecated, and will stop working soon.
Waiting for verification...
Cleaning up challenges

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed with reload of apache server; fullchain is
/etc/letsencrypt/live/sheepsystems.com/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/sheepsystems.com/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

I did this twice – same result both times.

So, now I am worried again. The Certbot documentation implies that a cli.ini file might also affect configuration. I did find a cli.ini file but it only has one key/value pair, max-log-backups = 0.

Do you think I should try and explicitly set the challenge/validation method in my .conf file? If so, please tell me the exact key name. I cannot reverse-engineer it from bmw’s sed one liner because it contains a wildcard, and I cannot find key names defined in the Certbot documentation either.

6

Well that is what all the hubbub is about; TLS-SNI-01 works today in production but will soon be removed.
So the best test today is to use --dry-run which will simulate a renewal but use the staging system which has already removed TLS-SNI-01.
If you can pass the --dry-run test, then you will pass the "real run" when it comes time to renew.

On a positive note, now that you have "forced a renewal", you have 90 days until it expires :slight_smile:

7

I hope you are correct, rg. But what about the following three lines in the --force-renewal transcript?

Performing the following challenges:
tls-sni-01 challenge for sheepsystems.com
tls-sni-01 challenge for www.sheepsystems.com

You know, tls-sni-01 is the bad guy.

8

Yes, --force-renewal uses the current production system.
Which has NOT yet removed TLS-SN-01.
But it will start removing it real soon (on Feb 13, 2019).

9

Have you passed the --dry-run test?
Did the --dry-run say "Congratulations..."?

10

OK, I thought that the current production system supported tls-sni-01, http-01, and others. I think you are saying it does not, so it ignores my Certbot’s request for http-01 and falls back to the only thing it can currently do, tls-sn-01.

11

The current production system (means LE server side).
And it does still support TLS-SNI-01, HTTP-01, DNS-01, TLS-ALPN-01.
The current production certbot client also supports most of those - but it will default to HTTP-01 (unless there is an overriding pref_challenge in the renewal.conf file).

The test is simple.
Lets start with your daily check/renew process.
Please show:
crontab -l

12

I think that my server uses systemctl timers instead of cron jobs…

jk@bird:/etc/letsencrypt$ crontab -l
no crontab for jk
jk@bird:/etc/letsencrypt$ sudo crontab -l
[sudo] password for jk: 
no crontab for root
jk@bird:/etc/letsencrypt$ systemctl list-timers
NEXT                         LEFT          LAST                         PASSED       UNIT                         ACTIVATES
Tue 2019-01-29 03:46:51 PST  3h 40min left Mon 2019-01-28 15:38:16 PST  8h ago       certbot.timer                certbot.service
Tue 2019-01-29 06:59:32 PST  6h left       Mon 2019-01-28 06:01:59 PST  18h ago      apt-daily-upgrade.timer      apt-daily-upgrade.service
Tue 2019-01-29 16:39:43 PST  16h left      Mon 2019-01-28 22:46:00 PST  1h 20min ago apt-daily.timer              apt-daily.service
Tue 2019-01-29 22:55:00 PST  22h left      Mon 2019-01-28 22:55:00 PST  1h 11min ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service

4 timers listed.
13

OK, please show:
grep ExecStart /lib/systemd/system/certbot.service

Or if not there (unlikely)
find / -name certbot.service

14 
ExecStart=/usr/bin/certbot -q renew
15

OK. So, when left alone, that is what is run:
/usr/bin/certbot -q renew

Lets try that with --dry-run (and not so quiet):
sudo /usr/bin/cerbot renew --dry-run

This should show HTTP-01 challenges and “Congratulations”.

16

Yes, it does show http-01 challenges and Congratulations. It’s only with --force-renewal that the result says it used tls-sni-01 for some reason.

17

That will change in due time unless:

So lets review those as well:
grep -Eri 'pref|chall|tls' /etc/letsencrypt/renewal

Or also if in the cli.ini file (rare - but possble):
grep -Ei 'pref|chall|tls' /etc/letsencrypt/cli.ini

18

Both of those greps return an empty result, as expected. And I have previously inspected with my eyes the contents of both of those files (I only have one domain on this server) and found no key/values including challenge or tls-sni-01.

19

Since you have now told me the key, I just tried the test I suggested in Post 4. That is, I appended these two lines to the .conf file:

# To ensure that deprecated tls-sni-01 challenge is not used
pref_challenge = http-01

and then executed another certbot renew with --force-renewal. Result: The transcript says it used tls-sni-01 :frowning:

20

You won't need to change anything manually.

It will just happen...
All on its' own (in due time):
HTTP-01
[because there is nothing stopping it from happening and there is no other way for it to happen]