Received 2nd email but certbot test passed?

Help
1

I got the first email on Jan 27th, upgraded the client and run the dry-run which succeeded.
Today I got a new notification email claiming I still had a renewal on 2019-02-09 using ACME TLS-SNI-01.
Not sure who is lying here…

My domain is:utopia14.org

I ran this command:sudo certbot renew --dry-run

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/polny-shkaf.utopia14.org.conf

Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator apache, Installer apache
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for polny-shkaf.utopia14.org
http-01 challenge for www.utopia14.org
Waiting for verification…
Cleaning up challenges
Resetting dropped connection: acme-staging-v02.api.letsencrypt.org

new certificate deployed with reload of apache server; fullchain is
/etc/letsencrypt/live/polny-shkaf.utopia14.org/fullchain.pem

Processing /etc/letsencrypt/renewal/utopia14.org.conf

Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator apache, Installer apache
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for polny-shkaf.utopia14.org
http-01 challenge for utopia14.org
http-01 challenge for www.utopia14.org
Waiting for verification…
Cleaning up challenges
Resetting dropped connection: acme-staging-v02.api.letsencrypt.org

new certificate deployed with reload of apache server; fullchain is
/etc/letsencrypt/live/utopia14.org/fullchain.pem

** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:
/etc/letsencrypt/live/polny-shkaf.utopia14.org/fullchain.pem (success)
/etc/letsencrypt/live/utopia14.org/fullchain.pem (success)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)

My web server is (include version):
httpd-2.4.6-88.el7.centos.x86_64

The operating system my web server runs on is (include version):
CentOS Linux release 7.6.1810 (Core)

My hosting provider, if applicable, is:
linode

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.29.1

2

Pretty sure no one is lying.
You might be misinterpreting the email or the results your actions (thus far) should have produced.

The --dry-run doesn't undue the fact that a cert was recently obtained using TLS-SNI-01 on 2019-02-09. The email is merely to make you everyone is aware of that and help guide them towards using HTTP-01 (or any other method before the current cert expires).

This is good news:

But it is a simulated renewal (against the "staging" environment) - not an actual "production" renewal.
So the current (in use) cert was issued with... TLS-SNI-01.
And until an actual renewal is seen via any other method, that is the latest information.
[unfortunately "staging" and "production" are very separate systems - for the obvious, and some not so obvious, reasons]

All-in-all you should be fine and you can (generally) ignore current emails related to that IP.
But you should keep an eye on upcoming renewals
via: certbot certificates
and ensure they all get renewed (at least once) after TLS-SNI-01 is fully deprecated.

3

I don't see how this can be misinterpreted:

Your Let's Encrypt client used ACME TLS-SNI-01 domain validation to issue a
certificate in the past 7 days. Below is a list of names and IP addresses
validated (max of one per account):

www.utopia14.org (66.228.35.49) on 2019-02-09

It say that my domain used the old domain validation on the 9th of February. Which means that my configuration is still broken. No?

4

I’ve just double-checked our logs, and I can confirm that there was definitely a validation from Certbot 0.29.1 on 2019-02-09, using the Apache authenticator. That version of Certbot, using that authenticator, should definitely default to using the HTTP-01 challenge even if TLS-SNI-01 is offered.

Can you check if tls-sni-01 is explicitly configured in your renewal config? This could happen if you used the --preferred-challenges=tls-sni-01 flag in the past. You can run

  grep tls-sni /etc/letsencrypt/renewal/*

If you do find an example, follow these instructions to fix.

Otherwise, it’s a bit of a mystery what’s going on. It’s 95% likely things will work just fine at your next renewal, but I’d like to understand this better, and see if other people are likely to have the same issue. Can you upload your log file from 2019-02-09? It would be one of the files in /var/log/letsencrypt/.

Thanks,
Jacob

5

Well I have great news, I have the same or very similar situation.
I got the dreaded Action Required email earlier this month, with the relevant line " vectorjohn.com (173.230.158.112) on 2018-12-14". I received it on 2019-01-28.

Since that time, I have not touched certbot in any way. No updates. Also:

root@localhost:/var/log/letsencrypt# certbot --version
certbot 0.21.1

Which according to other forum posts, is the old version that uses the soon deprecated tls-sni. I kept the old one because the instructions here say that I should expect errors if I run sudo certbot renew --dry-run. I get no errors at all. I verified I can get errors by setting the challenge explicitly to the deprecated one:

sudo certbot renew --dry-run --preferred-challenges tls-sni-01

But without doing that, the renew completes without issue.

Anyway, long story short, I am wary that anything I do actually fixes the potential problem since I can’t even reproduce it now with the old version. That, and my system (Ubuntu 17.10) says 0.21 is the newest so I can’t even update it.

For what it’s worth, I’m also on Linode, which is either a clue or a red herring.

Edit: One more thing. grep tls-sni /etc/letsencrypt/renewal/* finds nothing. I have not configured anything in there.
One more detail if it helps, I have three domains on the same system using letsencrypt. I only got the email about one of them.

6

Grep command returns nothing.
I can’t upload files here (new user) so here’s the log file
http://www.utopia14.org/~meshko/letsencrypt.log.9
this is the one from Feb 9th and judging by its size it has the actual renewal. I don’t have time to look at it myself now.

7

Perhaps you have multiple acme clients and they are running from different jobs?

which certbot
which certbot-auto
crontab -l
sudo crontab -l
systemctl list-timers

8

Your log show TLS-SNI-01 use on 2019.02.09:

2019-02-09 12:41:07,038:INFO:certbot.auth_handler:Performing the following challenges:
2019-02-09 12:41:07,039:INFO:certbot.auth_handler:tls-sni-01 challenge for polny-shkaf.utopia14.org
2019-02-09 12:41:07,040:INFO:certbot.auth_handler:tls-sni-01 challenge for www.utopia14.org
2019-02-09 12:41:07,040:WARNING:certbot.auth_handler:TLS-SNI-01 is deprecated, and will stop working soon.
2019-02-09 12:41:07,974:DEBUG:certbot_apache.parser:Adding Include /etc/httpd/conf.d/le_tls_sni_01_cert_challenge.conf to /files/etc/httpd/conf/httpd.conf
2019-02-09 12:41:07,975:DEBUG:certbot_apache.tls_sni_01:writing a config file with text:
 <IfModule mod_ssl.c>
<VirtualHost *:443>
    ServerName 674311e73b76df0cbd3968e176ff2758.7d55b60135b43fb22c716e03164454bd.acme.invalid
    UseCanonicalName on
    SSLStrictSNIVHostCheck on

    LimitRequestBody 1048576

    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /var/lib/letsencrypt/qGZb1iFHilpVpFFXN7orxWv_Qjyj27Nmap3--3MrpYU.crt
    SSLCertificateKeyFile /var/lib/letsencrypt/qGZb1iFHilpVpFFXN7orxWv_Qjyj27Nmap3--3MrpYU.pem

    DocumentRoot /var/lib/letsencrypt/tls_sni_01_page/
</VirtualHost>

<VirtualHost *:443>
    ServerName 0c58c08a99312a44f31c7516c7a8ebd8.821952d2aa63ea3fe7badf932469e72d.acme.invalid
    UseCanonicalName on
    SSLStrictSNIVHostCheck on

    LimitRequestBody 1048576

    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /var/lib/letsencrypt/oM_7KKf9_aAhMW1HJE8cSiRtXijCR5V5FyjwWv7qtVc.crt
    SSLCertificateKeyFile /var/lib/letsencrypt/oM_7KKf9_aAhMW1HJE8cSiRtXijCR5V5FyjwWv7qtVc.pem

    DocumentRoot /var/lib/letsencrypt/tls_sni_01_page/
</VirtualHost>

</IfModule>

2019-02-09 12:41:20,320:INFO:certbot.auth_handler:tls-sni-01 challenge for polny-shkaf.utopia14.org
2019-02-09 12:41:20,321:INFO:certbot.auth_handler:tls-sni-01 challenge for www.utopia14.org
2019-02-09 12:41:20,321:INFO:certbot.auth_handler:tls-sni-01 challenge for utopia14.org
2019-02-09 12:41:20,322:WARNING:certbot.auth_handler:TLS-SNI-01 is deprecated, and will stop working soon.
2019-02-09 12:41:21,279:DEBUG:certbot_apache.parser:Adding Include /etc/httpd/conf.d/le_tls_sni_01_cert_challenge.conf to /files/etc/httpd/conf/httpd.conf
2019-02-09 12:41:21,281:DEBUG:certbot_apache.tls_sni_01:writing a config file with text:
 <IfModule mod_ssl.c>
<VirtualHost *:443>
    ServerName 674311e73b76df0cbd3968e176ff2758.7d55b60135b43fb22c716e03164454bd.acme.invalid
    UseCanonicalName on
    SSLStrictSNIVHostCheck on

    LimitRequestBody 1048576

    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /var/lib/letsencrypt/qGZb1iFHilpVpFFXN7orxWv_Qjyj27Nmap3--3MrpYU.crt
    SSLCertificateKeyFile /var/lib/letsencrypt/qGZb1iFHilpVpFFXN7orxWv_Qjyj27Nmap3--3MrpYU.pem

    DocumentRoot /var/lib/letsencrypt/tls_sni_01_page/
</VirtualHost>

<VirtualHost *:443>
    ServerName 0c58c08a99312a44f31c7516c7a8ebd8.821952d2aa63ea3fe7badf932469e72d.acme.invalid
    UseCanonicalName on
    SSLStrictSNIVHostCheck on

    LimitRequestBody 1048576

    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /var/lib/letsencrypt/oM_7KKf9_aAhMW1HJE8cSiRtXijCR5V5FyjwWv7qtVc.crt
    SSLCertificateKeyFile /var/lib/letsencrypt/oM_7KKf9_aAhMW1HJE8cSiRtXijCR5V5FyjwWv7qtVc.pem

    DocumentRoot /var/lib/letsencrypt/tls_sni_01_page/
</VirtualHost>

<VirtualHost *:443>
    ServerName 83d8e70c50d8a263c77361456b4d8fd7.3ceb731f8e38775f32f5d533a8dc677c.acme.invalid
    UseCanonicalName on
    SSLStrictSNIVHostCheck on

    LimitRequestBody 1048576

    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /var/lib/letsencrypt/x8MD1odQiAVB9MmA2kQfvYCop5pccDo4S2MM7ubw5wQ.crt
    SSLCertificateKeyFile /var/lib/letsencrypt/x8MD1odQiAVB9MmA2kQfvYCop5pccDo4S2MM7ubw5wQ.pem

    DocumentRoot /var/lib/letsencrypt/tls_sni_01_page/
</VirtualHost>

</IfModule>
9

Well, yes, I think we all agree that it used the wrong protocol. The question is why.
I doulble checked and I only have one version installed. Also the log file contains the line

2019-02-09 12:34:05,615:DEBUG:certbot.main:certbot version: 0.29.1

10

I don't agree...
TLS-SNI-01 is still valid even today.

No; It is NOT broken.
It will work with HTTP-01 when TLS-SNI-01 fails.
[unless you have restricted it to only use TLS-SNI-01 - which I don't think is the case here]

11

OK, then this just means that the scheme for notifying people is broken, because everyone with the correct config will still get the email.

12

How so?

LE can't be sure you have a client capable of validating via HTTP-01.
Nor that your ISP isn't blocking port 80, etc.
Only you can verify those things.
The email states only verifiable facts.

Where did it go wrong?
[if it can be improved please help LE help others - better]

13

Hi there - we are having similar issues on all of our machines (Ubuntu 16.04, nginx, certbot 0.28.0). It appears that the default behavior between actual certificate renewal and dry run is different.

Here’s the dry run output which shows http-01:

Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator nginx, Installer nginx
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for <redacted>
Waiting for verification...
Cleaning up challenges

Here’s a forced renewal from a few minutes ago which shows tls-sni-01 (which worked, I thought this was supposed to be temporarily disabled today?)

Plugins selected: Authenticator nginx, Installer nginx
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Renewing an existing certificate
Performing the following challenges:
tls-sni-01 challenge for <redacted>
TLS-SNI-01 is deprecated, and will stop working soon.
Waiting for verification...
Cleaning up challenges

There is no reference to tls-sni-01 nor pref_challs in any of the renewal confs. We have also received warning emails since updating certbot to 0.28.

Please advise if you think anything needs to be done, or if there’s nothing to worry about.

Thanks for your help, it is much appreciated.

1 Like
14

This has been postponed until some day in March (don't recall exactly).
[the latest emails should show the exact end date]

They may continue, so long as the client is still renewing via TLS-SNI-01.

The logs cutoff before the "Congrats", but assuming both did show it, you are good to go!

15

It can be improved by clearly stating that this email does not mean that my configuration is broken.
Or by suggesting a configuration which would disable the scheme which is going away.
I can’t believe I’m the only one confused by the messaging (there is at least one other person here it seems).

16

Are you sure all of Certbot's packages are up-to-date?

dpkg -l '*certbot*' '*letsencrypt*'
17

The only visible difference is (expected):

18

Thanks @paraxial, @meshko, @vectorjohn for the extra info. I've bumped your trust levels on the forum so you should be able to upload files now if you need to.

@rg305 is missing a bit of context in his comments: Certbot 0.28+ is supposed to prefer HTTP-01 over TLS-SNI-01 when using the Apache or Nginx plugins, even if TLS-SNI-01 is available. Certbot 0.21-0.27 supports both but prefers TLS-SNI-01. So if you have 0.28+ and Certbot is selecting TLS-SNI-01, that sounds like a bug. @bmw do you have any idea what could cause that?

0.21 supports HTTP-01 but will prefer TLS-SNI. You should be fine after the switchover. However I definitely recommend you upgrade your system to the latest Ubuntu. If you're not on an LTS release (16.04, 18.04), you aren't getting any security updates.

Yes, that will happen today, but hasn't happened quite yet.

I think you shouldn't worry too much, but I do appreciate your help tracking down what's going on. I don't want to be sending emails to people who don't need to take any action. I'll keep following up with my colleagues and take a deeper look at the logs provided.

1 Like
19

@paraxial is there anything in the renewal.conf file that would make certbot prefer TLS-SNI-01 ?
[/etc/letsencryot/renewal/{cert-name}.conf]

20

Ok, this seems to have been my problem - python3-certbot-nginx was still 0.22. I admit not knowing my way around the packaging.

Once updated, both dry run and actual renewal use http-01.

Thanks for your help!

2 Likes