Email from Slack regarding ISRG Root X1

We have received a few reports from folks who have received an email from Slack about updating their ISRG root certificates. We have confirmed that the email text at the bottom of this post is legitimately sent by Slack.

Here is background on ISRG Root X1.

ISRG Root X1 is trusted in many root stores, so updating to a current version of your operating system or firmware may suffice for resolving this issue. As a quick test, please try accessing https://slack.com in your browser. If you are able to do so, then you don’t need to take further action.

If you need to manually update your trusted root certificates, they may be downloaded from our website. Please note that we generally do not encourage people to manually add trusted root certificates because this process can be error-prone.

The text of the email is:

As an Owner or a Primary Owner of Slack instance name, you're receiving this email to ensure Slack continues to function within your network environment.

Please work with your network or IT team to ensure a new root certificate is installed in your infrastructure for – slack-edge.com – by May 9th, 2023. Specifically, you will need to ensure the "ISRG Root X1" certificate from Let's Encrypt is installed and trusted, which can be downloaded from Let's Encrypt: https://letsencrypt.org/certificates/

Any clients connecting to Slack should have this certificate installed. We ask that this be done as soon as possible, as it will be necessary for Slack to function properly in the coming months. If this root certificate is already installed and trusted, no action is needed at this time.

If you have any questions, reply to this email or contact us at feedback@slack.com.

Thank you,

The team at Slack

5 Likes

They are trying to use short chain? long chain expires at 2024-10-1 so it makes sense to prepare about it.

P.S trying slack.com in browser won't be enough, that site sends long chain.
try https://valid-isrgrootx1.letsencrypt.org/

7 Likes

I don't see any evidence they're choosing the short chain; I assume they'll use the same one they recently switched to for slack.com. They're currently using a Digicert certificate on *.slack-edge.com.

The "quick test" line comes from Slack support, so we assume it's accurate.

7 Likes

It is not a bad idea to test both chains - the Internet is filed with lots of each :wink:

4 Likes

I was looking for that link yesterday! I remembered that site.

There is also https://valid-isrgrootx2.letsencrypt.org/

Were any test sites made for long vs short chains?

3 Likes

When going to this test link where does it denote that it is valid vs invalid? If what I am seeing is the valid side then it doesn't clearly denote it. Do you know what it shows if invalid, would I see a big red box with an error/alert?

That depends on the client you are using (browser for example).

You can check what revoked or expired certs look like with your client using the samples on this page:

Look for the section "We’ve set up websites to test certificates chaining to our active roots."

4 Likes

Thanks, so the links noted here are just linking to a page that always say valid. So there is no real test page you can send people to that will determine if the user's certificate is valid, revoked, or expired?

1 Like

You're looking for TLS connection errors. For a browser that means, when you can see the website, the certificate validation is fine. If you get an error message (like the one below), the validation is not fine.

grafik

Browsers are unlikely to exhibit any issues, as they have trusted ISRG Root X1 for years. Only significantly outdated clients are prone to errors. For non-browser clients the situation (both in terms of validation and error messages) will be different.

8 Likes

Small nuance here: if a client does NOT ship with it's own Trusted Root Store, it will rely on the Operating System store (or some sort of packaged distribution storage/library, such as Python apps utilizing Certifi). In these scenarios you can have a very recent client, but the operating system or library must be updated. This is increasingly rare as more clients ship with their own trust stores, but it happens frequently enough.

5 Likes

Only the short chain one is actually "official" as in required by the BR to be a test site chaining to Root X1. The "helloworld" site has been around for a long time and currently sends the long chain but Let's Encrypt isn't under any obligation to keep it running or have any particular uptime or anything. That is, I think it only sends the long chain because that's just what whatever client Let's Encrypt uses for it does by default.

4 Likes

Maybe it's just me but...
Wouldn't something like this make life simpler:

  • https://ShortChain.letsencrypt.org
  • https://LongChain.letsencrypt.org
3 Likes

A fun set of these is at

https://badssl.com/

You can click on any of the failing ones if you want to see what a browser error related to a specific problem would look like. :slight_smile:

6 Likes

most of things there(badssl) are expired even when it's supposed to be valid, don't think they are maintained

6 Likes

The only example I think I can see like that at the moment is the EV certificate (extended-validation); do you see others?

(It would still be good for that to be unexpired!)

3 Likes

I just spot-checked a couple and found these 3 with expired dates

https://no-common-name.badssl.com/
https://no-subject.badssl.com/
https://incomplete-chain.badssl.com/

(no-common-name expired almost 3 years ago)

4 Likes

Oh, I see—some of the intentionally-invalid ones are also expired!

6 Likes

I don't think anyone really needs to waste any time on this slack issue, they're just saying (in an awkward way) that they will switch their own certs over to LE with the ISRG Root X1 chain. It's a roundabout way of saying they no longer support old android (without actually saying it) and if anyone else was holding out on updating their trust stores then could they do it now.

Storm in a teacup :slight_smile: everyone actively using the internet has done this already in 2021 and anyone who hasn't can enjoy a little downtime. Some people using webhook posts to the slack API from old servers will still run into issues but they've been notified and it was going to happen eventually anyway. We really need more stuff like this to encourage people (and their systems) to have better maintained trust stores.

8 Likes

That's a great interpretation!

5 Likes

So as a small business who have no IT - do we need to do anything with this email ? We all just use slack app on our laptops and desktops and very limited integrations really if any ?