Can not issue X3 chain certificate even with the --preferred-chain "DST Root CA X3" option

We're not able to issue certificates for any lifetime other than 90 days, and we don't have any way to limit issuance to just your company. I'm sorry to hear this is happening, around the holidays in particular, I wish we could help but I don't see a way.

1 Like

Josh

I understand the 90 days, I just thought that it might be easier to request for a lower expiry date to fall inside the lifetime. We have misunderstood the March deadline which is our fault we thought we had until that point to resolve this, not that we could not get any renewed certificates beyond today.

I appreciate the other point on limiting it to us also. I am at a real loss to try to suggest technically what you can do but I thought there must be a way that it could be done technically, even within a small time window where such a certificate generation would be limited or perhaps using some tools internally with whatever information you might need from us.

Would it be possible to speak please?

Thanks
Mike

1 Like

We have been trying to think what we can do alone on this the whole day and have hit a brick wall. If anyone has any good ideas that cannot involve the devices we’re all ears

1 Like

I don't understand your setup.

Why these clients are so limited?

If every client update requires a connection to a server with a not working certificate and if there isn't something like a kill switch to ignore the expired certificate or to switch to another server, the general update configuration is fatal.

But if that situation exists, a manual update is required. With additional code to start the update.

1 Like

@JuergenAuer Thank you for responding. Our client devices do not have that non ssl fallback so that’s why we are stuck with the broken and expired cert chain.

1 Like

I don't say "non ssl fallback". An expired certificate with the correct domain name is better then http.

Then create and deploy such code -> looks like manual actions (one per device) are required.

1 Like

They first establish an ssl connection with the server and after handshake they then validate the certificate which also checks the chain which is where a newly generated cert fails as the intermediate check fails (now R3 and expecting X3, which we know is our error). If this ssl fails the socket is closed and that’s the end of the session.

1 Like

So you have two options:

  • Deploy the new intermediate certificate
  • Change that code
1 Like

I'm guessing neither of those options is available at this moment, as the devices can't connect at all. So the company has lost all remote communication with those devices.

I don't know how many devices we're talking about, but I'm afraid it'll require manual/physical updating the devices.. Or swap out the old ones for new ones.

1 Like

I wrote that already.

Certificate update or code update.

1 Like

Without any technical assistance or help that Lets Encrypt might be able to offer I think that is the conclusions we are reaching too. But we were still hopeful a cert with the old chain could be generated for an expiry in the future when we can then fix the bug

1 Like

@josh Can you explain what would be the problem for you guys if you were to allow what you said for a period of time but not specifically for us?

That wouldn't technically prevent us from issuing from X3 but most of our subscribers do not want a certificate with a lifetime that outlives the intermediate and we have no way to issue from X3 specifically to you

Would it be that since no one else has this issue, all others would not request from X3 anymore.

Sorry I don’t quite understand the concern you have but I do understand the rationale for not being able to use any different tools to generate just for us.

We are perhaps clutching at straws but as you identify, this is our problem but we cannot solve it by any other server side method than our requests to you.

Regards
Mike

1 Like

Most people cannot consume X3 certs any more because the cross-sign they rely on will expire before the certs expire. All certs are 90 days, we have no way to change that, and there are only 84 days until cross sign expiration.

Because of this we cannot issue from X3 any more by default, even for a short period of time. We have no way to issue from X3 specifically to you either.

@josh So the fact that we can consume it and need to to fix our problem is that if others requested a certificate they could/would get such a certificate and then couldn’t use it. But they could request a certificate with the correct R3 intermediate as their solution and as could we once we fix our device side bug in a matter of days plus rollout?

Where do your devices get their time from?
If you can control TIME, you can set their clocks back a month and just use an old cert to have them connect.

4 Likes

Chiming in quick!

So if we look at our issuance, it's about 1.5M certificates/day. If we were to move back to the cert that you need with a few days plus rollout, we would issue likely around 9M certificates in that time. For these 9M certificates, they would need to reissue once we have the R3 intermediate back in place. That's a lot of breakage for a lot of folks. In addition, we are a privacy-respecting organization and do not require e-mail addresses to use Let's Encrypt certificates. Meaning that even if we did change back for 6 days, we couldn't get ahold of all of the subscribers we would be breaking in March.

I'm not highly technical and am more software than hardware based, but I think, with the help of our community (thanks @rg305!) we should try and find a workaround if y'all are up for some brainstorming.

2 Likes

@rg305 thanks for the feedback idea. We are looking into this idea too as they get time over UDP from another source but it will almost certainly come after the cert check happens. We are digging around though to check.

3 Likes

@jple I appreciate the chime in. I wasn’t as clear in my post perhaps. We only need the switch made for the period of time for us to make the request for a cert, so if coordinated could be as low as a few minutes perhaps

3 Likes

Also one more thing! I would love to make sure this doesn't happen again for other folks in IoT and around the Web. @meddle0x53 and @dokie I know you are in the midst of a technical challenge right now, but I would love to get some time with y'all after the beginning of the year to talk through what docs you looked at when setting up your environment and how we can make those more clear or add more information to be sure this doesn't happen in the future.

6 Likes

@jple of course we would welcome that.

2 Likes