IIS 8.5 building incorrect chain with Lets Encrypt Authority X3

I should also mention, I copied the certificates to a new server, and that server sent out the correct X3 certificate chain.

So, in other words, it doesn’t seem to be a problem with the browser/client, it doesn’t seem to be a problem with the certificates/LE client, which to me leaves only the server software itself as the issue.

I’ve used this client for generating certs - https://github.com/Lone-Coder/letsencrypt-win-simple

However, others have also used it (not on IIS 8.5) and been able to get things back to working just fine once removing the X1 intermediate cert - https://github.com/Lone-Coder/letsencrypt-win-simple/issues/177

Plus, as noted above, the certs work just fine on a new server.

Looking at the issue ( and the links you provide, thanks) it would seem that the issue is related to ACMESharp / windows - and the authors are looking to resolve that issue.

I assume you mainly just want to fix your one server.

On that server, what intermediate cert is within the configuration itself ? ( not what you see when remotely checking with openssl, rather what about the actual file on the windows server ) From what you say they are X3 on there,

Are you happy to provide your domain name ?

Looking at the issue ( and the links you provide, thanks) it would seem that the issue is related to ACMESharp / windows - and the authors are looking to resolve that issue.

From my tests, I don't believe this is the case at all. Like I said, if I take the exact certs that are generated by ACMESharp, and put them on a different Windows server, they work just fine. This purely seems to be an issue with the existing server caching the old intermediate.

On the server cert store itself (locally), it happily displays X3 as the intermediate in the cert chain.

At this point I'm 99% certain the issue is with some sort of cert caching mechanism in Server 2012 R2 and/or IIS 8.5. Not to say that the issue can't be fixed elsewhere (for example, having a new intermediate issued, with completely different key). Just not sure LE will do that to fix an issue that is seemingly happening with only one particular type of server setup.

As far as my domain, it is behind CloudFlare and so can't be tested publicly (it will return the CloudFlare certs, not the LE ones). Access to origin server requires client auth certs, which I'd rather not give out :stuck_out_tongue:. The only reason I'm trying to get this working again is so that CloudFlare's SSL strict mode will accept the certificates as valid again (since right now they show as invalid due to X1 being in the chain presented).

Are you happy to PM the IP address or anything ?

Sorry, would rather not, since like I said would have to also provide my client-cert for access authorization.

Thanks for the offer to help either way!

Edit: Just an update, I temporarily bypassed CloudFlare and ran the SSL labs test here - SSL Server Test (Powered by Qualys SSL Labs)

In the results it shows an error -

Chain issues Incomplete, Extra certs

Essentially it shows that the server is distributing the X1 certificate still, in addition to X3. Which, seems to be the root of the problem. Some clients deal with this more gracefully than others. Why the server is still hosting out that X1 intermediate, and where it is pulling it from, I have no idea still.

Hi,

I think you are right: I just created a new certificate for one domain which I deployed on a Windows Server 2016 TP4 (IIS 10.0) Server, which previously had a Let’s Encrypt certificate with the X1 intermediate. With both the X1 and the X3 intermediate present, the server sent the X1 intermediate, which leads to an Error in Firefox, but works on IE and Chrome on Windows (since they use SChannel to verify the certificate chain).

After I deleted the X1 certificate on the server’s intermediate certificate store (opening mmc.exe, adding the certificate snap-in for “computer account”, then opening the “Intermediate Certification Authorities” and removing the X1 cert there), IIS still served it, so I had to restart IIS to get rid of it.

After that, IIS served the correct X3 certificate, and all browsers (Firefox, IE, Chrome) then displayed the correct X3 intermediate when I opened the page again. So in that case the clients didn’t seem to buffer the X1 intermediate cert (or buffered it but didn’t use it).

1 Like

I’m glad some people are managing to solve this issue on their servers.

On my server the X1 certificate keeps being served. I’ve tried all the steps in the post above. The X1 certificate is nowhere to be found on my server (as far as i can tell…).

I agree the the issue seems to be a Windows Server issue more than a LE issue. still. this is very anoying and will become an big problem as the old certificates on some production machines get refreshed…

Our (Win 2012) server also suffers of this weird behavior, X1 cert in nowhere to be found on the server, i searched using this command, which got no results:

Get-ChildItem -Path cert: -Recurse | Where-Object {$_.Subject -eq "CN=Let's Encrypt Authority X1, O=Let's Encrypt, C=US"}

This second command finds the X3 cert just fine, I get results in both the CurrentUser\CA and LocalMachine\CA stores.

Get-ChildItem -Path cert: -Recurse | Where-Object {$_.Subject -eq "CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US"}

Cleaned up the whole certificate store, reinstalled all certificates and still IIS (8.x) serves the X1 cert to clients, i’m clueless how to proceed now :frowning:

Hello. I see and read topic very carefully a 4 days ago, because I have exactly same problem. Is there any fix provided? We have a very unpleasant situation - we cant login via mobile devices in our web system.

Same here, Chrome and IE work just fine, but the checking sites like SSLabs gives an chain error. Also Mozilla Firefox doesn’t work. We just recently went for SSL because it was free. So was quite happy that it works so easily.

We work on Windows Server 2012 R2 with IIS 8.5.

Similar issue but for me Chrome on Android displays the invalid error.
Running sites on Windows 2012R2 IIS 8.5

Because of the limit of the number of images per post for new users :unamused: I need to post several replies.

Last image can be found here because of the post limit for new users I cannot post the last one :unamused:

The situation is getting too confusing for me. In that topic:

https://github.com/Lone-Coder/letsencrypt-win-simple/issues/177#issuecomment-207996172

Have a discussion for same problem. There is user which say that the only action that he make is to remove old X1 intermediate certificate and everything is work fine. So - there IS a way to make all in order without full IIS rebuild, but honestly - i can’t find any logic now.

@rmhrisk confirmed to me that IIS, by way of CryptoAPI, does chain building based on the Authority Key Identifier / Subject Key Identifier fields, even if Issuer / Subject don’t match. Because Let’s Encrypt Authority X1 and Let’s Encrypt Authority X3 share the same Subject Key Identifier, IIS may incorrectly build a chain if it has Let’s Encrypt Authority X1 anywhere in its cache.

There are multiple levels at which resources may be cached. This blog post describes how to clear the CryptNet cache: https://unmitigatedrisk.com/?p=23. Note that one of the options for clearing it is certutil -URLcache * delete, which is also mentioned in the ServerFault thread, so I’m assuming @actyler1001 has already tried it. @rmhrisk, can you comment on whether there is a significant difference between certutil -URLcache * delete and the longer form of commands you list in your blog post?

I’ve tried using the certutil -URLcache * delete command multiple times, without any success (after stumbling on that SF thread).

Just tried also manually clearing the folders mentioned in the blog post you linked, without any change (even after reboot). Maybe someone else will have luck though?

Did you try rebooting afterwords? Cache may be in memory also.

You can also see what cert is being used : http://unmitigatedrisk.com/?p=297

After log creation - it’s obviously that there is problem with some sort of cache. Here is XML output from Build chain event:
`-

  • 11 0 4 11 2 0x4000000000000003 3 Microsoft-Windows-CAPI2/Operational WIN-T2ACO5T9J52
  • `

I have NO any idea from where come this
<Certificate fileRef="3EAE91937EC85D74483FF4B77B07B43E2AF36BF4.cer" subjectName="Let's Encrypt Authority X1" />

@_xentia, if you search for the “3EAE91937EC85D74483FF4B77B07B43E2AF36BF4.cer” file