Helloworld.letsencrypt.org can only find certificate with DST X3 loaded

Hi all,

I have a test project using Play’s HTTP client configured to use Letsencrypt certificates. It works fine when I have all the certificates loaded, but JSSE can’t find a certificate path unless I have the Identrust root X3 certificate loaded.

It seems like there’s something wrong about this – shouldn’t it need the “isrgrootx1.pem” certificate to connect?

Will.

The ISRG root certificate is currently not included in most root programs. For that reason, Let’s Encrypt currently defaults to using the issuer certificate cross-signed by IdenTrust, which leads back to DST Root CA X3. Once the ISRG root becomes trusted, this is bound to change - server operators will be encouraged to include both issuer certificates. Right now, this would essentially be wasted bandwidth because the majority of users won’t trust that root certificate.

This sort of makes sense, but the irony is that the DST Root CA X3 certificate isn’t in JDK 1.8,

And the canonical location of DST Root CA X3 is: certificates/trustid/root-download-x3 dot html

which has the root certificate of… DST Root CA X3. So if you’re on Java, you can’t download the Letsencrypt certificates from a server that passes the server identity check.

The only way I’ve found past this is to bundle the root certificate with the application itself, which is a bit silly.

DST Root CA X3 is not loaded into the JDK 1.8 cacerts file, and I can’t find a place that has the “DST Root CA X3” certificate which isn’t hosted from a place that isn’t served over an HTTPS connection that uses that same certificate or the Letsencrypt one.

Can you point me to a well known HTTPS location of root certificates that I can download from?

I’ve found the DST X3 root certificate available on Mozilla’s bug tracker:

https://bugzilla.mozilla.org/show_bug.cgi?id=359069

If you know of a more definitive location, please let me know.

Hello @wsargent,

If you have already a certificate issued by Let’s Encrypt you could check which CA issued the Let’s Encrypt Intermediate Xn certificate.

$ openssl x509 -in /etc/letsencrypt/live/yourdomain/chain.pem -noout -text | grep "CA Issuers"
                CA Issuers - URI:http://apps.identrust.com/roots/dstrootcax3.p7c

Then you get the exact url from IdenTrust to download the root certificate (I don’t know the reason but it is http not https)

Then you can download it:
wget http://apps.identrust.com/roots/dstrootcax3.p7c

If you want to convert it from PKCS7 to PEM format:
openssl pkcs7 -inform der -in dstrootcax3.p7c -print_certs -out dstrootcax3.pem

If you want to view the cert:
openssl x509 -in dstrootcax3.p7c -noout -text

Cheers,
sahsanu

That doesn’t solve the problem – I want to download everything over HTTPS that starts with a trusted root certificate. Downloading from HTTP is just as bad as downloading a root cert from a host using that cert.

Did you not bother looking on identrust yourself?

https://www.identrust.com/certificates/ie.html

1 Like

You can parse the output to get the cert file: https://www.identrust.com/certificates/trustid/root-download-x3.html

You can extract the root cert from this chain file: https://www.identrust.com/doc/SSLTrustIDCAA5_DSTCAX3.p7b

You're not understanding. I want to download the Identrust root certificate using Java.

If I try downloading the path "Downloads and Drivers | IdenTrust" or any other path belonging to the identrust.com domain, then the root certificate isn't listed in JDK 1.8's cacerts file:

Caused by: play.api.libs.ws.ssl.CompositeCertificateException: No trust manager was able to validate this certificate chain: # of exceptions = 1
at play.api.libs.ws.ssl.CompositeX509TrustManager.checkServerTrusted(CompositeX509TrustManager.scala:88)
at sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:936)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:919)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:916)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1369)
at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1124)

I know I can get the root certificate through a browser that has the root certificate bundled. I want to bootstrap it over HTTPS from a well known source that's trusted by Java.

If I use https://bugzilla.mozilla.org/attachment.cgi?id=276893 then I CAN download it using Java's cacerts, because it's signed by Digicert:

certpath: Cert path validation succeeded. (PKIX validation algorithm)
certpath: --------------------------------------------------------------
Found trusted certificate:
[
[
Version: V3
Subject: CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

Key: Sun RSA public key, 2048 bits
modulus: 28559384442792876273280274398620578979733786817784174960112400169719065906301471912340204391164075730987771255281479191858503912379974443363319206013285922932969143082114108995903507302607372164107846395526169928849546930352778612946811335349917424469188917500996253619438384218721744278787164274625243781917237444202229339672234113350935948264576180342492691117960376023738627349150441152487120197333042448834154779966801277094070528166918968412433078879939664053044797116916260095055641583506170045241549105022323819314163625798834513544420165235412105694681616578431019525684868803389424296613694298865514217451303
public exponent: 65537
Validity: [From: Thu Nov 09 16:00:00 PST 2006,
To: Sun Nov 09 16:00:00 PST 2031]
Issuer: CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
SerialNumber: [ 083be056 904246b1 a1756ac9 5991c74a]

So the question is "how do I securely bootstrap a Java client with letsencrypt and download the DST CA X3 root certificate from an officially approved source"?

I’m not aware of a well-known URL for the DST root that’s not using an IdenTrust cert itself.

That being said, I would recommend bundling the root certificate with your application. That’s what operating systems, browsers and all other TLS clients which ship their own trust stores do.

If you don’t want to include the certificate, you could also just store a hash and download the file from some unauthenticated URL, verifying the content with that hash.

The application is sample code, so it’s likely to be forked and passed around. It’s not like a well known binary or an OS.

In that case I would probably go with the hash approach.

Hello @wsargent,

You are right, I didn't understand your request.

If that is the case, bugzilla site is a well known source to download DST Root CA X3 cert. Another well known source that you could use is https://crt.sh. crt.sh is a site to query CTL (Certificate Transparency Logs) created by COMODO. If you want to use it, the url to get DST Root CA X3 cert is https://crt.sh/?d=8395

Cheers,
sahsanu

I am still having issues.

If I go to whatsmychaincert.com and run a check against it, I get the following:

https://whatsmychaincert.com/?helloworld.letsencrypt.org

helloworld.letsencrypt.org is misconfigured. This is the chain it should be using.”

The same with ssllabs:

https://www.ssllabs.com/ssltest/analyze.html?viaform=on&d=helloworld.letsencrypt.org

It seems to be saying that in addition to DST X3 CA Cert, the “Let’s Encrypt Authority X1” certificate must also be in the trust store as well.

So now I have two questions:

  • Where can I find a trusted location for the “Let’s Encrypt Authority X1” certificate?
  • What can I do to ensure that the certificate chain won’t break again in the future?

Can I ask what your aim is ?

helloworld.letsencrpyt.org deliberately has a chain that isn't trusted by all browsers at the moment - but that in no way affects your site.

When you get a certificate for your site, the client will also provide you with the chain, so will be fine. In answer to your questions ...

In what way has it broken for any sites ? When you are issued a certificate you are also provided with the appropriate chain files.

Maybe I'm just not understanding your issue.

Can I ask what your aim is ?

The goal is that, starting with the certificates that are in the standard Java 1.8 cacerts trust store, to be able to get the root and intermediate certificates for letsencrypt, over HTTPS, using those already well known certificates.

If you look at Path #2 for SSL Server Test: helloworld.letsencrypt.org (Powered by Qualys SSL Labs) then you can see that "Let's Encrypt Authority X1" is not in the trust store for SSLLabs, so the certificate chain can't be validated.

Chain of Trust - Let's Encrypt isn't a trusted location to download the intermediate certificates if you're using Java to download them, because you can't validate the certificate chain with a client that doesn't have those intermediate certificates to begin with.

A good URL for "Let's Encrypt Authority X1" is https://crt.sh/?d=9314792 -- it uses a COMODO certificate which is in the JDK, so it's not trying to lift itself up by its bootstraps.

Thanks, That makes things clearer.

I guess the only area of confusion I have left, is why you are using helloworld.letsencrpyt.org - since that isn’t using the same standard set of chains that you would use if you were correctly obtaining a certificate from LE at the moment.

Since you have the crt at crt.sh - isnt’ that an acceptable place to download it from ?

I guess the only area of confusion I have left, is why you are using helloworld.letsencrpyt.org - since that isn't using the same standard set of chains that you would use if you were correctly obtaining a certificate from LE at the moment.

I'm not downloading the certificates from helloworld -- I'm trying to set up a wrapper that makes letsencrypt transparent to Java clients.

Also, you say "correctly obtaining a certificate" -- please see the previous posts detailing how identrust also provides its certs from a site that presents that cert for authenticity. Given that TLS goes to some lengths to establish identity, it's a little strange that the process of establishing identity is essentially "use a browser that already has this" without any thought put into those systems that don't have access to those browsers.

Since you have the crt at crt.sh - isnt' that an acceptable place to download it from ?

That's what I'm doing right now -- the problem is that it's a very manual process and there is no well defined process / location for clients that want to bootstrap in this way. Every time the call to helloworld fails, I have to look at the chain and go spelunking for a valid cert location.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.