Keep getting Fake LE Intermediate X1 Issuer

Please fill out the fields below so we can help you better.

My domain is: jira.cspeed.com

I ran this command: I ran a script that I got from http://blog.ivantichy.cz/blogpost/view/74. I believe the pertinent lines are:

#./letsencrypt-auto certonly --standalone --test-cert --break-my-certs -d $mydomain --standalone-supported-challenges http-01 --http-01-port 9999 --renew-by-default --email $myemail --agree-tos
./letsencrypt-auto certonly --standalone -d $mydomain --standalone-supported-challenges http-01 --http-01-port 9999 --renew-by-default --email $myemail --agree-tos

I was originally using the first line, but once it started working, I commented it out and started using the second line. I also updated the passwords seen in the script from “changeit” to a new password. This caused problems with the .keystore file, so I renamed the .keystore file to .keystore2 and just had the script generate a new .keystore file.

It produced this output:

The standalone specific supported challenges flag is deprecated. Please use the --preferred-challenges flag instead.
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for jira.cspeed.com
Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/jira.cspeed.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/jira.cspeed.com/privkey.pem
   Your cert will expire on 2017-11-12. To obtain a new or tweaked
   version of this certificate in the future, simply run
   letsencrypt-auto again. To non-interactively renew *all* of your
   certificates, run "letsencrypt-auto renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

I believe I grabbed the right output here, the rest seems to be from the git command and the keytool commands and starting jira.

My web server is (include version): Apache Tomcat 6.0.20

The operating system my web server runs on is (include version): Ubuntu 12.04.5

My hosting provider, if applicable, is: ???

I can login to a root shell on my machine (yes or no, or I don’t know): No? I can sudo things but I can’t log into the server as root

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

I know both our Ubuntu and Tomcat are very old. We’re going to update soon but we want to figure out how to get https working before doing so

Are you sure Apache is pointing to the right certificate? Can you paste the output of openssl x509 -in /etc/letsencrypt/live/jira.cspeed.com/fullchain.pem -noout -text? (None of this output will display sensitive information, it’s exactly the same information in the public certificate presented to anyone who visits that domain or looks at the certificate transparency logs.)

Speaking of which, I don’t see jira.cspeed.com in the certificate transparency logs, so it appears this was never actually issued. Is this your real domain?

It looks like this advice is very old because the names of letsencrypt-auto, --standalone-supported-challenges, and --renew-by-defaulthave all changed since then (tocertbot-auto ,--preferred-challenges, and --force-renewal`).

Anyway, I agree with @jared.m that it would be useful to see that information, and also you could look at /etc/letsencrypt/renewal/jira.cspeed.com.conf to see if it's still hard-coded to point to the staging server (I'm not sure why the second command wouldn't have changed that).

Maybe it was only ever issued by the staging server and never by the production server? The staging server doesn't log in CT.

Yes, jira.cspeed.com is my real domain (I think - I’m doing all this because I have downtime at work, not because I am well suited to do this). I had to sudo the command to get more than permission denied, but once I did:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            03:05:87:20:d7:69:5f:aa:d2:3e:91:9c:a8:c2:cb:e9:5d:6d
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3
        Validity
            Not Before: Aug 14 19:02:00 2017 GMT
            Not After : Nov 12 19:02:00 2017 GMT
        Subject: CN=jira.cspeed.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:e9:26:d7:7a:52:20:79:7c:ce:9b:36:6c:5e:80:
                    1f:44:37:fd:ce:bf:fc:86:5a:74:3b:f8:15:9c:4f:
                    35:45:15:98:b7:a1:60:4c:27:bd:64:27:2c:e8:37:
                    75:07:f7:3d:7b:34:19:c7:b9:2e:04:e6:e2:67:61:
                    58:30:7f:64:cb:93:04:ef:0f:37:86:80:31:b7:d0:
                    70:cd:0d:7a:e9:17:66:e7:6a:a1:86:e7:dd:2c:8a:
                    18:42:e2:3a:27:98:b5:a3:0a:e3:1c:e7:24:8a:eb:
                    d3:ac:7f:d8:bc:ba:77:66:6f:9f:e5:10:76:9d:b4:
                    9b:14:09:e7:b0:5e:1e:a7:fb:61:5b:a2:8e:78:b5:
                    99:21:bc:22:da:13:4e:39:59:80:c4:cb:63:77:09:
                    2f:76:a4:0f:fe:27:a5:1b:12:07:f1:49:8b:57:44:
                    14:31:17:ae:0b:66:fe:11:a1:10:1e:00:d5:b8:20:
                    1e:87:77:32:e2:ac:3a:25:e6:8f:9d:e7:bf:29:f6:
                    dd:b3:74:e4:be:f0:a1:0b:c4:2e:11:a7:4b:dd:d5:
                    a8:90:77:69:9f:ab:f4:33:00:5c:97:dd:10:2d:6a:
                    aa:fe:6f:0b:05:8f:62:a5:79:f8:a7:96:67:03:0d:
                    a7:3c:f7:e1:96:4f:9e:33:4f:7d:93:64:1b:47:d9:
                    46:1b
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier:
                C1:49:EC:D7:32:2E:D3:5D:8B:A4:16:C9:3D:82:4C:05:A6:F0:A9:8B
            X509v3 Authority Key Identifier:
                keyid:A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1

            Authority Information Access:
                OCSP - URI:http://ocsp.int-x3.letsencrypt.org
                CA Issuers - URI:http://cert.int-x3.letsencrypt.org/

            X509v3 Subject Alternative Name:
                DNS:jira.cspeed.com
            X509v3 Certificate Policies:
                Policy: 2.23.140.1.2.1
                Policy: 1.3.6.1.4.1.44947.1.1.1
                  CPS: http://cps.letsencrypt.org
                  User Notice:
                    Explicit Text: This Certificate may only be relied upon by Relying Parties and only in accordance with the Certificate Policy found at https://letsencrypt.org/repository/

    Signature Algorithm: sha256WithRSAEncryption
         37:60:bd:b0:b7:2b:25:1d:35:b5:35:7b:cf:35:f5:4a:4c:0c:
         97:05:17:66:8f:d9:0d:d7:12:4d:8e:1e:6b:3c:6d:71:26:6d:
         71:fc:25:89:d7:4e:87:30:ed:db:28:2c:be:fd:5a:7b:e7:0d:
         5d:4b:4d:32:0f:da:e7:2f:6a:69:c5:14:5b:f7:04:64:f6:1d:
         29:ef:f8:7e:2a:3e:a3:1c:dd:d4:bc:7f:df:2e:24:7c:17:8b:
         7f:5e:97:61:12:dd:c5:e7:40:5b:55:0e:a5:79:e5:ac:b1:56:
         41:a0:4d:5e:97:3a:34:74:df:80:cf:40:ab:18:e0:c8:8f:4a:
         a6:ec:3f:5d:b8:f0:d2:34:fc:41:8d:bb:df:2a:9b:08:bd:57:
         6a:6a:d0:17:c7:2c:29:56:ec:f1:ad:2d:b9:27:4f:59:80:24:
         ae:e6:36:97:c1:52:a0:62:44:5b:5c:9b:a0:73:f2:4a:02:54:
         63:76:05:65:02:52:79:f2:43:e5:1a:7e:39:1d:ac:82:51:51:
         06:77:08:63:73:7d:23:f6:41:2b:40:a5:a4:01:e3:7b:1d:06:
         63:58:29:dc:50:3e:8f:17:8f:ed:f8:65:f9:b3:9f:01:1d:ac:
         66:13:77:64:f6:a5:b3:48:0c:58:e8:11:50:4c:08:94:56:00:
         3a:f3:4e:79

That looks like a real certificate to me. Can you take a look at your Apache configs and verify that it’s set to use that certificate? It’s also possible you need to reload Apache to force it to use this certificate. Usually this is done by virtue of the fact that Apache must be stopped before and restarted after using --standalone, as Apache and Certbot cannot listen on port 80 simultaneously, but if you’re running on a nonstandard HTTP port, then you may not have reloaded Apache after issuance of the real certificate.

Yeah, I know it’s a little old, but it has comments from earlier this year that indicate it still works, and it was the best thing I could find.

jira.cspeed.com.conf contains:
# renew_before_expiry = 30 days
version = 0.17.0
archive_dir = /etc/letsencrypt/archive/jira.cspeed.com
cert = /etc/letsencrypt/live/jira.cspeed.com/cert.pem
privkey = /etc/letsencrypt/live/jira.cspeed.com/privkey.pem
chain = /etc/letsencrypt/live/jira.cspeed.com/chain.pem
fullchain = /etc/letsencrypt/live/jira.cspeed.com/fullchain.pem

# Options used in the renewal process
[renewalparams]
standalone_supported_challenges = http-01
account = 364054c5dfecc15aca13dddb2982fe4c
http01_port = 9999
authenticator = standalone
installer = None
server = https://acme-v01.api.letsencrypt.org/directory

I have been restarting the web server. In Jira’s install folder in /opt/, there’s /bin/startup.sh and /bin/shutdown.sh which is what I’ve been using to restart the server (again, I assume - I know after running shutdown.sh I can’t access the site at all). Jira runs on 8080 (HTTP) and 8334 (HTTPS). I’m not sure how to check Apache’s configs

There are six certificates in CT for this name today and none before today. So, you’re going to be rate-limited and unable to issue any more certificates like this for a week!

As we saw from the openssl output, your current certificate in /etc/letsencrypt/live/jira.cspeed.com/cert.pem is valid and was issued by the production server, not by the test server. So the problem now is not issuing any more certificates but getting Jira to use the new certificate. This probably calls for debugging or refining your JKS import process to correctly get the data from /etc/letsencrypt/live/jira.cspeed.com into the JKS file that Jira is using. That’s my impression, anyway.

CT = Certificate Transparency? Jared said he couldn’t see any entries. How do I even check?

https://crt.sh/?Identity=jira.cspeed.com&iCAID=16418

1 Like

Yes.

Google's logs lag behind a bit, and the earliest certificate is barely 2 hours old. They only just showed up.

Well, there are a lot of ways, like downloading them yourself. But https://crt.sh/ is good.

https://crt.sh/?q=jira.cspeed.com

They also display the delay:

https://crt.sh/monitored-logs

This was enough to point me in the right direction and everything is up and working now. Thanks!

Regarding the number of certificates I can be issued within a certain time period, the script I have that gets a new certificate is placed in cron.monthly - is renewing once every month going to be a problem?

While once a month will certainly not hit any rate limits, the best practices method for this is to remove the ‘renew-by-default’ flag (also making sure not to have the synonymous ‘force-renewal’ flag set) and set the renewal to run in cron twice daily. Certbot, without the flags mentioned, will only attempt renewal of certificates expiring within 30 days. Running twice a day will help catch intermittent failures early and give the system plenty of chances to attempt renewal at different times of day.

We really encourage a very different approach — our certbot renew script is meant to be run at least once per day, but only attempts to renew a certificate when that particular certificate is less than 30 days from expiry. There are various arguments in favor of this, for example the fact that the certificate authority service itself has intermittent outages (I would estimate that it's only been up around 99.5% of the time, although the people running it may have more accurate numbers), or that a misconfiguration could cause a renewal to fail. In that case, it would be good to have more lead time before the expiry in order to notice and be able to retry or remedy the problem.

Also, if you have many certificates, there can be benefits to spreading out their renewals because of rate limits (although if you have very many certificates, you'll probably need to plan the exact renewal schedule).

Under Let's Encrypt's rules, you're allowed to renew a given certificate about once per week if you feel you really need to.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.