Only being issued with cert with example.org

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:
Some of the domains that I have are is
https://kibana.tanscloud.com
I ran this command:
I use letsencrypt in a container that has traefik and it is producing certificates with example.org. and it is not being validated by letsencrypt.
It produced this output:

My web server is (include version):

The operating system my web server runs on is (include version):

My hosting provider, if applicable, is:
I'm hosting my own
I can login to a root shell on my machine (yes or no, or I don't know):
yes
I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
no
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): i use traefik in kubernetes which has the letsencrypt in traefik.

I am wondering whether this is because of the issue that is going at the moment with the secondary certvalidation?

In my attempts to troubleshoot it I attempted to rename the acme.json. I wonder if I have caused an issue to get a new cert? Many thanks.

I see that "traefik" responds to http and https.
I'm not too familiar with it.
Can you show the configuration, and how it handles certificates?

1 Like

Thank you for the reply.

I do get a certificate but it is issued from example.org.

My setting are:
[ping]
entryPoint = "http"
[kubernetes]
[traefikLog]
format = "json"
[acme]
KeyType = "RSA4096"
email = "xxxxxxx@gmail.com"
storage = "/acme/acme.json"
entryPoint = "https"
onHostRule = true
acmeLogging = true
[acme.httpChallenge]

We just supply an ingress and Lets Encrypt handles the rest. I did however create a wrong host entry on the ingress and because I use http challenge I may have triggered a setting or something?

2 Likes

So I restored my previous acme.json as i have a backup. My existing certificates that were generated previously are now working, but any new certs are now having issues.....

Can you show how traefik handles the inbound http and https requests?

I am still picking it up myself but Here is a diagram on how it works. I have a router that portforwards 443 and 80 to traefik. everytime I create a ingress for my container in kubernetes traefik creates a frontend. As soon as traefik creates the front end letsencrypt spins up a certificate and does it magic and authentication.

I am getting a certificate but it is being signed by example.org instead of my my domain name.

1 Like

I wonder if it is related to this?

LOVE the diagram!

That degradation can't possibly cause traefik to serve the wrong cert.
That said, I dont see the wrong cert:
openssl s_client -connect kibana.tanscloud.com:443 -servername kibana.tanscloud.com
returns:

CONNECTED(00000190)
depth=1 C = US, O = Let's Encrypt, CN = R3
verify error:num=20:unable to get local issuer certificate
Server did acknowledge servername extension.
---
Certificate chain
 0 s:/CN=kibana.tanscloud.com
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGMDCCBRigAwIBAgISBCvL02vuFPsoMlsvDhnDcXfWMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMDEyMjQyMTA5MzNaFw0yMTAzMjQyMTA5MzNaMB8xHTAbBgNVBAMT
FGtpYmFuYS50YW5zY2xvdWQuY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIIC
CgKCAgEAySmSuc+nS0nYJVodLtE0jBiRUNVEMVY74GRSMcg0rD07qlUTV1V23pUM
1xm/OJM4I/vqLVCFDpmfznn7bWmxzEw+KqTnsgRa32pNTBvk+T1McKbA8QOCUZLP
d2urkXjb3dck7kbyAT0C2g7yXtdqvz4HfL5nEstoPk8UqxJ6R1FAfHw0COCGxtxA
OSTrpcuMb2rOwQZJduXeLbowJo4ISdHsuKrIjnXEO3ejAFyoxcHKLfdGIAavrvnG
REQWYL5cnRDM03DwVGKBQ0FN2/72DMXcEm+TZyjUsleNgiAclWlV6X4E4j95gzxw
6SDmf1J0lYZxJWdMkthBtYhHHtR7MPkc7A7EyzNlx6nqQIjYt6Au8TQwhQhBW9WL
K/PYJ5v0NujckCypr4/w6rkxJI0UO6Kv8XxnZao5m8SZuu5t3nKtMQ236BaCvoSX
9ZLZDjh74AAaLEkM9FI34xvzHs+V0caAbv5Qsud8NA6hsy1vsIpfg8i5p+8tbIR7
xaUD4I892UMLn6dI8eC5RwL3kiYaIMmACxXY1+e1r8qwX1Ht8GgoV3USowp0pFeU
sLMYshmkFHoXwlCJMeFTsi/jxWnlCV7tiaA3AAGWKLp35gaYgDX6qt+psakaPodg
CXRhCN3Fpp7V1H4xbGrXXWsbHVA3mBM0xM81R5ivl5TvZAs7bD8CAwEAAaOCAlEw
ggJNMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUH
AwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQU/RLPNR8LnaiGRZDdoay91BwUBOgw
HwYDVR0jBBgwFoAUFC6zF7dYVsuuUAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBH
MCEGCCsGAQUFBzABhhVodHRwOi8vcjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKG
Fmh0dHA6Ly9yMy5pLmxlbmNyLm9yZy8wHwYDVR0RBBgwFoIUa2liYW5hLnRhbnNj
bG91ZC5jb20wTAYDVR0gBEUwQzAIBgZngQwBAgEwNwYLKwYBBAGC3xMBAQEwKDAm
BggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwggEGBgorBgEE
AdZ5AgQCBIH3BIH0APIAdwCUILwejtWNbIhzH4KLIiwN0dpNXmxPlD1h204vWE2i
wgAAAXaWzM9wAAAEAwBIMEYCIQDtAUAA4R4IHvf0HLCAVOpDXa5L4x13oUSSPwmV
EdRfkgIhAJpnz9pmZBlw7VYF/Enf+WfnAyM3GKnXJBT2LMiG8y+1AHcAfT7y+I//
iFVoJMLAyp5SiXkrxQ54CX8uapdomX4i8NcAAAF2lszPlwAABAMASDBGAiEA3LQO
rwJp8qzrodlK0bl2xEcFM72FjhXdN87dpRSru+4CIQDF+bA7g1I36KasVc03NpuE
IsRsn5d2QuNoo56W38h8OjANBgkqhkiG9w0BAQsFAAOCAQEACxhV84ItAQ40XbMw
UF7qNCsnoc8bgN4aJH8rkLLt2wC86atc40tT1fqsXzaCNkij1qbfeUnIO6Rr+siL
W2JM0TX1eYHwnwivySrbqebha/ZGUtDAEeqSmksu5YNjYyJKyDTn3+dGdxnABr1x
w2/SAELG7+6jdlVUz18qN3W5qkJ75bUBU8jbV46OqUhQvr2QNI2iep7D+vkG5z/q
DVppOK/iRGm71BiSy7SXF0KFMP67TDhXRFQLgQyHYMYnRL498Fjtupxbvqjq7CZA
b6vGWPx5xYRI0nGIjKkuDNBM1Hd9YC8Vv6/46uF7/o8ZjV1rUeKLae4uKkcLxEwM
jW8umw==
-----END CERTIFICATE-----
subject=/CN=kibana.tanscloud.com
issuer=/C=US/O=Let's Encrypt/CN=R3
---
...

I do see the *.example.com only when SNI is NOT used [or a unmatched SNI name is used]:
Like:
openssl s_client -connect kibana.tanscloud.com:443 -servername non.existent.domain

openssl s_client -connect kibana.tanscloud.com:443
CONNECTED(00000190)
depth=0 C = US, ST = Colorado, L = Boulder, O = ExampleCorp, OU = IT, CN = *.example.com, emailAddress = admin@example.com
verify error:num=18:self signed certificate
verify return:1
depth=0 C = US, ST = Colorado, L = Boulder, O = ExampleCorp, OU = IT, CN = *.example.com, emailAddress = admin@example.com
verify error:num=10:certificate has expired
notAfter=Oct 24 21:09:52 2017 GMT
verify return:1
depth=0 C = US, ST = Colorado, L = Boulder, O = ExampleCorp, OU = IT, CN = *.example.com, emailAddress = admin@example.com
notAfter=Oct 24 21:09:52 2017 GMT
verify return:1
---
Certificate chain
 0 s:/C=US/ST=Colorado/L=Boulder/O=ExampleCorp/OU=IT/CN=*.example.com/emailAddress=admin@example.com
   i:/C=US/ST=Colorado/L=Boulder/O=ExampleCorp/OU=IT/CN=*.example.com/emailAddress=admin@example.com
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIEmzCCA4OgAwIBAgIJAJAGQlMmC0kyMA0GCSqGSIb3DQEBBQUAMIGPMQswCQYD
VQQGEwJVUzERMA8GA1UECBMIQ29sb3JhZG8xEDAOBgNVBAcTB0JvdWxkZXIxFDAS
BgNVBAoTC0V4YW1wbGVDb3JwMQswCQYDVQQLEwJJVDEWMBQGA1UEAxQNKi5leGFt
cGxlLmNvbTEgMB4GCSqGSIb3DQEJARYRYWRtaW5AZXhhbXBsZS5jb20wHhcNMTYx
MDI0MjEwOTUyWhcNMTcxMDI0MjEwOTUyWjCBjzELMAkGA1UEBhMCVVMxETAPBgNV
BAgTCENvbG9yYWRvMRAwDgYDVQQHEwdCb3VsZGVyMRQwEgYDVQQKEwtFeGFtcGxl
Q29ycDELMAkGA1UECxMCSVQxFjAUBgNVBAMUDSouZXhhbXBsZS5jb20xIDAeBgkq
hkiG9w0BCQEWEWFkbWluQGV4YW1wbGUuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEAtuJ9mww9Bap6H4NuHXLPzwSUdZi4bra1d7VbEBZYfCI+Y64C
2uu8pu3aU5sauMbD97jQaoyW6G98OPreWo8oyfndIctErlnxjqzU2UTV7qDTy4nA
5OZeoReLfeqRxllJ14Via5QdgywGLhE9jg/c7e4YJznh9KWY2qcVxDuGD3iehsDn
aNzV4WF9cIfms8zwPvONNLfsAmw7uHT+3bK13IIhx27fevquVpCs41P6psu+VLn2
5HDy41thBCwOL+N+albtfKSqs7LAs3nQN1ltzHLvy0a5DhdjJTwkPrT+UxpoKB9H
4ZYk1+EDt7OPlhyo3741QhN/JCY+dJnALBsUjQIDAQABo4H3MIH0MB0GA1UdDgQW
BBRpeW5tXLtxwMroAs9wdMm53UUILDCBxAYDVR0jBIG8MIG5gBRpeW5tXLtxwMro
As9wdMm53UUILKGBlaSBkjCBjzELMAkGA1UEBhMCVVMxETAPBgNVBAgTCENvbG9y
YWRvMRAwDgYDVQQHEwdCb3VsZGVyMRQwEgYDVQQKEwtFeGFtcGxlQ29ycDELMAkG
A1UECxMCSVQxFjAUBgNVBAMUDSouZXhhbXBsZS5jb20xIDAeBgkqhkiG9w0BCQEW
EWFkbWluQGV4YW1wbGUuY29tggkAkAZCUyYLSTIwDAYDVR0TBAUwAwEB/zANBgkq
hkiG9w0BAQUFAAOCAQEAcGXMfk8NZsB+t9KBzl1Fl6yIjEkjHO0PVUlEcSD2B4b7
PxnMOjdmgPrauHb9unXEaL7zyAqaD6tbXWU6RxCAmgLajVJNZHOw45N0hrDkWgB8
EvZtQ56ammwC1qIhAiA6390D3Csex7gL6nJo7kbr1YWUG3zIvoxdz8YDrZNeWKLD
pRvWen0lMbpjIRP4XZsnC45C9gVXdh3LRe1+wyQq6h9QPiloxmD6NpE9imTOn2A5
/bJ3VKIzAMudeU6kpyYlJBzdG1uaHTjQOWosGiweCKVUXF6UtisVddrxQth6ENyW
vIFqaZx84+DlSCc93yfk/GlBt+SKG46zEHMB9hqPbA==
-----END CERTIFICATE-----
subject=/C=US/ST=Colorado/L=Boulder/O=ExampleCorp/OU=IT/CN=*.example.com/emailAddress=admin@example.com
issuer=/C=US/ST=Colorado/L=Boulder/O=ExampleCorp/OU=IT/CN=*.example.com/emailAddress=admin@example.com
---
...

That's because I restored the acme.json. Try https://tautulli.tanscloud.com/

I tested by recreating the acme.json file. When i did recreate the acme.json file, the Web sites that used to work now come back as example.org. When I restore the acme.json file that worked all the existing certs work but the newly generated one does not...

Can you show the json file that fails?
[If there is anything private in it, please obfuscate, or redact, that info - before posting]

There is certificate details with my domain details. Shall I still send? Or can I send to you?

If it contains the private key, then NO.
Don't ever send a private key to anyone or post it anywhere.
Simply cut that part out and replace it with:
"this part contains my private key"

1 Like

This very telling. I backed up my acme.json , nerfed it and restarted the container.

I get this in my container logs
{"level":"debug","msg":"Try to challenge certificate for domain [requests.tanscloud.com] founded in Host rule","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Try to challenge certificate for domain [codeserver.tanscloud.com] founded in Host rule","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Try to challenge certificate for domain [traefik.tanscloud.com] founded in Host rule","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Try to challenge certificate for domain [gateway.tanscloud.com] founded in Host rule","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Try to challenge certificate for domain [radarr.tanscloud.com] founded in Host rule","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Try to challenge certificate for domain [kibana.tanscloud.com] founded in Host rule","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Try to challenge certificate for domain [sonarr.tanscloud.com] founded in Host rule","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Try to challenge certificate for domain [tautulli.tanscloud.com] founded in Host rule","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Try to challenge certificate for domain [artifacts.tanscloud.com] founded in Host rule","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Looking for provided certificate(s) to validate ["artifacts.tanscloud.com"]...","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"No ACME certificate generation required for domains ["artifacts.tanscloud.com"].","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Looking for provided certificate(s) to validate ["requests.tanscloud.com"]...","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"No ACME certificate generation required for domains ["requests.tanscloud.com"].","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Looking for provided certificate(s) to validate ["codeserver.tanscloud.com"]...","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"No ACME certificate generation required for domains ["codeserver.tanscloud.com"].","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Looking for provided certificate(s) to validate ["traefik.tanscloud.com"]...","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"No ACME certificate generation required for domains ["traefik.tanscloud.com"].","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Looking for provided certificate(s) to validate ["gateway.tanscloud.com"]...","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"No ACME certificate generation required for domains ["gateway.tanscloud.com"].","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Looking for provided certificate(s) to validate ["radarr.tanscloud.com"]...","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"No ACME certificate generation required for domains ["radarr.tanscloud.com"].","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Looking for provided certificate(s) to validate ["kibana.tanscloud.com"]...","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"No ACME certificate generation required for domains ["kibana.tanscloud.com"].","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Looking for provided certificate(s) to validate ["sonarr.tanscloud.com"]...","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"No ACME certificate generation required for domains ["sonarr.tanscloud.com"].","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"Looking for provided certificate(s) to validate ["tautulli.tanscloud.com"]...","time":"2021-02-10T09:15:50Z"}

2/10/2021 10:15:50 PM {"level":"debug","msg":"No ACME certificate generation required for domains ["tautulli.tanscloud.com"].","time":"2021-02-10T09:15:50Z"}

and this in my acme.json:

{
"Account": {
"Email": "weiyen.tan@gmail.com",
"Registration": {
"body": {
"status": "valid",
"contact": [
"mailto:weiyen.tan@gmail.com"
]
},
"uri": "https://acme-v02.api.letsencrypt.org/acme/acct/112280964"
},
"PrivateKey": private key
"KeyType": "4096"
},
"Certificates": null,
"HTTPChallenges": {},
"TLSChallenges": {}
}

Apologies about formatting. I am a bit new to the interface.

1 Like

I'm not familiar with how this should be setup to work.
I see minor differences in that some domains are enclosed in quotes and some are not.
But they seem to come from different logged messages.

The first group of messages shows:
Try to challenge certificate for domain [{domain}] founded in Host rule
[not sure if this is just a log entry for a routine action or an actual error/concern]

The second group of messages shows:
Looking for provided certificate(s) to validate
&
No ACME certificate generation required for domains
[Not sure but it sounds like it doesn't need to even try to renew the checked domain - probably not close enough to expiration yet]

But, again, I'm not a traefik expert.

As for the formatting, if you enclose the "difficult text" within three backticks - that usually does the trick.
Like:

```
funky text with much
indents and such
```

Ahh those entries you mention is the container spinning up. Https://tautulli.tanscloud.com never got a proper cert. What I will do is to flick back to the old acme.json but try a new host name. I suspect that because I tried to get the cert too many times let's encrypt is not giving a new one.

ok so i loaded in a newhost brought in the necessary values:


{"level":"debug","msg":"Domains [\"plexmonitoring.tanscloud.com\"] need ACME certificates generation for domains \"plexmonitoring.tanscloud.com\".","time":"2021-02-10T10:12:32Z"}

{"level":"debug","msg":"Loading ACME certificates [plexmonitoring.tanscloud.com]...","time":"2021-02-10T10:12:32Z"} 
{"level":"info","msg":"legolog: [INFO] [plexmonitoring.tanscloud.com] acme: Obtaining bundled SAN certificate","time":"2021-02-10T10:12:32Z"} 

we can see that its loading in the certs. but wheni browse the website it says it is insecure....

Is there anything marked against my username in letsencrypt?

It also doesn't have a DNS entry:

 dan@Dan-Hack-Mini  ~  curl https://tautulli.tanscloud.com/
curl: (6) Could not resolve host: tautulli.tanscloud.com

Codeserver works. Plexmonitoring works. Radarr works, but it doesn't redirect http->https. Same with traefik, gateway, and in fact everything else except tautulli--which fails because of the aforementioned DNS problem. Everything else has a cert, and serves the right cert, if you specify https, but none of them redirect to https.