Get https://acme-staging-v02.api.letsencrypt.org/directory: dial tcp: i/o timeout - kubernetes cert-manager

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: blog.paloitcloud.com.sg

I ran this command: kubectl describe clusterissuer letsencrypt-staging

It produced this output:

Blockquote
root@DESKTOP-UCLGSHT:~# kubectl describe clusterissuer letsencrypt-staging
Name: letsencrypt-staging
Namespace:
Labels: io.cattle.field/appId=cert-manager
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"certmanager.k8s.io/v1alpha1","kind":"ClusterIssuer","metadata":{"annotations":{},"name":"letsencrypt-staging","namespace":"...
API Version: certmanager.k8s.io/v1alpha1
Kind: ClusterIssuer
Metadata:
Creation Timestamp: 2018-08-30T03:29:04Z
Generation: 1
Resource Version: 451839516
Self Link: /apis/certmanager.k8s.io/v1alpha1/clusterissuers/letsencrypt-staging
UID: d88a1df0-ac04-11e8-bf31-d4ae52bc8b57
Spec:
Acme:
Email: sg-hive-tt@palo-it.com
Http 01:
Private Key Secret Ref:
Key:
Name: letsencrypt-private-key
Server: https://acme-staging-v02.api.letsencrypt.org/directory
Status:
Acme:
Uri: https://acme-staging-v02.api.letsencrypt.org/acme/acct/6834096
Conditions:
Last Transition Time: 2019-07-30T04:06:19Z
Message: The ACME account was registered with the ACME server
Reason: ACMEAccountRegistered
Status: True
Type: Ready
Events:
Type Reason Age From Message

Warning ErrVerifyACMEAccount 40m cert-manager Failed to verify ACME account: context canceled
Warning ErrVerifyACMEAccount 36m cert-manager Failed to verify ACME account: Get https://acme-staging-v02.api.letsencrypt.org/directory: dial tcp: i/o timeout
Warning ErrInitIssuer 36m cert-manager Error initializing issuer: Get https://acme-staging-v02.api.letsencrypt.org/directory: dial tcp: i/o timeout
Warning ErrVerifyACMEAccount 33m cert-manager Failed to verify ACME account: context canceled
Warning ErrVerifyACMEAccount 13m cert-manager Failed to verify ACME account: context canceled
Warning ErrVerifyACMEAccount 4m29s cert-manager Failed to verify ACME account: context canceled
Warning ErrInitIssuer 4m29s cert-manager Error initializing issuer: context canceled

Blockquote

I already tried to add nameserver 8.8.8.8 and 8.8.4.4 and add an entry to /etc/hosts to the cert-manager pod but still no luck.. Seems our network is okay as well..

Pls help.

Thanks!

2

Hi @mariesheilag,

May I ask what version of cert-manager you’re using?

1 Like
3

The image version of the cert-manager is 0.3.0. Hope this helps.

root@DESKTOP-UCLGSHT:~# kubectl -n cert-manager describe pods cert-manager-cert-manager-78df5474b5-pj84x
Name: cert-manager-cert-manager-78df5474b5-pj84x
Namespace: cert-manager
Node: node8/10.0.0.28
Start Time: Tue, 30 Jul 2019 12:05:59 +0800
Labels: app=cert-manager
pod-template-hash=3489103061
release=cert-manager
Annotations:
Status: Running
IP: 10.42.2.192
Controlled By: ReplicaSet/cert-manager-cert-manager-78df5474b5
Containers:
cert-manager:
Container ID: docker://08c8fef8cfead7bc2b62138590c21437ae38642beab8a449618d202be617c705
Image: Quay
Image ID: docker-pullable://quay.io/jetstack/cert-manager-controller@sha256:16572331caf8906867fc997d1f1ef4637edb94f2808c1006221fd6f5a6ed9534

1 Like
4

Hi @mariesheilag,

I don't know whether this is related to your problem, but cert-manager versions have serious bugs in their Let's Encrypt integration—so serious that Let's Encrypt is planning to block everything prior to 0.8.0 this autumn.

Since this blocking hasn't been implemented yet, it's not itself the reason for the error message that you're seeing, but this is an important indication that your cert-manager should be updated to a newer version.

1 Like
5

(To me, the error itself looks as though that container doesn’t have working outbound Internet access, even though the host machine that it’s running on does. You could check this by running curl on various sites from inside that container, including the Let’s Encrypt API URL that was mentioned in the error.)

2 Likes
6

Unfortunately, the curl is not installed inside the container. I hope the below results will help.

/ $ nslookup acme-staging-v02.api.letsencrypt.org
nslookup: can't resolve '(null)': Name does not resolve

Name: acme-staging-v02.api.letsencrypt.org
Address 1: 184.50.102.158 acme-staging-v02.api.letsencrypt.org
/ $ curl
sh: curl: not found
/ $ telnet acme-staging-v02.api.letsencrypt.org 443
^C
Console escape. Commands are:

l go to line mode
c go to character mode
z suspend telnet
e exit telnet
/ $

Also.. auto-renew of cert is not working because of this. The weird thing is, if I create a new ingress.. everything is working fine and sometimes not.

7

Do you have ping inside that container? Could you ping 8.8.8.8 and also the IP address of the Let’s Encrypt API endpoint that you found with the nslookup?

I’m about 95% sure that this is a totally different part of Let’s Encrypt’s infrastructure and also can’t be related to your problem, but there is an ongoing routing problem reported currently:

https://letsencrypt.status.io/pages/incident/55957a99e800baa4470002da/5d40614b3df4b70c69ed16dd

8

Kindly see below output.

/bin $ /bin/busybox ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
ping: permission denied (are you root?)
/bin $ /bin/busybox ping acme-staging-v02.api.letsencrypt.org
PING acme-staging-v02.api.letsencrypt.org (184.50.102.158): 56 data bytes
ping: permission denied (are you root?)
/bin $ nslookup acme-staging-v02.api.letsencrypt.org
nslookup: can't resolve '(null)': Name does not resolve

Name: acme-staging-v02.api.letsencrypt.org
Address 1: 184.50.102.158 acme-staging-v02.api.letsencrypt.org
/bin $ nslookup 8.8.8.8
nslookup: can't resolve '(null)': Name does not resolve

Name: 8.8.8.8
Address 1: 8.8.8.8 dns.google
/bin $

9

Can you share your kind: ingress, kind: issuer, and kind: certificate manifests please?

Can you launch a fedora container that will have the ability to install curl/ping/telnet/whatever?

1 Like
10

I deployed the ubuntu image in the cert-manager namespace. Please see below output.

curl https://acme-staging-v02.api.letsencrypt.org/directory

{
"keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
"meta": {
"caaIdentities": [
"letsencrypt.org"
],
"termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
"website": "Staging Environment - Let's Encrypt"
},
"newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
"newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
"newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
"revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert",
"rowsGKH3bqc": "Adding random entries to the directory"
}#

telnet acme-staging-v02.api.letsencrypt.org 443

Trying 184.50.102.158...
Connected to e14990.dscx.akamaiedge.net.
Escape character is '^]'.
^].
Connection closed by foreign host.

11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.