Certificate renewal error: EOF occurred in violation of protocol

Our renewal process started to fail after moving to new DNS provider with the following error:

An unexpected error occurred:
requests.exceptions.SSLError: HTTPSConnectionPool(host='acme-staging-v02.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))

Is that a block on Let's Encrypt side? I cannot figure out what the problem is.

Joey

1 Like

Hi @cotjoey, and welcome to the LE community forum :slight_smile:

While you wait for a response, please also send a PM to @JamesLE with the IP of the system in question.

1 Like

Thank you, just did!

2 Likes

Here's the thread with more info on what the problem probably is:

2 Likes

Unfortunately, that's not it: it looks like we did not block the IP address here.

1 Like

Well that wasn't it :frowning:
Can it?:
curl -I https://acme-staging-v02.api.letsencrypt.org/

1 Like
issuer = _create_tuple_for_X509_NAME(X509_get_issuer_name(certificate));
if (issuer == NULL)
  goto fail0;

Try running one of these then trying again:

python3 -m pip install certifi

pip install certifi
1 Like

Hello,

To keep you in the loop re: my issue.

Yesterday, after I created my post, I noticed there was a service disruption to most Let's Encrypt services. I was doing my test against the staging server, and that one was listed as down. When it came back up, I tested again, but the error was different than "EOF occurred in violation of protocol". I think the EOF error was related to the service disruption.

The new error was:

2021-07-19 21:20:48,685:ERROR:certbot._internal.log:requests.exceptions.SSLError: HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1131)')))

Because I was testing with a new method for the first time (snapd), I think there is an issue related to leftover certbot installations or I don't understand where the client trust store is picked from... If you have a fix for that error, or guidance on trust store used by snapd, I will gladly take it.

When I used my old method, and install the Cloudflare pip module, it works (no API access token yet though):

Error determining zone_id: 9109 Invalid access token. Please confirm that you have supplied valid Cloudflare API credentials. (Did you enter a valid Cloudflare Token?)

If you could provide guidance on the snapd issue, I'd be happy. If not, then I think my problem is fixed. I was able to renew the certificates manually this morning.

Joey

1 Like

Did you uninstall (the old) certbot before installing (the new) certbot via snapd?

Generally: ca-certificates
For APT type systems:
sudo apt install ca-certificates

1 Like

On Ubuntu 18.04.5 LTS, I get:

sudo apt install ca-certificates
Reading package lists... Done
Building dependency tree
Reading state information... Done
ca-certificates is already the newest version (20210119~18.04.1).
0 upgraded, 0 newly installed, 0 to remove and 31 not upgraded.
1 Like

OK, I do have ca-certificates installed.

I downloaded all the certs in the chain that show up with openssl:

openssl s_client -showcerts -verify 5 -connect acme-staging-v02.api.letsencrypt.org:443 < /dev/null

And added them in the system trust store by first adding them here:

/usr/share/pki/ca-trust-source/anchors/mycert.pem

... and running:

 update-ca-trust

I then ran trust list and could see all the certs listed. So I think it was not using that trust store for a reason I ignore.

2 Likes

What I meant from my last reply is that I added all the certificates, but I am still having the issue. Sorry, my wording wasn't clear.

2 Likes

Then I refer you back to Tomcat.
I don't how/why it works as it does(n't).

If all else fails, I would just put it all behind a real proxy.
And let that proxy handle all the HTTP & HTTPS (certs and redirections).

1 Like

OK, so what you're saying is that the snapd certbot should be picking up its certificates from ca-certificates.

When I browse the snap directory, I see this folder with invalid links to .pem files:

/var/lib/snapd/snap/core20/1026/etc/ssl/certs/

I also found this file, which doesn't contain the certificates I added manually:

/var/lib/snapd/snap/core20/1026/etc/ssl/certs/ca-certificates.crt

Is there a service I need to restart after a manual certificate addition?

1 Like

I don't really know what those files do.

Not normally.

As for Ubuntu 18.04 with snapd, the two .crt files are identical:

-rw-r--r-- 1 root root 200313 Apr 29 21:46 /snap/core20/current/etc/ssl/certs/ca-certificates.crt
-rw-r--r-- 1 root root 200313 Feb  4 06:05 /etc/ssl/certs/ca-certificates.crt
1 Like

Please show:
find / -name DST_Root_CA_X3.*
find / -name ISRG_Root_X1.*

1 Like

Did you try the solution that I suggested from two hours of research?

1 Like

@rg305:

$ sudo find / -name DST_Root_CA_X3.*
/var/lib/snapd/snap/core/11316/etc/ssl/certs/DST_Root_CA_X3.pem
/var/lib/snapd/snap/core/11316/usr/share/ca-certificates/mozilla/DST_Root_CA_X3.crt
/var/lib/snapd/snap/core20/1026/etc/ssl/certs/DST_Root_CA_X3.pem
/var/lib/snapd/snap/core20/1026/usr/share/ca-certificates/mozilla/DST_Root_CA_X3.crt
$ sudo find / -name ISRG_Root_X1.*
/var/lib/snapd/snap/core/11316/etc/ssl/certs/ISRG_Root_X1.pem
/var/lib/snapd/snap/core/11316/usr/share/ca-certificates/mozilla/ISRG_Root_X1.crt
/var/lib/snapd/snap/core20/1026/etc/ssl/certs/ISRG_Root_X1.pem
/var/lib/snapd/snap/core20/1026/usr/share/ca-certificates/mozilla/ISRG_Root_X1.crt

@rg305:

I don't have the two files - maybe that's my issue:

$ ls -lart /var/lib/snapd/snap/core20/1026/etc/ssl/certs/ca-certificates.crt
-rw-r--r--. 1 root root 200313 Apr 29 18:46 /var/lib/snapd/snap/core20/1026/etc/ssl/certs/ca-certificates.crt
$ ls -lart /etc/ssl/certs/ca-certificates.crt
ls: cannot access '/etc/ssl/certs/ca-certificates.crt': No such file or directory

@griffin:

I'll trying installing certifi as you suggested.

1 Like

sudo find / -name ca-certificates.crt

1 Like
$ sudo find / -name ca-certificates.crt
/var/lib/snapd/snap/core/11316/etc/ssl/certs/ca-certificates.crt
/var/lib/snapd/snap/core20/1026/etc/ssl/certs/ca-certificates.crt

My Operating System is Oracle Linux 8. I don't think it's using ca-certificates.crt, even when I install the ca-certificates package (yum install ca-certificates)

We have a SSL proxy that intercepts traffic, and I know what to do normally to add the certificates to the trust store (which I did already), but the same trust store I updated is not used by snap. I think my OS uses ca-bundle.crt, but not sure how snapd handles that one under the hood.

2 Likes