Certbot renew problem

Help
1

Hi, on my website www.atakama-studio.ca, I installed the certificates using the webroot method, because it was not working with the automatic way (plugin). I’m using Cloudflare and also have a virtual host configuration. I’m under Apache 16.0.4. I also moved the certificates from the default locations.

It’s been around 80 days now, and I have to renew two certificates, one for atakama-studio.ca/www.atakama-studio.ca and one for cloud.atakama-studio.ca

I tried sudo certbot renew --dry-run and here’s the results :

Processing /etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf

Cert is due for renewal, auto-renewing…
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for cloud.atakama-studio.ca
Waiting for verification…
Cleaning up challenges
Attempting to renew cert (cloud.atakama-studio.ca) from /etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf produced an unexpected error: (4, ‘Interrupted system call’). Skipping.

Processing /etc/letsencrypt/renewal/atakama-studio.ca.conf

expected /etc/letsencrypt/live/atakama-studio.ca/fullchain.pem to be a symlink
Renewal configuration file /etc/letsencrypt/renewal/atakama-studio.ca.conf is broken. Skipping.
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/cloud.atakama-studio.ca/fullchain.pem (failure)

Additionally, the following renewal configuration files were invalid:
/etc/letsencrypt/renewal/atakama-studio.ca.conf (parsefail)
** DRY RUN: simulating ‘certbot renew’ close to cert expiry
** (The test certificates above have not been saved.)
1 renew failure(s), 1 parse failure(s)

I need help with that, I would like not to spend many days like I did for the initial configuration !

2

Hi @PatricF,

Why?, well, you can do whatever you want but copying them is enough, no need to move them or you are broking the /etc/letsencrypt/ structure.

Please, show the output of these commands:

cat /etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf
cat /etc/letsencrypt/renewal/atakama-studio.ca.conf
ls -la /etc/letsencrypt/live/cloud.atakama-studio.ca/
ls -la /etc/letsencrypt/live/atakama-studio.ca/

Cheers,
sahsanu

3

Hi sahsanu, thanks for the follow up.

Moved or copy, i’m not sure anymore but we can check. I did note my new paths though.

Here’s the outputs :

cat /etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf

renew_before_expiry = 30 days

version = 0.14.2
archive_dir = /etc/letsencrypt/archive/cloud.atakama-studio.ca
cert = /etc/letsencrypt/live/cloud.atakama-studio.ca/cert.pem
privkey = /etc/letsencrypt/live/cloud.atakama-studio.ca/privkey.pem
chain = /etc/letsencrypt/live/cloud.atakama-studio.ca/chain.pem
fullchain = /etc/letsencrypt/live/cloud.atakama-studio.ca/fullchain.pem

Options used in the renewal process

[renewalparams]
authenticator = webroot
installer = None
account = 243c1645cca2ed815103f7df41919fb4
[[webroot_map]]
cloud.atakama-studio.ca = /var/www/cloud.atakama-studio.ca/public_html

cat /etc/letsencrypt/renewal/atakama-studio.ca.conf

renew_before_expiry = 30 days

version = 0.14.2
archive_dir = /etc/letsencrypt/archive/atakama-studio.ca
cert = /etc/letsencrypt/live/atakama-studio.ca/cert.pem
privkey = /etc/letsencrypt/live/atakama-studio.ca/privkey.pem
chain = /etc/letsencrypt/live/atakama-studio.ca/chain.pem
fullchain = /etc/letsencrypt/live/atakama-studio.ca/fullchain.pem

Options used in the renewal process

[renewalparams]
authenticator = webroot
installer = None
account = 243c1645cca2ed815103f7df41919fb4
[[webroot_map]]
atakama-studio.ca = /var/www/atakama-studio.ca/public_html
www.atakama-studio.ca = /var/www/atakama-studio.ca/public_html

sudo ls -la /etc/letsencrypt/live/cloud.atakama-studio.ca/
total 12
drwxr-xr-x 2 root root 4096 Jul 25 00:10 .
drwx------ 4 root root 4096 Jul 25 00:10 …
lrwxrwxrwx 1 root root 47 Jul 25 00:10 cert.pem -> …/…/archive/cloud.atakama-studio.ca/cert1.pem
lrwxrwxrwx 1 root root 48 Jul 25 00:10 chain.pem -> …/…/archive/cloud.atakama-studio.ca/chain1.pem
lrwxrwxrwx 1 root root 52 Jul 25 00:10 fullchain.pem -> …/…/archive/cloud.atakama-studio.ca/fullchain1.pem
lrwxrwxrwx 1 root root 50 Jul 25 00:10 privkey.pem -> …/…/archive/cloud.atakama-studio.ca/privkey1.pem
-rw-r–r-- 1 root root 543 Jul 25 00:10 README

sudo ls -la /etc/letsencrypt/live/atakama-studio.ca/
total 12
drwxr-xr-x 2 root root 4096 Jul 22 02:05 .
drwx------ 4 root root 4096 Jul 25 00:10 …
lrwxrwxrwx 1 root root 41 Jul 21 16:45 cert.pem -> …/…/archive/atakama-studio.ca/cert1.pem
lrwxrwxrwx 1 root root 42 Jul 21 16:45 chain.pem -> …/…/archive/atakama-studio.ca/chain1.pem
lrwxrwxrwx 1 root root 44 Jul 21 16:45 privkey.pem -> …/…/archive/atakama-studio.ca/privkey1.pem
-rw-r–r-- 1 root root 543 Jul 21 16:45 README

4

Regarding the first issue:

Are you sure didn't cancel the process right?. Well, show the output of these commands:

cat -A /etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf
cat -A /etc/letsencrypt/renewal/atakama-studio.ca.conf

And when you paste the outputs here on the forum, select the pasted text and click on icon </> so the output won't be formatted ;).

Regarding the second issue:

So you moved fullchain.pem :frowning: Well, if you have this file yet:

ls -l /etc/letsencrypt/archive/atakama-studio.ca/fullchain1.pem

then (as root user):

cd /etc/letsencrypt/live/atakama-studio.ca/
ln -s ../../archive/atakama-studio.ca/fullchain1.pem fullchain.pem

If the file /etc/letsencrypt/archive/atakama-studio.ca/fullchain1.pem is missing then, recreate it (as root user):

cd /etc/letsencrypt/archive/atakama-studio.ca/
cat cert1.pem chain1.pem > fullchain1.pem
cd /etc/letsencrypt/live/atakama-studio.ca/
ln -s ../../archive/atakama-studio.ca/fullchain1.pem fullchain.pem

And you could try to renew again using --dry-run to see if this second domain gives errors or it is solved now.

Cheers,
sahsanu

5

I tried the renew command 2-3 times with the same result, didn't cancel the process.

Here's the output :

cat -A /etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf
# renew_before_expiry = 30 days$
version = 0.14.2$
archive_dir = /etc/letsencrypt/archive/cloud.atakama-studio.ca$
cert = /etc/letsencrypt/live/cloud.atakama-studio.ca/cert.pem$
privkey = /etc/letsencrypt/live/cloud.atakama-studio.ca/privkey.pem$
chain = /etc/letsencrypt/live/cloud.atakama-studio.ca/chain.pem$
fullchain = /etc/letsencrypt/live/cloud.atakama-studio.ca/fullchain.pem$
$
# Options used in the renewal process$
[renewalparams]$
authenticator = webroot$
installer = None$
account = 243c1645cca2ed815103f7df41919fb4$
[[webroot_map]]$
cloud.atakama-studio.ca = /var/www/cloud.atakama-studio.ca/public_html$

cat -A /etc/letsencrypt/renewal/atakama-studio.ca.conf
# renew_before_expiry = 30 days$
version = 0.14.2$
archive_dir = /etc/letsencrypt/archive/atakama-studio.ca$
cert = /etc/letsencrypt/live/atakama-studio.ca/cert.pem$
privkey = /etc/letsencrypt/live/atakama-studio.ca/privkey.pem$
chain = /etc/letsencrypt/live/atakama-studio.ca/chain.pem$
fullchain = /etc/letsencrypt/live/atakama-studio.ca/fullchain.pem$
$
# Options used in the renewal process$
[renewalparams]$
authenticator = webroot$
installer = None$
account = 243c1645cca2ed815103f7df41919fb4$
[[webroot_map]]$
atakama-studio.ca = /var/www/atakama-studio.ca/public_html$
www.atakama-studio.ca = /var/www/atakama-studio.ca/public_html$

File is still there, so I run the commands as root user with no error.

Here's the output now for a new test with --dry-run

Processing /etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for cloud.atakama-studio.ca
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (cloud.atakama-studio.ca) from /etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf produced an unexpected error: Failed authorization procedure. cloud.atakama-studio.ca (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching https:/cloud.atakama-studio.ca/.well-known/acme-challenge/A24GJuGeev4AXDV4dzo0SQLKMGkRp7HdscjiPFymTgI: Error getting validation data. Skipping.

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/atakama-studio.ca.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for atakama-studio.ca
http-01 challenge for www.atakama-studio.ca
Waiting for verification...
Cleaning up challenges

-------------------------------------------------------------------------------
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/atakama-studio.ca/fullchain.pem
-------------------------------------------------------------------------------
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

The following certs were successfully renewed:
  /etc/letsencrypt/live/atakama-studio.ca/fullchain.pem (success)

The following certs could not be renewed:
  /etc/letsencrypt/live/cloud.atakama-studio.ca/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: cloud.atakama-studio.ca
   Type:   connection
   Detail: Fetching
   https:/cloud.atakama-studio.ca/.well-known/acme-challenge/A24GJuGeev4AXDV4dzo0SQLKMGkRp7HdscjiPFymTgI:
   Error getting validation data

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.
6

Ok, so we have resolved 1 problem, you would be able to renew your cert for atakama-studio.ca so let’s see what is the problem with cloud.atakama-studio.ca.

Now there are no conf errors, the problem is that you have a wrong redirection on cloudflare.

Take a look to this command:

$ curl -IkL http://cloud.atakama-studio.ca/
HTTP/1.1 302 Found
Date: Mon, 02 Oct 2017 16:14:21 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: keep-alive
Set-Cookie: __cfduid=d5813ad717fcf687197299c0f8d3261041506960861; expires=Tue, 02-Oct-18 16:14:21 GMT; path=/; domain=.atakama-studio.ca; HttpOnly
Location: https:/cloud.atakama-studio.ca/
Server: cloudflare-nginx
CF-RAY: 3a78f446b4ff34fa-LHR

And pay attention to the header Location which is redirecting your domain… you are using https:/ instead of https:// so there is a missing / and you should fix that.

Cheers,
sahsanu

7

I see… this should be in the virtual host config right ?

8

@PatricF, I bet you created a page rule on cloudflare to perform the redirection but if you didn’t create a page rule then, the redirection should be on your VirtualHost conf or in some .htaccess .

9

No page rules, i'll check my VirtualHost and .htaccess. I'll post the follow up. Thanks :slight_smile:

1 Like
10

Just a tip:

Change /etc/apache2/ by your current conf dir for apache:
grep -Re 'https:/[^/]' /etc/apache2/

grep -Re 'https:/[^/]' /var/www/cloud.atakama-studio.ca/public_html/

11

Ok, found the config error in my VirtualHost config file. Made the change, restart apache and run curl -IkL http://cloud.atakama-studio.ca/ and it’s now fixed.

I tried certbot renew --dry-run and there’s no error anymore so yeaaa

Now, I can see that it would renew the fullchain.pem file. This is the only one i need ?

I’m asking that because my configuration use cert.pem, privkey.pem and chain.pem.

Do I have to copy the new file to my custom directory ?

12

Currently, Certbot will also change privkey.pem in a renewal. You’ll need both privkey.pem and fullchain.pem.

Edit: @sahsanu’s advice below is more complete.

13

I'm glad it is working fine now :clap: :grinning:

No, it will renew all 4 files, well strictly it will change just 2 files, cert.pem and privkey.pem, the chain.pem won't change but it could in a future and fullchain.pem is created joining cert.pem and chain.pem.

If you are using those 3 files then you should copy those 3 files (pay attention, copy not move :wink: )

You said you are using Apache 16.0.4 and I suppose you mixed the OS version which should be Ubuntu 16.04 and the Apache version should be 2.4.18. So if you are using apache 2.4.8+ then the directive SSLCertificateChainFile has been deprecated and should not be used, instead in SSLCertificateFile you should change cert.pem by fullchain.pem.

So, if your Apache version is 2.4.7 or lower this conf is correct:

SSLCertificateFile    /path/to/cert.pem
SSLCertificateKeyFile /path/to/privkey.pem
SSLCertificateChainFile /path/to/chain.pem

if your Apache version is 2.4.8 or higher the right conf should be:

SSLCertificateFile    /path/to/fullchain.pem
SSLCertificateKeyFile /path/to/privkey.pem

I hope this helps.

Cheers,
sahsanu

14

Yes, sorry, Ubuntu 16.04.3 LTS.

And my Apache is 2.4.18

So I’ll try the new configuration.

And thanks a lot for your help.

1 Like
15

Well, i’ve waited to long before renewing my certificates. I guess they are expired. Here’s what I have when I run sudo certbot renew --dry-run

Processing /etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf
-------------------------------------------------------------------------------
Cert not due for renewal, but simulating renewal for dry run
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for cloud.atakama-studio.ca
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (cloud.atakama-studio.ca) from /etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf produced an unexpected error: (4, 'Interrupted system call'). Skipping.

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/atakama-studio.ca.conf
-------------------------------------------------------------------------------
Cert not due for renewal, but simulating renewal for dry run
Attempting to renew cert (atakama-studio.ca) from /etc/letsencrypt/renewal/atakama-studio.ca.conf produced an unexpected error: HTTPSConnectionPool(host='acme-staging.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f8a695f4810>: Failed to establish a new connection: [Errno 101] Network is unreachable',)). Skipping.
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/cloud.atakama-studio.ca/fullchain.pem (failure)
  /etc/letsencrypt/live/atakama-studio.ca/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
2 renew failure(s), 0 parse failure(s)

What would be the best way to have my new certificates ? I used webroot for the first configuration. Should I just run it again and copy my new certificates in the right place, or I have to delete the current certificates before ? Thanks !

16

Hi @PatricF,

You have 3 valid certificates:

CRT ID     DOMAIN (CN)              VALID FROM             VALID TO               EXPIRES IN  SANs
222780243  atakama-studio.ca        2017-Oct-02 23:24 UTC  2017-Dec-31 23:24 UTC  72 days     atakama-studio.ca
                                                                                              www.atakama-studio.ca

222780221  cloud.atakama-studio.ca  2017-Oct-02 23:24 UTC  2017-Dec-31 23:24 UTC  72 days     cloud.atakama-studio.ca

178359091  cloud.atakama-studio.ca  2017-Jul-24 23:10 UTC  2017-Oct-22 23:10 UTC  2 days      cloud.atakama-studio.ca

1 for cloud.atakama-studio.ca will expire in 2 days, but you have one more valid cert for this domain that will expire in 72 days, the same for cert covering atakama-studio.ca and www.atakama-studio.ca.

As you are using cloudflare we can’t check the current cert used by your web server but you can check them yourself.

echo | openssl s_client -connect localhost:443 -servername atakama-studio.ca 2>/dev/null | openssl x509 -noout -text | grep -E '(Issuer:|Not After|DNS:)'

echo | openssl s_client -connect localhost:443 -servername cloud.atakama-studio.ca 2>/dev/null | openssl x509 -noout -text | grep -E '(Issuer:|Not After|DNS:)'

Regarding the renewal issue for cloud.atakama-studio.ca :

/etc/letsencrypt/renewal/cloud.atakama-studio.ca.conf produced an unexpected error: (4, 'Interrupted system call'). Skipping.

I think we resolved this, don’t know the reason for this issue. Did you touch the renewal conf or moved again the certificate files?

Regarding the second issue trying to renew atakama-studio.ca:

HTTPSConnectionPool(host='acme-staging.api.letsencrypt.org' ... Failed to establish a new connection: [Errno 101] Network is unreachable'

So seems certbot can’t connect to Let’s Encrypt staging server, seems a network error or a firewall dropping/rejecting the connection from your machine to acme-staging.api.letsencrypt.org on port 443.

Maybe the first error is related too this issue too.

From your server try to execute this command:

curl -IkLv https://acme-staging.api.letsencrypt.org/directory

So, seems you have network/firewall issues and also seems you already have valid certificates but maybe your sites are not using the last valid certs… yes I think so because cloudflare gives an error trying to reach your site on atakama-studio.ca and I suppose it is because the cert used in your server expired yesterday:

176410307  atakama-studio.ca        2017-Jul-21 15:45 UTC  2017-Oct-19 15:45 UTC  -1 day      atakama-studio.ca
                                                                                              www.atakama-studio.ca

175770700  atakama-studio.ca        2017-Jul-20 10:47 UTC  2017-Oct-18 10:47 UTC  -2 days     atakama-studio.ca
                                                                                              www.atakama-studio.ca

And I also suppose cloud.atakama-studio.ca will fail in two days because you are not using the renewed certs.

As I said, check your network/firewall and the certificates used by your web server.

Cheers,
sahsanu

17

Hi Sahsanu, thanks again for the help. I didn’t though i’ve renewed the certificate, but it seems i did on oct-02. So I guess, I didn’t moved the certificate in the right place yet.

I just need one certificate for cloud.atakama-studio.ca. Should I revoke CRT 178359091 ?

18

No, there is no need to revoke the certificate.

20

I’ve copy all files (cert.pem, privkey.pem, chain.pem) by using cp -rp /etc/letsencrypt/live/atakama-studio.ca/cert.pem /etc/apache2/ssl/cert.pem to keep the same permission, but the site is not back online. I can see that the file are there and that modification date is good. I’ve reboot the server. Maybe it’s a permission issue ?

I’ve taken note of what the old permission was on the old files. Should I try to change them ?

21

I’ve notice that cert.pem, chain.pem and privkey.pem are in red with black background. So there are no actual files ?