Automatic Renewal of certifications: what's the correct procedure?

Hi @marcoippolito

use only

without a domain name. But your cron has already the correct command.

So it looks that your renew didn’t work.

There is a check of your domain - some days old. Looks bad, two different ip addresses with different answers, timeout and blocking firewall.

What’s the output of the command?

PS: Rechecked your domain - https://check-your-website.server-daten.de/?q=ggc.world

Now there is only one ip address. But http is blocked - you have to change that.

These two commands need to be on their own. (And usually certbot is run as root, but I don’t know your setup so you may not need privilege elevation.)

marco@pc01:~$ sudo certbot renew
marco@pc01:~$ sudo logrotate -f certbot

If they must be on the same line, e.g. for cron, you can use the && construct (or a semicolon if you want the logs to rotate whether or not certbot successfully renews your certs) to run them on the same line.

marco@pc01:~$ sudo certbot renew && sudo logrotate -f certbot

Your cron job isn’t running because Ubuntu uses systemd as its init system. (Take a peek at the comments in /etc/cron.d/certbot.) Also, there isn’t a root command available for Ubuntu (did a quick search on the Ubuntu Packages site and I can’t see anything that provides a root executable).

You’ll have to manually create the cron job yourself by using sudo cron and entering in something like:

0 */12 * * * sleep $(awk 'BEGIN{;srand();print int(1+rand()*30);}')m;certbot renew && logrotate -f certbot

(The above must be entered into root's crontab!)

1 Like

Hi @JuergenAuer.
Thanks for replying promptly.

I just checked my domain https://check-your-website.server-daten.de/?q=ggc.world
and it seems ok, apart from the expiration dates of the certificates.

How to unblock the http?

This is the entire /etc/nginx/conf.d/default.conf file, which hasn’t changed since when I got the certificates:

server {
    listen 443 http2 default_server;
    server_name ggc.world www.ggc.world;

    ssl on;
    ssl_certificate /etc/ssl/certs/chained.pem;
    ssl_certificate_key /etc/ssl/private/domain.key;
    ssl_session_timeout 5m;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers EECDH+CHACHA20:EECDH+CHACHA20-    

draft:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DE
S:RSA+3DES:!MD5;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:50m;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
ssl_stapling on;
ssl_stapling_verify on;

    access_log /var/log/nginx/ggcworld-access.log combined;

   add_header Strict-Transport-Security "max-age=31536000";
   location = /favicon.ico { access_log off; log_not_found off; }
    location / {
        proxy_pass http://192.168.1.7:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

server {
    #listen 80 default server;
    listen [::]:80 default_server;
    server_name ggc.world www.ggc.world;
    return 443 https://$host$request_uri;

    access_log /var/log/nginx/ggcworld-access.log combined;

    add_header Strict-Transport-Security "max-age=31536000";
    location = /favicon.ico { access_log off; log_not_found off; }
    location / {
        proxy_pass http://192.168.1.7:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}


marco@pc01:~$ sudo certbot renew
[sudo] password for marco: 
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

In /var/log/letsencrypt/letsencrypt.log :

2019-07-19 15:49:00,679:DEBUG:certbot.renewal:no renewal failures
2019-07-19 15:49:58,657:DEBUG:certbot.main:certbot version: 0.31.0

It’s your router or your firewall, not your webserver configuration.

Where do you see a working http / port 80?

That’s (your new check) bad:

Domainname Http-Status redirect Sec. G
http://ggc.world/
109.116.168.162 -2 1.097 V
ConnectFailure - Unable to connect to the remote server No connection could be made because the target machine actively refused it 109.116.168.162:80
http://www.ggc.world/
109.116.168.162 -2 1.100 V
ConnectFailure - Unable to connect to the remote server No connection could be made because the target machine actively refused it 109.116.168.162:80
https://ggc.world/
109.116.168.162 200 0.340 I
small content:
https://www.ggc.world/
109.116.168.162 200 0.357 I
small content:
http://ggc.world/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
109.116.168.162 -2 1.087 V
ConnectFailure - Unable to connect to the remote server No connection could be made because the target machine actively refused it 109.116.168.162:80
Visible Content:
http://www.ggc.world/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
109.116.168.162 -2 1.100 V
ConnectFailure - Unable to connect to the remote server No connection could be made because the target machine actively refused it 109.116.168.162:80

Connect failure -> http doesn’t work.

Compare it with other domains (Grade E and better).

Looks like HTTPS works fine. You may want to correct your redirect block:

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name ggc.world www.ggc.world;
    return 301 https://$host$request_uri;
}

I disagree with @JuergenAuer, I don’t think this is related to a firewall. You just don’t have anything listening on port 80 with the config above.

1 Like

There

is a long topic. There you have used dns-01 validation, so port 80 isn't relevant.

So do that again, perhaps with the --manual - option.

But if you use --manual, you can't use an automatic cron job.

Maybe correct.

@marcoippolito : Does your http work internal?

curl http://ggc.world/

from your webserver?

In /etc/nginx/conf.d/default.conf in the redirect block I modified the line:

return 443 https://$host$request_uri;

to

return 301 https://$host$request_uri;

I also checked the firewalls:

in my PC-WebServer:

marco@pc01:/etc/nginx/conf.d$ sudo ufw status
Status: inactive

and in the Internet Service Provider (Vodafone):

But still when checking https://check-your-website.server-daten.de/?q=ggc.world
Http-Status: -2

You also have to uncomment the listen line. Change

#listen 80 default server;

to

listen 80 default_server;

Don’t forget to reapply your config!

marco@pc01:~$ sudo nginx -t && sudo service nginx reload

Thanks a lot for your kind explanation.
I’m following now these hints to setup cron job on Ubuntu 18.04 : https://linux4one.com/how-to-set-cron-jobs-on-ubuntu-18-04/

If it's easier, I wrote a little bit about that here.

Quoted below:

The above will run the update command every 12 hours (randomized between +1 and +30 minutes to save the servers a little headache) and update if needed. If there's an error, the logs won't rotate, and if the renewal succeeds, logs will be rotated.

marco@pc01:/etc/nginx/conf.d$ sudo nginx -t && sudo service nginx reload
nginx: [warn] the “ssl” directive is deprecated, use the “listen … ssl” directive instead in /etc/nginx/conf.d/default.conf:5
nginx: [emerg] invalid parameter “server” in /etc/nginx/conf.d/default.conf:30
nginx: configuration file /etc/nginx/nginx.conf test failed

line 30 : listen 80 default server;

You can remove the ssl on; line from your SSL config, and instead add it to the listen line:

listen 443 ssl http2 default_server;

Change default server to default_server. The underscore is important.

Now with the resulting /etc/nginx/conf.d/default.conf file :

server {
    listen 443 ssl http2 default_server;
    server_name ggc.world www.ggc.world;

    ssl_certificate /etc/ssl/certs/chained.pem;
    ssl_certificate_key /etc/ssl/private/domain.key;
    ssl_session_timeout 5m;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers EECDH+CHACHA20:EECDH+CHACHA20- 
draft:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:50m;
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
    ssl_stapling on;
    ssl_stapling_verify on;

    access_log /var/log/nginx/ggcworld-access.log combined;

    add_header Strict-Transport-Security "max-age=31536000";
    location = /favicon.ico { access_log off; log_not_found off; }
    location / {
        proxy_pass http://192.168.1.7:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name ggc.world www.ggc.world;
    #return 443 https://$host$request_uri;
    return 301 https://$host$request_uri;

    access_log /var/log/nginx/ggcworld-access.log combined;

    add_header Strict-Transport-Security "max-age=31536000";
    location = /favicon.ico { access_log off; log_not_found off; }
    location / {
        proxy_pass http://192.168.1.7:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

marco@pc01:/etc/nginx/conf.d$ sudo nginx -t && sudo service nginx reload
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Checking with https://check-your-website.server-daten.de/?q=ggc.world

Http-Status : 301

Now the only thing to fix is the auto-renewal of certificates with sudo chron

Since you’re redirecting all traffic to HTTPS, you can remove these lines from your HTTP server (the one listening on 80):

add_header Strict-Transport-Security "max-age=31536000";
location = /favicon.ico { access_log off; log_not_found off; }
location / {
    proxy_pass http://192.168.1.7:8080;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
}

Since you’re pushing traffic to the other server, headers never get sent and your proxy is never used. I don’t think it’ll actually cause any problems as-is, but it just looks cleaner and it’s easier to debug if things are as simple as possible. :slight_smile: You don’t need to do that if you don’t want to.

Actually I do want to redirect all traffic to https.
Removing those lines -> HTTP_STATUS : 301

Now … the cron job…

Great, a working port 80 :+1:

Try

sudo certbot --nginx -d ggc.world -d www.ggc.world

to see, if that works. If yes, that changes your config file, so you can use renew.

PS: No, not the cron job. First the new certificate creation.

@EmberHeartshine

marco@pc01:/etc/cron.d$ su -
Password: 
root@pc01:~# sudo cron 0 */12 * * * sleep $(awk 'BEGIN{;srand();print    
int(1+rand()*30);}')m;certbot renew && logrotate -f certbot
cron: can't lock /var/run/crond.pid, otherpid may be 942: Resource temporarily 
unavailable
 Saving debug log to /var/log/letsencrypt/letsencrypt.log

 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

 No renewals were attempted.
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 error: cannot stat certbot: No such file or directory
 root@pc01:~# 

What do you mean as “entering into root’s crontab”?

@JuergenAuer
Unfortunately something again has to be fixed:

marco@pc01:/etc/nginx/conf.d$ sudo certbot --nginx -d ggc.world -d 
www.ggc.world
[sudo] password for marco: 
Saving debug log to /var/log/letsencrypt/letsencrypt.log
The requested nginx plugin does not appear to be installed
marco@pc01:/etc/nginx/conf.d$

@EmberHeartshine
Following the indications found here:

I added this line

0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep
 int(rand(43200))' && certbot -q renew

to the crontab file

Based on the hints found here: https://serverfault.com/questions/790772/cron-job-for-lets-encrypt-renewal, I also systemctl enabled certbot.timer :

marco@pc01:/etc/cron.d$ systemctl enable certbot.timer
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-unit-files ===
Authentication is required to manage system service or unit files.
Authenticating as: marco
Password: 
==== AUTHENTICATION COMPLETE ===
==== AUTHENTICATING FOR org.freedesktop.systemd1.reload-daemon ===
Authentication is required to reload the systemd state.
Authenticating as: marco
Password: 
==== AUTHENTICATION COMPLETE ===

marco@pc01:/etc/cron.d$ systemctl start certbot.timer
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to start 'certbot.timer'.
Authenticating as: marco
Password: 
==== AUTHENTICATION COMPLETE ===


marco@pc01:/etc/cron.d$ sudo systemctl list-timers
[sudo] password for marco: 
NEXT                         LEFT        LAST                         PASSED       UNIT  
                         ACTIVATES
Sat 2019-07-20 02:53:15 UTC  8h left     Fri 2019-07-19 14:14:58 UTC  3h 40min 
  ago motd-news.timer              motd-news.service
Sat 2019-07-20 05:30:07 UTC  11h left    Fri 2019-07-19 13:26:36 UTC  4h 28min 
 ago certbot.timer                certbot.service
Sat 2019-07-20 05:33:22 UTC  11h left    Fri 2019-07-19 13:26:36 UTC  4h 28min
 ago apt-daily.timer              apt-daily.service
Sat 2019-07-20 06:34:30 UTC  12h left    Fri 2019-07-19 13:26:36 UTC  4h 28min
 ago apt-daily-upgrade.timer      apt-daily-upgrade.service
Sat 2019-07-20 13:41:58 UTC  19h left    Fri 2019-07-19 13:41:58 UTC  4h 13min
 ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
Mon 2019-07-22 00:00:00 UTC  2 days left Mon 2019-07-15 06:23:55 UTC  4
 days ago   fstrim.timer                 fstrim.service
n/a                          n/a         Fri 2019-07-19 13:28:50 UTC  4h 26min ago   
ureadahead-stop.timer        ureadahead-stop.service

7 timers listed.
Pass --all to see loaded but inactive timers, too.

As far as I understand the renewal process should now restart properly… am I right or is there something more to do and / or to fix?

And another thing I do not understand is that in this file

 nano /lib/systemd/system/certbot.timer 

[Unit]
Description=Run certbot twice daily

[Timer]
OnCalendar=*-*-* 00,12:00:00
RandomizedDelaySec=43200
Persistent=true

[Install]
WantedBy=timers.target

seems that it is already set to run certbot twice a day… why it didn’t happen then?

Thanks again for your kind help.
Marco