Creating certificates with --nginx works. Renew or certonly doesn't

Help
21

The two certificates are ALMOST identical - in that they cover all the same names except:

Certificate Name: vidyut.info-0001
Has domain: india.aamjanata.com

Certificate Name: vidyut.net-0001
Has domain: map.aamjanata.com

So you need to ensure those two sites are all using their correct cert.
For now, all other sites can use either cert.

At some point, you may want to either:

  • consolidate them all into one new cert (and delete these two certs)
  • separate the sites into grouped certs or even individual certs

[whichever makes most sense to you and your configuration and workflow]

22

Before running the --dry-run test again…
We need to first ensure we know which certs are actually being used.
Please show:
grep -Eri 'ssl_cert|server_name' /etc/nginx

23

I thought I had.... and it wasn't empty - it had the cert that expired which triggered this emergency. It seems to have got recreated and empty. I wonder why and if that has something to do with failed renewals....

The output of this is very convoluted - it goes through sites-available and lists out non-existent certificates from obsolete configs also.

All the servers are currently using /etc/letsencrypt/live/vidyut.info-0001/fullchain.pem; (and privkey).

The india/map is a test site thing. Not really using it for anything much so far. Ignore the difference.

24

One thing worth investigating may be that certbot certonly --dry-run failed, but certbot certonly worked.

Certbot renew is obviously not working right now with new cert, but if it is issued using webroot, maybe certbot renew will work when cert is due for renewal, but fails for certbot renew --dry-run similar to certonly?

Something different about how --dry-run handles authentication?

If so, short of waiting for certs to be due to expire, is there any other way I can make sure that it will work?

25

Then you can (and should) delete the other cert:
certbot delete --cert-name vidyut.net-0001

Then recheck certs with:
certbot certificates

and ensure this folder looks OK (please show):
ls -l /etc/letsencrypt/renewal/

26

We are trying to correct the problems preventing it from renewing...

If it fails now, it will fail later too.
[we will get it fixed for you]

Today, no more than staging vs production.
[all else is exactly the same - both should be forced to use HTTP-01 right now]

Yes, cleaning up the certs and unused files it a very good start.

27

Done.

certbot certificates

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Found the following certs:
Certificate Name: vidyut.info-0001
Domains: vidyut.info aadhaar.fail aamjanata.com aamjanata.in digitalindia.watch fekle.in homeschoolingindia.in india.aamjanata.com nisarga.info vidyut.net wide-aware.com www.aadhaar.fail www.aamjanata.com www.aamjanata.in www.digitalindia.watch www.fekle.in www.homeschoolingindia.in www.nisarga.info www.vidyut.info www.vidyut.net www.wide-aware.com
Expiry Date: 2019-05-15 15:16:22+00:00 (VALID: 89 days)
Certificate Path: /etc/letsencrypt/live/vidyut.info-0001/fullchain.pem
Private Key Path: /etc/letsencrypt/live/vidyut.info-0001/privkey.pem

ls -l /etc/letsencrypt/renewal/

total 4
-rw-r--r-- 1 root root 1340 Feb 14 21:46 vidyut.info-0001.conf

===============================

Not sure what Total 4 means.

28

OK now it looks much better.

[the “4” is the real count of all items in that folder - some aren’t normally shown (like ones that start with a dot) - Just as an FYI, to see more try: ls -la /etc/letsencrypt/renewal/]

Now we try:
cerbot renew --dry-run

29

Already did. No change. Timeout.

30

Please show files:

and
/etc/letsencrypt/renewal/vidyut.info-0001.conf

31

cat /etc/letsencrypt/renewal/vidyut.info-0001.conf

renew_before_expiry = 30 days

version = 0.28.0
archive_dir = /etc/letsencrypt/archive/vidyut.info-0001
cert = /etc/letsencrypt/live/vidyut.info-0001/cert.pem
privkey = /etc/letsencrypt/live/vidyut.info-0001/privkey.pem
chain = /etc/letsencrypt/live/vidyut.info-0001/chain.pem
fullchain = /etc/letsencrypt/live/vidyut.info-0001/fullchain.pem

Options used in the renewal process

[renewalparams]
account = 07ca9221d37e86d5be25c5c043e50af0
server = https://acme-v02.api.letsencrypt.org/directory
authenticator = webroot
webroot_path = /var/www/vin, /var/www/vn, /var/www/aj, /var/www/diwdru, /var/www/af, /var/www/fl, /var/www/ajin, /var/www/hs, /var/www/ni, /var/www/map
[[webroot_map]]
vidyut.net = /var/www/vn
aadhaar.fail = /var/www/af
www.vidyut.info = /var/www/vin
aamjanata.in = /var/www/ajin
india.aamjanata.com = /var/www/map
nisarga.info = /var/www/ni
vidyut.info = /var/www/vin
www.fekle.in = /var/www/fl
www.aamjanata.in = /var/www/ajin
www.aadhaar.fail = /var/www/af
www.digitalindia.watch = /var/www/diwdru
www.wide-aware.com = /var/www/aj
digitalindia.watch = /var/www/diwdru
www.homeschoolingindia.in = /var/www/hs
www.aamjanata.com = /var/www/aj
homeschoolingindia.in = /var/www/hs
wide-aware.com = /var/www/aj
fekle.in = /var/www/fl
www.nisarga.info = /var/www/ni
aamjanata.com = /var/www/aj
www.vidyut.net = /var/www/vn

I have to go out of town on urgent work tomorrow and replies may be erratic - network issues + super hectic schedule. Though I still will be checking - this is important. I will be back on 24th after which replies will be prompt again.

The letsencrypt log is over 8MB!!!? Any way to weed out the info you need from it?

32

Just show the most recent lines.
You can see when the date/time changes - show the last group.

Or move that log file to backup name/location and then run certbot again; which create a new log file.

33

https://pastebin.com/DK7Wc2St

34 
CertStorageError: expected /etc/letsencrypt/live/aamjanata.com/cert.pem to be a symlink
CertStorageError: expected /etc/letsencrypt/live/wide-aware.com/cert.pem to be a symlink
CertStorageError: expected /etc/letsencrypt/live/india.aamjanata.com/cert.pem to be a symlink
CertStorageError: expected /etc/letsencrypt/live/aamjanata.in/cert.pem to be a symlink
CertStorageError: expected /etc/letsencrypt/live/aadhaar.fail/cert.pem to be a symlink
CertStorageError: expected /etc/letsencrypt/live/fekle.in/cert.pem to be a symlink
CertStorageError: expected /etc/letsencrypt/live/nisarga.info/cert.pem to be a symlink
CertStorageError: expected /etc/letsencrypt/live/vidyut.net/cert.pem to be a symlink
CertStorageError: expected /etc/letsencrypt/live/homeschoolingindia.in/cert.pem to be a symlink
Error: 0 renew failure(s), 9 parse failure(s)

Please show:
ls -lR /etc/letsencrypt/live/
ls -lR /etc/letsencrypt/archive/

You probably copied the /etc/letsencyrpt folder from another system or restored it from a backup.
But lost the symbolic links in the process.

35

These certs shouldn’t exist. These domains are now in the combined cert. How to fix this?

I don’t see the certs or the renewal info anywhere - you saw the ls results. Where is this coming from?

36

ls -lR /etc/letsencrypt/live/

/etc/letsencrypt/live/:
total 8
-rw-r--r-- 1 root root 740 Feb 13 05:04 README
drwxr-xr-x 2 root root 4096 Feb 14 21:46 vidyut.info-0001

/etc/letsencrypt/live/vidyut.info-0001:
total 4
lrwxrwxrwx 1 root root 40 Feb 14 21:46 cert.pem -> ../../archive/vidyut.info-0001/cert1.pem
lrwxrwxrwx 1 root root 41 Feb 14 21:46 chain.pem -> ../../archive/vidyut.info-0001/chain1.pem
lrwxrwxrwx 1 root root 45 Feb 14 21:46 fullchain.pem -> ../../archive/vidyut.info-0001/fullchain1.pem
lrwxrwxrwx 1 root root 43 Feb 14 21:46 privkey.pem -> ../../archive/vidyut.info-0001/privkey1.pem
-rw-r--r-- 1 root root 692 Feb 14 21:46 README

ls -lR /etc/letsencrypt/archive/

/etc/letsencrypt/archive/:
total 4
drwxr-xr-x 2 root root 4096 Feb 14 21:46 vidyut.info-0001

/etc/letsencrypt/archive/vidyut.info-0001:
total 16
-rw-r--r-- 1 root root 2382 Feb 14 21:46 cert1.pem
-rw-r--r-- 1 root root 1647 Feb 14 21:46 chain1.pem
-rw-r--r-- 1 root root 4029 Feb 14 21:46 fullchain1.pem
-rw-r--r-- 1 root root 1704 Feb 14 21:46 privkey1.pem

Nothing in renewal, archive or live with those separate domains anymore. Is there anywhere else letsencrypt keeps track?

38

Never copied or restored anything related to letsencrypt. Have been on same server since I started using. There indeed used to be these separate certificates before combined certificates were possible and simplified configs, but they no longer exist. I have no idea why they would be present now in any form - symlink or otherwise.

39

The entries were from your pastebin file.
But I did not notice they were from Jan 14.
Just ignore those errors.

Now reviewing the more recent log entries…

40

I deleted the certs recently, but they were not in use for a while.

41

The pastebin upload doesn’t show the problem.

Please move the letsencrypt.log file elsewhere (if you want to keep it) or delete it (if you don’t).
And rerun
certbot renew --dryrun -vv

then upload the newly created log file.