Error on renewal after migrating a certificate

sig · March 11, 2021, 10:17am

Hi everyone,
My domain is: planning.inforlife.ch

Recently, I migrated the web application from an Ubuntu 16.10 machine to a one running Ubuntu 20.04. In the process, I migrated the certificate as indicated here.

The migration seemed to work out fine: the new certificate were correctly loaded.
However, now I need to renew it and I'm running into such issues.

My renewal process uses the certbot/cerbot:latest Docker image.

I ran this command:

sudo docker run -it --rm \
-v /docker-volumes/etc/letsencrypt:/etc/letsencrypt \
-v /docker-volumes/var/lib/letsencrypt:/var/lib/letsencrypt \
-v /docker-volumes/var/log/letsencrypt:/var/log/letsencrypt \
-v /usr/deploys/letsencrypt/temp-site:/data/letsencrypt \
certbot/certbot \
renew

It produced this output:

2021-03-11 09:36:28,091:DEBUG:certbot._internal.main:certbot version: 1.12.0
2021-03-11 09:36:28,091:DEBUG:certbot._internal.main:Location of certbot entry point: /usr/local/bin/certbot
2021-03-11 09:36:28,091:DEBUG:certbot._internal.main:Arguments: []
2021-03-11 09:36:28,091:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2021-03-11 09:36:28,113:DEBUG:certbot._internal.log:Root logging level set at 20
2021-03-11 09:36:28,113:INFO:certbot._internal.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2021-03-11 09:36:28,116:DEBUG:certbot.display.util:Notifying user: Processing /etc/letsencrypt/renewal/planning.inforlife.ch.conf
2021-03-11 09:36:28,119:WARNING:certbot._internal.renewal:
Traceback (most recent call last):
  File "/opt/certbot/src/certbot/certbot/_internal/renewal.py", line 71, in _reconstitute
    renewal_candidate = storage.RenewableCert(full_path, config)
  File "/opt/certbot/src/certbot/certbot/_internal/storage.py", line 471, in __init__
    self._check_symlinks()
  File "/opt/certbot/src/certbot/certbot/_internal/storage.py", line 541, in _check_symlinks
    raise errors.CertStorageError("target {0} of symlink {1} does "
certbot.errors.CertStorageError: target /docker-volumes/etc/letsencrypt/archive/planning.inforlife.ch/cert10.pem of symlink /etc/letsencrypt/live/planning.inforlife.ch/cert.pem does not exist
2021-03-11 09:36:28,123:WARNING:certbot._internal.renewal:Renewal configuration file /etc/letsencrypt/renewal/planning.inforlife.ch.conf is broken. Skipping.
2021-03-11 09:36:28,124:DEBUG:certbot._internal.renewal:Traceback was:
Traceback (most recent call last):
  File "/opt/certbot/src/certbot/certbot/_internal/renewal.py", line 71, in _reconstitute
    renewal_candidate = storage.RenewableCert(full_path, config)
  File "/opt/certbot/src/certbot/certbot/_internal/storage.py", line 471, in __init__
    self._check_symlinks()
  File "/opt/certbot/src/certbot/certbot/_internal/storage.py", line 541, in _check_symlinks
    raise errors.CertStorageError("target {0} of symlink {1} does "
certbot.errors.CertStorageError: target /docker-volumes/etc/letsencrypt/archive/planning.inforlife.ch/cert10.pem of symlink /etc/letsencrypt/live/planning.inforlife.ch/cert.pem does not exist

2021-03-11 09:36:28,124:DEBUG:certbot.display.util:Notifying user:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2021-03-11 09:36:28,125:DEBUG:certbot.display.util:Notifying user: No renewals were attempted.
2021-03-11 09:36:28,126:DEBUG:certbot.display.util:Notifying user:
Additionally, the following renewal configurations were invalid:
2021-03-11 09:36:28,126:DEBUG:certbot.display.util:Notifying user:   /etc/letsencrypt/renewal/planning.inforlife.ch.conf (parsefail)
2021-03-11 09:36:28,127:DEBUG:certbot.display.util:Notifying user: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2021-03-11 09:36:28,128:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 11, in <module>
    load_entry_point('certbot', 'console_scripts', 'certbot')()
  File "/opt/certbot/src/certbot/certbot/main.py", line 15, in main
    return internal_main.main(cli_args)
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 1413, in main
    return config.func(config, plugins)
  File "/opt/certbot/src/certbot/certbot/_internal/main.py", line 1317, in renew
    renewal.handle_renewal_request(config)
  File "/opt/certbot/src/certbot/certbot/_internal/renewal.py", line 510, in handle_renewal_request
    raise errors.Error("{0} renew failure(s), {1} parse failure(s)".format(
certbot.errors.Error: 0 renew failure(s), 1 parse failure(s)
2021-03-11 09:36:28,129:ERROR:certbot._internal.log:0 renew failure(s), 1 parse failure(s)

From my understanding, the error is

certbot.errors.CertStorageError: target /docker-volumes/etc/letsencrypt/archive/planning.inforlife.ch/cert10.pem of symlink /etc/letsencrypt/live/planning.inforlife.ch/cert.pem does not exist

Here the content of the directory

/docker-volumes/etc/letsencrypt/archive/planning.inforlife.ch$ ls
cert1.pem   cert3.pem  cert6.pem  cert9.pem    chain2.pem  chain5.pem  chain8.pem      fullchain10.pem  fullchain4.pem  fullchain7.pem  privkey1.pem   privkey3.pem  privkey6.pem  privkey9.pem
cert10.pem  cert4.pem  cert7.pem  chain1.pem   chain3.pem  chain6.pem  chain9.pem      fullchain2.pem   fullchain5.pem  fullchain8.pem  privkey10.pem  privkey4.pem  privkey7.pem
cert2.pem   cert5.pem  cert8.pem  chain10.pem  chain4.pem  chain7.pem  fullchain1.pem  fullchain3.pem   fullchain6.pem  fullchain9.pem  privkey2.pem   privkey5.pem  privkey8.pem

The configuration file looks like

# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/planning.inforlife.ch
cert = /etc/letsencrypt/live/planning.inforlife.ch/cert.pem
privkey = /etc/letsencrypt/live/planning.inforlife.ch/privkey.pem
chain = /etc/letsencrypt/live/planning.inforlife.ch/chain.pem
fullchain = /etc/letsencrypt/live/planning.inforlife.ch/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = webroot
account = XXXXXXXXXXXXXXXXXXX
server = https://acme-v02.api.letsencrypt.org/directory
[[webroot_map]]
planning.inforlife.ch = /data/letsencrypt

As far as I know, the only thing that changed between the old machine (where the renewal process worked fine) and the new machine (besides the OS) is the version of the certbot/cerbot Docker image.

Am I missing anything here?

Thanks,
Sig

bruncsak · March 11, 2021, 2:24pm

Inside the container the name docker-volumes should not be visible. That is probably a mistake in copying the files from your previous system, especially the symbolic links.

sig · March 11, 2021, 5:05pm

Thanks for your reply. I agree with you docker-volumes should not be visible within the container.
However, I don't understand where the issue is since I believe I mount the directories in the correct way.

bruncsak · March 11, 2021, 7:19pm

I do not think that the problem is with the mounting of directories, that seem to be correct. The problem is with the symbolic links, that you could check outside of the containers. The requirement that symbolic links should work properly within the container itself, that makes them looking broken outside the container. I have the impression that at this moment the situation is the reverse: they point properly outside the container, and broken inside.

Osiris · March 11, 2021, 7:21pm

The way certbot generates the symbolic links is relative to their location. I.e., if you'd ls -l /etc/letsencrypt/live/example.com, you would see:

cert.pem -> ../../archive/example.com/cert1.pem

bruncsak · March 12, 2021, 8:37am

You may wish to provide us the output of the file listing with -l option.

sig · March 12, 2021, 8:55am

Thanks,
I had absolute symlinks instead of relative ones.

system · April 11, 2021, 8:55am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.