Both webroot_path and webroot_map removed from renewal config

Help
1

I’ve just experienced a strange issue whereby certbot removes both webroot_path and webroot_map from the renewal configuration file. Forgive me from deviating from the issue template; the relevant details are all here, but hopefully in a way which is concise and makes most sense for being able to reproduce this particular issue.

I had the following renewal configuration:

# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/leadsheet-publications.co.uk
cert = /etc/letsencrypt/live/leadsheet-publications.co.uk/cert.pem
privkey = /etc/letsencrypt/live/leadsheet-publications.co.uk/privkey.pem
chain = /etc/letsencrypt/live/leadsheet-publications.co.uk/chain.pem
fullchain = /etc/letsencrypt/live/leadsheet-publications.co.uk/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = manual
account = x
pref_challs = dns-01,
manual_public_ip_logging_ok = True
server = https://acme-v02.api.letsencrypt.org/directory

I ran:

certbot certonly --webroot -w /var/www/vhosts/leadsheet-publications.co.uk/current/web -d leadsheet-publications.co.uk -d www.leadsheet-publications.co.uk -d new.leadsheet-publications.co.uk

And got:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Cert not yet due for renewal

You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /etc/letsencrypt/renewal/leadsheet-publications.co.uk.conf)

What would you like to do?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Keep the existing certificate for now
2: Renew & replace the cert (limit ~5 per 7 days)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2

Which produced:

# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/leadsheet-publications.co.uk
cert = /etc/letsencrypt/live/leadsheet-publications.co.uk/cert.pem
privkey = /etc/letsencrypt/live/leadsheet-publications.co.uk/privkey.pem
chain = /etc/letsencrypt/live/leadsheet-publications.co.uk/chain.pem
fullchain = /etc/letsencrypt/live/leadsheet-publications.co.uk/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = webroot
account = x
server = https://acme-v02.api.letsencrypt.org/directory
webroot_path = /var/www/vhosts/leadsheet-publications.co.uk/current/web,
[[webroot_map]]

I then ran certbot renew --force-renewal to test something and this replaced the renewal config with:

# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/leadsheet-publications.co.uk
cert = /etc/letsencrypt/live/leadsheet-publications.co.uk/cert.pem
privkey = /etc/letsencrypt/live/leadsheet-publications.co.uk/privkey.pem
chain = /etc/letsencrypt/live/leadsheet-publications.co.uk/chain.pem
fullchain = /etc/letsencrypt/live/leadsheet-publications.co.uk/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = webroot
account = x
server = https://acme-v02.api.letsencrypt.org/directory

All references to the web root have vanished. Yet if I force renew again, it appears to work.

Some of the renewal configurations on my servers have a [[webroot_map]] section with nothing underneath but have a webroot_path with a value. Others have no webroot_path but entries under [[webroot_map]]. I’m not sure what is by design and what is deprecated and what is a bug.

Domain: leadsheet-publications.co.uk
Certbot version: certbot 0.31.0
OS: CentOS 7.6

2

@bmw, can you make any sense of this situation?

3

Hi @jamieburchell,

Does it appear to work in the sense of getting a new certificate? Does it prompt you interactively for the webroot in this case, or does it just go ahead and succeed?

Do you have an /etc/letsencrypt/cli.ini file?

4

I wouldn't worry about this piece. You should not modify these files by hand, but the two options serve a similar purpose and as long as one is set, you should be fine.

The file you showed with no webroot settings is troubling though. What happens when you run certbot renew --dry-run?

5

It seems to get a new certificate without prompting for anything and succeeds. Nobody was more surprised than me.

No.

6

With this as my renewal config:

# renew_before_expiry = 30 days
version = 0.31.0
archive_dir = /etc/letsencrypt/archive/leadsheet-publications.co.uk
cert = /etc/letsencrypt/live/leadsheet-publications.co.uk/cert.pem
privkey = /etc/letsencrypt/live/leadsheet-publications.co.uk/privkey.pem
chain = /etc/letsencrypt/live/leadsheet-publications.co.uk/chain.pem
fullchain = /etc/letsencrypt/live/leadsheet-publications.co.uk/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = webroot
account = x
server = https://acme-v02.api.letsencrypt.org/directory

$ certbot renew --dry-run

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/leadsheet-publications.co.uk.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator webroot, Installer None
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org
Renewing an existing certificate
Resetting dropped connection: acme-staging-v02.api.letsencrypt.org

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/leadsheet-publications.co.uk/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Note that I've exceeded my rate limit in testing this, but I'm pretty sure that I can reproduce it if that limit is lifted. I reproduced it twice already. I think it might be something around the certificate already existing and replacing it that gets confused. Also note this was originally a certificate obtained by dns-01 (first config above).

Excerpt from the debug log during the dry-run:

	{
  "status": "valid",
  "expires": "2019-05-15T09:33:52Z",
  "identifiers": [
    {
      "type": "dns",
      "value": "leadsheet-publications.co.uk"
    },
    {
      "type": "dns",
      "value": "new.leadsheet-publications.co.uk"
    },
    {
      "type": "dns",
      "value": "www.leadsheet-publications.co.uk"
    }
  ],

Note the DNS entries do not exist (and haven't done for a week at least).

7

Thanks for the additional info.

I have a few more suggestions now:

  1. What OS are you on and how did you install Certbot?
  2. Run sudo rm -rf /etc/letsencrypt/accounts/acme-staging-v02.api.letsencrypt.org/ (or temporarily move that directory somewhere else) and try running sudo certbot renew --dry-run again. Providing the full log of this run would be the most helpful in debugging this.

To explain the reasoning behind the second suggestion, based on your comments about DNS entries, I think you’re getting hit by Let’s Encrypts order/authorization reuse so your setup isn’t actually being tested properly. Deleting the staging account should stop this and allow you to better test your setup.

8

It’s CentOS 7.6, with certbot installed via yum.

I moved that directory out, cleared the log file and tried again with the dry run.

This time I got something more like what I would expect to see:

Attempting to renew cert (leadsheet-publications.co.uk) from /etc/letsencrypt/renewal/leadsheet-publications.co.uk.conf produced an unexpected error: Missing command line flag or config entry for this setting: Input the webroot for leadsheet-publications.co.uk:.

OK, so that sort of clears up why it was magically working without a webroot - now why is certbot removing those details from the renewal file? I’m still unable to test that part due to the rate limit.

9

The scenario that needs to be tested is:

With a valid certificate obtained via dns-01, in which Let’s Encrypt’s order/authorization reuse is active:

Replace certificate and switch to webroot authentication:

certbot certonly --webroot -w /path/to/public/webroot -d example.com
certbot renew --force-renewal

(webroot_path and [[webroot_map]] vanish from renewal config).

Edit: I just did exactly this to test on a different domain and reproduced the issue.

10

Thanks again @jamieburchell. I was able to reproduce the problem.

I wrote up what I think is going on at https://github.com/certbot/certbot/issues/7048 and the issue will be fixed in our next release.

1 Like
11

Excellent, glad I could help.

1 Like
12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.