Attempting to renew cert produced an unexpected error: Select the webroot for

Help
1

A surprise. I have a site, with certs that have been renewed successfully in past, but suddenly it fails with the most puzzling error. Here is the output:

$ sudo certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/leaderboard.space.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for leaderboard.space
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/leaderboard.space.conf produced an unexpected error: Missing command line flag or config entry for this setting:
Select the webroot for leaderboard.space:
Choices: ['Enter a new webroot']

(You can set this with the --webroot-path flag). Skipping.

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/leaderboard.space/fullchain.pem (failure)
1 renew failure(s), 0 parse failure(s)

And here is the conf file:

$ cat /etc/letsencrypt/renewal/leaderboard.space.conf 
# renew_before_expiry = 30 days
version = 0.10.2
archive_dir = /etc/letsencrypt/archive/leaderboard.space
cert = /etc/letsencrypt/live/leaderboard.space/cert.pem
privkey = /etc/letsencrypt/live/leaderboard.space/privkey.pem
chain = /etc/letsencrypt/live/leaderboard.space/chain.pem
fullchain = /etc/letsencrypt/live/leaderboard.space/fullchain.pem

# Options used in the renewal process
[renewalparams]
# authenticator = standalone
# path to the public_html / webroot folder being served by your web server.
authenticator = webroot
webroot-path = /mnt/passport/www/html/leaderboard.space

installer = None
account = 52abb1522ceb0b5fb67f17c68d1bec67
pref_challs = http-01,

webroot-path is clearly specified, and as I said, this was working, it’s been renewed in past, not a new cert or site. What happened?

Oh and webroot-path:

$ ll -d /mnt/passport/www/html/leaderboard.space
drwxrwxr-x 7 weaver www-data 4096 Apr 10 13:42 /mnt/passport/www/html/leaderboard.space
$ ll /mnt/passport/www/html/leaderboard.space
total 44
drwxr-xr-x 3 weaver www-data 4096 Mar 30 21:08 CoGs
-rwxr-xr-x 1 weaver www-data  476 Mar 30 21:08 DumpData
drwxr-xr-x 7 weaver www-data 4096 Mar 30 21:08 Leaderboards
drwxr-xr-x 3 weaver www-data 4096 Mar 30 21:08 django_generic_view_extensions
drwxr-xr-x 3 weaver www-data 4096 Mar 30 21:08 django_lighttpd_middleware
-rw-r--r-- 1 weaver www-data   17 Aug 19  2017 index.html
-rwxr-xr-x 1 weaver www-data  143 Aug 24  2017 index.py
-rwxr-xr-x 1 weaver www-data  248 Mar 30 21:08 manage.py
drwxr-xr-x 6 weaver www-data 4096 Mar 30 21:08 static
-rw-r--r-- 1 weaver www-data  330 Oct 10 20:45 uwsgi.ini
-rwxr-xr-x 1 weaver www-data  881 Oct  9  2017 uwsgi_test

I’m stuck identifying changes to the system. It’s been pretty static and not much action or movement there certainly none on the cert front I can identify. But then I can’t be sure clearly, something seems to have changed!

Some more detail asked for in template:

The operating system my web server runs on is (include version): Raspbian GNU/Linux 9

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): nope. ssh.

2

certbot seems to be confused on where “leaderboard.space” should be found.
Is it possible that you have multiple vhost files with overlapping servernames?

or, you could try:
cerbot renew --webroot-path /mnt/passport/www/html/leaderboard.space/
or, maybe:
cerbot renew --webroot-path /mnt/passport/www/html/leaderboard.space/ -d leaderboard.space
…
or maybe just reboot the dang thing - lol

3

If all else fails, remove and retype these two lines in the leaderboard.space.conf file:
authenticator = webroot
webroot-path = /mnt/passport/www/html/leaderboard.space/

I say this because of the extra line found between webroot-path and installer - which may indicate that something was modified.

last but not least:
find / -name leaderboard.space
if a file exists by that name in an in-opportune location, it might be causing the problem.

4

@bmw, this is kind of confusing to me—can you understand what could cause this condition?

5

Hey thanks for a speedy reply. Alas no good news:

  1. Rebooted. Same issue.

  2. Not sure what a vhost is, so unlikely to be an issue.

  3. specifying webroot on the CLI provides more feedback which may be useful:

    $ sudo certbot renew --webroot-path /mnt/passport/www/html/leaderboard.space
    Saving debug log to /var/log/letsencrypt/letsencrypt.log

    Processing /etc/letsencrypt/renewal/leaderboard.space.conf

    Cert is due for renewal, auto-renewing…
    Renewing an existing certificate
    Performing the following challenges:
    http-01 challenge for leaderboard.space
    Using the webroot path /mnt/passport/www/html/leaderboard.space for all unmatched domains.
    Waiting for verification…
    Cleaning up challenges
    Attempting to renew cert from /etc/letsencrypt/renewal/leaderboard.space.conf produced an unexpected error: Failed authorization procedure. leaderboard.space (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://leaderboard.space/.well-known/acme-challenge/PXWpr7gKRvRmCruTOiAqvySeQ2ZRqeD9HcWHkXfI8AI: "

    Not Found

    The requested URL /.well-known/acme-challenge/PXWpr7gKRvRmCruTOiAqvySeQ2ZRqeD9HcWHkXfI8AI was not found on ". Skipping.

    All renewal attempts failed. The following certs could not be renewed:
    /etc/letsencrypt/live/leaderboard.space/fullchain.pem (failure)
    1 renew failure(s), 0 parse failure(s)

    IMPORTANT NOTES:

    • The following errors were reported by the server:

      Domain: leaderboard.space
      Type: unauthorized
      Detail: Invalid response from
      http://leaderboard.space/.well-known/acme-challenge/PXWpr7gKRvRmCruTOiAqvySeQ2ZRqeD9HcWHkXfI8AI:
      "

      Not Found

      The requested URL
      /.well-known/acme-challenge/PXWpr7gKRvRmCruTOiAqvySeQ2ZRqeD9HcWHkXfI8AI
      was not found on "

      To fix these errors, please make sure that your domain name was
      entered correctly and the DNS A record(s) for that domain
      contain(s) the right IP address.

it’s referring to a URL: http://leaderboard.space/.well-known/…

and:

$ ll -a /mnt/passport/www/html/leaderboard.space/.well-known/
total 8
drwxr-xr-x 2 root   root     4096 Apr 10 14:28 .
drwxrwxr-x 8 weaver www-data 4096 Apr 10 14:26 ..

the .well_known dir exists but is empty. Now there’s a hint. I don’t understand enough of what’s happening to interpret this well, but if certbot is expecting this file:

/mnt/passport/www/html/leaderboard.space/.well-known/acme-challenge/PXWpr7gKRvRmCruTOiAqvySeQ2ZRqeD9HcWHkXfI8AI

to be served it clearly isn’t, and I haven’t got it. If that’s meant to be there, I don’t know where it went but can imagine that publishing a web site update some time in past three months may have blown it away inadvertently - need to look closely, only reason that’s even a possibility is I have no recollection of ever seeing this folder before and don’t know what role it plays.

Is this a goo clue? And if so, what should be in that file? Haven’t been good with backups there alas as that whole web root is just published from a dev box anyhow and hasn’t any original data (or at least I didn’t know it did :wink:

(boy Preformatted text isn’t really being honoured in that copy/paste above)

6

Hi @thumbone,

Certbot makes and later deletes the necessary challenge files inside of .well-known, so you wouldn’t expect to find them there later on. The necessary file is different for every certificate request because its name and contents are chosen randomly by the certificate authority (as a way of proving that the person who requested the certificate really controls the associated site).

Could you try making a file test.txt in /mnt/passport/www/html/leaderboard.space/.well-known and seeing whether you can see it on the web at http://leaderboard.space/.well-known/test.txt`?

1 Like
7

Also try:
Making the folder /mnt/passport/www/html/leaderboard.space/.well-known/acme-challenge/
and place a test.txt file there to see it can be seen from the Internet
http://leaderboard.space/.well-known/acme-challenge/test.txt

8

Thanks for suggestions. That was in fact not working. So I fixed it (easy enough, just serve up /.well-known as plain static files, along with /static which I was doing - anything else gets pumped through a a uwsgi service). Still, given I’d never done that and had successful renewals before no surprise that it didn’t fix things. But for what it’s worth:

http://leaderboard.space/.well-known/test.txt
http://leaderboard.space/.well-known/acme-challenge/test.txt

Both serve up a trivial text file now.

As an aside, I had in fact accidentally nuked my whole alpha site in recent days. Not a crisis, just mean the uwsgi served site was down. Fixed that today (tedious, but did - basically I nuked it by accidentally publishing my development branch to the server, so rather than try and recover it I just did a proper database migration as well and got it going with the new, even rougher build as it’s in the middle of some overhauls).

Still, after all that, rebuilding the site and all, now:

$ sudo certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/leaderboard.space.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for leaderboard.space
Cleaning up challenges
Attempting to renew cert from /etc/letsencrypt/renewal/leaderboard.space.conf produced an unexpected error: Missing command line flag or config entry for this setting:
Select the webroot for leaderboard.space:
Choices: ['Enter a new webroot']

(You can set this with the --webroot-path flag). Skipping.

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/leaderboard.space/fullchain.pem (failure)
1 renew failure(s), 0 parse failure(s)

BUT! Figure this:

$ sudo certbot renew --webroot-path /mnt/passport/www/html/leaderboard.space
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/leaderboard.space.conf
-------------------------------------------------------------------------------
Cert is due for renewal, auto-renewing...
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for leaderboard.space
Using the webroot path /mnt/passport/www/html/leaderboard.space for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Unable to clean up challenge directory /mnt/passport/www/html/leaderboard.space/.well-known/acme-challenge
Generating key (2048 bits): /etc/letsencrypt/keys/0002_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0002_csr-certbot.pem

-------------------------------------------------------------------------------
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/leaderboard.space/fullchain.pem
-------------------------------------------------------------------------------

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/leaderboard.space/fullchain.pem (success)

That worked! And it couldn’t clean up acme-challenge because I have a test.txt in there of course. But I also notice it rewrote my conf file! Now it reads:

$ cat /etc/letsencrypt/renewal/leaderboard.space.conf # renew_before_expiry = 30 days
version = 0.10.2
archive_dir = /etc/letsencrypt/archive/leaderboard.space
cert = /etc/letsencrypt/live/leaderboard.space/cert.pem
privkey = /etc/letsencrypt/live/leaderboard.space/privkey.pem
chain = /etc/letsencrypt/live/leaderboard.space/chain.pem
fullchain = /etc/letsencrypt/live/leaderboard.space/fullchain.pem

# Options used in the renewal process
[renewalparams]
# authenticator = standalone
# path to the public_html / webroot folder being served by your web server.
authenticator = webroot

installer = None
account = 52abb1522ceb0b5fb67f17c68d1bec67
pref_challs = http-01,
webroot_path = /mnt/passport/www/html/leaderboard.space,
[[webroot_map]]
leaderboard.space = /mnt/passport/www/html/leaderboard.space

That is, it added a [[webroot_map]] section to the end, and a comma.

Hmmm, twilight zone and I begin to suspect an update to certbot changed practices somehow and is even a little buggy. But it seems I have renewed certs. I’ll test them now (have to publish them to my gateway as it serves them up).

Addendum: Done, tested certs and all is good. Hopefully next renewal goes smoothly with the new conf file.

9

My theory about the renewal trouble is /etc/letsencrypt/renewal/leaderboard.space.conf got modified manually at some point in a way that Certbot doesn’t understand. Both the old version of the file and the new one were written by Certbot 0.10.2 as you can see by the version number in the file. In the old version of the file:

  • The key webroot-path should be webroot_path.
  • The value for webroot_path should have a trailing comma.
  • There should be a webroot_map.

The last two of these Certbot works around, but not webroot-path with a hyphen instead of an underscore. Every version of Certbot should have always written the values this way and I just tested and verified this with the oldest released version of Certbot (letsencrypt 0.1.0).

If you don’t believe this file was manually modified, I’m not sure what happened, but I’m glad it’s working now and please report back if you hit a similar problem.

3 Likes
10

Good theory. And possible. Didn’t jump out at me as likely at first because really no-one is in a position to edit that bar me (so chance someone else did it is near zero). I am veyr busy doing many many things and don’t always remember everything I’ve done or leave a good doco trail ((fairly common problem alas I suspect) which makes it far more possible, even likely that I did edit it manually some time in past. But then it is hard to imagine and less likely again that I’d have made edits and left them lying there untested. That is, the primary and perhaps only, reason I would make mods to such a conf file is, to get something working ;-).

But indeed. A confusion like webroot-path vs. webroot_path, smacks of typo, aka human edit.

Take homes from this experience:

  1. Pay closer attention to underscores and dashes, and trailing commas (forums like this rock because they leave their own doco trail so to speak).

  2. Make sure the webserver can serve up /.well-known/acme-challenge/ files …

Thanks again for all the kind support and attention from the community here. Working together to make SSL more accessible for all! Which may rise in significance soon with lots of murmurs about browsers flagging ALL http: sites as untrusted …

1 Like
11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.