Renewal error 404 failed authorization

[ktys1]

Please fill out the fields below so we can help you better.

My domain is: audio.phoncert.com

I ran this command: sudo letsencrypt renew

It produced this output:Processing /etc/letsencrypt/renewal/audio.phoncert.com.conf
2017-07-12 17:03:41,541:WARNING:letsencrypt.cli:Attempting to renew cert from /etc/letsencrypt/renewal/audio.phoncert.com.conf produced an unexpected error: Failed authorization procedure. audio.phoncert.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://audio.phoncert.com/.well-known/acme-challenge/-6ExySvl3Kig406lyXxsh1JrEtxhDtaiR7pQxvS3tV0: "

404 Not Found

404 Not Found

---

". Skipping.

My web server is (include version): nginx

The operating system my web server runs on is (include version): ubuntu 16.04

My hosting provider, if applicable, is: AWS

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): yes AWS ec2

[schoen]

Hi @ktys1,

What webroot directory is specified in /etc/letsencrypt/renewal/audio.phoncert.com.conf? Does it seem correct to you?

[ktys1]

Yes. I read most of the previous similar reports on 404 problems, and checked that the webroot line in .conf looked right - it points to the folder that actually hosts the app:

Options and defaults used in the renewal process

[renewalparams]
no_self_upgrade = False
no_verify_ssl = False
ifaces = None
register_unsafely_without_email = False
uir = None
installer = none
config_dir = /etc/letsencrypt
text_mode = False
staging = False
dry_run = False
work_dir = /var/lib/letsencrypt
tos = False
init = False
http01_port = 80
duplicate = False
noninteractive_mode = False
key_path = None
nginx = False
fullchain_path = /home/ubuntu/apps/chain.pem
email = XXXX@gmail.com
csr = None
agree_dev_preview = None
redirect = None
verbose_count = -3
config_file = None
renew_by_default = False
hsts = False
authenticator = webroot
domains = audio.phoncert.com,
rsa_key_size = 2048
verb = certonly
checkpoints = 1
manual_test_mode = False
apache = False
cert_path = /home/ubuntu/apps/cert.pem
webroot_path = /home/ubuntu/apps/soundctl,
reinstall = False
expand = False
strict_permissions = False
account = b69393987d93fe17c78080913cc9408f
prepare = False
manual_public_ip_logging_ok = False
chain_path = /home/ubuntu/apps/chain.pem
break_my_certs = False
standalone = False
manual = False
server = https://acme-v01.api.letsencrypt.org/directory
standalone_supported_challenges = "tls-sni-01,http-01"
webroot = False
os_packages_only = False
func = <function obtain_cert at 0x7f1bd18f0c80>
user_agent = None
debug = False
tls_sni_01_port = 443
logs_dir = /var/log/letsencrypt
configurator = None
[[webroot_map]]
audio.phoncert.com = /home/ubuntu/apps/soundctl

[schoen]

@ktys1

/home/ubuntu/apps/soundctl

Great! So the first test would be: if you create a text file /home/ubuntu/apps/soundctl/test.txt, can you see it at http://audio.phoncert.com/test.txt? (note: not https)

Second, if you create a text file /home/ubuntu/apps/soundctl/.well-known/acme-challenge/test2.txt, can you see it at http://audio.phoncert.com/.well-known/acme-challenge/test2.txt?

[ktys1]

OK, I realize my node server will not let anything thru that is not expected, therefore both my test.txt and I suppose whatever letsencrypt .well-known wants to do has been rerouted - it defaults to the main page

Do I need to make a specific route for the .well-known folder?

[schoen]

Well, that should explain the 404 error; glad you were able to figure that out.

If you can pass through /.well-known/acme-challenge from HTTP requests into /home/ubuntu/apps/soundctl/.well-known/acme-challenge on the filesystem, the renewal should go through successfully. (This refers to a directory path, not just a single file.)

[ktys1]

Still same error

I have tested that my path is displaying the 2 text files you suggested as a test.

The permissions seem correct on folders .well-known and acme-challenge (755)

Once the challenge begins, what is happening? My server expects to respond to a request for an existing file in the folder, and the response to be the content-type text/html - but the renew process sends something like this - kt7uA7LoM7aOqMfvhh-bKRl9AqGt1uhcTnhRv9qANpc:

[schoen]

The certificate authority is telling the client (Certbot, previously called letsencrypt) “if you really control the domain name audio.phoncert.com, you can prove it by making a file at http://audio.phoncert.com/.well-known/acme-challenge/kt7uA7LoM7aOqMfvhh-bKRl9AqGt1uhcTnhRv9qANpc containing the content [whatever]”. Then Certbot tries to create such a file within the webroot directory that was previously specified to it. It tells the CA that it’s done so, and the CA makes an inbound web connection to check whether that content is really in place as requested.

The webroot directory needs to be the directory that contains .well-known/acme-challenge, not the acme-challenge subdirectory itself.

[ktys1]

I do have acme-challenge in the .well-known dir

So permissions?

start in: /home/ubuntu/apps/soundctl
do: ls -al
drwxr-xr-x 3 root root 4096 Jul 12 20:23 .well-known

for .well-known do: ls -al
drwxr-xr-x 2 root root 4096 Jul 12 22:34 acme-challenge

[schoen]

Can you give me an example of a test file that you made in .well-known/acme-challenge that’s accessible over the web?

Alternatively, you can run with --debug-challenges, which will pause Certbot after setting up the challenge but before telling the certificate authority to validate it, so that you can take a look for yourself at what it’s tried to do.

[ktys1]

OK, this is my test:

https://audio.phoncert.com/well-known/acme-challenge/testS.txt

I realized I have dropped the dot on the well-known - the server is not handling it so I will fix that and try again.

also the latest letsencrypt.log:
sudo cat /var/log/letsencrypt/letsencrypt.log

2017-07-12 22:34:57,694:DEBUG:letsencrypt.cli:Exiting abnormally:
Traceback (most recent call last):
File “/usr/bin/letsencrypt”, line 9, in
load_entry_point(‘letsencrypt==0.4.1’, ‘console_scripts’, ‘letsencrypt’)()
File “/usr/lib/python2.7/dist-packages/letsencrypt/cli.py”, line 1986, in main
return config.func(config, plugins)
File “/usr/lib/python2.7/dist-packages/letsencrypt/cli.py”, line 1034, in renew
len(renew_failures), len(parse_failures)))
Error: 1 renew failure(s), 0 parse failure(s)

[ktys1]

Q: Is there some kind of necessary reason for the well-known to be a hidden dir?

[schoen]

@ktys1

Q: Is there some kind of necessary reason for the well-known to be a hidden dir?

Compliance with RFC 5785:

RFC 5785: Defining Well-Known Uniform Resource Identifiers (URIs)

[ktys1]

Hi Schoen,

Wanted to follow up with you on this particular problem help request.

Got the renewal to work.

I added this to replace my config in default for nginx:

location ^~ /.well-known {
allow all;
auth_basic off;
alias /path/to/.well-known/;
}
from: https://github.com/letsencrypt/acme-spec/issues/221

That did it - my dotfolder was being 404’ed

[schoen]

Thanks for sharing your solution, @ktys1.

[ismaelirc]

Hi, @schoen and @ktys1 I have the same problem and I made a little test: I rename my .htaccess and run the renew command and works well… so my question is: What kind of rule a must have in my .htaccess for the renew command works well?
I got try many different rules

Here is my htaccess now:

RewriteEngine On

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond $1 !^(index.php|public|images|robots.txt|css)
RewriteRule ^(.*)$ index.php/$1 [L]
RewriteRule “/.|^.” - [F]
RewriteRule “/.|^.(?!well-known/)” - [F]

[schoen]

Hi @ismaelirc,

I don’t have enough familiarity with mod_rewrite to answer your question offhand. You might have better luck starting a new thread and mentioning in the topic that it relates to mod_rewrite; maybe that will attract someone else to jump in and help figure out your problem!

[ismaelirc]

OK, thanks @schoen!