[BUG] Cannot achieve manual URL challenge

Hi all. First of all I'm not here just to complain, I've done my research and my work and I've reached a conclusion, that's what I want to share with you.

Well, I'm using letsencrypt-auto in CentOs with Python2.6 in order to get a free certificate for a domain that's beyond my computer scope, so I use the manual mode specifying a CSR (Certificate Signing Request) as follows.

./letsencrypt-auto certonly --authenticator manual --email admin@xxx.xxx.xxx --csr signreq.der --text --debug

The error that I always get Is -> WARNING:letsencrypt.plugins.manual:Self-verify of challenge failed... The client lacks sufficient authorization :: Error parsing key authorization file

Now, I'm very sure that I've created the challenge file successfully since I can go to the URL and see the challenge content there. What I've realised is that when I CURL the challenge URL It tells my that's forbidden (Error 403), thus I need to set the User-Agent to Mozilla's and also fill the main cookie. With those parameters filled I'm able to see the challenge content via CURL.

Proofs of CURL with/without Cookie & User-Agent:

Without -> command: curl -v http://xxx.xxx.xxx/.well-known/acme-challenge/ujox7fUMV6Dk6DsCJILDxYopIVR41fJmzOM5x5L2mVw

GET /.well-known/acme-challenge/ujox7fUMV6Dk6DsCJILDxYopIVR41fJmzOM5x5L2mVw HTTP/1.1

Host: xxx.xxx.xxx
User-Agent: curl/7.43.0
Accept: /
< HTTP/1.1 403 Forbidden

With UAgent and Cookie -> command: curl -v http://xxx.xxx.xxx/.well-known/acme-challenge/ujox7fUMV6Dk6DsCJILDxYopIVR41fJmzOM5x5L2mVw --cookie "__test=a3dc21f75aca2595302ecaea9948b2a0" -H "User-Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0"

...HEADERS....
ujox7fUMV6Dk6DsCJILDxYopIVR41fJmzOM5x5L2mVw.BON7JEeVn9miBfADQ34eQIcUTsSTVsHRQqCeXuVUGVs

As you see first command doesn't specify neither cookies nor User-Agent so it throws 403 forbidden error, but the second complies with User-Agent and Cookies so it prints the challenge (that token below "...HEADERS...".

So here's my approach, letsencrypt-auto doesn't validate the challenge because my website gives an error 403 (Forbidden) since letsencrypt-auto doesn't handle cookies and user-agent correctly. I've seen over there that --user-agent parameter is available in the input, but I couldn't make it work.

I just wanted to know what you people think about this issue, if you have suffered also from this or just I'm missing something I can't see? Well at this point I think that's letsencrypt's fault since it doesn't handle all the aforementioned stuff.

Thanks for your patience and support! Have a nice day!

The 403 error you describe sounds like it may be coming from mod_security. We’ve seen some similarly aggressive rejections based on User-Agent from other folks using mod_security. I’d recommend disabling it if you have it enabled.

More generally: If your server is replying 403 Forbidden to Let’s Encrypt’s self-verify or to the CA’s validation request, that is something that you will need to fix in your server configuration.

Thanks,
Jacob

1 Like

Hello Jacob, I appreciate your help, it’s glad to see that I’m not the only one with this kind of trouble.

Thing is that my server is hosted in a free plan, and also it uses Nginx. I’ve open a ticket in their forum requesting some help, but surelly at this point I’ve no more chance to deal with this issue.

What do you recommend to me? Is there any trick for disabling the security in Nginx via .htaccess file? I’m aware that in apache this option is not available anymore. I would like to secure my website in order to circumvent MITM attacks.

Sorry for the inconvenience. Regards

I’m afraid if your hoster doesn’t allow you to change your web server configuration in some way, there’s no way to fix this. nginx doesn’t have anything similar to .htaccess.

If you have control over the DNS for your domain, you can wait until Let’s Encrypt supports dns-01 challenges (hopefully soon!), at which point you can solve challenges by adding a DNS record instead of replying to a specific HTTP request.

Thanks for the fast reply! Yeah, definitely I’ll have to wait until DNS challenges are included in LE.

By the way I’ve obtained a certificate for my VPS and works like a charm, it was time to enjoy free privacy over the internet :slight_smile: this is a right that everyone deserves.

Kind regards.

1 Like