Certbot receiving 503, but wget suggests the URL is good

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: mealplan.dashwood.world

I ran this command: sudo certbot renew --dry-run

It produced this output:

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: mealplan.dashwood.world
Type: unauthorized
Detail: 185.112.146.199: Invalid response from http://mealplan.dashwood.world/.well-known/acme-challenge/: 503

My web server is (include version): Nginx 1.18.0

The operating system my web server runs on is (include version): Raspberry Pi OS 6.1, based on Debian 11 (Bullseye)

My hosting provider, if applicable, is: N/A

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 2.10.0

==================

Hi. I'm having some problems configuring Certbot.

My setup is: Raspberry Pi, running Nginx reverse proxy, to an application (happens to be ASP.Net Core application, but I don't think that's relevant). I've used Certbot to generate a certificate, and now I've got my site up and running... but when I run sudo certbot renew --dry-run, I get an error. I'm using Certbot's webroot option, so that I can renew certificates while my site is running.

The error is:

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: mealplan.dashwood.world
Type: unauthorized
Detail: 185.112.146.199: Invalid response from http://mealplan.dashwood.world/.well-known/acme-challenge/: 503

However, if I run wget <the URL from the error message>, I get a 404, not a 503. I'm assuming that a 404 is correct because Certbot removed the file once the dry-run finished.

So, the question is, why does Certbot see a 503, when I'm seeing a 404.

Some more information:

My full Nginx configuration is as follows:

server {
    listen 80;
    server_name mealplan.dashwood.world;

    location /.well-known {
        root /home/<my user>/<folder>/dist/wwwroot;
    }

    location / {
        return 307 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name mealplan.dashwood.world;

    ssl_certificate     /etc/letsencrypt/live/mealplan.dashwood.world/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/mealplan.dashwood.world/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:5000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Also, I'm using pagekite to tunnel from my domain to my Rasperry Pi. The pagekite configuration is:

###[ Current settings for pagekite.py v1.5.2.201011. ]#########
#
## NOTE: This file may be rewritten/reordered by pagekite.py.
#

##[ Default kite and account details ]##
kitename   = <my-kite-name>.pagekite.me
kitesecret = <redacted>

##[ Front-end settings: use pagekite.net defaults ]##
defaults

##[ Back-ends and local services ]##
service_on  = https:<my domain>    : localhost:443     : @kitesecret
service_on  = http:<my domain>     : localhost:80      : @kitesecret



##[ Miscellaneous settings ]##
savefile = /home/<my user>/.pagekite.rc
max_read_bytes = 16256x3.100

###[ End of pagekite.py configuration ]#########
END

Finally, my application enables HSTS, which means that if I try to paste the the .well-known/acme-challenge/ URL into a browser, the browser immediately redirects to HTTPS, which means that I can't test how a browser behaves on this HTTP URL - but I don't believe that wget is affected by this?

Any ideas what I've done wrong would be great, thanks!

Edit: I just tried placing a test file in .well-known, and checked that I could download that using wget. It worked fine, no sign of any redirects and the file downloaded correctly. So it seems like Nginx is configured correctly with regards to accessing .well-known - but Certbot can't use it for some reason

Your nginx looks okay but requests from the public internet are getting intercepted by your pagekite system. What happens if you try from the public internet like a mobile phone with wifi disabled so using the carrier network?

Because I get this from my own test server

curl -i http://mealplan.dashwood.world
HTTP/1.1 503 Unavailable
X-PageKite-UUID: 45328c7ad206d5636aac6ee692c9a4fc0c40c5b2
Content-Type: text/html; charset=utf-8
Pragma: no-cache
Expires: 0
Cache-Control: no-store
Connection: close

<html><frameset cols="*">
<frame target="_top" src="https://pagekite.net/offline/?&amp;where=FE&amp;proto=http&amp;domain=mealplan.dashwood.world&amp;relay=::ffff:185.112.146.199&amp;rejected=ips_per_sec" /><noframes>
<h1>Sorry! (fe)</h1>
<p>The HTTP <a href="https://pagekite.net/"><i>PageKite</i></a> for <b>mealplan.dashwood.world</b>
 is unavailable at the moment.</p>
<p>Please try again later.</p>
<!--  --></noframes></frameset></html>
3 Likes
Name:    mealplan-deand.pagekite.me
Address: 185.112.146.199
Aliases: mealplan.dashwood.world
1 Like

Yes, you are right - I get a 503 when I try to connect from any computer other than the Rasperry Pi that I'm using as a server.

I'm not sure what that means though - I'll take a closer look at the pagekite settings

I think I've fixed it.

In .pagekite.rc, I had two lines (one for HTTP and one for HTTPS) which linked to my actual domain.

The domain is a CNAME which points at my pagekite URL (the "kite"). I needed to add another two lines - again, one for HTTP and one for HTTPS - which linked to the kite. It's not enough for .pagekite.rc to link to the actual domain name that's being used, it must also link to the kite, otherwise it doesn't work.

Thanks for pointing me in the right direciton.

4 Likes