Certbot standalone failure

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: bitwarden.bp2215.info

I ran this command: certbot certonly --standalone

It produced this output: Requesting a certificate for bitwarden.bp2215.info

Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
Domain: bitwarden.bp2215.info
Type: unauthorized
Detail: Invalid response from http://bitwarden.bp2215.info/.well-known/acme-challenge/SUTjCbmLYqpKO9OShoBICyG5HHXPZEliMC5EABltpe4 [69.245.126.81]: 503

Hint: The Certificate Authority couldn't exterally verify that the standalone plugin completed the required http-01 challenges. Ensure the plugin is configured correctly and that the changes it makes are accessible from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version): Bitwarden standalone server installed as docker containers using the bitwarden install script.

The operating system my web server runs on is (include version): Ubuntu 20.04.2 LTS

My hosting provider, if applicable, is: N/A

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.16.0 cert

Port 80/443 are open and available for the standalone server to bind.
Firewall is completely disabled for testing/debugging
When I install Apache2 and initiate certbot using the apache plugin it works fine.

Test system: domain nextcloudtest.bp2215.info points to IP 69.245.126.81
New Ubuntu 20.04.2 LTS spun up at local ip address 192.168.1.245 with haproxy as a reverse passthrough
Firewall disabled. Port 80/443 unused, open, and available for standalone server to bind.
certbot certonly
Saving debug log to /var/log/letsencrypt/letsencrypt.log

How would you like to authenticate with the ACME CA?

1: Apache Web Server plugin (apache)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)

Select the appropriate number [1-3] then [enter] (press 'c' to cancel): 2
Please enter the domain name(s) you would like on your certificate (comma and/or
space separated) (Enter 'c' to cancel): nextcloudtest.bp2215.info
Requesting a certificate for nextcloudtest.bp2215.info

Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
Domain: nextcloudtest.bp2215.info
Type: unauthorized
Detail: Invalid response from http://nextcloudtest.bp2215.info/.well-known/acme-challenge/uhTAFjhd0VOeTvEfkiPRPFBtzkPUt23ZR0jU2MQxk1g [69.245.126.81]: 503

Hint: The Certificate Authority couldn't exterally verify that the standalone plugin completed the required http-01 challenges. Ensure the plugin is configured correctly and that the changes it makes are accessible from the internet.

Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

Installed apache2. http://nextcloudtest.bp2215.info accessible from outside local network.
again. Firewall disabled, ports 80/443 now bound to apache.
Now run certbot certonly on same domain using apache plugin:
root@nextcloudtest:/home/michael# certbot certonly
Saving debug log to /var/log/letsencrypt/letsencrypt.log

How would you like to authenticate with the ACME CA?

1: Apache Web Server plugin (apache)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)

Select the appropriate number [1-3] then [enter] (press 'c' to cancel): 1
Please enter the domain name(s) you would like on your certificate (comma and/or
space separated) (Enter 'c' to cancel): nextcloudtest.bp2215.info
Requesting a certificate for nextcloudtest.bp2215.info

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/nextcloudtest.bp2215.info/fullchain.pem
Key is saved at: /etc/letsencrypt/live/nextcloudtest.bp2215.info/privkey.pem
This certificate expires on 2021-08-31.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

If you like Certbot, please consider supporting our work by:

And it works.
So why is the standalone server not able to finish the process???

2

Hello @micahels2408,

When visiting http://bitwarden.bp2215.info/.well-known/acme-challenge/SUTjCbmLYqpKO9OShoBICyG5HHXPZEliMC5EABltpe4, it looks like haproxy is responding with an HTTP 503.

Usually one of the following needs to happen:

  1. haproxy needs to stop and Certbot's standalone server needs to take its port, or
  2. Certbot's standalone server needs to run on a different port (using the --http-01-port option) and haproxy needs to be told to route requests to that port.

Could you describe your setup in more detail? Which of those options sounds applicable to you?

3

Doesn't fit my scenario at all. haproxy is running on a separate router/gateway and forwards all requests to the appropriate in house server. Again, nothing is running/binding to port 80/443.

4

Do you have any health checks set on the backend in haproxy?

If haproxy is bouncing the request with an 503, it either really can't reach the destination server, or it has marked the backend as down and didn't try.

Something that might help debug is to launch this on the target server:

sudo certbot certonly -d example.com --debug-challenges --standalone --dry-run

That will keep the Certbot standalone server up indefinitely.

From there, you can try see whether you can make an HTTP request from the haproxy server and try to make the backend work.

5

I do have checks in place on my http backends. However, I noticed certbot is spinning up the standalone as ip6
root@bitwarden:/home/michael# netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 699/systemd-resolve
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 765/sshd: /usr/sbin
tcp6 0 0 :::80 :::* LISTEN 47703/python3
tcp6 0 0 :::22 :::* LISTEN 765/sshd: /usr/sbin

I'm not using ip6 on my router or this server.
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether a2:df:19:01:99:75 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.201/24 brd 192.168.1.255 scope global dynamic ens18
valid_lft 57516sec preferred_lft 57516sec
I was under the impression the standalone was supposed to default to ip6 and when it wasn't active drop down to ip4

6

The IPv6 bind is expected on Linux:

By default, Certbot first attempts to bind to the port for all interfaces using IPv6 and then bind to that port using IPv4; Certbot continues so long as at least one bind succeeds. On most Linux systems, IPv4 traffic will be routed to the bound IPv6 port and the failure during the second bind is expected.

To exclude IPv6 being an issue, you could, from the haproxy server, try check that Certbot's standalone server is reachable with IPv4:

curl -4 http://192.168.1.201

Personally I'd try disabling the health check, or try ACL routing /.well-known/acme-challenge/ to a separate backend (same IP) which lacks any health checks.

7

It was the check feature. I will sit down down later and write some ACL lines to route past the normal check routines. Thanks for all the help!

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.