Fails to recognize my DNS record

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is:

jenkins.yourmessagedelivered.com

I ran this command:

docker run -i --rm \

--name certcont \
--publish 209.195.11.75:8888:8888 \
--volume letsencrypt-etc:/etc/letsencrypt \
--volume letsencrypt-var-lib:/var/lib/letsencrypt \
--volume letsencrypt-var-log:/var/log/letsencrypt \
certbot/certbot:latest \
certonly \
--standalone \
-d jenkins.yourmessagedelivered.com \
--non-interactive \
--preferred-challenges http \
--agree-tos \
--email hesco@yourmessagedelivered.com \
--staging --http-01-port=8888

It produced this output:

full output:

I also get errors about missing files, but these are the permissions on that path:

root@dessalines021:/exports/data/letsencrypt# ls -alht /exports/data/letsencrypt/etc/letsencrypt | grep live

drwx------ 5 root root 4.0K Aug 25 2019 live

yet no new path is created for this domain.
the result written for the haproxy certificate
winds up 0 length, and it must be removed to restart the haproxy.

My web server is (include version):

I am running a docker cluster, implementing a recipe I found here:

relevant portions of my haproxy.cfg are available here:

The operating system my web server runs on is (include version):

My docker hosts are all debian.
most of my inventory is built on a base image derived from an ubuntu containers

My hosting provider, if applicable, is:

I am running a docker cluster, I am my hosting provider,
renting a partial rack in a local data center.

I can login to a root shell on my machine (yes or no, or I don’t know):

YES,
I can use ssh to access as root, my haproxy and the docker hosts in my docker cluster.
except that I have not been daemonizing the certbot, I suppose I could use docker -it certbot attach to probably access root on the certbot node.

I’m using a control panel to manage my site (no, or provide the name and version of the control panel):

no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot):

certbot 1.3.0

2

Hi,

My bad…
The error message indicates that there’s a redirection that redirected the request to a login page…
I’m not sure why that happened, but you need to check your webpages.

You have another option since you hosted your domain on Dreamhost: Obtain certificate with DNS API. This will require you to use TXT record instead of HTTP validation and might save you from these troubles (as you can do it outside docker and share it to your docker).
Currently, certbot doesn’t support DreamHost API, but you can use acme.sh with your Dreamhost API.
acme.sh: https://github.com/acmesh-official/acme.sh/wiki/dnsapi#40-use-dreamhost-dns-api

Thank you

Thank you

3

stevezhu:

thanks for taking a look at this.

my haproxy is already listening on port 80 at that IP.
haproxy here is a docker container,
whose port 80 is bound to port 80 on the docker host.

This haproxy configuration directs matching, like so:

frontend http_proxy
bind 172.17.0.2:80
acl letsencrypt-acl path_beg -i /.well-known/acme-challenge/

mode http
option httplog
option forwardfor
use_backend letsencrypt if letsencrypt-acl

backend letsencrypt
mode http
server letsencrypt_node_01 192.168.51.121:8888
server letsencrypt_node_02 209.195.11.75:8888
timeout connect 1h
timeout server 1h

my docker run command is binding port 8888 like so:
letsencrypt_listen_ip=‘209.195.11.75’ # <-- my public IP

--publish "${letsencrypt_listen_ip}:8888:8888" \

Seems I should already be listening to port 80. Am I missing something, here?

4

I’ve updated my original comment.
It looks like there’s a redirect that tried to send the request back to http://jenkins.yourmessagedelivered.com/login .

5

OK, for this test I disabled the jenkins server with service jenkins stop, run on the jenkins node.

And then I ran my usual shell script, and got this is response:

Challenge failed for domain jenkins.yourmessagedelivered.com
http-01 challenge for jenkins.yourmessagedelivered.com
Cleaning up challenges
Some challenges have failed.
IMPORTANT NOTES: - The following errors were reported by the server:
Domain: jenkins.yourmessagedelivered.com
Type: unauthorized
Detail: Invalid response from
http://jenkins.yourmessagedelivered.com/.well-known/acme-challenge/FgzFJBU8r4yp21WJYCrvVbpZWHgtG8tXfynIPE_sS-Q
[209.195.11.75]: 503

To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.

  • cat /exports/data/letsencrypt/etc/letsencrypt/live/jenkins.yourmessagedelivered.com/fullchain.pem /exports/data/letsencrypt/etc/letsencrypt/live/jenkins.yourmessagedelivered.com/privkey.pem
    cat: /exports/data/letsencrypt/etc/letsencrypt/live/jenkins.yourmessagedelivered.com/fullchain.pem: No such file or directory
    cat: /exports/data/letsencrypt/etc/letsencrypt/live/jenkins.yourmessagedelivered.com/privkey.pem: No such file or directory
6

I can leave this jenkins server down for a while if you want to try it.
I tried to daemonize the letsecrypt node, so it would remain up
long enough for you to test that link yourself.

But could not figure out how to send it a new csr once it was --detach’d.

8

Can you please point to some sample code demonstrating what you advise?

The only thing I host at Dreamhost is my DNS. Everything described above is in a quarter rack a 30 minute drive away.

I do not really need to share it with my docker cluster per se, so much as with my haproxy which reverse proxies everything in my network.

9

The first thing that stands out to me about your haproxy config is that this rule:

use_backend letsencrypt if letsencrypt-acl

is located well below the the <snip>, which is where I assume you have a different rule to select Jenkins as a backend.

If that’s the case, Jenkins is always going to get selected as the backend. haproxy will use whichever use_backend evaluates first.

From all of the symptoms you have posted, it sounds like the letsencrypt backend is never being selected.

Perhaps try moving the line above somewhere above the <snip>, preferably just after the line where you declare letsencrypt-acl (L34-ish in your gist)?

e: Also, path_beg is not a regex - that would be path_reg. Unescape your slashes.

10

my use backend letsencrypt is the very first use backend statement in my haproxy.cfg file.
I tried moving it to immediately after the acl rule which routes to it. I tried removing the escape, I tried using path_reg, rather than path_beg.

Having restored all of those tests, which either made things worse, or the same, I am still getting:

IMPORTANT NOTES:                                                                                                                                                                                 - The following errors were reported by the server:
                                                                                                                                                                                                   Domain: jenkins.yourmessagedelivered.com
   Type:   unauthorized                                                                     
   Detail: Invalid response from                                 
   http://jenkins.yourmessagedelivered.com/.well-known/acme-challenge/rDYJyMp7fT2_Y4p0D-fhFKHbsqP07h8b6sGR6b-iInw
   [209.195.11.75]: 503                      

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.
+ cat /exports/data/letsencrypt/etc/letsencrypt/live/jenkins.yourmessagedelivered.com/fullchain.pem /exports/data/letsencrypt/etc/letsencrypt/live/jenkins.yourmessagedelivered.com/privkey.pem
cat: /exports/data/letsencrypt/etc/letsencrypt/live/jenkins.yourmessagedelivered.com/fullchain.pem: No such file or directory
cat: /exports/data/letsencrypt/etc/letsencrypt/live/jenkins.yourmessagedelivered.com/privkey.pem: No such file or directory

That is my IP address, that 503 is coming from my haproxy. Not sure why it is complaining about DNS. This seems to me to be an issue of haproxy configuration, though.

11

And this, from the haproxy logs tells me that the request is being routed appropriately:

http_proxy letsencrypt/letsencrypt_node_01 0/9126/-1/-1/10150 503 212 - - SC-- 3/3/2/1/+3 0/0 “GET /.well-known/acme-challenge/R-R07bNZx6KfKlLS60NChViNF_NxxiBu1OHEqZzb4xQ HTTP/1.1”

That suggests to me that the 503 is being thrown because the letsencrypt service I am running with certbot is not listening for requests.

12

Well, that certainly helps narrow it down!

Maybe you can temporarily run a full-time webserver, rather than Certbot's temporary standalone server, to try narrow down whether there's a connectivity issue unrelated to Certbot:

docker run --rm --publish 192.168.51.121:8888:80 nginx:alpine

(assuming that you've got that backend server uncommented in haproxy)

and try see how haproxy responds to a manual request:

curl -i jenkins.yourmessagedelivered.com/.well-known/acme-challenge/test

If it's working, you should see an HTTP 404 with an nginx server banner, as well as log entries in the stdout for the Docker container.

It'd be helpful to see whether this minimal test works, so we know whether to start looking at Certbot or not.

13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.