"Error getting validation data"

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: www.peregrineit.net

I ran this command: /usr/bin/certbot certonly --dry-run --agree-tos --force-renewal --standalone --preferred-challenges http --http-01-port 54321 -d www.peregrineit.net

It produced this output: Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for www.peregrineit.net
Waiting for verification…
Challenge failed for domain www.peregrineit.net
http-01 challenge for www.peregrineit.net
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: www.peregrineit.net
    Type: connection
    Detail: Fetching
    https://www.peregrineit.net/.well-known/acme-challenge/-wgCZFcbpQfPha10v_bQ9E2nzkFezDRLhyhOubvV16g:
    Error getting validation data

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address. Additionally, please check that
    your computer has a publicly routable IP address and that no
    firewalls are preventing the server from communicating with the
    client. If you’re using the webroot plugin, you should also verify
    that you are serving files from the webroot path you provided.

My web server is (include version): niginx (latest update)

The operating system my web server runs on is (include version): centos 7 (latest update)

My hosting provider, if applicable, is: N/A

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.34.2

Other Relevant Info:
1) Certbox is sitting on a HAProxy server
2) The Nginx Webserver is a separate server to the HAProxy server
3) The (obsqured) IP Address of the servers are:
a) HAProxy: 10.999.999.10
b) Nginx: 10.999.999.11
4) Public IP Addrees of Port Forwarding Router to HAProxy: 218.214.86.96
5) (Obscured) HAProxy Config is:

  1 global
  2         log 127.0.0.1 local
  3         chroot /var/lib/haproxy
  4         pidfile /var/run/haproxy.pid
  5         maxconn 4000
  6         stats socket /var/lib/haproxy/stats mode 660 level admin
  7         stats timeout 30s
  8         user haproxy
  9         group haproxy
 10         daemon
 11         ca-base **********
 12         crt-base **********
 13         ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNUL        L:!MD5:!DSS
 14         ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11
 15         tune.ssl.default-dh-param 4096
 16
 17 defaults
 18         log global
 19         mode http
 20         option tcpka
 21         option httplog
 22         option dontlognull
 23         option tcp-smart-connect
 24         option splice-auto
 25         timeout connect 5000
 26         timeout client  50000
 27         timeout server  50000
 28         retries 3
 29         timeout http-request 10s
 30         timeout queue 1m
 31 #       errorfile 400 /etc/haproxy/errors/400.http
 32 #       errorfile 403 /etc/haproxy/errors/403.http
 33 #       errorfile 408 /etc/haproxy/errors/408.http
 34 #       errorfile 500 /etc/haproxy/errors/500.http
 35 #       errorfile 502 /etc/haproxy/errors/502.http
 36 #       errorfile 503 /etc/haproxy/errors/503.http
 37 #       errorfile 504 /etc/haproxy/errors/504.http
 38
 39 frontend hap
 40         bind *:80
 41 bind *:443 # temporary setting until cert is obtained 
 42 #       bind *:443 ssl crt www.peregrineit.net.pem ecdhe secp384r1 alpn h2,http/1.1
 43         acl letsencryptrequest path_beg -i /.well-known/acme-challenge/
 44         acl acl_secure dst_port eq 443
 45         use_backend letsencrypt if letsencryptrequest
 46         redirect scheme https if !{ ssl_fc }
 47         redirect prefix http://www.peregrineit.net code 301 if { hdr(host) -i domain.com }
 48         rspadd Strict-Transport-Security:\ max-age=31536000;\ includeSubDomains;\ preload
 49         rsprep ^Set-Cookie:\ (.*) Set-Cookie:\ \1;\ Secure if acl_secure
 50         default_backend nginx
 51
 52 backend nginx
 53         http-request set-header X-Forwarded-Port %[dst_port]
 54         http-request add-header X-Forwarded-Proto https if { ssl_fc }
 55         server nginx 10.999.999.11:80
 56
 57 backend letsencrypt
 58         mode http
 59         server letsencrypt 127.0.0.1:54321
 60
61
 62 listen stats
 63         bind *:60443
 64         stats enable
 65         stats uri /
 66         stats refresh 5s
 67         stats auth domadmin:st@ts
 68         stats admin if TRUE
 69         stats show-legends
 70

Any help is greatly appreciated - thanks

2

Hi @dulux-oz

it's nearly impossible to debug a configuration which uses --standalone. That spins up an own webserver.

But your current configuration is buggy

You have redirects http - https: And https doesn't work ( https://check-your-website.server-daten.de/?q=peregrineit.net ):

Domainname Http-Status redirect Sec. G
• http://peregrineit.net/
218.214.86.96 302 https://peregrineit.net/ 0.660 A
• http://www.peregrineit.net/
218.214.86.96 302 https://www.peregrineit.net/ 0.650 A
• https://peregrineit.net/
218.214.86.96 -4 1.360 W
SendFailure - The underlying connection was closed: An unexpected error occurred on a send. The handshake failed due to an unexpected packet format.
• https://www.peregrineit.net/
218.214.86.96 -4 1.360 W
SendFailure - The underlying connection was closed: An unexpected error occurred on a send. The handshake failed due to an unexpected packet format.
• http://peregrineit.net:443/
218.214.86.96 302 https://peregrineit.net/ 0.640 Q
Visible Content:
• http://www.peregrineit.net:443/
218.214.86.96 302 https://www.peregrineit.net/ 0.666 Q
Visible Content:
• http://peregrineit.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
218.214.86.96 302 https://peregrineit.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.680 A
Visible Content:
• http://www.peregrineit.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de
218.214.86.96 302 https://www.peregrineit.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de 0.674 A
Visible Content:
• https://peregrineit.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de -4 1.363 W
SendFailure - The underlying connection was closed: An unexpected error occurred on a send. The handshake failed due to an unexpected packet format.
Visible Content:
• https://www.peregrineit.net/.well-known/acme-challenge/check-your-website-dot-server-daten-dot-de -4 1.357 W
SendFailure - The underlying connection was closed: An unexpected error occurred on a send. The handshake failed due to an unexpected packet format.
Visible Content:

Looks like your haproxy redirects to https. But your https doesn't work. Perhaps your haproxy catches https - but can't handle that. Certbot creates via --standalone a http port.

The result: The unexpected packet format error. Perhaps not from the --standalone - server, instead from your haproxy.

Reconfigure your haproxy so /.well-known/acme-challenge isn't redirected to https.

So you redirect the http request directly to your port 54321.

3

Thanks for replying

So, I commented out the “redirect” lines of the haproxy.conf file (see below) and reran the same command - relevant output is alose below:

HAproxy config (revised, extract):

39 frontend hap
 40         bind *:80
 41 bind *:443 # temporary setting until cert is obtained 
 42 #       bind *:443 ssl crt www.peregrineit.net.pem ecdhe secp384r1 alpn h2,http/1.1
 43         acl letsencryptrequest path_beg -i /.well-known/acme-challenge/
 44         acl acl_secure dst_port eq 443
 45         use_backend letsencrypt if letsencryptrequest
 46#        redirect scheme https if !{ ssl_fc }
 47         redirect prefix http://www.peregrineit.net code 301 if { hdr(host) -i domain.com }
 48         rspadd Strict-Transport-Security:\ max-age=31536000;\ includeSubDomains;\ preload
 49         rsprep ^Set-Cookie:\ (.*) Set-Cookie:\ \1;\ Secure if acl_secure
 50         default_backend nginx
 51
 52 backend nginx
 53  #      http-request set-header X-Forwarded-Port %[dst_port]
 54  #      http-request add-header X-Forwarded-Proto https if { ssl_fc }
 55         server nginx 10.999.999.11:80
 56
 57 backend letsencrypt
 58         mode http
 59         server letsencrypt 127.0.0.1:54321

Resulting Output:

IMPORTANT NOTES:

4

Looks like you have a buggy configuration.

503 - ServerError:

503 Service Unavailable

No server is available to handle this request.

Maybe a wrong haproxy config, maybe the way haproxy -> certbot doesn't work. That's the problem of standalone, terrible to debug. Start an own, running webserver, so you can debug that. Then use --webroot.

5

Yes, thanks - that’s why I’m asking for help - the website loads perfectly well under http: but not under https: - partially because I can’t get a certificate :slight_smile:

6

Then it's wrong if you have already redirects http -> https.

7

Your haproxy+Certbot config looks fine to me. I use a very similar one.

Can you post the full log file following a failure? /var/log/letsencrypt/letsencrypt.log .

8

Yes, but I’ve turned off http->https redirects - and, as you can see, I’m now getting a differnt error when I run the certbot command, which now give the 503 error.

So I either don’t do the redirect and certbot gives me a 503, or I do the redirect and certbot gives me the Error getting validation data error

HELP!!! :smile:

9

Current log - aas requested (thanks):

  1 2019-07-04 18:20:30,839:DEBUG:certbot.main:certbot version: 0.34.2
  2 2019-07-04 18:20:30,839:DEBUG:certbot.main:Arguments: ['--dry-run', '--agree-tos', '--force-renewal', '--standalone', '-        -preferred-challenges', 'http', '--http-01-port', '54321', '-d', 'www.peregrineit.net']
  3 2019-07-04 18:20:30,839:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#        null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
  4 2019-07-04 18:20:30,885:DEBUG:certbot.log:Root logging level set at 20
  5 2019-07-04 18:20:30,886:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
  6 2019-07-04 18:20:30,888:DEBUG:certbot.plugins.selection:Requested authenticator standalone and installer None
  7 2019-07-04 18:20:30,889:DEBUG:certbot.plugins.selection:Single candidate plugin: * standalone
  8 Description: Spin up a temporary webserver
  9 Interfaces: IAuthenticator, IPlugin
 10 Entry point: standalone = certbot.plugins.standalone:Authenticator
 11 Initialized: <certbot.plugins.standalone.Authenticator object at 0x7effc83cc090>
 12 Prep: True
 13 2019-07-04 18:20:30,890:DEBUG:certbot.plugins.selection:Selected authenticator <certbot.plugins.standalone.Authenticator         object at 0x7effc83cc090> and installer None
 14 2019-07-04 18:20:30,890:INFO:certbot.plugins.selection:Plugins selected: Authenticator standalone, Installer None
 15 2019-07-04 18:20:30,940:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(status=None,         terms_of_service_agreed=None, agreement=None, only_return_existing=None, contact=(), key=None, external_account_binding=        None), uri=u'https://acme-staging-v02.api.letsencrypt.org/acme/acct/9620947', new_authzr_uri=None, terms_of_service=None        ), 26ba1dcd914bd4958cecf7593c78de63, Meta(creation_host=u'hap01.mjb-co.pri', creation_dt=datetime.datetime(2019, 6, 16,         6, 17, 7, tzinfo=<UTC>)))>
 16 2019-07-04 18:20:30,981:DEBUG:acme.client:Sending GET request to https://acme-staging-v02.api.letsencrypt.org/directory.
 17 2019-07-04 18:20:30,996:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-staging-v0        2.api.letsencrypt.org
 18 2019-07-04 18:20:31,706:DEBUG:requests.packages.urllib3.connectionpool:"GET /directory HTTP/1.1" 200 724
 19 2019-07-04 18:20:31,708:DEBUG:acme.client:Received response:
 20 HTTP 200
 21 content-length: 724
 22 expires: Thu, 04 Jul 2019 08:20:31 GMT
 23 strict-transport-security: max-age=604800
 24 server: nginx
 25 connection: keep-alive
 26 pragma: no-cache
 27 cache-control: max-age=0, no-cache, no-store
 28 date: Thu, 04 Jul 2019 08:20:31 GMT
 29 x-frame-options: DENY
 30 content-type: application/json
 31
 32 {
 33   "8H6Di_nwHx0": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
 34   "keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
 35   "meta": {
 36     "caaIdentities": [
 37       "letsencrypt.org"
 38     ],
 39     "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
 40     "website": "https://letsencrypt.org/docs/staging-environment/"
 41   },
 42   "newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
 43   "newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
 44   "newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
 45   "revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert"
 46 }
 47 2019-07-04 18:20:31,712:INFO:certbot.main:Obtaining a new certificate
 48 2019-07-04 18:20:31,911:DEBUG:acme.client:Requesting fresh nonce
 49 2019-07-04 18:20:31,912:DEBUG:acme.client:Sending HEAD request to https://acme-staging-v02.api.letsencrypt.org/acme/new-        nonce.
 50 2019-07-04 18:20:32,173:DEBUG:requests.packages.urllib3.connectionpool:"HEAD /acme/new-nonce HTTP/1.1" 200 0
 51 2019-07-04 18:20:32,174:DEBUG:acme.client:Received response:
 52 HTTP 200
 53 content-length: 0
 54 expires: Thu, 04 Jul 2019 08:20:32 GMT
 55 strict-transport-security: max-age=604800
  56 server: nginx
 57 connection: keep-alive
 58 link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
 59 pragma: no-cache
 60 cache-control: max-age=0, no-cache, no-store
 61 date: Thu, 04 Jul 2019 08:20:32 GMT
 62 x-frame-options: DENY
 63 replay-nonce: oVPL4mm4XQIGy0nojTq5W3p4zOnIhgqTxQGdZ5k1WO8
 64
 65
 66 2019-07-04 18:20:32,175:DEBUG:acme.client:Storing nonce: oVPL4mm4XQIGy0nojTq5W3p4zOnIhgqTxQGdZ5k1WO8
 67 2019-07-04 18:20:32,176:DEBUG:acme.client:JWS payload:
 68 {
 69   "identifiers": [
 70     {
 71       "type": "dns",
 72       "value": "www.peregrineit.net"
 73     }
 74   ]
 75 }
 76 2019-07-04 18:20:32,185:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/new-        order:
 77 {
 78   "protected": "eyJub25jZSI6ICJvVlBMNG1tNFhRSUd5MG5valRxNVczcDR6T25JaGdxVHhRR2RaNWsxV084IiwgInVybCI6ICJodHRwczovL2FjbWUt        c3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL25ldy1vcmRlciIsICJraWQiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRz        ZW5jcnlwdC5vcmcvYWNtZS9hY2N0Lzk2MjA5NDciLCAiYWxnIjogIlJTMjU2In0",
 79   "payload": "ewogICJpZGVudGlmaWVycyI6IFsKICAgIHsKICAgICAgInR5cGUiOiAiZG5zIiwgCiAgICAgICJ2YWx1ZSI6ICJ3d3cucGVyZWdyaW5laX        QubmV0IgogICAgfQogIF0KfQ",
 80   "signature": "fze3Q6lWSqQpipxgtVqKtMqS9krXY1Qc5atstkpvqC6rKHiNSFx4NycbBdk_8Ilbw3iLDUBdxkdGV4L3IXabkMsMmGwmtghDV7a_CTcK        nM5pj3_d4wLm-K_mjprco_a2o5GiU29LMQoOA4zLPsVKWtpgDNYeazNm7hZ5CtJtcdUGBb2RV56PvQC--S4Ef6c3Bi5bmhGUqAs4WMs0FuJ2oh6hTOg9kyPT        pqZilykdyhtPTQcC7XgknKehnTE635ps21wv4jpe6OLwqy141qwDFOzg4w-9-dY1G6MSyiJk6DqpLJVFgJYc_aBRKCespmyhzxTUJy9QJ8Vy5W986_D-lQ"
 81 }
 82 2019-07-04 18:20:32,503:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/new-order HTTP/1.1" 201 358
 83 2019-07-04 18:20:32,504:DEBUG:acme.client:Received response:
 84 HTTP 201
 85 content-length: 358
 86 expires: Thu, 04 Jul 2019 08:20:32 GMT
 87 cache-control: max-age=0, no-cache, no-store
 88 strict-transport-security: max-age=604800
 89 server: nginx
 90 connection: keep-alive
 91 link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
 92 location: https://acme-staging-v02.api.letsencrypt.org/acme/order/9620947/39295480
 93 pragma: no-cache
 94 boulder-requester: 9620947
 95 date: Thu, 04 Jul 2019 08:20:32 GMT
 96 x-frame-options: DENY
 97 content-type: application/json
 98 replay-nonce: Wb1FP8dTWnrBbUVeNqHQePlWvIghCI-1XiyUQHXRxgs
 99
100 {
101   "status": "pending",
102   "expires": "2019-07-11T08:20:32.333603281Z",
103   "identifiers": [
104     {
105       "type": "dns",
106       "value": "www.peregrineit.net"
107     }
108   ],
109   "authorizations": [
110     "https://acme-staging-v02.api.letsencrypt.org/acme/authz/v2/518308"
111   ],
      111   ],
112   "finalize": "https://acme-staging-v02.api.letsencrypt.org/acme/finalize/9620947/39295480"
113 }
114 2019-07-04 18:20:32,504:DEBUG:acme.client:Storing nonce: Wb1FP8dTWnrBbUVeNqHQePlWvIghCI-1XiyUQHXRxgs
115 2019-07-04 18:20:32,505:DEBUG:acme.client:JWS payload:
116
117 2019-07-04 18:20:32,509:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/auth        z/v2/518308:
118 {
119   "protected": "eyJub25jZSI6ICJXYjFGUDhkVFduckJiVVZlTnFIUWVQbFd2SWdoQ0ktMVhpeVVRSFhSeGdzIiwgInVybCI6ICJodHRwczovL2FjbWUt        c3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2F1dGh6L3YyLzUxODMwOCIsICJraWQiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFw        aS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hY2N0Lzk2MjA5NDciLCAiYWxnIjogIlJTMjU2In0",
120   "payload": "",
121   "signature": "siCl3kdcfZbCRpJ55MWeAViyKByOefv5QjvwU5-cy0S3Ltm0zblk5eSqURk4ba-tWoMYMFfibjidmmi58a7QOQfUbpsZudD6o1_ieEzA        sJBGc-kU2WgeCL-AdhcpprM8RW6-k8FzGhRYt65la4SRBnHoYAhTvug-LzmGCeYeNzKF2VtRCSzVWFSoHA1Ag9OorZ4ilGMx27z-IomiiIvpY8ZMKyyUjFpq        JD3gqH-CweSyxhuRPLeqcJFAfmsddfp_i1kwBxyzxCXA65vpgLciooEQibQ6cYdP5ZPBxa4U3n_22MmvdxgUZgx9TTe53gOnDhOKJFoc8n-q1oJ0L-eqEg"
122 }
123 2019-07-04 18:20:32,846:DEBUG:requests.packages.urllib3.connectionpool:"POST /acme/authz/v2/518308 HTTP/1.1" 200 821
124 2019-07-04 18:20:32,846:DEBUG:acme.client:Received response:
125 HTTP 200
126 content-length: 821
127 expires: Thu, 04 Jul 2019 08:20:32 GMT
128 cache-control: max-age=0, no-cache, no-store
129 strict-transport-security: max-age=604800
130 server: nginx
131 connection: keep-alive
132 link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
133 pragma: no-cache
134 boulder-requester: 9620947
135 date: Thu, 04 Jul 2019 08:20:32 GMT
136 x-frame-options: DENY
137 content-type: application/json
138 replay-nonce: 1IQoBihtDguaihnN8ywyw6H0pdbTuCUJirtll_aOgLU
139
140 {
141   "identifier": {
142     "type": "dns",
143     "value": "www.peregrineit.net"
144   },
145   "status": "pending",
146   "expires": "2019-07-11T08:20:32Z",
147   "challenges": [
148     {
149       "type": "http-01",
150       "status": "pending",
151       "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/v2/518308/To8ncg",
152       "token": "87FbY9gvIKdVKpq7yVjsYzyZlDphO_2CgGLPV9-uRmA"
153     },
154     {
155       "type": "dns-01",
156       "status": "pending",
157       "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/v2/518308/vnHlsA",
158       "token": "87FbY9gvIKdVKpq7yVjsYzyZlDphO_2CgGLPV9-uRmA"
159     },
160     {
161       "type": "tls-alpn-01",
162       "status": "pending",
163       "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/v2/518308/4tc2CQ",
164       "token": "87FbY9gvIKdVKpq7yVjsYzyZlDphO_2CgGLPV9-uRmA"
165     }
166   ]
167 }

168 2019-07-04 18:20:32,847:DEBUG:acme.client:Storing nonce: 1IQoBihtDguaihnN8ywyw6H0pdbTuCUJirtll_aOgLU
169 2019-07-04 18:20:32,848:INFO:certbot.auth_handler:Performing the following challenges:
170 2019-07-04 18:20:32,849:INFO:certbot.auth_handler:http-01 challenge for www.peregrineit.net
171 2019-07-04 18:20:32,857:DEBUG:acme.standalone:Failed to bind to :54321 using IPv6
172 2019-07-04 18:20:32,863:DEBUG:acme.standalone:Successfully bound to :54321 using IPv4
173 2019-07-04 18:20:32,871:INFO:certbot.auth_handler:Waiting for verification…
174 2019-07-04 18:20:32,872:DEBUG:acme.client:JWS payload:
175 {
176 “type”: “http-01”,
177 “resource”: “challenge”
178 }
179 2019-07-04 18:20:32,877:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/chal lenge/v2/518308/To8ncg:
180 {
181 “protected”: “eyJub25jZSI6ICIxSVFvQmlodERndWFpaG5OOHl3eXc2SDBwZGJUdUNVSmlydGxsX2FPZ0xVIiwgInVybCI6ICJodHRwczovL2FjbWUt c3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2NoYWxsZW5nZS92Mi81MTgzMDgvVG84bmNnIiwgImtpZCI6ICJodHRwczovL2FjbWUtc3Rh Z2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvOTYyMDk0NyIsICJhbGciOiAiUlMyNTYifQ”,
182 “payload”: “ewogICJ0eXBlIjogImh0dHAtMDEiLCAKICAicmVzb3VyY2UiOiAiY2hhbGxlbmdlIgp9”,
183 “signature”: “FbXjGT-PUicoWBFvknEbQYXvoVB1I674dI2sqoGzdycxmoPCFKvqobO2VCLLsAKKr1xnz8OKFE5NZ3V7a2nL8S1LgxJglAgB_zXqxNz5 A8VPnD_HFzg6vXEmyBRjuSyNTetKgxRDWDOh2PC5ti5v8ej1Iqf1iWERouznVYFkO30zWtRHOr4cWuQgSQhNEKq3-SVMW5Kb5DFzg6mx3fG93PnqqTYe6Kqi E7Hyg3U_dXRhVvOEFtFQ5FHFLDiUMqbRgxTdq3elesqMGYdrAxbx8J_GZLMP2PXHNDcLlhUuP8JFJUoYCFmmvURn932B0WfDX2wIMFcF7vD-MPPPPjG-aQ”
184 }
185 2019-07-04 18:20:33,159:DEBUG:requests.packages.urllib3.connectionpool:“POST /acme/challenge/v2/518308/To8ncg HTTP/1.1” 200 193
186 2019-07-04 18:20:33,160:DEBUG:acme.client:Received response:
187 HTTP 200
188 content-length: 193
189 expires: Thu, 04 Jul 2019 08:20:33 GMT
190 cache-control: max-age=0, no-cache, no-store
191 strict-transport-security: max-age=604800
192 server: nginx
193 connection: keep-alive
194 link: https://acme-staging-v02.api.letsencrypt.org/directory;rel=“index”, <https://acme-staging-v02.api.letsencrypt.or g/acme/authz/v2/518308>;rel=“up”
195 location: https://acme-staging-v02.api.letsencrypt.org/acme/challenge/v2/518308/To8ncg
196 pragma: no-cache
197 boulder-requester: 9620947
198 date: Thu, 04 Jul 2019 08:20:33 GMT
199 x-frame-options: DENY
200 content-type: application/json
201 replay-nonce: HvdrjM9vBZMNfc9NGpw2yWwBpMJNfulxFuE9ZwpsevQ
202
203 {
204 “type”: “http-01”,
205 “status”: “pending”,
206 “url”: “https://acme-staging-v02.api.letsencrypt.org/acme/challenge/v2/518308/To8ncg”,
207 “token”: “87FbY9gvIKdVKpq7yVjsYzyZlDphO_2CgGLPV9-uRmA”
208 }
209 2019-07-04 18:20:33,161:DEBUG:acme.client:Storing nonce: HvdrjM9vBZMNfc9NGpw2yWwBpMJNfulxFuE9ZwpsevQ
210 2019-07-04 18:20:34,162:DEBUG:acme.client:JWS payload:
211
212 2019-07-04 18:20:34,167:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/auth z/v2/518308:
213 {
214 “protected”: “eyJub25jZSI6ICJIdmRyak05dkJaTU5mYzlOR3B3MnlXd0JwTUpOZnVseEZ1RTlad3BzZXZRIiwgInVybCI6ICJodHRwczovL2FjbWUt c3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2F1dGh6L3YyLzUxODMwOCIsICJraWQiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFw aS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hY2N0Lzk2MjA5NDciLCAiYWxnIjogIlJTMjU2In0”,
215 “payload”: “”,
216 “signature”: “HO4bUohZ4u4jBC2G78p8RI_gfFTUrBrnrS14iayPF5mvGTg-xBxBAX7_FCKkdO_2FYPv_LefT8iMB2paBQsgd0vTLkedtWawTj-gx0zD Kn7MPByBCyM1S0ogOlPJxe61-5r2oLSlpquXHgM1yAH1V_m2kbhpHsW9p78qWxuD14N1KnkyFb7n9iZBWlZw2lEBhSH-1Wpve2armvucvXY4OvwYemn_Tu9v TspfzPG7L9i_FS9caDi593-IakfTBmoC8FGPE1xLx2y3SWUDbDdEpBl2ligHlOc5PPDGQnY5krVpJ6RZ-3gQWWQ13JzYdgayF4kqTSNOZpAg49U5pLUfEg”
217 }
218 2019-07-04 18:20:34,443:DEBUG:requests.packages.urllib3.connectionpool:“POST /acme/authz/v2/518308 HTTP/1.1” 200 1449
219 2019-07-04 18:20:34,444:DEBUG:acme.client:Received response:
220 HTTP 200
221 content-length: 1449
222 expires: Thu, 04 Jul 2019 08:20:34 GMT
223 cache-control: max-age=0, no-cache, no-store
224 strict-transport-security: max-age=604800
225 server: nginx
226 connection: keep-alive
227 link: https://acme-staging-v02.api.letsencrypt.org/directory;rel=“index”
228 pragma: no-cache
229 boulder-requester: 9620947
230 date: Thu, 04 Jul 2019 08:20:34 GMT
231 x-frame-options: DENY
232 content-type: application/json
233 replay-nonce: 0l_ENMIsj0s-ziGmYH9CgIU0y_z9LTY4hw5QZwqJBiw
234
235 {
236 “identifier”: {
237 “type”: “dns”,
238 “value”: “www.peregrineit.net”
239 },
240 “status”: “invalid”,
241 “expires”: “2019-07-11T08:20:32Z”,
242 “challenges”: [
243 {
244 “type”: “http-01”,
245 “status”: “invalid”,
246 “error”: {
247 “type”: “urn:ietf:params:acme:error:unauthorized”,
248 “detail”: “Invalid response from http://www.peregrineit.net/.well-known/acme-challenge/87FbY9gvIKdVKpq7yVjsYzyZl DphO_2CgGLPV9-uRmA [218.214.86.96]: 503”,
249 “status”: 403
250 },
251 “url”: “https://acme-staging-v02.api.letsencrypt.org/acme/challenge/v2/518308/To8ncg”,
252 “token”: “87FbY9gvIKdVKpq7yVjsYzyZlDphO_2CgGLPV9-uRmA”,
253 “validationRecord”: [
254 {
255 “url”: “http://www.peregrineit.net/.well-known/acme-challenge/87FbY9gvIKdVKpq7yVjsYzyZlDphO_2CgGLPV9-uRmA”,
256 “hostname”: “www.peregrineit.net”,
257 “port”: “80”,
258 “addressesResolved”: [
260 ],
261 “addressUsed”: “218.214.86.96”
262 }
263 ]
264 },
265 {
266 “type”: “dns-01”,
267 “status”: “invalid”,
268 “url”: “https://acme-staging-v02.api.letsencrypt.org/acme/challenge/v2/518308/vnHlsA”,
269 “token”: “87FbY9gvIKdVKpq7yVjsYzyZlDphO_2CgGLPV9-uRmA”
270 },
271 {
272 “type”: “tls-alpn-01”,
273 “status”: “invalid”,
274 “url”: “https://acme-staging-v02.api.letsencrypt.org/acme/challenge/v2/518308/4tc2CQ”,
275 “token”: “87FbY9gvIKdVKpq7yVjsYzyZlDphO_2CgGLPV9-uRmA”
276 }
277 ]
278 }
279 2019-07-04 18:20:34,445:DEBUG:acme.client:Storing nonce: 0l_ENMIsj0s-ziGmYH9CgIU0y_z9LTY4hw5QZwqJBiw
280 2019-07-04 18:20:34,446:WARNING:certbot.auth_handler:Challenge failed for domain www.peregrineit.net
281 2019-07-04 18:20:34,446:INFO:certbot.auth_handler:http-01 challenge for www.peregrineit.net
282 2019-07-04 18:20:34,447:DEBUG:certbot.reporter:Reporting to user: The following errors were reported by the server:
283
284 Domain: www.peregrineit.net
285 Type: unauthorized
286 Detail: Invalid response from http://www.peregrineit.net/.well-known/acme-challenge/87FbY9gvIKdVKpq7yVjsYzyZlDphO_2CgGLP V9-uRmA [218.214.86.96]: 503
287
288 To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address.
289 2019-07-04 18:20:34,448:DEBUG:certbot.error_handler:Encountered exception:
290 Traceback (most recent call last):
291 File “/usr/lib/python2.7/site-packages/certbot/auth_handler.py”, line 90, in handle_authorizations
292 self._poll_authorizations(authzrs, max_retries, best_effort)
293 File “/usr/lib/python2.7/site-packages/certbot/auth_handler.py”, line 154, in _poll_authorizations
294 raise errors.AuthorizationError(‘Some challenges have failed.’)
295 AuthorizationError: Some challenges have failed.
296
297 2019-07-04 18:20:34,448:DEBUG:certbot.error_handler:Calling registered functions
298 2019-07-04 18:20:34,448:INFO:certbot.auth_handler:Cleaning up challenges
299 2019-07-04 18:20:34,449:DEBUG:certbot.plugins.standalone:Stopping server at 0.0.0.0:54321…
300 2019-07-04 18:20:34,867:DEBUG:certbot.log:Exiting abnormally:
301 Traceback (most recent call last):
302 File “/usr/bin/certbot”, line 9, in
303 load_entry_point(‘certbot==0.34.2’, ‘console_scripts’, ‘certbot’)()
304 File “/usr/lib/python2.7/site-packages/certbot/main.py”, line 1379, in main
305 return config.func(config, plugins)
306 File “/usr/lib/python2.7/site-packages/certbot/main.py”, line 1262, in certonly
307 lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
308 File “/usr/lib/python2.7/site-packages/certbot/main.py”, line 120, in _get_and_save_cert
309 lineage = le_client.obtain_and_enroll_certificate(domains, certname)
310 File “/usr/lib/python2.7/site-packages/certbot/client.py”, line 406, in obtain_and_enroll_certificate
311 cert, chain, key, _ = self.obtain_certificate(domains)
312 File “/usr/lib/python2.7/site-packages/certbot/client.py”, line 349, in obtain_certificate
313 orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
314 File “/usr/lib/python2.7/site-packages/certbot/client.py”, line 385, in _get_order_and_authorizations
315 authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
316 File “/usr/lib/python2.7/site-packages/certbot/auth_handler.py”, line 90, in handle_authorizations
317 self._poll_authorizations(authzrs, max_retries, best_effort)
318 File “/usr/lib/python2.7/site-packages/certbot/auth_handler.py”, line 154, in _poll_authorizations
319 raise errors.AuthorizationError(‘Some challenges have failed.’)
320 AuthorizationError: Some challenges have failed.

10

:\

If you run in one terminal:

nc -vvv -l -p 54321

and in another terminal you run:

curl -X GET -I www.peregrineit.net:54321/.well-known/acme-challenge/x

Do you see the request arrive in the nc terminal?

11

Sorry, I get a Command not found - its probably ( :smile:) not in my path - could I have the full path, please?

12 
yum -y install nmap-ncat
13

There is no need to actually install a web server on a computer not having one, there is already a web server on every computer having certbot installed: python. That's the way standalone runs, but it's perfectly possible to make it permanent.

/etc/systemd/system$ cat acme-challenge.service
[Unit]
Description=acme-challenge
After=network.target
ConditionPathExists=/var/www/letsencrypt

[Service]
WorkingDirectory=/var/www/letsencrypt
ExecStart=/usr/bin/python3 -m http.server
Type=simple
SyslogIdentifier=acme-challenge.http.server
User=acme
Group=acme

[Install]
WantedBy=multi-user.target
Alias=acme-challenge.service

in the case of something having to run with haproxy, that could be python -m http.server 1234 to listen on a non standard port.

14

I’ve always used (rarely) ncat from my Windows laptop - I’ve never done it from a Linux Terminal before - I’ve learnt somthing - thanklyou

ncat result:

Ncat: Version 7.50 ( https://nmap.org/ncat )
NCAT DEBUG: Initialized fdlist with 103 maxfds
NCAT DEBUG: do_listen("::"): Address family not supported by protocol
Ncat: Listening on 0.0.0.0:54321
NCAT DEBUG: Added fd 3 to list, nfds 1, maxfd 3
NCAT DEBUG: Added fd 0 to list, nfds 2, maxfd 3
NCAT DEBUG: Initialized fdlist with 100 maxfds
NCAT DEBUG: selecting, fdmax 3

curl result:

HTTP/1.0 503 Service Unavailable
Cache-Control: no-cache
Connection: close
Content-Type: text/html

15

Okay. From that, we learn that something is busted with your haproxy config, or even networking in general. (You can curl 127.0.0.1:54321 while nc is running, right?)

I think from here I would probably take the approach of simplifying your haproxy config until it begins to work. Maybe take a backup and then it with something very basic:

global
    daemon
    maxconn 1024

defaults
    mode http
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

frontend ft_http
    bind :80 
    bind :::80
    use_backend be_certbot if { path_beg /.well-known/acme-challenge/ }
    default_backend be_nginx

backend be_nginx
    server s_nginx 10.123.123.11:80

backend be_certbot
    server s_certbot 127.0.0.1:54321
16

Actuall, no, I can’t curl 127… I get either a conection reset by peer, (from the 2nd terminal), or a connection refused (from the 1st terminal).

I don’t belive it is a IP issue - mainly because the website comes up just fine if its http: - try it, you’ll see.

As the path/stack is: Gateway (Router) => HAProxy (Server) => Nginx (Server) the IP has to be correct. Its only this damned certificate (ie certbot) that is not working :frowning:

17

Then start a website on your nginx, so you can use --webroot. So you don't need --standalone.

PS: So a check

http://www.peregrineit.net/.well-known/acme-challenge/87FbY9gvIKdVKpq7yVjsYzyZlDphO_2CgGLPV9-uRmA

should send a http status 404 - Not Found.

18

To clarify:

  • Two terminals one the same server
  • First terminal has nc -vvv -l -p 54321 running
  • Then (after nc has started), second terminal runs curl 127.0.0.1:54321

and that gives you a “Connection reset by peer” error on the second terminal?

19

Correct - you sense a problem, master?

20

Not possible - hence the use of the HAProxy server in the first place :frowning: