I get a validation to www sub domain, but I want the validation to arrive at the domain I specify

Hello,

(I have filled in the form below, and even included parts of /var/log/letsencrypt/letsencrypt.log at the end, but I don't think much of it is interesting).

I am trying to get a certificate for imo2022.org, but the challenge seems to be sent to the sub domain www.imo2022.org, which I do not want a certificate for. I run Varnish and I synthesize a 301 redirect to www.imo2022.org for all requests to the host name imo2022.org whose path do not start with /.well-known. The www subdomain is maintained by an external company, and they manage their own certificate independently of me. Running varnishlog on the server for imo2022.org clearly shows that letsencrypt does not send a request to imo2022.org. I control DNS, but I prefer to use a HTTP for validation, as it works great with Varnish.

Do I have to change to DNS to be able to get a certificate for imo2022.org?

I can't see this documented anywhere. If I am blind, a link to the relevant docs will be fine.

My domain is: imo2022.org

I ran this command: certbot certonly --test-cert --standalone --preferred-challenges http --http-01-port 888 -d imo2022.org --renew-hook="/usr/local/bin/hitch-renew-hook" --post-hook="service hitch reload"

It produced this output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Requesting a certificate for imo2022.org
Performing the following challenges:
http-01 challenge for imo2022.org
Waiting for verification...
Challenge failed for domain imo2022.org
http-01 challenge for imo2022.org

Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems:
Domain: imo2022.org
Type: unauthorized
Detail: Invalid response from https://www.imo2022.org/.well-known/acme-challenge/Q4TFOBC8SXZ3wivB6YNtwxAEPuoRv67dTQ7zilLFyAk [52.213.47.89]: "\r\n<html lang="en-US">\r\n\r\n <meta charset="UTF-8" />\r\n <meta name="viewport" content="width=device-widt"

Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 888. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet.

Cleaning up challenges
Running post-hook command: service hitch reload
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version): varnish-plus-6.0.9r3 with a VCL which routes .well-known to port 888.

The operating system my web server runs on is (include version): Ubuntu 18.04.6 LTS

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.22.0

From /var/log/letsencrypt/letsencrypt.log

2022-01-22 13:33:39,040:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Sat, 22 Jan 2022 13:33:38 GMT
Content-Type: application/json
Content-Length: 1979
Connection: keep-alive
Boulder-Requester: 41417418
Cache-Control: public, max-age=0, no-cache
Link: https://acme-staging-v02.api.letsencrypt.org/directory;rel="index"
Replay-Nonce: 0001mNgheHp_ePq9UNt_NDHGs6KKKycoqil1bR0v16u2HAQ
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
"identifier": {
"type": "dns",
"value": "imo2022.org"
},
"status": "invalid",
"expires": "2022-01-29T13:33:34Z",
"challenges": [
{
"type": "http-01",
"status": "invalid",
"error": {
"type": "urn:ietf:params:acme:error:unauthorized",
"detail": "Invalid response from https://www.imo2022.org/.well-known/acme-challenge/-VVYGtEixrQO9YYWp8Kw3dHysCrHAYueOFCNYtYJFA4 [52.213.47.89]: "\u003c!DOCTYPE html\u003e\r\n\u003chtml lang=\"en-US\"\u003e\r\n\u003chead\u003e\r\n \u003cmeta charset=\"UTF-8\
" /\u003e\r\n \u003cmeta name=\"viewport\" content=\"width=device-widt"",
"status": 403
},
"url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1496701498/KOIkZQ",
"token": "-VVYGtEixrQO9YYWp8Kw3dHysCrHAYueOFCNYtYJFA4",
"validationRecord": [
{
"url": "http://imo2022.org/.well-known/acme-challenge/-VVYGtEixrQO9YYWp8Kw3dHysCrHAYueOFCNYtYJFA4",
"hostname": "imo2022.org",
"port": "80",
"addressesResolved": [
"165.227.172.28",
"2001:700:100:118::130"
],
"addressUsed": "2001:700:100:118::130"
},
{
"url": "https://imo2022.org/.well-known/acme-challenge/-VVYGtEixrQO9YYWp8Kw3dHysCrHAYueOFCNYtYJFA4",
"hostname": "imo2022.org",
"port": "443",
"addressesResolved": [
"165.227.172.28",
"2001:700:100:118::130"
],
"addressUsed": "2001:700:100:118::130"
},
{
"url": "https://www.imo2022.org/.well-known/acme-challenge/-VVYGtEixrQO9YYWp8Kw3dHysCrHAYueOFCNYtYJFA4",
"hostname": "www.imo2022.org",
"port": "443",
"addressesResolved": [
"52.213.47.89",
"3.248.123.157"
],
"addressUsed": "52.213.47.89"
}
],
"validated": "2022-01-22T13:33:34Z"
}
]
}

1 Like

Welcome @hermunn

Requests to /.well-known are also redirecting. As shown in your post and see below.

Better would be to not redirect the original http request and pass that to port 888 for the certbot standalone to process.

curl -I http://imo2022.org/.well-known/acme-challenge/Forum123

HTTP/1.1 301 Moved Permanently
Content-length: 0
Location: https://imo2022.org/.well-known/acme-challenge/Forum123

+++++++

curl -Ik https://imo2022.org/.well-known/acme-challenge/Forum123

HTTP/2 301
server: nginx
date: Sat, 22 Jan 2022 14:35:01 GMT
content-type: text/html
content-length: 162
location: https://www.imo2022.org/.well-known/acme-challenge/Forum123
strict-transport-security: max-age=31536000
content-security-policy: upgrade-insecure-requests
permissions-policy: interest-cohort=()
3 Likes

Hello, and thank you for the reply. This does not make sense at all, or there is something seriously bad going on.

As shown in your post and see below.

I am not sure who it is show in my post (except that the response in the log indicates that are quest was made for https://www.imo2022.org/.well-known/acme-challenge/-VVYGtEixrQO9YYWp8Kw3dHysCrHAYueOFCNYtYJFA4

When it comes to your curl command, I get something very different (tried from a few locations):

curl -I http://imo2022.org/.well-known/acme-challenge/Forum123
HTTP/1.1 503 Backend fetch failed
Date: Sat, 22 Jan 2022 17:28:32 GMT
content-type: text/html; charset=utf-8
Retry-After: 5
Connection: close
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Frame-Options: DENY
Age: 0
Cache-Control: max-age=1
Link: <http://hermunn.no/foo?bar=baz>; secret="Not a secret"

The above is as expected - when I run this command in a normal setting, nothing is listening on :888, and a synthetic 503 Backend fetch failed is generated by my Varnish.

The response you are pasting is generated neither by my Varnish (which does not add Server: nginx), my nginx behind varnish (which generates Server: nginx/1.14.0) nor the web server at www.imo2022.org (which does not generate a Server header at all.

The obvious question is then, what IP hides behind imo2022.org from your location. I have this:

$ dig imo2022.org

; <<>> DiG 9.16.1-Ubuntu <<>> imo2022.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8002
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;imo2022.org.			IN	A

;; ANSWER SECTION:
imo2022.org.		1421	IN	A	165.227.172.28

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: lø. jan. 22 18:42:51 CET 2022
;; MSG SIZE  rcvd: 56

Please note that I was successful in generating a certificate for hermunn.no using the exact same configuration (VCL), and the same ISP is used for hermunn.no's DNS, but there I am also requesting alternate names in the same command like this:

sudo certbot certonly --standalone --preferred-challenges http --http-01-port 888 \
     -d www.hermunn.no \
     -d imo2022.no \
     -d www.imo2022.no \
     -d frankfurt.hermunn.no \
     -d imo2022.net \
     -d www.imo2022.net \
     -d imo2022.com \
     -d www.imo2022.com \
     -d hermunn.no \
     -d skraphaugen.hermunn.no \
     -d openasocial.com \
     -d www.openasocial.com \
     -d annakristine.no \
     -d www.annakristine.no \
     -d fb2.hermunn.no \
	-d balticway2009.hermunn.no \
     --renew-hook="/usr/local/bin/hitch-renew-hook" --post-hook="service hitch reload"

The certificate from the command above is in use (by hitch), as you can see from the start date in the output of curl -Iv https://www.hermunn.no

3 Likes

Hmm. The DNS lookup from my server shows IPv4 and IPv6 addresses. So, Let's Encrypt server would favor the IPv6 IP as would my curl since I have active IPv6.

If I force IPv4 on my curl (-I4) I also see the 503 backend fetch failure. So, resolving the DNS is step 1.

nslookup imo2022.org

Name:   imo2022.org
Address: 165.227.172.28
Name:   imo2022.org
Address: 2001:700:100:118::130

UPDATE: Oddly, the Let's Encrypt message shows it was trying IP: 52.213.47.89
And, yes, I said your post showed redirect as Let's Encrypt must have been redirected as it starts with an http request.
UPDATE2: Ah, I just realized the DNS for www is different so that explains the IP in LE message:

nslookup www.imo2022.org

www.imo2022.org canonical name = elb-1457-imo2022-org.swoogo.com.
Name:   elb-1457-imo2022-org.swoogo.com
Address: 52.213.47.89
Name:   elb-1457-imo2022-org.swoogo.com
Address: 3.248.123.157
3 Likes

Thank you so much. This solved everything. I failed hard when it came to updating the IPv6 DNS record, which should have been deleted. Now I just have to wait for 14400 seconds and retry.

I really appreciate the help, and I feel really silly for not realizing that this could be a IPv6 issue.

3 Likes

No worries. Cheers

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.