Done and checked - yes it is reachable.
I had to generate the .well-known directory structure - I assume that certbot removes it after use?
Yes, it will remove it after it creates it.
[if it already exists, it will be left alone]
This does throw a monkey wrench in the equation...
curl http://apex-test.dnv.com/.well-known/acme-challenge/Test_File-1234
TEST
hmm...
Try the test cert, but this time change the order of the parameters, like:
sudo certbot certonly --webroot -w /usr/share/nginx/html -d apex-test.dnv.com -v --test-cert
[and remove the "/" at the end of the webroot path]
1 Like
sudo certbot certonly --webroot -w /usr/share/nginx/html -d apex-test.dnv.com -v --test-cert
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Requesting a certificate for apex-test.dnv.com
Performing the following challenges:
http-01 challenge for apex-test.dnv.com
Using the webroot path /usr/share/nginx/html for all unmatched domains.
Waiting for verification...
Challenge failed for domain apex-test.dnv.com
http-01 challenge for apex-test.dnv.com
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: apex-test.dnv.com
Type: unauthorized
Detail: Invalid response from http://apex-test.dnv.com/.well-known/acme-challenge/F42a88ujR5j8zLTT1v-QdKSZPwAjk_S43VJoN7CPBdA [51.124.73.17]: "<html><head><title>Request Rejected</title></head><body>The requested URL was rejected. Please consult with your administrator.<"
Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.
Cleaning up challenges
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
grep Test_File-1234 /var/log/nginx/access.log
10.234.2.54 - - [10/Feb/2022:07:55:59 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 200 18 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36 OPR/82.0.4227.58" "80.231.12.90"
10.234.2.54 - - [10/Feb/2022:07:58:30 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 200 5 "-" "Mozilla/5.0 (Linux; Android 10; moto g(7)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.87 Mobile Safari/537.36" "-"
10.234.2.54 - - [10/Feb/2022:08:01:06 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 200 0 "-" "curl/7.58.0" "-"
10.234.2.54 - - [10/Feb/2022:08:01:12 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 200 5 "-" "curl/7.58.0" "-"
10.234.2.54 - - [10/Feb/2022:08:06:20 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 206 5 "-" "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)" "-"
10.234.2.54 - - [10/Feb/2022:08:06:20 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 206 5 "-" "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)" "-"
10.234.2.54 - - [10/Feb/2022:08:06:20 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 416 213 "-" "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)" "-"
10.234.2.54 - - [10/Feb/2022:08:06:20 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 416 213 "-" "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)" "-"
So, this time nothing from the webroot call (around :08:17).
Network people have confirmed that they're not seeing any denied inbound traffic and that inbound traffic for apex-test.dnv.com is being forwarded to azure1548.
Show:
grep acme-challenge /var/log/nginx/access.log | tail -n 20
1 Like
azurewe1548:/etc # grep acme-challenge /var/log/nginx/access.log | tail -n 20
10.234.2.54 - - [09/Feb/2022:19:29:53 +0000] "GET /.well-known/acme-challenge/Dba3tk5zWH670COWWirTQo8dY9gcdi9KwWdbgkaE3Xk HTTP/1.1" 404 169 "-" "curl/7.79.1" "-"
10.234.2.54 - - [09/Feb/2022:23:29:20 +0000] "GET /.well-known/acme-challenge/KNxDLqCs6q_9l0fQqImEHB8gYRh2dDumafEGs5wH6Ug HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36 OPR/82.0.4227.58" "80.231.12.90"
10.234.2.54 - - [10/Feb/2022:02:43:11 +0000] "GET /.well-known/acme-challenge/Dba3tk5zWH670COWWirTQo8dY9gcdi9KwWdbgkaE3Xk HTTP/1.1" 404 169 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:75.0) Gecko/20100101 Firefox/75.0" "-"
10.234.2.54 - - [10/Feb/2022:02:43:11 +0000] "GET /.well-known/acme-challenge/Dba3tk5zWH670COWWirTQo8dY9gcdi9KwWdbgkaE3Xk HTTP/1.1" 404 169 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:75.0) Gecko/20100101 Firefox/75.0" "-"
10.234.2.54 - - [10/Feb/2022:02:47:24 +0000] "GET /.well-known/acme-challenge/Dba3tk5zWH670COWWirTQo8dY9gcdi9KwWdbgkaE3Xk HTTP/1.0" 404 169 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:75.0) Gecko/20100101 Firefox/75.0" "-"
10.234.2.54 - - [10/Feb/2022:02:47:24 +0000] "GET /.well-known/acme-challenge/Dba3tk5zWH670COWWirTQo8dY9gcdi9KwWdbgkaE3Xk HTTP/1.0" 404 169 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:75.0) Gecko/20100101 Firefox/75.0" "-"
10.234.2.54 - - [10/Feb/2022:02:47:26 +0000] "GET /.well-known/acme-challenge/Dba3tk5zWH670COWWirTQo8dY9gcdi9KwWdbgkaE3Xk HTTP/1.1" 404 169 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:75.0) Gecko/20100101 Firefox/8156" "-"
10.234.2.54 - - [10/Feb/2022:02:47:26 +0000] "GET /.well-known/acme-challenge/Dba3tk5zWH670COWWirTQo8dY9gcdi9KwWdbgkaE3Xk HTTP/1.1" 404 169 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:75.0) Gecko/20100101 Firefox/75.0" "-"
10.234.2.54 - - [10/Feb/2022:07:04:26 +0000] "GET /.well-known/acme-challenge/Hk0Xm9BRE-83pZkE43qZYXTbAqUCSh1Oqb5TJP7RqXc HTTP/1.1" 404 169 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Safari/605.1.15" "-"
10.234.2.54 - - [10/Feb/2022:07:55:14 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 200 5 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36 OPR/82.0.4227.58" "80.231.12.90"
10.234.2.54 - - [10/Feb/2022:07:55:59 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 200 18 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36 OPR/82.0.4227.58" "80.231.12.90"
10.234.2.54 - - [10/Feb/2022:07:58:30 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 200 5 "-" "Mozilla/5.0 (Linux; Android 10; moto g(7)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.87 Mobile Safari/537.36" "-"
10.234.2.54 - - [10/Feb/2022:08:01:06 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 200 0 "-" "curl/7.58.0" "-"
10.234.2.54 - - [10/Feb/2022:08:01:08 +0000] "GET /.well-known/acme-challenge/Hk0Xm9BRE-83pZkE43qZYXTbAqUCSh1Oqb5TJP7RqXc HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36" "-"
10.234.2.54 - - [10/Feb/2022:08:01:12 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 200 5 "-" "curl/7.58.0" "-"
10.234.2.54 - - [10/Feb/2022:08:01:41 +0000] "GET /.well-known/acme-challenge/Hk0Xm9BRE-83pZkE43qZYXTbAqUCSh1Oqb5TJP7RqXc HTTP/1.1" 404 169 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0" "-"
10.234.2.54 - - [10/Feb/2022:08:06:20 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 206 5 "-" "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)" "-"
10.234.2.54 - - [10/Feb/2022:08:06:20 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 206 5 "-" "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)" "-"
10.234.2.54 - - [10/Feb/2022:08:06:20 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 416 213 "-" "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)" "-"
10.234.2.54 - - [10/Feb/2022:08:06:20 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 416 213 "-" "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)" "-"
'10.' is our LB, '80.' is my desktop, 'Android' is my phone (checking 'external' network vs office network)
Well this has gotten interesting. I also see your test file with curl from a test server in the AWS East Coast region.
The Let's Debug site fails in the same way as the actual Let's Encrypt servers. Its summary is that all is OK but if you look at the detailed results it gets the same 403 error saying "... <title>Request Rejected</title></head><body>The requested URL was rejected. ..."
Perhaps Let's Debug should treat a 403 as a failure instead of a successful test as it should be getting a 404 as the file name it uses does not exist. But, it doesn't.
See that result here. Click Rerun test on top of page to retry.
The user agent is similar but slightly different than actual LE servers:
Mozilla/5.0 (compatible; Let's Debug emulating Let's Encrypt validation server; +https://letsdebug.net)
2 Likes
Mike,
I appreciate your help, but I've decided to kill this server.
I've built another server in 'standard' MS azure land and that had no issues with getting certified by Lets Encrypt.
I'm assuming that it's an issue with our network blocking access from Let's Encrypt, but I've no idea why.
Again, thanks for your help everyone.
2 Likes
Update: Found cause. The single quote in Let's in the user agent causes the failure.
I recreated the problem using curl and the same user agent string as Let's Encrypt. My initial request failed same as we see with LE servers.
curl -A "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" http://apex-test.dnv.com/.well-known/acme-challenge/Test_File-1234
Response:
<html><head><title>Request Rejected</title></head><body>The requested URL was rejected. Please consult with your administrator.<br><br>Your support ID is: 18339266511516962734<br><br><a href='javascript:history.back();'>[Go Back]</a></body></html>
Does that Support ID help?
Changing Let's to Lets results in a successful get.
2 Likes
Thanks, I'll check with the network people.
2 Likes
Yes, despite their previous statement they were blocking the LE user agent (not sure if it was by design or by accident!).
Thanks for all your help,
2 Likes
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.