Renewal is failing with acme challenge 404

Help
22

I think the problem is within Apache
[but I always think that]

Please show the ENTIRE output of:
sudo apachectl -t -D DUMP_VHOSTS

1 Like
23

root@ip-172-31-44-194:/etc/apache2/sites-available# sudo apachectl -t -D DUMP_VHOSTS
VirtualHost configuration:
*:443 admin.wizpms.com (/etc/apache2/sites-enabled/000-default-le-ssl.conf:2)
*:80 ip-172-31-44-194.us-east-2.compute.internal (/etc/apache2/sites-enabled/000-default.conf:1)
root@ip-172-31-44-194:/etc/apache2/sites-available#

24

hmm...

Please show:
ls -l /var/www.html/.well-known/acme-challenge/

1 Like
25

root@ip-172-31-44-194:/etc/apache2/sites-available# ls -l /var/www.html/.well-known/acme-challenge/
total 4
-rw-r--r-- 1 root root 5 Apr 15 17:47 Test_File-1234
root@ip-172-31-44-194:/etc/apache2/sites-available#

26

OK we scroll it all the way back and...
begin at the begining!

Please show the output of:
curl -4 ifconfig.co

1 Like
27

root@ip-172-31-44-194:/etc/apache2/sites-available# curl -4 ifconfig.co
52.14.105.212
root@ip-172-31-44-194:/etc/apache2/sites-available#

28

OK, that's a match:

Name:    admin.wizpms.com
Address: 52.14.105.212

now...
I would suggest we handle the challenge requests in HTTP (rather than redirecting them to HTTPS).
For that, please replace file: /etc/apache2/sites-enabled/000-default.conf
with the following:

<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/html
  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined
  #skip challenge requests and
  <LocationMatch "^/(?!\.well-known)">
    #send all other requests to HTTPS
    RewriteEngine On
    RewriteCond %{HTTPS} !=on
    RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1
  </LocationMatch>
</VirtualHost>

Then restart Apache

1 Like
29

Made changes, and restarted.

30

Please show the tail of the Apache error.log and access.log files.

I'm getting:

curl -Ii http://admin.wizpms.com/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 404 Not Found
Date: Fri, 15 Apr 2022 20:56:06 GMT
Server: Apache/2.4.29 (Ubuntu)
Content-Type: text/html; charset=iso-8859-1
1 Like
31

access log

17.121.204.209 - - [15/Apr/2022:20:58:21 +0000] "GET /.well-known/acme-challenge/W6Mw80GXwXMhsBuvAJ85Livxe3HAtqGnSvN2j6agA5w HTTP/1.1" 404 498 "-" "AppleNewsBot"
17.121.204.209 - - [15/Apr/2022:20:58:21 +0000] "GET /.well-known/acme-challenge/g_p6YJuOKhIFFkpeiE9Luypy25Zf2iDBLFZ5l2ly4ro HTTP/1.1" 404 5169 "-" "AppleNewsBot"
17.121.204.209 - - [15/Apr/2022:20:58:21 +0000] "GET /.well-known/acme-challenge/gIewdfL4m6tkbKt7QZY8CCRQcUoV6GxEZfy4TSkPfPI HTTP/1.1" 404 493 "-" "AppleNewsBot"
17.121.204.209 - - [15/Apr/2022:20:58:22 +0000] "GET /.well-known/acme-challenge/aDBI0e-v8Hbv9AzzO90Rtg2QOmilsXhWC-jeJhhZLAM HTTP/1.1" 404 495 "-" "AppleNewsBot"
17.121.204.209 - - [15/Apr/2022:20:58:22 +0000] "GET /.well-known/acme-challenge/nA19Sbz_7a2sr4jG2X_5pBkmQlLBqjND62h9iT6Q608 HTTP/1.1" 404 494 "-" "AppleNewsBot"
17.121.204.209 - - [15/Apr/2022:20:58:22 +0000] "GET /.well-known/acme-challenge/m5gQGgLSPrzV-gizndXORSViinyY16v7inpu5Ta1zBA HTTP/1.1" 404 495 "-" "AppleNewsBot"
17.121.204.209 - - [15/Apr/2022:20:58:22 +0000] "GET /.well-known/acme-challenge/7WJ9ebplKAwM-ZYvLk8hRM9tewGZxiF5wUdVirlVoD8 HTTP/1.1" 404 5166 "-" "AppleNewsBot"
45.155.204.20 - - [15/Apr/2022:20:59:22 +0000] "GET /js/chunk-78a5aa55.e794d1c0.js HTTP/1.1" 200 7180 "-" "osee2unifiedRelease/1466 osee2unifiedReleaseVersion/6.9.1 Mozilla/5.0 (iPhone; CPU iPhone OS 12_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148"
52.14.105.212 - - [15/Apr/2022:20:59:55 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 140 "-" "curl/7.58.0"

Nothing in the error log

33

That is very strange!
The IP shown is the IP of the server.
As if there is some local proxy... to itself.

Or did you try that curl request from the server itself?

1 Like
34

Yes. I did run the curl from the same server

35

What's this fetching those for?

36

OK that explains it.
Show:
grep Test_File-1234 access.log

1 Like
37

because it fetches every link posted in discord.

1 Like
38

root@ip-172-31-44-194:/var/log/apache2# grep Test_File-1234 access.log

70.226.121.235 - - [15/Apr/2022:17:47:37 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 5399 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"
18.211.147.95 - - [15/Apr/2022:17:59:45 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 301 659 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
35.172.241.34 - - [15/Apr/2022:18:00:07 +0000] "GET /.well-known/acme-challenge/Test_File-1234%60 HTTP/1.1" 301 661 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
38.205.191.197 - - [15/Apr/2022:18:00:10 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 301 603 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36"
38.205.191.215 - - [15/Apr/2022:18:00:19 +0000] "GET /.well-known/acme-challenge/Test_File-1234%60 HTTP/1.1" 301 605 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.84 Safari/537.36"
185.178.95.254 - - [15/Apr/2022:18:02:05 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 301 659 "-" "Mozilla/5.0 (Linux; Android 11; M2007J20CG) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Mobile Safari/537.36"
75.51.0.154 - - [15/Apr/2022:20:43:51 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 301 226 "-" "curl/7.58.0"
75.51.0.154 - - [15/Apr/2022:20:43:56 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 5059 "-" "curl/7.58.0"
75.51.0.154 - - [15/Apr/2022:20:52:06 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 301 226 "-" "curl/7.58.0"
75.51.0.154 - - [15/Apr/2022:20:52:13 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 301 226 "-" "curl/7.58.0"
75.51.0.154 - - [15/Apr/2022:20:52:33 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 301 226 "-" "curl/7.58.0"
75.51.0.154 - - [15/Apr/2022:20:52:40 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 301 226 "-" "curl/7.58.0"
75.51.0.154 - - [15/Apr/2022:20:52:49 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 301 226 "-" "curl/7.58.0"
75.51.0.154 - - [15/Apr/2022:20:54:47 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 140 "-" "curl/7.58.0"
75.51.0.154 - - [15/Apr/2022:20:56:06 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 140 "-" "curl/7.58.0"
75.51.0.154 - - [15/Apr/2022:20:56:48 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 140 "-" "curl/7.58.0"
52.14.105.212 - - [15/Apr/2022:20:59:55 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 140 "-" "curl/7.58.0"
52.14.105.212 - - [15/Apr/2022:21:02:17 +0000] "HEAD /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 140 "-" "curl/7.58.0"
70.226.121.235 - - [15/Apr/2022:21:03:03 +0000] "GET /.well-known/acme-challenge/Test_File-1234 HTTP/1.1" 404 495 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"
70.226.121.235 - - [15/Apr/2022:21:03:04 +0000] "GET /favicon.ico HTTP/1.1" 302 588 "http://admin.wizpms.com/.well-known/acme-challenge/Test_File-1234" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36"
39

That's not it. 17.121.204.209 belongs to Apple, Inc.

The Discourse fetcher uses IPs belonging to them explicitly. Maybe it works differently on iOS.

40

What don't you get?

Apple fetches every link posted in discord.

1 Like
41

I get that Apple is fetching.

I don't get why it does that, or what Discourse has to do with it.

42

Discourse has nothing to do with Apple querying public information from a public site.

1 Like