Cert renew failing, self-hosted CentOS 7 with Apache

Help
1

My domain is: blog.elkhart.k12.in.us

I ran this command:

certbot renew -v --dry-run

It produced this output:

Verbose output

Certificate is due for renewal, auto-renewing...
Plugins selected: Authenticator apache, Installer apache
Simulating renewal of an existing certificate for blog.elkhart.k12.in.us
Performing the following challenges:
http-01 challenge for blog.elkhart.k12.in.us
Waiting for verification...
Challenge failed for domain blog.elkhart.k12.in.us
http-01 challenge for blog.elkhart.k12.in.us

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
  Domain: blog.elkhart.k12.in.us
  Type:   unauthorized
  Detail: Invalid response from http://blog.elkhart.k12.in.us/.well-known/acme-challenge/PpJWV-YfK40Jb_hwMUjhqWS1_y5BtVv6oeHpildEm-k [173.245.253.4]: 503

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

Cleaning up challenges
Failed to renew certificate blog.elkhart.k12.in.us with error: Some challenges have failed.

My web server is (include version):

Server version: Apache/2.4.6 (CentOS)
Server built:   Jan 25 2022 14:08:43

The operating system my web server runs on is (include version):

CentOS 7.9.2009

My hosting provider, if applicable, is: N/A

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):

certbot 1.26.0

My webserver is up and running with several VirtualHost configs, so I'm not sure where the 503 error is coming from. I've also confirmed:

Checking that file with curl also works:

❯ curl -I -m10 http://blog.elkhart.k12.in.us/.well-known/acme-challenge/testfile
HTTP/1.1 200 OK
Date: Fri, 15 Apr 2022 18:12:38 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips Phusion_Passenger/6.0.4 mod_wsgi/3.4 Python/2.7.5 PHP/7.3.33
Last-Modified: Fri, 15 Apr 2022 17:58:33 GMT
ETag: "0-5dcb528ad70b0"
Accept-Ranges: bytes
Content-Type: text/plain; charset=UTF-8

I'm at a loss for what to do at this point. Any tips on where to look next?

2

Welcome to the community @bbennett

Looks like the 503 is coming from "ECS" (see the title portion below). I assume you know them? Was your attempt at that file from the public internet? Because I got:

curl -i  http://blog.elkhart.k12.in.us/.well-known/acme-challenge/testfile

HTTP/1.1 503 Service Unavailable
Content-Type: text/html; charset=UTF-8
Content-Length: 1853
Connection: close
P3P: CP="CAO PSA OUR"
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache

<html>
<head>
<title>Application Restricted by ECS</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

</head>
(... rest omitted for brevity ...)

We have had a number of people recently whose providers started blocking paths for the acme challenge. You might want to check with them because this seems ok (note the missing challenge file name):

curl -i http://blog.elkhart.k12.in.us/.well-known/acme-challenge

HTTP/1.1 301 Moved Permanently
Date: Fri, 15 Apr 2022 19:01:10 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips Phusion_Passenger/6.0.4 mod_wsgi/3.4 Python/2.7.5 PHP/7.3.33
Location: http://blog.elkhart.k12.in.us/.well-known/acme-challenge/
Content-Length:
6 Likes
3

Oh good grief. Yeah, we're a school district and there's another layer of filters from outside traffic I always forget about.

I'll see if I can get my IT department to look into it.

2 Likes
4

Sorry @MikeMcQ, one more question:

Is there a URL that LE uses to issue acme challenges? Or is there documentation somewhere I can send to our network techs?

5

They all have this format:

http://blog.elkhart.k12.in.us/.well-known/acme-challenge/ChallengeFileName

ChallengeFileName will be a mixture of letters, numbers, and maybe some symbols

It will always start with http but will follow redirects if you send it to like https

7 Likes
6

Hi @bbennett
I am seeing 2 recent LE cert issued today.
@rg305 pointed out that I may need "slightly thicker glasses"

I'll make the appointment :face_with_monocle:

5 Likes
7

Expiring today!

7 Likes
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.