CentOS Stream 9 Error

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: youngones.net.nz

I ran this command: sudo certbot -v --apache

It produced this output: [sudo] password for youngone:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache

Which names would you like to activate HTTPS for?
We recommend selecting either all domains, or all domains in a VirtualHost/server block.


1: youngones.net.nz


Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel):
Requesting a certificate for youngones.net.nz
Performing the following challenges:
http-01 challenge for youngones.net.nz
Waiting for verification...
Challenge failed for domain youngones.net.nz
http-01 challenge for youngones.net.nz

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
Domain: youngones.net.nz
Type: unauthorized
Detail: 115.188.30.169: Invalid response from https://youngones.net.nz: "\n<html lang="en-US">\n\n\t<meta charset="UTF-8" />\n\t<meta name="viewport" content="width=device-width, initial"

Hint: The Certificate Authority failed to verify the temporary Apache configuration changes made by Certbot. Ensure that the listed domains point to this Apache server and that it is accessible from the internet.

Cleaning up challenges
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

My web server is (include version):Apache 2.4.57 (CentOS Stream)

The operating system my web server runs on is (include version): CentOS Stream 9

My hosting provider, if applicable, is: No-Ip

I can login to a root shell on my machine (yes or no, or I don't know): Yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): 2.7.4

Installed Certbot using snap, as per instructions.
The weird bit I'd like help with if possible is:

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
** Domain: youngones.net.nz**
** Type: unauthorized**
** Detail: 115.188.30.169: Invalid response from https://youngones.net.nz: "\n<html lang="en-US">\n\n\t<meta charset="UTF-8" />\n\t<meta name="viewport" content="width=device-width, initial"**
That's clearly coming from some config file, but I'm blowed if I can find it.
httpd -t gives Syntax OK

Hi @PartTimeZombie, and welcome to the LE community forum :slight_smile:

That shows that the HTTP request was redirected to HTTPS.
[it could have been handled in HTTP]
And the HTTPS site doesn't handle the ACME challenge requests as expected.

That said, and since you are using Apache, I would start by ensuring that everything is configured as you expect.
Starting with the output of:
sudo apachectl -t -D DUMP_VHOSTS

1 Like

Thanks for the reply.
I'm running CentOS so apachectl doesn't work, but httpd does:
VirtualHost configuration:
*:443 youngones.net.nz (/etc/httpd/conf.d/ssl.conf:40)
*:80 is a NameVirtualHost
default server youngones.net.nz (/etc/httpd/conf.d/vhost.conf:3)
port 80 namevhost youngones.net.nz (/etc/httpd/conf.d/vhost.conf:3)
port 80 namevhost youngones.net.nz (/etc/httpd/conf.d/vhost.conf:9)
I'm a bit unsure what that means

1 Like

This looks like there is a "name:port" overlap:

We should have a look at that file.

3 Likes

Thanks for tip.
I have messed about with httpd.conf and vhost.conf and wound up with this output:
$ sudo httpd -t -D DUMP_VHOSTS
VirtualHost configuration:
*:443 youngones.net.nz (/etc/httpd/conf.d/ssl.conf:40)
*:80 youngones.net.nz (/etc/httpd/conf.d/vhost.conf:9)
which looks promising but I get the same error:
Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
Domain: youngones.net.nz
Type: unauthorized
Detail: 115.188.30.169: Invalid response from https://youngones.net.nz: "\n<html lang="en-US">\n\n\t<meta charset="UTF-8" />\n\t<meta name="viewport" content="width=device-width, initial"

I'm not sure where to go with this at all, so I might just go ahead and reinstall httpd

Since this is were that failure occurs:

It is clear to me that the redirection is part of the problem.
Please show us this file:

3 Likes

settings for original domain

#<VirtualHost *:80>
#    DocumentRoot /var/www/wordpress
#    ServerName youngones.net.nz
#</VirtualHost>

# settings for new domain
<VirtualHost *:80>
    DocumentRoot /var/www/wordpress
    ServerName youngones.net.nz
    ServerAdmin alistair.young@xtra.co.nz
    ErrorLog logs/youngones-error_log
    CustomLog logs/youngones-access_log combined
    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</VirtualHost>

<Directory "/var/www/wordpress">
    Options FollowSymLinks
    AllowOverride All
</Directory>
curl -Ii youngones.net.nz
HTTP/1.1 302 Moved Temporarily
Date: Thu, 30 Nov 2023 06:11:41 GMT
Content-Type: text/html
Content-Length: 154
Connection: keep-alive
Location: https://youngones.net.nz     <<<<<<<<<< EXPECTED
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff

curl -Ii youngones.net.nz/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 302 Moved Temporarily
Date: Thu, 30 Nov 2023 06:12:02 GMT
Content-Type: text/html
Content-Length: 154
Connection: keep-alive
Location: https://youngones.net.nz     <<<<<<<<<< UNEXPECTED
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff

I think you might not be showing us the vhost that is actually serving that site.
I suspect there might be some "name:port" overlapping.
Show us the output of:
httpd -t -D DUMP_VHOSTS

3 Likes

sudo httpd -t -D DUMP_VHOSTS
[sudo] password for youngone:

VirtualHost configuration:
*:443                  youngones.net.nz (/etc/httpd/conf.d/ssl.conf:40)
*:80                   youngones.net.nz (/etc/httpd/conf.d/vhost.conf:9)

youngone@sol ~]$ cat /etc/httpd/conf.d/ssl.conf

#
# When we also provide SSL we have to listen to the 
# standard HTTPS port in addition.
#
Listen 443 https

##
##  SSL Global Context
##
##  All SSL configuration in this context applies both to
##  the main server and all SSL-enabled virtual hosts.
##

#   Pass Phrase Dialog:
#   Configure the pass phrase gathering process.
#   The filtering dialog program (`builtin' is a internal
#   terminal dialog) has to provide the pass phrase on stdout.
SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog

#   Inter-Process Session Cache:
#   Configure the SSL Session Cache: First the mechanism 
#   to use and second the expiring timeout (in seconds).
SSLSessionCache         shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout  300

#
# Use "SSLCryptoDevice" to enable any supported hardware
# accelerators. Use "openssl engine -v" to list supported
# engine names.  NOTE: If you enable an accelerator and the
# server does not start, consult the error logs and ensure
# your accelerator is functioning properly. 
#
SSLCryptoDevice builtin
#SSLCryptoDevice ubsec

##
## SSL Virtual Host Context
##

<VirtualHost _default_:443>

# General setup for the virtual host, inherited from global configuration
DocumentRoot "/var/www/wordpress"
ServerName youngones.net.nz

# Use separate log files for the SSL virtual host; note that LogLevel
# is not inherited from httpd.conf.
ErrorLog logs/ssl_error_log
TransferLog logs/ssl_access_log
LogLevel warn

#   SSL Engine Switch:
#   Enable/Disable SSL for this virtual host.
SSLEngine on

#   List the protocol versions which clients are allowed to connect with.
#   The OpenSSL system profile is used by default.  See
#   update-crypto-policies(8) for more details.
#SSLProtocol all -SSLv3
#SSLProxyProtocol all -SSLv3

#   User agents such as web browsers are not configured for the user's
#   own preference of either security or performance, therefore this
#   must be the prerogative of the web server administrator who manages
#   cpu load versus confidentiality, so enforce the server's cipher order.
SSLHonorCipherOrder on

#   SSL Cipher Suite:
#   List the ciphers that the client is permitted to negotiate.
#   See the mod_ssl documentation for a complete list.
#   The OpenSSL system profile is configured by default.  See
#   update-crypto-policies(8) for more details.
SSLCipherSuite PROFILE=SYSTEM
SSLProxyCipherSuite PROFILE=SYSTEM

#   Point SSLCertificateFile at a PEM encoded certificate.  If
#   the certificate is encrypted, then you will be prompted for a
#   pass phrase.  Note that restarting httpd will prompt again.  Keep
#   in mind that if you have both an RSA and a DSA certificate you
#   can configure both in parallel (to also allow the use of DSA
#   ciphers, etc.)
#   Some ECC cipher suites (http://www.ietf.org/rfc/rfc4492.txt)
#   require an ECC certificate which can also be configured in
#   parallel.
SSLCertificateFile /etc/pki/tls/certs/localhost.crt

#   Server Private Key:
#   If the key is not combined with the certificate, use this
#   directive to point at the key file.  Keep in mind that if
#   you've both a RSA and a DSA private key you can configure
#   both in parallel (to also allow the use of DSA ciphers, etc.)
#   ECC keys, when in use, can also be configured in parallel
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

#   Server Certificate Chain:
#   Point SSLCertificateChainFile at a file containing the
#   concatenation of PEM encoded CA certificates which form the
#   certificate chain for the server certificate. Alternatively
#   the referenced file can be the same as SSLCertificateFile
#   when the CA certificates are directly appended to the server
#   certificate for convenience.
#SSLCertificateChainFile 

#   Certificate Authority (CA):
#   Set the CA certificate verification path where to find CA
#   certificates for client authentication or alternatively one
#   huge file containing all of them (file must be PEM encoded)
#SSLCACertificateFile /etc/pki/tls/certs/ca-bundle.crt

#   Client Authentication (Type):
#   Client certificate verification type and depth.  Types are
#   none, optional, require and optional_no_ca.  Depth is a
#   number which specifies how deeply to verify the certificate
#   issuer chain before deciding the certificate is not valid.
#SSLVerifyClient require
#SSLVerifyDepth  10

#   Access Control:
#   With SSLRequire you can do per-directory access control based
#   on arbitrary complex boolean expressions containing server
#   variable checks and other lookup directives.  The syntax is a
#   mixture between C and Perl.  See the mod_ssl documentation
#   for more details.
#<Location />
#SSLRequire (    %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
#            and %{SSL_CLIENT_S_DN_O} eq "Snake Oil, Ltd." \
#            and %{SSL_CLIENT_S_DN_OU} in {"Staff", "CA", "Dev"} \
#            and %{TIME_WDAY} >= 1 and %{TIME_WDAY} <= 5 \
#            and %{TIME_HOUR} >= 8 and %{TIME_HOUR} <= 20       ) \
#           or %{REMOTE_ADDR} =~ m/^192\.76\.162\.[0-9]+$/
#</Location>

#   SSL Engine Options:
#   Set various options for the SSL engine.
#   o FakeBasicAuth:
#     Translate the client X.509 into a Basic Authorisation.  This means that
#     the standard Auth/DBMAuth methods can be used for access control.  The
#     user name is the `one line' version of the client's X.509 certificate.
#     Note that no password is obtained from the user. Every entry in the user
#     file needs this password: `xxj31ZMTZzkVA'.
#   o ExportCertData:
#     This exports two additional environment variables: SSL_CLIENT_CERT and
#     SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
#     server (always existing) and the client (only existing when client
#     authentication is used). This can be used to import the certificates
#     into CGI scripts.
#   o StdEnvVars:
#     This exports the standard SSL/TLS related `SSL_*' environment variables.
#     Per default this exportation is switched off for performance reasons,
#     because the extraction step is an expensive operation and is usually
#     useless for serving static content. So one usually enables the
#     exportation for CGI and SSI requests only.
#   o StrictRequire:
#     This denies access when "SSLRequireSSL" or "SSLRequire" applied even
#     under a "Satisfy any" situation, i.e. when it applies access is denied
#     and no other module can change it.
#   o OptRenegotiate:
#     This enables optimized SSL connection renegotiation handling when SSL
#     directives are used in per-directory context. 
#SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
<FilesMatch "\.(cgi|shtml|phtml|php)$">
    SSLOptions +StdEnvVars
</FilesMatch>
<Directory "/var/www/cgi-bin">
    SSLOptions +StdEnvVars
</Directory>

#   SSL Protocol Adjustments:
#   The safe and default but still SSL/TLS standard compliant shutdown
#   approach is that mod_ssl sends the close notify alert but doesn't wait for
#   the close notify alert from client. When you need a different shutdown
#   approach you can use one of the following variables:
#   o ssl-unclean-shutdown:
#     This forces an unclean shutdown when the connection is closed, i.e. no
#     SSL close notify alert is sent or allowed to be received.  This violates
#     the SSL/TLS standard but is needed for some brain-dead browsers. Use
#     this when you receive I/O errors because of the standard approach where
#     mod_ssl sends the close notify alert.
#   o ssl-accurate-shutdown:
#     This forces an accurate shutdown when the connection is closed, i.e. a
#     SSL close notify alert is sent and mod_ssl waits for the close notify
#     alert of the client. This is 100% SSL/TLS standard compliant, but in
#     practice often causes hanging connections with brain-dead browsers. Use
#     this only for browsers where you know that their SSL implementation
#     works correctly. 
#   Notice: Most problems of broken clients are also related to the HTTP
#   keep-alive facility, so you usually additionally want to disable
#   keep-alive for those clients, too. Use variable "nokeepalive" for this.
#   Similarly, one has to force some clients to use HTTP/1.0 to workaround
#   their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
#   "force-response-1.0" for this.
BrowserMatch "MSIE [2-5]" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0

#   Per-Server Logging:
#   The home of a custom SSL log file. Use this when you want a
#   compact non-error SSL logfile on a virtual host basis.
CustomLog logs/ssl_request_log \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

In the file:

Try changing:

To:
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]

Then reload/restart httpd

2 Likes

No joy:
Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
Domain: youngones.net.nz
Type: unauthorized
Detail: 115.188.30.169: Invalid response from https://youngones.net.nz: "\n<html lang="en-US">\n\n\t<meta charset="UTF-8" />\n\t<meta name="viewport" content="width=device-width, initial"

Please use the testing/staging environment (not production) until we are done testing.
Better yet, you can check if the redirection directly without involving certbot.
Try:

2 Likes

curl -Ii youngones.net.nz/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 404 Not Found
Date: Thu, 30 Nov 2023 07:11:39 GMT
Server: Apache/2.4.57 (CentOS Stream) OpenSSL/3.0.7 mod_fcgid/2.3.9
Content-Type: text/html; charset=iso-8859-1

Try that from the Internet.
hmm...
If you get different results from within the same network, then there may be more than one system responding to those requests.
Is there a reverse proxy type device in front of your web server?
Is there a NAT or port forwarding router in line?
If so, we may need to review those settings.

3 Likes

Yes, there is NAT and port forwarding, but I'm pretty sure they're fine.
curl -Ii youngones.net.nz/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Thu, 30 Nov 2023 07:42:14 GMT
Content-Type: text/html
Content-Length: 162
Connection: keep-alive
Location: https://youngones.net.nz/.well-known/acme-challenge/Test_File-1234
X-Frame-Options: SAMEORIGIN
I did remove one old entry and got the above

1 Like

Show the NAT/port foirwarding settings.

3 Likes

This:

does NOT equal this:

curl -Ii youngones.net.nz/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 302 Moved Temporarily
Date: Thu, 30 Nov 2023 08:10:41 GMT
Content-Type: text/html
Content-Length: 154
Connection: keep-alive
Location: https://youngones.net.nz
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff

301 Moved Permanently | 302 Moved Temporarily
162 bytes | 154 bytes
Server: nginx | "{nothing}"
Location: https://youngones.net.nz/.well-known/acme-challenge/Test_File-1234 | Location: https://youngones.net.nz
"{nothing}" | X-XSS-Protection: 1; mode=block
"{nothing}" | X-Content-Type-Options: nosniff

3 Likes

This is from work this morning:
curl -Ii youngones.net.nz/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 302 Moved Temporarily
Date: Fri, 01 Dec 2023 00:05:11 GMT
Content-Type: text/html
Content-Length: 154
Connection: close
Location: https://youngones.net.nz
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff

This is from inside the network tonight:

curl -Ii youngones.net.nz/.well-known/acme-challenge/Test_File-1234
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Fri, 01 Dec 2023 06:50:36 GMT
Content-Type: text/html
Content-Length: 162
Connection: keep-alive
Location: https://youngones.net.nz/.well-known/acme-challenge/Test_File-1234
X-Frame-Options: SAMEORIGIN
This is the port forward rules


Firewall rules:

Let me know if you need anything else.

Yes, one more thing:
[let's confirm the IP in the forwarding]

  • what shows?:
    [either will do]
    ifconfig | grep -Ei 'inet|addr'
    ip addr | grep inet
2 Likes

You need to find the network device that is redirecting your HTTP requests. Are you using some sort of DNS URL redirect service maybe? Or, maybe your router or something is doing this.

#I can see your Apache server with HTTPS
curl -Ik https://youngones.net.nz/Test404
HTTP/1.1 404 Not Found
Server: Apache/2.4.57 (CentOS Stream) 

# It uses a self-signed cert (for now okay)
openssl s_client -connect youngones.net.nz:443
---
Certificate chain
 0 s:C = US, O = Unspecified, CN = sol, emailAddress = root@sol
   i:C = US, O = Unspecified, OU = ca-2909082762099798941, CN = sol, emailAddress = root@sol
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Nov 21 07:13:34 2023 GMT; NotAfter: Nov 20 07:13:34 2024 GMT

.
#But, HTTP handled by some other device
#And, it redirects any request to the home page at HTTPS
#Notice no Apache server header

curl -I youngones.net.nz/Test404
HTTP/1.1 302 Moved Temporarily
Date: Fri, 01 Dec 2023 16:39:47 GMT
Content-Type: text/html
Content-Length: 154
Connection: keep-alive
Location: https://youngones.net.nz
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff

3 Likes