Challenge failed for domain - Invalid response (404)

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: maxproxies.com

I ran this command:
certbot --apache -d maxproxies.com

It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for maxproxies.com
Waiting for verification...
Challenge failed for domain maxproxies.com
http-01 challenge for maxproxies.com
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:

My web server is (include version):
Server version: Apache/2.4.6 (CentOS)

The operating system my web server runs on is (include version):
CentOS 7 (x64)

My hosting provider, if applicable, is:
VPS

I can login to a root shell on my machine (yes or no, or I don't know):
yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel):
No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot):
certbot 1.9.0

Note - Initially I had no virtual hosts enabled/created which certbot didn't like so created the virtual hosts files needed for maxproxies.com which seem to be working fine

Also, I did manually create the .well-known/acme-challenge directory and created a 1234 file within it which loads fine.
http://maxproxies.com/.well-known/acme-challenge/1234

1 Like

If you want, you can just use the webroot authenticator:

certbot -a webroot -w /path/to/webroot -i apache -d maxproxies.com

but to figure out why --apache isn't working, I think we'd need to see how the virtual host you created it set up:

apachectl -t -D DUMP_VHOSTS
3 Likes

Would rather fix whatever is causing the issue otherwise I may run into issues with renewing etc down the track.

Here's the output of apachectl -t -D DUMP_VHOSTS

Passing arguments to httpd using apachectl is no longer supported.
You can only start/stop/restart httpd using this script.
If you want to pass extra arguments to httpd, edit the
/etc/sysconfig/httpd config file.
VirtualHost configuration:
51.81.161.0:80         ip0.ip-51-81-161.us (/etc/httpd/sites-enabled/maxproxies.com.conf:1)
*:443                  maxproxies.com (/etc/httpd/conf.d/ssl.conf:56)
2 Likes

I don't see a straightforward reason why this wouldn't be working.

Do you have RewriteEngine off inside the contents of the virtualhost? There's a couple of bugs like that which can break things.

Changing the bind address to *:80 and including a ServerName might help too (though it isn't strictly necessary afaik).

If you want to post the full file, I can try reproduce it for you.

1 Like

Here is the entire virtual host file;

<VirtualHost 51.81.161.0:80>
    ServerName www.maxproxies.com
    ServerAlias maxproxies.com
    DocumentRoot /var/www/maxproxies.com/html
    ErrorLog /var/www/maxproxies.com/log/error.log
    CustomLog /var/www/maxproxies.com/log/requests.log combined
   <Directory /var/www/maxproxies.com/html>
     Order allow,deny
    Allow from all
   </Directory>

   </VirtualHost>
1 Like

In your pre-edited post, you had a missing line break:

<VirtualHost 51.81.161.0:80>    ServerName www.maxproxies.com

Was it actually like this in your file? Apache doesn't handle this properly and it could certainly explain your problem getting a certificate.

1 Like

Yes, I've now corrected this and am not getting a slightly different reason for the challenge failing, going from 404 to 403 error;

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for maxproxies.com
Waiting for verification...
Challenge failed for domain maxproxies.com
http-01 challenge for maxproxies.com
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: maxproxies.com
   Type:   unauthorized
   Detail: Invalid response from
   http://maxproxies.com/.well-known/acme-challenge/D16ZsukVze0PMZdrcefsQeMghUtsP-kVFoy-QL0UIKU
   [51.81.161.0]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>403
   Forbidden</title>\n</head><body>\n<h1>Forbidden</h1>\n<p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.
1 Like

Anything in Apache's error_log about that 403?

Running SELinux?

You can post code in backticks like:

```
code goes here
```

or just use a pastebin, either way is fine.

2 Likes

When you paste code, add three backticks ``` on lines by themselves above and below the code.

Damn you're fast, _az. :slightly_smiling_face:

1 Like

You can edit your existing post to add them.

SELinux status:                 disabled

http.conf;

Header append X-FRAME-OPTIONS "SAMEORIGIN"
FileETag None
TraceEnable off

<IfModule mod_headers.c>
    Header set X-XSS-Protection "1; mode=block"
</IfModule>

<IfModule mod_headers.c>
    Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
</IfModule>

# This is the main Apache HTTP server configuration file.  It contains the
# configuration directives that give the server its instructions.
# See <URL:http://httpd.apache.org/docs/2.4/> for detailed information.
# In particular, see 
# <URL:http://httpd.apache.org/docs/2.4/mod/directives.html>
# for a discussion of each configuration directive.
#
# Do NOT simply read the instructions in here without understanding
# what they do.  They're here only as hints or reminders.  If you are unsure
# consult the online docs. You have been warned.  
#
# Configuration and logfile names: If the filenames you specify for many
# of the server's control files begin with "/" (or "drive:/" for Win32), the
# server will use that explicit path.  If the filenames do *not* begin
# with "/", the value of ServerRoot is prepended -- so 'log/access_log'
# with ServerRoot set to '/www' will be interpreted by the
# server as '/www/log/access_log', where as '/log/access_log' will be
# interpreted as '/log/access_log'.

#
# ServerRoot: The top of the directory tree under which the server's
# configuration, error, and log files are kept.
#
# Do not add a slash at the end of the directory path.  If you point
# ServerRoot at a non-local disk, be sure to specify a local disk on the
# Mutex directive, if file-based mutexes are used.  If you wish to share the
# same ServerRoot for multiple httpd daemons, you will need to change at
# least PidFile.
#
ServerRoot "/etc/httpd"

#
# Listen: Allows you to bind Apache to specific IP addresses and/or
# ports, instead of the default. See also the <VirtualHost>
# directive.
#
# Change this to Listen on specific IP addresses as shown below to 
# prevent Apache from glomming onto all bound IP addresses.
#
Listen 51.81.161.0:80
#Listen 80

#
# Dynamic Shared Object (DSO) Support
#
# To be able to use the functionality of a module which was built as a DSO you
# have to place corresponding `LoadModule' lines at this location so the
# directives contained in it are actually available _before_ they are used.
# Statically compiled modules (those listed by `httpd -l') do not need
# to be loaded here.
#
# Example:
# LoadModule foo_module modules/mod_foo.so
#
Include conf.modules.d/*.conf

#
# If you wish httpd to run as a different user or group, you must run
# httpd as root initially and it will switch.  
#
# User/Group: The name (or #number) of the user/group to run httpd as.
# It is usually good practice to create a dedicated user and group for
# running httpd, as with most system services.
#
User apache
Group apache

# 'Main' server configuration
#
# The directives in this section set up the values used by the 'main'
# server, which responds to any requests that aren't handled by a
# <VirtualHost> definition.  These values also provide defaults for
# any <VirtualHost> containers you may define later in the file.
#
# All of these directives may appear inside <VirtualHost> containers,
# in which case these default settings will be overridden for the
# virtual host being defined.
#

#
# ServerAdmin: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
ServerAdmin root@localhost

#
# ServerName gives the name and port that the server uses to identify itself.
# This can often be determined automatically, but we recommend you specify
# it explicitly to prevent problems during startup.
#
# If your host doesn't have a registered DNS name, enter its IP address here.
#
ServerName maxproxies.com:80

#
# Deny access to the entirety of your server's filesystem. You must
# explicitly permit access to web content directories in other 
# <Directory> blocks below.
#
<Directory />
    Options None
    Order deny,allow
    Deny from all
</Directory>

#
# Note that from this point forward you must specifically allow
# particular features to be enabled - so if something's not working as
# you might expect, make sure that you have specifically enabled it
# below.
#

#
# DocumentRoot: The directory out of which you will serve your
# documents. By default, all requests are taken from this directory, but
# symbolic links and aliases may be used to point to other locations.
#
DocumentRoot "/var/www/html"

#
# Relax access to content within /var/www.
#
<Directory "/var/www">
    AllowOverride None
  Options -Indexes -FollowSymLinks 
   # Allow open access:
    Require all granted
</Directory>

# Further relax access to the default document root:
<Directory "/var/www/html">
    #
    # Possible values for the Options directive are "None", "All",
    # or any combination of:
    #   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
    #
    # Note that "MultiViews" must be named *explicitly* --- "Options All"
    # doesn't give it to you.
    #
    # The Options directive is both complicated and important.  Please see
    # http://httpd.apache.org/docs/2.4/mod/core.html#options
    # for more information.
    #
   Options -Indexes -FollowSymLinks -ExecCGI -Includes
    #
    # AllowOverride controls what directives may be placed in .htaccess files.
    # It can be "All", "None", or any combination of the keywords:
    #   Options FileInfo AuthConfig Limit
    #
    AllowOverride None

    #
    # Controls who can get stuff from this server.
    #
    Require all granted
        <LimitExcept GET POST HEAD>
            deny from all
        </LimitExcept>
</Directory>

#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
<IfModule dir_module>
 DirectoryIndex index.php index.html
</IfModule>

#
# The following lines prevent .htaccess and .htpasswd files from being 
# viewed by Web clients. 
#
<Files ".ht*">
    Require all denied
</Files>

#
# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here.  If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog "logs/error_log"

#
# LogLevel: Control the number of messages logged to the error_log.
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
#
LogLevel warn

<IfModule log_config_module>
    #
    # The following directives define some format nicknames for use with
    # a CustomLog directive (see below).
    #
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common

    <IfModule logio_module>
      # You need to enable mod_logio.c to use %I and %O
      LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
    </IfModule>

    #
    # The location and format of the access logfile (Common Logfile Format).
    # If you do not define any access logfiles within a <VirtualHost>
    # container, they will be logged here.  Contrariwise, if you *do*
    # define per-<VirtualHost> access logfiles, transactions will be
    # logged therein and *not* in this file.
    #
    #CustomLog "logs/access_log" common

    #
    # If you prefer a logfile with access, agent, and referer information
    # (Combined Logfile Format) you can use the following directive.
    #
    CustomLog "logs/access_log" combined
</IfModule>

<IfModule alias_module>
    #
    # Redirect: Allows you to tell clients about documents that used to 
    # exist in your server's namespace, but do not anymore. The client 
    # will make a new request for the document at its new location.
    # Example:
    # Redirect permanent /foo http://www.example.com/bar

    #
    # Alias: Maps web paths into filesystem paths and is used to
    # access content that does not live under the DocumentRoot.
    # Example:
    # Alias /webpath /full/filesystem/path
    #
    # If you include a trailing / on /webpath then the server will
    # require it to be present in the URL.  You will also likely
    # need to provide a <Directory> section to allow access to
    # the filesystem path.

    #
    # ScriptAlias: This controls which directories contain server scripts. 
    # ScriptAliases are essentially the same as Aliases, except that
    # documents in the target directory are treated as applications and
    # run by the server when requested rather than as documents sent to the
    # client.  The same rules about trailing "/" apply to ScriptAlias
    # directives as to Alias.
    #
    ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

</IfModule>

#
# "/var/www/cgi-bin" should be changed to whatever your ScriptAliased
# CGI directory exists, if you have that configured.
#
<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options None
    Require all granted
</Directory>

<IfModule mime_module>
    #
    # TypesConfig points to the file containing the list of mappings from
    # filename extension to MIME-type.
    #
    TypesConfig /etc/mime.types

    #
    # AddType allows you to add to or override the MIME configuration
    # file specified in TypesConfig for specific file types.
    #
    #AddType application/x-gzip .tgz
    #
    # AddEncoding allows you to have certain browsers uncompress
    # information on the fly. Note: Not all browsers support this.
    #
    #AddEncoding x-compress .Z
    #AddEncoding x-gzip .gz .tgz
    #
    # If the AddEncoding directives above are commented-out, then you
    # probably should define those extensions to indicate media types:
    #
    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz

    #
    # AddHandler allows you to map certain file extensions to "handlers":
    # actions unrelated to filetype. These can be either built into the server
    # or added with the Action directive (see below)
    #
    # To use CGI scripts outside of ScriptAliased directories:
    # (You will also need to add "ExecCGI" to the "Options" directive.)
    #
    #AddHandler cgi-script .cgi

    # For type maps (negotiated resources):
    #AddHandler type-map var

    #
    # Filters allow you to process content before it is sent to the client.
    #
    # To parse .shtml files for server-side includes (SSI):
    # (You will also need to add "Includes" to the "Options" directive.)
    #
    AddType text/html .shtml
    AddOutputFilter INCLUDES .shtml
</IfModule>

#
# Specify a default charset for all content served; this enables
# interpretation of all content as UTF-8 by default.  To use the 
# default browser choice (ISO-8859-1), or to allow the META tags
# in HTML content to override this choice, comment out this
# directive:
#
AddDefaultCharset UTF-8

<IfModule mime_magic_module>
    #
    # The mod_mime_magic module allows the server to use various hints from the
    # contents of the file itself to determine its type.  The MIMEMagicFile
    # directive tells the module where the hint definitions are located.
    #
    MIMEMagicFile conf/magic
</IfModule>

#
# Customizable error responses come in three flavors:
# 1) plain text 2) local redirects 3) external redirects
#
# Some examples:
#ErrorDocument 500 "The server made a boo boo."
#ErrorDocument 404 /missing.html
#ErrorDocument 404 "/cgi-bin/missing_handler.pl"
#ErrorDocument 402 http://www.example.com/subscription_info.html
#

#
# EnableMMAP and EnableSendfile: On systems that support it, 
# memory-mapping or the sendfile syscall may be used to deliver
# files.  This usually improves server performance, but must
# be turned off when serving from networked-mounted 
# filesystems or if support for these functions is otherwise
# broken on your system.
# Defaults if commented: EnableMMAP On, EnableSendfile Off
#
#EnableMMAP off
EnableSendfile on

# Supplemental configuration
#
# Load config files in the "/etc/httpd/conf.d" directory, if any.
IncludeOptional conf.d/*.conf
ServerSignature Off
ServerTokens Prod
IncludeOptional sites-enabled/*.conf

Apache error.log for maxproxies.com;

[Mon Nov 16 15:26:08.033636 2020] [access_compat:error] [pid 30336] [client 34.209.232.166:64544] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/YonEDk_HbfZ81jqgXUn1kUQCV8UFzj4pdYhz1c-4bV8
[Mon Nov 16 15:26:08.130386 2020] [access_compat:error] [pid 30338] [client 66.133.109.36:48938] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/YonEDk_HbfZ81jqgXUn1kUQCV8UFzj4pdYhz1c-4bV8
[Mon Nov 16 15:26:08.170373 2020] [access_compat:error] [pid 30340] [client 3.128.26.105:47180] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/YonEDk_HbfZ81jqgXUn1kUQCV8UFzj4pdYhz1c-4bV8
[Mon Nov 16 15:26:08.233587 2020] [access_compat:error] [pid 30339] [client 18.196.96.172:43776] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/YonEDk_HbfZ81jqgXUn1kUQCV8UFzj4pdYhz1c-4bV8
[Mon Nov 16 15:31:50.548629 2020] [access_compat:error] [pid 30444] [client 17.58.82.171:57666] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/ANHSr8CAbYMlvOQ5T5Pz2yzd3tbtrX6sxI6nx22Ix74
[Mon Nov 16 15:31:50.656735 2020] [access_compat:error] [pid 30443] [client 17.58.82.171:57670] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/1234
[Mon Nov 16 15:31:51.980726 2020] [access_compat:error] [pid 30445] [client 34.209.232.166:28794] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/D16ZsukVze0PMZdrcefsQeMghUtsP-kVFoy-QL0UIKU
[Mon Nov 16 15:31:52.041463 2020] [access_compat:error] [pid 30446] [client 66.133.109.36:35230] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/D16ZsukVze0PMZdrcefsQeMghUtsP-kVFoy-QL0UIKU
[Mon Nov 16 15:33:59.292386 2020] [access_compat:error] [pid 30526] [client 66.133.109.36:50748] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/4EJ76lTyDzF7KZ5soAcjRJkMgO9u0qvE3bC9yU9wvUE
[Mon Nov 16 15:33:59.303909 2020] [access_compat:error] [pid 30527] [client 34.211.6.84:44542] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/4EJ76lTyDzF7KZ5soAcjRJkMgO9u0qvE3bC9yU9wvUE
[Mon Nov 16 15:33:59.348926 2020] [access_compat:error] [pid 30529] [client 3.22.70.135:14244] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/4EJ76lTyDzF7KZ5soAcjRJkMgO9u0qvE3bC9yU9wvUE
[Mon Nov 16 15:33:59.452573 2020] [access_compat:error] [pid 30530] [client 52.28.236.88:59280] AH01797: client denied by server configuration: /var/lib/letsencrypt/http_challenges/4EJ76lTyDzF7KZ5soAcjRJkMgO9u0qvE3bC9yU9wvUE

Looks like something in my httpd.conf has broken it.

1 Like

Perhaps what's going on is that the combination of:

  • Denying access to /
  • Allowing access to /var/www
  • Certbot using /var/lib/letsencrypt/http_challenges to store the --apache challenge responses

results in an overall denial for these requests.

To fix it, I guess you would have to carve another exception for that directory, just as you did with /var/www.

1 Like

Changed;

<Directory />
    Options None
    Order deny,allow
    Deny from all
</Directory>

To

<Directory />
    Options None
    Order allow,deny
    Deny from all
</Directory>

Restarted Apache, same error

Added these lines to httpd.conf and restarted httpd;

<Directory "/var/lib/letsencrypt/http_challenges">
    AllowOverride None
  Options -Indexes -FollowSymLinks
   # Allow open access:
    Require all granted
</Directory>

Same error

1 Like

Further change from;

<Directory />
    Options None
    Order allow,deny   
   Deny from all
</Directory>

To;

<Directory />
    Options None
    Order allow,deny   
   Allow from all
</Directory>

Now Let's Encrypt is unhappy with me;

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
Obtaining a new certificate
An unexpected error occurred:
There were too many requests of a given type :: Error creating new order :: too many failed authorizations recently: see https://letsencrypt.org/docs/rate-limits/
Please see the logfiles in /var/log/letsencrypt for more details.
1 Like

Use --dry-run while testing, to avoid production rate limits.

You get 5 attempts/hr without it.

2 Likes

Thanks. Ok my last change I think did the trick.

certbot certonly --apache -d maxproxies.com --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for maxproxies.com
Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - The dry run was successful.

I guess I'll wait an hour and attempt to run it again in normal mode. Note that I edited my previous post to show the change actually made.

2 Likes

@_az

Uh, what the!?

certonly and Installer apache!?

Since when?

That effectively merges to

certonly -a apache

The installer is not invoked.

1 Like

@Cranky

I believe you want to use:
certbot --cert-name maxproxies.com --apache -d "maxproxies.com,www.maxproxies.com"

1 Like

So I'm wondering if I can revert back to;

<Directory />
    Options None
    Order deny,allow
    Deny from  all
</Directory>

Once the verification is done and certificate is installed, or if I need to keep it open for renewals as well. I guess I'll change it back once the cert is installed and do a dry run on renewing the cert.