Authentication failure for domains

Hi, I am getting 404 errors when using certbot to set up https. I am using a cloned server instance on which I had successfully set up certificates for a different domain. On this cloned server I removed previous certbot settings, before starting again, but perhaps this has contributed to the errors I'm getting?

My index.html for mikekelly.org is accessible via the browser. I also tried setting up a test file here http://mikekelly.org/.well-known/acme-challenge/test.txt which I can also access via the browser. I manually created the .well-known/acme-challenge folder.

Thanks for any suggestions:


My domain is: mikekelly.org

I ran this command: sudo certbot --nginx

It produced this output:

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: www.mikekelly.org
  Type:   unauthorized
  Detail: Invalid response from http://www.mikekelly.org/.well-known/acme-challenge/GdA9yuQEkX4Fp-ihb4w01bHgVo-vCtvgaXLIUt8Y8tk [2a06:1002::5d5a:926b]: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was"

  Domain: mikekelly.org
  Type:   unauthorized
  Detail: Invalid response from http://mikekelly.org/.well-known/acme-challenge/W5yErklSz2dWRqpP_9uR4GkaEu2nXwsNbMHXIwImp3o [2a06:1002::5d5a:926b]: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was"

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

My web server is (include version): nginx 1.18.0

The operating system my web server runs on is (include version): Ubuntu 20.04

My hosting provider, if applicable, is: vultr.com

I can login to a root shell on my machine (yes or no, or I don't know): yes

I'm using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 1.19.0

Hi @mikekelly, welcome to the LE community forum :slight_smile:

LE prefers IPv6 over IPv4 when present.
As seen in the error/failure message, the IP used is the IPv6 address.
You need to ensure the IPv6 address works (or remove it from DNS).

curl -Iki4 www.mikekelly.org
HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Date: Thu, 30 Sep 2021 13:39:02 GMT
Content-Type: text/html
Content-Length: 281
Last-Modified: Tue, 28 Sep 2021 16:15:58 GMT
Connection: keep-alive
ETag: "61533fbe-119"
Accept-Ranges: bytes

curl -Iki6 www.mikekelly.org
HTTP/1.1 403 Forbidden
Connection: close
Content-Type: text/html
Cache-Control: no-cache
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Security-Policy: frame-ancestors 'self'
Content-Length: 3449
Name:    m6-mh-https-all.newsquestdigital.co.uk
Address: 93.174.10.103
Aliases: www.dailyecho.co.uk
2 Likes

The IPv4 and IPv6 address are hosted by different AS networks, so I'm guessing one or the other is incorrect.

3 Likes

Good find. Likely: stale DNS info (from a previous hosting plan).
So many people overlook IPv6.

Ah, thank you! I will try updating the DNS settings and will confirm if successful.

I believe the IPv6 addresses are now correct. However now I get a 401 error when trying to create the certificates:

Certbot failed to authenticate some domains (authenticator: nginx). The Certificate Authority reported these problems:
  Domain: mikekelly.org
  Type:   unauthorized
  Detail: Invalid response from http://mikekelly.org/.well-known/acme-challenge/zkFcLUcg2m8lq4LwOMCOjrGqZiq3a9EiubIQdJq-4uQ [2001:19f0:6c01:1afb:5400:3ff:fe97:8219]: "<html>\r\n<head><title>401 Authorization Required</title></head>\r\n<body>\r\n<center><h1>401 Authorization Required</h1></center>\r\n<h"

  Domain: www.mikekelly.org
  Type:   unauthorized
  Detail: Invalid response from http://www.mikekelly.org/.well-known/acme-challenge/gNR9eV0uLkGpMUlcfzPK1eS4kFbBFiQ_C7eeaiuc7Kc [2001:19f0:6c01:1afb:5400:3ff:fe97:8219]: "<html>\r\n<head><title>401 Authorization Required</title></head>\r\n<body>\r\n<center><h1>401 Authorization Required</h1></center>\r\n<h"

Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.

I guess I could simply remove the DNS AAAA entries? I would be happier if I could get IPv6 up and running.

I still get:

curl -Iki6 www.mikekelly.org
HTTP/1.1 403 Forbidden
Connection: close
Content-Type: text/html
Cache-Control: no-cache
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Security-Policy: frame-ancestors 'self'
Content-Length: 3449

Let's have a look at your nginx config:
sudo nginx -T

I get this:

curl -Iki6 www.mikekelly.org
HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Date: Thu, 30 Sep 2021 17:15:20 GMT
Content-Type: text/html
Content-Length: 281
Last-Modified: Tue, 28 Sep 2021 16:15:58 GMT
Connection: keep-alive
ETag: "61533fbe-119"
Accept-Ranges: bytes

Here's the output from sudo nginx -T:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
	worker_connections 768;
	# multi_accept on;
}

http {

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	##
	# Logging Settings
	##

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;

	##
	# Gzip Settings
	##

	gzip on;

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}


#mail {
#	# See sample authentication script at:
#	# http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
#	# auth_http localhost/auth.php;
#	# pop3_capabilities "TOP" "USER";
#	# imap_capabilities "IMAP4rev1" "UIDPLUS";
#
#	server {
#		listen     localhost:110;
#		protocol   pop3;
#		proxy      on;
#	}
#
#	server {
#		listen     localhost:143;
#		protocol   imap;
#		proxy      on;
#	}
#}

# configuration file /etc/nginx/modules-enabled/50-mod-http-image-filter.conf:
load_module modules/ngx_http_image_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-xslt-filter.conf:
load_module modules/ngx_http_xslt_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-mail.conf:
load_module modules/ngx_mail_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-stream.conf:
load_module modules/ngx_stream_module.so;

# configuration file /etc/nginx/mime.types:

types {
    text/html                             html htm shtml;
    text/css                              css;
    text/xml                              xml;
    image/gif                             gif;
    image/jpeg                            jpeg jpg;
    application/javascript                js;
    application/atom+xml                  atom;
    application/rss+xml                   rss;

    text/mathml                           mml;
    text/plain                            txt;
    text/vnd.sun.j2me.app-descriptor      jad;
    text/vnd.wap.wml                      wml;
    text/x-component                      htc;

    image/png                             png;
    image/tiff                            tif tiff;
    image/vnd.wap.wbmp                    wbmp;
    image/x-icon                          ico;
    image/x-jng                           jng;
    image/x-ms-bmp                        bmp;
    image/svg+xml                         svg svgz;
    image/webp                            webp;

    application/font-woff                 woff;
    application/java-archive              jar war ear;
    application/json                      json;
    application/mac-binhex40              hqx;
    application/msword                    doc;
    application/pdf                       pdf;
    application/postscript                ps eps ai;
    application/rtf                       rtf;
    application/vnd.apple.mpegurl         m3u8;
    application/vnd.ms-excel              xls;
    application/vnd.ms-fontobject         eot;
    application/vnd.ms-powerpoint         ppt;
    application/vnd.wap.wmlc              wmlc;
    application/vnd.google-earth.kml+xml  kml;
    application/vnd.google-earth.kmz      kmz;
    application/x-7z-compressed           7z;
    application/x-cocoa                   cco;
    application/x-java-archive-diff       jardiff;
    application/x-java-jnlp-file          jnlp;
    application/x-makeself                run;
    application/x-perl                    pl pm;
    application/x-pilot                   prc pdb;
    application/x-rar-compressed          rar;
    application/x-redhat-package-manager  rpm;
    application/x-sea                     sea;
    application/x-shockwave-flash         swf;
    application/x-stuffit                 sit;
    application/x-tcl                     tcl tk;
    application/x-x509-ca-cert            der pem crt;
    application/x-xpinstall               xpi;
    application/xhtml+xml                 xhtml;
    application/xspf+xml                  xspf;
    application/zip                       zip;

    application/octet-stream              bin exe dll;
    application/octet-stream              deb;
    application/octet-stream              dmg;
    application/octet-stream              iso img;
    application/octet-stream              msi msp msm;

    application/vnd.openxmlformats-officedocument.wordprocessingml.document    docx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet          xlsx;
    application/vnd.openxmlformats-officedocument.presentationml.presentation  pptx;

    audio/midi                            mid midi kar;
    audio/mpeg                            mp3;
    audio/ogg                             ogg;
    audio/x-m4a                           m4a;
    audio/x-realaudio                     ra;

    video/3gpp                            3gpp 3gp;
    video/mp2t                            ts;
    video/mp4                             mp4;
    video/mpeg                            mpeg mpg;
    video/quicktime                       mov;
    video/webm                            webm;
    video/x-flv                           flv;
    video/x-m4v                           m4v;
    video/x-mng                           mng;
    video/x-ms-asf                        asx asf;
    video/x-ms-wmv                        wmv;
    video/x-msvideo                       avi;
}

# configuration file /etc/nginx/sites-enabled/default:
#
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# https://www.nginx.com/resources/wiki/start/
# https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/
# https://wiki.debian.org/Nginx/DirectoryStructure
#
# In most cases, administrators will remove this file from sites-enabled/ and
# leave it as reference inside of sites-available where it will continue to be
# updated by the nginx packaging team.
#
# This file will automatically load configuration files provided by other
# applications, such as Drupal or Wordpress. These applications will be made
# available underneath a path with that package name, such as /drupal8.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##

# Default server configuration
#
server {
    listen 80 default_server;
    listen [::]:80 default_server;

    # SSL configuration
    #
    # listen 443 ssl default_server;
    # listen [::]:443 ssl default_server;
    #
    # Note: You should disable gzip for SSL traffic.
    # See: https://bugs.debian.org/773332
    #
    # Read up on ssl_ciphers to ensure a secure configuration.
    # See: https://bugs.debian.org/765782
    #
    # Self signed certs generated by the ssl-cert package
    # Don't use them in a production server!
    #
    # include snippets/snakeoil.conf;

    root /home/mike/doubleelephant/dist/;

    # Add index.php to the list if you are using PHP
    index index.html index.htm index.nginx-debian.html;

    server_name _;

    location / {
        # First attempt to serve request as file, then
        # as directory, then fall back to displaying a 404.
        try_files $uri $uri/ =404;
        auth_basic "Password required";
        auth_basic_user_file /etc/nginx/.htpasswd;
    }

    # pass PHP scripts to FastCGI server
    #
    #location ~ \.php$ {
    #    include snippets/fastcgi-php.conf;
    #
    #    # With php-fpm (or other unix sockets):
    #    fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
    #    # With php-cgi (or other tcp sockets):
    #    fastcgi_pass 127.0.0.1:9000;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny all;
    #}
}


# Virtual Host configuration for example.com
#
# You can move that to a different file under sites-available/ and symlink that
# to sites-enabled/ to enable it.
#
#server {
#    listen 80;
#    listen [::]:80;
#
#    server_name example.com;
#
#    root /var/www/example.com;
#    index index.html;
#
#    location / {
#        try_files $uri $uri/ =404;
#    }
#}

# configuration file /etc/nginx/sites-enabled/mikekelly.org:
upstream backend {
  server localhost:3000; #node app address
}

server {
listen 80;
server_name mikekelly.org www.mikekelly.org;

root /home/www/html/;

location /api {
  rewrite ^/api(/.*) $1 break;
  proxy_pass http://backend;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header Host $host;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  # Following is necessary for Websocket support
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";
}

location /research {
    alias /var/www/html/research/server/public/;
    index index.html index.htm;
    try_files $uri $uri/ /index.html =404;
}

location / {
    root /var/www/html/;
    index index.html index.htm;
    try_files $uri $uri/ /index.html =404;
}

    client_max_body_size 20M;
}

Try changing:

server {
listen 80;
server_name mikekelly.org www.mikekelly.org;

To:

server {
listen 80;
listen [::]:80;   ### add this line ###
server_name mikekelly.org www.mikekelly.org;

Also, please show:
sudo netstat -pant | grep Ei 'nginx|80|443'

That fixed it, thanks so much for your help with this!

I didn't have success running the netstat line: grep: nginx|80|443: No such file or directory

Beautiful!
Glad to have helped out :slight_smile:
Cheers from Miami :beers:

#FreeCUBA :cuba:

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.