Sudo certbot renew fails

My domain is: s4.lister-studios.com

I ran this command: sudo certbot renew

It produced this output: output is below

My web server is (include version): nginx/1.10.3 (Ubuntu)

The operating system my web server runs on is (include version): Ubuntu 16.04

My hosting provider, if applicable, is: not applicable. It’s a vps. Not a domain hoster. Not hosting a website.

I can login to a root shell on my machine (yes or no, or I don’t know): yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): no

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.31.0

Command and output:

$ sudo certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/s4.lister-studios.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for s4.lister-studios.com
Waiting for verification...
Cleaning up challenges
Attempting to renew cert (s4.lister-studios.com) from /etc/letsencrypt/renewal/s4.lister-studios.com.conf produced an unexpected error: Failed authorization procedure. s4.lister-studios.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://s4.lister-studios.com/.well-known/acme-challenge/zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4 [2a03:4000:17:6fa:b40b:18ff:fe91:df5b]: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>". Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/s4.lister-studios.com/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/s4.lister-studios.com/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: s4.lister-studios.com
   Type:   unauthorized
   Detail: Invalid response from
   http://s4.lister-studios.com/.well-known/acme-challenge/zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4
   [2a03:4000:17:6fa:b40b:18ff:fe91:df5b]: "<html>\r\n<head><title>404
   Not Found</title></head>\r\n<body
   bgcolor=\"white\">\r\n<center><h1>404 Not
   Found</h1></center>\r\n<hr><center>"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

Debug log is posted in the following reply.

I created the certificate about 2 months ago with sudo certbot --nginx. I believe I cut the added lines which were added to the nginx default config to my custom sym linked config. This might be why it’s now failing to renew. Auto renew has been failing for some time.
Dry run works:

$ sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/s4.lister-studios.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator nginx, Installer nginx
Renewing an existing certificate

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed with reload of nginx server; fullchain is
/etc/letsencrypt/live/s4.lister-studios.com/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/s4.lister-studios.com/fullchain.pem (success)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Thanks!

2019-06-30 20:26:47,479:DEBUG:certbot.main:certbot version: 0.31.0
2019-06-30 20:26:47,480:DEBUG:certbot.main:Arguments: []
2019-06-30 20:26:47,481:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2019-06-30 20:26:47,490:DEBUG:certbot.log:Root logging level set at 20
2019-06-30 20:26:47,491:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2019-06-30 20:26:47,500:DEBUG:certbot.plugins.selection:Requested authenticator <certbot.cli._Default object at 0x7f4054d26e80> and installer <certbot.cli._Default object at 0x7f4054d26e80>
2019-06-30 20:26:47,512:DEBUG:certbot.storage:Should renew, less than 30 days before certificate expiry 2019-07-20 17:53:41 UTC.
2019-06-30 20:26:47,512:INFO:certbot.renewal:Cert is due for renewal, auto-renewing...
2019-06-30 20:26:47,513:DEBUG:certbot.plugins.selection:Requested authenticator nginx and installer nginx
2019-06-30 20:26:47,696:DEBUG:certbot.plugins.selection:Single candidate plugin: * nginx
Description: Nginx Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx.configurator:NginxConfigurator
Initialized: <certbot_nginx.configurator.NginxConfigurator object at 0x7f4054d81668>
Prep: True
2019-06-30 20:26:47,698:DEBUG:certbot.plugins.selection:Single candidate plugin: * nginx
Description: Nginx Web Server plugin
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx.configurator:NginxConfigurator
Initialized: <certbot_nginx.configurator.NginxConfigurator object at 0x7f4054d81668>
Prep: True
2019-06-30 20:26:47,698:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_nginx.configurator.NginxConfigurator object at 0x7f4054d81668> and installer <certbot_nginx.configurator.NginxConfigurator object at 0x7f4054d81668>
2019-06-30 20:26:47,698:INFO:certbot.plugins.selection:Plugins selected: Authenticator nginx, Installer nginx
2019-06-30 20:26:47,702:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(external_account_binding=None, only_return_existing=None, contact=(), terms_of_service_agreed=None, key=None, agreement=None, status=None), new_authzr_uri=None, terms_of_service=None, uri='https://acme-v02.api.letsencrypt.org/acme/acct/55633253'), 3e97941e32782164c1b6a234d619585f, Meta(creation_dt=datetime.datetime(2019, 4, 21, 18, 53, 24, tzinfo=<UTC>), creation_host='v22019043906985996.hotsrv.de'))>
2019-06-30 20:26:47,704:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2019-06-30 20:26:47,706:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
2019-06-30 20:26:47,928:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 658
2019-06-30 20:26:47,928:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 658
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Sun, 30 Jun 2019 18:26:47 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Sun, 30 Jun 2019 18:26:47 GMT
Connection: keep-alive

{
  "VwmkaihXytI": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org"
  },
  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}
2019-06-30 20:26:47,929:INFO:certbot.main:Renewing an existing certificate
2019-06-30 20:26:48,043:DEBUG:certbot.crypto_util:Generating key (2048 bits): /etc/letsencrypt/keys/0023_key-certbot.pem
2019-06-30 20:26:48,046:DEBUG:certbot.crypto_util:Creating CSR: /etc/letsencrypt/csr/0023_csr-certbot.pem
2019-06-30 20:26:48,046:DEBUG:acme.client:Requesting fresh nonce
2019-06-30 20:26:48,046:DEBUG:acme.client:Sending HEAD request to https://acme-v02.api.letsencrypt.org/acme/new-nonce.
2019-06-30 20:26:48,213:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
2019-06-30 20:26:48,214:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: MV-8d36L5OQfOpjrBgol8aMLV5aJA7y08C4O01IOy5w
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Content-Length: 0
Expires: Sun, 30 Jun 2019 18:26:48 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Sun, 30 Jun 2019 18:26:48 GMT
Connection: keep-alive


2019-06-30 20:26:48,214:DEBUG:acme.client:Storing nonce: MV-8d36L5OQfOpjrBgol8aMLV5aJA7y08C4O01IOy5w
2019-06-30 20:26:48,215:DEBUG:acme.client:JWS payload:
b'{\n  "identifiers": [\n    {\n      "value": "s4.lister-studios.com",\n      "type": "dns"\n    }\n  ]\n}'
2019-06-30 20:26:48,217:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/new-order:
{
  "signature": "lIdy4L_40fAeuzRme7QNFNBNSKbTJwJHfcSFS6crEvGRCnDrktSkw8OPAR5LJg3iDaW5eDfbov9jQKcHH1niwZpcnkcR4zqruVAe3qG4lhY4hIw3Y2HRoYEFDX9GJFXz3ycxvBUVeItNzCgKP5BEk8CHspHLDcC5j9jnhwRZbYvfNYFy1LpMu4sj6JB5X-sEdGIkmoMx7zjbPh7n3xiu_pc7yodeRJHXwm7N1iSCPT0WHS3eL1GBG10EthvSHMHVhNr4mdjsvDCGOaRwZhrN5hEq6mZJ6TBfAAbupg7D8mEgaZ5Hd0A1KS5ZU4WHMW00lIZIybztd6S7SfV0HyFcYw",
  "protected": "eyJhbGciOiAiUlMyNTYiLCAidXJsIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL25ldy1vcmRlciIsICJraWQiOiAiaHR0cHM6Ly9hY21lLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC81NTYzMzI1MyIsICJub25jZSI6ICJNVi04ZDM2TDVPUWZPcGpyQmdvbDhhTUxWNWFKQTd5MDhDNE8wMUlPeTV3In0",
  "payload": "ewogICJpZGVudGlmaWVycyI6IFsKICAgIHsKICAgICAgInZhbHVlIjogInM0Lmxpc3Rlci1zdHVkaW9zLmNvbSIsCiAgICAgICJ0eXBlIjogImRucyIKICAgIH0KICBdCn0"
}
2019-06-30 20:26:48,425:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 201 380
2019-06-30 20:26:48,426:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Content-Type: application/json
Content-Length: 380
Boulder-Requester: 55633253
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Location: https://acme-v02.api.letsencrypt.org/acme/order/55633253/643899305
Replay-Nonce: MmXYvLLO165RY8MfEBeHg3D8jsbuyYCCb0swqDoPayk
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Sun, 30 Jun 2019 18:26:48 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Sun, 30 Jun 2019 18:26:48 GMT
Connection: keep-alive

{
  "status": "pending",
  "expires": "2019-07-07T18:26:48.332475738Z",
  "identifiers": [
    {
      "type": "dns",
      "value": "s4.lister-studios.com"
    }
  ],
  "authorizations": [
    "https://acme-v02.api.letsencrypt.org/acme/authz/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ"
  ],
  "finalize": "https://acme-v02.api.letsencrypt.org/acme/finalize/55633253/643899305"
}
2019-06-30 20:26:48,426:DEBUG:acme.client:Storing nonce: MmXYvLLO165RY8MfEBeHg3D8jsbuyYCCb0swqDoPayk
2019-06-30 20:26:48,426:DEBUG:acme.client:JWS payload:
b''
2019-06-30 20:26:48,428:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ:
{
  "signature": "fLB9M9vPZolMeCDpoE2HbxYPvfhDliYnyVq60PexqHfRl9tnuJ3JXR-oOOFwqMzyJaOc7-wIc8O3vT75EBOqQyecSX73c7aFd-F9RVbVC0RdBAQcuGl0153bfHi130x-RMQUsKncJ7ktdXozrHqesMYXSmD-vxfY8OAran_iWc_WD40CkmnBbUeyjajMT_55shmCEs1Az9mVqOelbU1DT84-Ufjc8TFwE9FVeyPCsWwu1fFAqf35DoutJVRR8YOOaI-Xz-RdzIJLCwazCRbNHIv5YWgUK9uF1jY__A5wn3nRKOS8O8wEJsR94hM6HWdEKPsrRFG8ElS3xOXi_8fBeA",
  "protected": "eyJhbGciOiAiUlMyNTYiLCAidXJsIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2F1dGh6L1p3QWdGYjhWVnBZbWpadVR5a1p5QVVkS1FDenB3MDNrREp0OWdzbVJGUFEiLCAia2lkIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvNTU2MzMyNTMiLCAibm9uY2UiOiAiTW1YWXZMTE8xNjVSWThNZkVCZUhnM0Q4anNidXlZQ0NiMHN3cURvUGF5ayJ9",
  "payload": ""
}
2019-06-30 20:26:48,612:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ HTTP/1.1" 200 916
2019-06-30 20:26:48,613:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 916
Boulder-Requester: 55633253
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: rHySGJ_MJItTJ-drmcntnqPp4qXfb-Dbao39Q5-Il-c
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Sun, 30 Jun 2019 18:26:48 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Sun, 30 Jun 2019 18:26:48 GMT
Connection: keep-alive

{
  "identifier": {
    "type": "dns",
    "value": "s4.lister-studios.com"
  },
  "status": "pending",
  "expires": "2019-07-07T18:26:48Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/challenge/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ/17705181945",
      "token": "zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/challenge/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ/17705181946",
      "token": "qgPC0Uy5w7qwQOberbP2ARCwrfHBA9tpUEF4zHlu7Dc"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/challenge/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ/17705181947",
      "token": "lriGfeYSTi9qj5G9s4KqChJYB78dqc1l6C7wCi1m4AI"
    }
  ]
}
2019-06-30 20:26:48,613:DEBUG:acme.client:Storing nonce: rHySGJ_MJItTJ-drmcntnqPp4qXfb-Dbao39Q5-Il-c
2019-06-30 20:26:48,614:INFO:certbot.auth_handler:Performing the following challenges:
2019-06-30 20:26:48,614:INFO:certbot.auth_handler:http-01 challenge for s4.lister-studios.com
2019-06-30 20:26:48,641:DEBUG:certbot_nginx.http_01:Generated server block:
[]
2019-06-30 20:26:48,642:DEBUG:certbot.reverter:Creating backup of /etc/letsencrypt/options-ssl-nginx.conf
2019-06-30 20:26:48,642:DEBUG:certbot.reverter:Creating backup of /etc/nginx/sites-enabled/sync_gateway
2019-06-30 20:26:48,642:DEBUG:certbot.reverter:Creating backup of /etc/nginx/sites-enabled/default
2019-06-30 20:26:48,643:DEBUG:certbot.reverter:Creating backup of /etc/nginx/nginx.conf
2019-06-30 20:26:48,643:DEBUG:certbot.reverter:Creating backup of /etc/nginx/mime.types
2019-06-30 20:26:48,645:DEBUG:certbot_nginx.parser:Writing nginx conf tree to /etc/nginx/sites-enabled/sync_gateway:
# read more here http://tautt.com/best-nginx-configuration-for-security/
# https://gist.github.com/plentz/6737338

# don't send the nginx version number in error pages and Server header
server_tokens off;

# config to don't allow the browser to render the page inside an frame or iframe
# and avoid clickjacking http://en.wikipedia.org/wiki/Clickjacking
# if you need to allow [i]frames, you can use SAMEORIGIN or even set an uri with ALLOW-FROM uri
# https://developer.mozilla.org/en-US/docs/HTTP/X-Frame-Options
add_header X-Frame-Options SAMEORIGIN;

# when serving user-supplied content, include a X-Content-Type-Options: nosniff header along with the Content-Type: header,
# to disable content-type sniffing on some browsers.
# https://www.owasp.org/index.php/List_of_useful_HTTP_headers
# currently suppoorted in IE > 8 http://blogs.msdn.com/b/ie/archive/2008/09/02/ie8-security-part-vi-beta-2-update.aspx
# http://msdn.microsoft.com/en-us/library/ie/gg622941(v=vs.85).aspx
# 'soon' on Firefox https://bugzilla.mozilla.org/show_bug.cgi?id=471020
add_header X-Content-Type-Options nosniff;

# This header enables the Cross-site scripting (XSS) filter built into most recent web browsers.
# It's usually enabled by default anyway, so the role of this header is to re-enable the filter for 
# this particular website if it was disabled by the user.
# https://www.owasp.org/index.php/List_of_useful_HTTP_headers
add_header X-XSS-Protection "1; mode=block";

# with Content Security Policy (CSP) enabled(and a browser that supports it(http://caniuse.com/#feat=contentsecuritypolicy),
# you can tell the browser that it can only download content from the domains you explicitly allow
# http://www.html5rocks.com/en/tutorials/security/content-security-policy/
# https://www.owasp.org/index.php/Content_Security_Policy
# I need to change our application code so we can increase security by disabling 'unsafe-inline' 'unsafe-eval'
# directives for css and js(if you have inline css or js, you will need to keep it too).
# more: http://www.html5rocks.com/en/tutorials/security/content-security-policy/#inline-code-considered-harmful
add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://ssl.google-analytics.com https://assets.zendesk.com https://connect.facebook.net; img-src 'self' https://ssl.google-analytics.com https://s-static.ak.facebook.com https://assets.zendesk.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://assets.zendesk.com; font-src 'self' https://themes.googleusercontent.com; frame-src https://assets.zendesk.com https://www.facebook.com https://s-static.ak.facebook.com https://tautt.zendesk.com; object-src 'none'";

upstream my_upstream_path {
    least_conn;
    server xx.x.x.xx:aaaa; 
    server yy.y.y.yy:aaaa; 
    keepalive 16;
}

server {rewrite ^(/.well-known/acme-challenge/.*) $1 break; # managed by Certbot


    # Certbot start
    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/s4.lister-studios.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/s4.lister-studios.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
    # Certbot end

    listen 80;
    server_name s4.lister-studios.com 188.68.41.164 s2.lister-studios.com;
    client_max_body_size 21m;
    location /my_upstream_path/ {
        proxy_pass http://my_upstream_path/my_upstream_path/;
        proxy_pass_header Accept;
        proxy_pass_header Server;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        keepalive_requests 1000;
        keepalive_timeout 360s;
        proxy_read_timeout 360s;
    }
location = /.well-known/acme-challenge/zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4{default_type text/plain;return 200 zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4.no2bx4wLS7LPR3l_Dd4J_nAcroGEfUvXa1C8YanIc0M;} # managed by Certbot

}

2019-06-30 20:26:48,646:DEBUG:certbot_nginx.parser:Writing nginx conf tree to /etc/nginx/nginx.conf:
user www-data;
worker_processes 2;
worker_rlimit_nofile 30000;
pid /run/nginx.pid;

events {
	worker_connections 65536;
        use epoll;
        multi_accept on;
}

http {
include /etc/letsencrypt/le_http_01_cert_challenge.conf;
server_names_hash_bucket_size 128;

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	##
	# Logging Settings
	##

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;

	##
	# Gzip Settings
	##

	gzip on;
	gzip_disable "msie6";

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}


#mail {
#	# See sample authentication script at:
#	# http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
# 
#	# auth_http localhost/auth.php;
#	# pop3_capabilities "TOP" "USER";
#	# imap_capabilities "IMAP4rev1" "UIDPLUS";
# 
#	server {
#		listen     localhost:110;
#		protocol   pop3;
#		proxy      on;
#	}
# 
#	server {
#		listen     localhost:143;
#		protocol   imap;
#		proxy      on;
#	}
#}

2019-06-30 20:26:49,664:INFO:certbot.auth_handler:Waiting for verification...
2019-06-30 20:26:49,665:DEBUG:acme.client:JWS payload:
b'{\n  "resource": "challenge",\n  "type": "http-01",\n  "keyAuthorization": "zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4.no2bx4wLS7LPR3l_Dd4J_nAcroGEfUvXa1C8YanIc0M"\n}'
2019-06-30 20:26:49,667:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/challenge/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ/17705181945:
{
  "signature": "h_K18elJKsFXw7n2_C-74qJOReG4N0C3hNluu73QlHpB3jWqLp7vEH8i5c0dAwnHf4rm7-F29FaINhev4vI3gV1egXxXsGPjcKPvjsecQqkasClAOGszKOvEggNR5JgYBOKnHURoCVwKrzAAES67lpml-DaROQR7p2490eRrBQon2YJz_V3A8M7gKUIPtvsAzNAavyXOKsVO0-tKRACF2wz2hfXFcsNvKsNMxOJdV3CV_YwgEumZqrUoXk5-N4LPmaDNZujm1H0Bw0HIpC3RyLllbqe-RlwPBzQoOw7x5bol4ecwrNnjAOptkHV8kkztf524RuyZqHL8Oa_CXlTL6A",
  "protected": "eyJhbGciOiAiUlMyNTYiLCAidXJsIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2NoYWxsZW5nZS9ad0FnRmI4VlZwWW1qWnVUeWtaeUFVZEtRQ3pwdzAza0RKdDlnc21SRlBRLzE3NzA1MTgxOTQ1IiwgImtpZCI6ICJodHRwczovL2FjbWUtdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hY2N0LzU1NjMzMjUzIiwgIm5vbmNlIjogInJIeVNHSl9NSkl0VEotZHJtY250bnFQcDRxWGZiLURiYW8zOVE1LUlsLWMifQ",
  "payload": "ewogICJyZXNvdXJjZSI6ICJjaGFsbGVuZ2UiLAogICJ0eXBlIjogImh0dHAtMDEiLAogICJrZXlBdXRob3JpemF0aW9uIjogInpqcXZFb2ViLV9aekxKcTJDUEx0bWdWZFdmMU9tSmwzWWF4UDdjQnY1VjQubm8yYng0d0xTN0xQUjNsX0RkNEpfbkFjcm9HRWZVdlhhMUM4WWFuSWMwTSIKfQ"
}
2019-06-30 20:26:49,847:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/challenge/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ/17705181945 HTTP/1.1" 200 224
2019-06-30 20:26:49,848:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 224
Boulder-Requester: 55633253
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index", <https://acme-v02.api.letsencrypt.org/acme/authz/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ>;rel="up"
Location: https://acme-v02.api.letsencrypt.org/acme/challenge/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ/17705181945
Replay-Nonce: cpilptwQGJzihHYPFQ__Az36-0jvz6kKCCXk-olAsiw
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Sun, 30 Jun 2019 18:26:49 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Sun, 30 Jun 2019 18:26:49 GMT
Connection: keep-alive

{
  "type": "http-01",
  "status": "pending",
  "url": "https://acme-v02.api.letsencrypt.org/acme/challenge/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ/17705181945",
  "token": "zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4"
}
2019-06-30 20:26:49,848:DEBUG:acme.client:Storing nonce: cpilptwQGJzihHYPFQ__Az36-0jvz6kKCCXk-olAsiw
2019-06-30 20:26:52,852:DEBUG:acme.client:JWS payload:
b''
2019-06-30 20:26:52,854:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ:
{
  "signature": "nW-yjU9sAQqarjz85YcoE5oFGUPjmCTpNoiamHPZ18siGLTcBR0QDzU1f5tcgtmaTt1mkaR6n_wSi1hel2iMmZtbBqk9y9THb20hjztE7IWcdzBKN__gw7MvaaytKGRVSOiYXQxvJ-TAWZc3-6VVFg01k5ZPQJuAnJ7H_HQF6dqNgJvZL6iwiQq_tM8yfJ6PzzTLI3rrCfFEHbdv4vtaKD5qTeC58BDMV3GEpJyB6SHXjKu6QQg3N3hUjJW9h0A6Tr7tr0mh5xwfmYc4VDlVyVgL7ozdZosMi2XDovpHzpESXVlcZn-HAWRKSMnpkIg1AOsMl9F8u7lLgSwovo5U9A",
  "protected": "eyJhbGciOiAiUlMyNTYiLCAidXJsIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2F1dGh6L1p3QWdGYjhWVnBZbWpadVR5a1p5QVVkS1FDenB3MDNrREp0OWdzbVJGUFEiLCAia2lkIjogImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2FjY3QvNTU2MzMyNTMiLCAibm9uY2UiOiAiY3BpbHB0d1FHSnppaEhZUEZRX19BejM2LTBqdno2a0tDQ1hrLW9sQXNpdyJ9",
  "payload": ""
}
2019-06-30 20:26:53,033:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ HTTP/1.1" 200 1919
2019-06-30 20:26:53,034:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 1919
Boulder-Requester: 55633253
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: nBEheFZP0HZFsE3QR-7bLLGOKIG6zQLMC263Y3FLnZA
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Sun, 30 Jun 2019 18:26:53 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Sun, 30 Jun 2019 18:26:53 GMT
Connection: keep-alive

{
  "identifier": {
    "type": "dns",
    "value": "s4.lister-studios.com"
  },
  "status": "invalid",
  "expires": "2019-07-07T18:26:48Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:unauthorized",
        "detail": "Invalid response from http://s4.lister-studios.com/.well-known/acme-challenge/zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4 [2a03:4000:17:6fa:b40b:18ff:fe91:df5b]: \"\u003chtml\u003e\\r\\n\u003chead\u003e\u003ctitle\u003e404 Not Found\u003c/title\u003e\u003c/head\u003e\\r\\n\u003cbody bgcolor=\\\"white\\\"\u003e\\r\\n\u003ccenter\u003e\u003ch1\u003e404 Not Found\u003c/h1\u003e\u003c/center\u003e\\r\\n\u003chr\u003e\u003ccenter\u003e\"",
        "status": 403
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/challenge/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ/17705181945",
      "token": "zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4",
      "validationRecord": [
        {
          "url": "http://s4.lister-studios.com/.well-known/acme-challenge/zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4",
          "hostname": "s4.lister-studios.com",
          "port": "80",
          "addressesResolved": [
            "188.68.41.164",
            "2a03:4000:17:6fa:b40b:18ff:fe91:df5b"
          ],
          "addressUsed": "2a03:4000:17:6fa:b40b:18ff:fe91:df5b"
        }
      ]
    },
    {
      "type": "dns-01",
      "status": "invalid",
      "url": "https://acme-v02.api.letsencrypt.org/acme/challenge/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ/17705181946",
      "token": "qgPC0Uy5w7qwQOberbP2ARCwrfHBA9tpUEF4zHlu7Dc"
    },
    {
      "type": "tls-alpn-01",
      "status": "invalid",
      "url": "https://acme-v02.api.letsencrypt.org/acme/challenge/ZwAgFb8VVpYmjZuTykZyAUdKQCzpw03kDJt9gsmRFPQ/17705181947",
      "token": "lriGfeYSTi9qj5G9s4KqChJYB78dqc1l6C7wCi1m4AI"
    }
  ]
}
2019-06-30 20:26:53,034:DEBUG:acme.client:Storing nonce: nBEheFZP0HZFsE3QR-7bLLGOKIG6zQLMC263Y3FLnZA
2019-06-30 20:26:53,036:DEBUG:certbot.reporter:Reporting to user: The following errors were reported by the server:

Domain: s4.lister-studios.com
Type:   unauthorized
Detail: Invalid response from http://s4.lister-studios.com/.well-known/acme-challenge/zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4 [2a03:4000:17:6fa:b40b:18ff:fe91:df5b]: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>"

To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address.
2019-06-30 20:26:53,037:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 82, in handle_authorizations
    self._respond(aauthzrs, resp, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 168, in _respond
    self._poll_challenges(aauthzrs, chall_update, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 239, in _poll_challenges
    raise errors.FailedChallenges(all_failed_achalls)
certbot.errors.FailedChallenges: Failed authorization procedure. s4.lister-studios.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://s4.lister-studios.com/.well-known/acme-challenge/zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4 [2a03:4000:17:6fa:b40b:18ff:fe91:df5b]: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>"

2019-06-30 20:26:53,038:DEBUG:certbot.error_handler:Calling registered functions
2019-06-30 20:26:53,038:INFO:certbot.auth_handler:Cleaning up challenges
2019-06-30 20:26:54,227:WARNING:certbot.renewal:Attempting to renew cert (s4.lister-studios.com) from /etc/letsencrypt/renewal/s4.lister-studios.com.conf produced an unexpected error: Failed authorization procedure. s4.lister-studios.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://s4.lister-studios.com/.well-known/acme-challenge/zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4 [2a03:4000:17:6fa:b40b:18ff:fe91:df5b]: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>". Skipping.
2019-06-30 20:26:54,230:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 452, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1193, in renew_cert
    renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 116, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 310, in renew_cert
    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 353, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 389, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 82, in handle_authorizations
    self._respond(aauthzrs, resp, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 168, in _respond
    self._poll_challenges(aauthzrs, chall_update, best_effort)
  File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 239, in _poll_challenges
    raise errors.FailedChallenges(all_failed_achalls)
certbot.errors.FailedChallenges: Failed authorization procedure. s4.lister-studios.com (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://s4.lister-studios.com/.well-known/acme-challenge/zjqvEoeb-_ZzLJq2CPLtmgVdWf1OmJl3YaxP7cBv5V4 [2a03:4000:17:6fa:b40b:18ff:fe91:df5b]: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>"

2019-06-30 20:26:54,230:ERROR:certbot.renewal:All renewal attempts failed. The following certs could not be renewed:
2019-06-30 20:26:54,231:ERROR:certbot.renewal:  /etc/letsencrypt/live/s4.lister-studios.com/fullchain.pem (failure)
2019-06-30 20:26:54,231:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.31.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1365, in main
    return config.func(config, plugins)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1272, in renew
    renewal.handle_renewal_request(config)
  File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 477, in handle_renewal_request
    len(renew_failures), len(parse_failures)))
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)

Hi @Spirit

if that

doesn't work, your configuration may be buggy.

What says

nginx -T

Hi @JuergenAuer,

thank you for getting in touch on a Sunday!

sudo nginx -T prints

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

The rest of the output prints the config files and mime types. To make sure that I don’t post the same info twice I believe I posted this in the debug log earlier but I’m happy to post this output, too, if it adds any needed info.

Thanks!

1 Like
    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot

    listen 80;

for the acme-challenge your server seems to listen on ipv6 and ipv4 for https but only on ipv4 for http - this is bad.

1 Like

@gpatel-fr thank you for your reply. My expertise in configuring nginx config files is not high. Could you kindly post what you would recommend please.

Thanks!

try to add
listen [::]:80;
after the listen 80;
restart nginx and retry certbot

1 Like

Thanks so much @gpatel-fr. The certicate was renewed!

Also thank you @JuergenAuer for helping out and making the web more secure one domain at a time!

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.