Dry Run works - Renewal Fails - Bad handshake

Help
1

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: https://sch-nagios.nynet.co.uk

I ran this command: sudo certbot renew

It produced this output:

certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/nycc-nagios.nynet.co.uk.conf

Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator apache, Installer apache
Attempting to renew cert (nycc-nagios.nynet.co.uk) from /etc/letsencrypt/renewal/nycc-nagios.nynet.co.uk.conf produced an unexpected error: (“bad handshake: Error([(‘SSL routines’, ‘SSL23_GET_SERVER_HELLO’, ‘tlsv1 alert internal error’)],)”,). Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/nycc-nagios.nynet.co.uk/fullchain.pem (failure)

My web server is (include version): Apache2.4.18-2ubuntu3.9

The operating system my web server runs on is (include version): Ubuntu 16.04.6 LTS

My hosting provider, if applicable, is: N/A

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.28.0

Additional: If i ssue the same command with --dry-run it works so I am rather confused.
Any help would be great.

Thanks

2

Can you post the traceback from /var/log/letsencrypt/letsencrypt.log and the contents of /etc/letsencrypt/renewal/nycc-nagios.nynet.co.uk.conf?

3 
2019-03-12 10:56:20,635:DEBUG:certbot.main:certbot version: 0.28.0
2019-03-12 10:56:20,635:DEBUG:certbot.main:Arguments: []
2019-03-12 10:56:20,636:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2019-03-12 10:56:20,642:DEBUG:certbot.log:Root logging level set at 20
2019-03-12 10:56:20,643:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2019-03-12 10:56:20,650:DEBUG:certbot.plugins.selection:Requested authenticator <certbot.cli._Default object at 0x7fa4d6064128> and installer <certbot.cli._Default object at 0x7fa4d6064128>
2019-03-12 10:56:20,656:DEBUG:certbot.storage:Should renew, less than 30 days before certificate expiry 2019-03-17 13:18:59 UTC.
2019-03-12 10:56:20,657:INFO:certbot.renewal:Cert is due for renewal, auto-renewing...
2019-03-12 10:56:20,657:DEBUG:certbot.plugins.selection:Requested authenticator apache and installer apache
2019-03-12 10:56:20,739:DEBUG:certbot_apache.configurator:Apache version is 2.4.18
2019-03-12 10:56:21,060:DEBUG:certbot.plugins.selection:Single candidate plugin: * apache
Description: Apache Web Server plugin - Beta
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT
Initialized: <certbot_apache.override_debian.DebianConfigurator object at 0x7fa4d603feb8>
Prep: True
2019-03-12 10:56:21,062:DEBUG:certbot.plugins.selection:Single candidate plugin: * apache
Description: Apache Web Server plugin - Beta
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT
Initialized: <certbot_apache.override_debian.DebianConfigurator object at 0x7fa4d603feb8>
Prep: True
2019-03-12 10:56:21,062:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_apache.override_debian.DebianConfigurator object at 0x7fa4d603feb8> and installer <certbot_apache.override_debian.DebianCon
figurator object at 0x7fa4d603feb8>
2019-03-12 10:56:21,063:INFO:certbot.plugins.selection:Plugins selected: Authenticator apache, Installer apache
2019-03-12 10:56:21,065:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, status=None, agreement=None, only_return_existing=None, terms_of_service_agreed=None, contact=
()), terms_of_service=None, new_authzr_uri=None, uri='https://acme-v02.api.letsencrypt.org/acme/acct/47848428'), 618ae3da65b7b8becfa1a73b50c10f92, Meta(creation_dt=datetime.datetime(2018, 12, 17, 16, 15, 59, tzi
nfo=<UTC>), creation_host='nagios-sch.nynet.co.uk'))>
2019-03-12 10:56:21,066:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2019-03-12 10:56:21,069:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
2019-03-12 10:56:21,134:WARNING:certbot.renewal:Attempting to renew cert (nycc-nagios.nynet.co.uk) from /etc/letsencrypt/renewal/nycc-nagios.nynet.co.uk.conf produced an unexpected error: ("bad handshake: Error(
[('SSL routines', 'SSL23_GET_SERVER_HELLO', 'tlsv1 alert internal error')],)",). Skipping.
2019-03-12 10:56:21,137:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 438, in wrap_socket
    cnx.do_handshake()
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1716, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1456, in _raise_ssl_error
    _raise_current_error()
  File "/usr/lib/python3/dist-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'SSL23_GET_SERVER_HELLO', 'tlsv1 alert internal error')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 846, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 326, in connect
    ssl_context=context)
  File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 325, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 445, in wrap_socket
    raise ssl.SSLError('bad handshake: %r' % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'tlsv1 alert internal error')],)",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 630, in urlopen
    raise SSLError(e)
urllib3.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'tlsv1 alert internal error')],)",)

During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 430, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1166, in renew_cert
    le_client = _init_le_client(config, auth, installer)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 611, in _init_le_client
    return client.Client(config, acc, authenticator, installer, acme=acme)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 248, in __init__
    acme = acme_from_config_key(config, self.account.key, self.account.regr)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 51, in acme_from_config_key
    return acme_client.BackwardsCompatibleClientV2(net, key, config.server)
  File "/usr/lib/python3/dist-packages/acme/client.py", line 763, in __init__
    directory = messages.Directory.from_json(net.get(server).json())
  File "/usr/lib/python3/dist-packages/acme/client.py", line 1097, in get
    self._send_request('GET', url, **kwargs), content_type=content_type)
  File "/usr/lib/python3/dist-packages/acme/client.py", line 1046, in _send_request
    response = self.session.request(method, url, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 502, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 612, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'tlsv1 alert internal error')],)",)

2019-03-12 10:56:21,138:ERROR:certbot.renewal:All renewal attempts failed. The following certs could not be renewed:
2019-03-12 10:56:21,138:ERROR:certbot.renewal:  /etc/letsencrypt/live/nycc-nagios.nynet.co.uk/fullchain.pem (failure)
2019-03-12 10:56:21,139:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.28.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1340, in main
    return config.func(config, plugins)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1247, in renew
    renewal.handle_renewal_request(config)
  File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 455, in handle_renewal_request
    len(renew_failures), len(parse_failures)))
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)

And the conf…

cat /etc/letsencrypt/renewal/nycc-nagios.nynet.co.uk.conf
# renew_before_expiry = 30 days
version = 0.28.0
archive_dir = /etc/letsencrypt/archive/nycc-nagios.nynet.co.uk
cert = /etc/letsencrypt/live/nycc-nagios.nynet.co.uk/cert.pem
privkey = /etc/letsencrypt/live/nycc-nagios.nynet.co.uk/privkey.pem
chain = /etc/letsencrypt/live/nycc-nagios.nynet.co.uk/chain.pem
fullchain = /etc/letsencrypt/live/nycc-nagios.nynet.co.uk/fullchain.pem

# Options used in the renewal process
[renewalparams]
server = https://acme-v02.api.letsencrypt.org/directory
authenticator = apache
installer = apache
account = 618ae3da65b7b8becfa1a73b50c10f92
4

Hi @sarlacpit

looks like you have used tls-sni-01 validation. That is deprecated. But --dry-run uses http-01 validation.

Your port 80 is open, that looks good.

So try

certbot renew --preferrend-challenges http

to switch your challenge method.

5

Thanks, interestingly, that doesn’t work either:

certbot renew --preferrend-challenges http
usage: 
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate. 
certbot: error: unrecognized arguments: --preferrend-challenges http
6

What happens if you run:

curl -v https://acme-v02.api.letsencrypt.org/directory

curl -v https://acme-staging-v02.api.letsencrypt.org/directory
7

Thanks, the first option gives an error…

curl -v https://acme-v02.api.letsencrypt.org/directory
*   Trying 23.36.209.29...
* Connected to acme-v02.api.letsencrypt.org (23.36.209.29) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 594 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* gnutls_handshake() failed: Internal error
* Closing connection 0
curl: (35) gnutls_handshake() failed: Internal error

The second curl seems to pull more information:

curl -v https://acme-staging-v02.api.letsencrypt.org/directory
*   Trying 184.87.187.237...
* Connected to acme-staging-v02.api.letsencrypt.org (184.87.187.237) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 594 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: acme-v02.api.letsencrypt.org (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: CN=acme-v02.api.letsencrypt.org
* 	 start date: Fri, 01 Mar 2019 04:24:29 GMT
* 	 expire date: Thu, 30 May 2019 04:24:29 GMT
* 	 issuer: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
* 	 compression: NULL
* ALPN, server accepted to use http/1.1
> GET /directory HTTP/1.1
> Host: acme-staging-v02.api.letsencrypt.org
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Content-Type: application/json
< Content-Length: 724
< X-Frame-Options: DENY
< Strict-Transport-Security: max-age=604800
< Expires: Tue, 12 Mar 2019 11:13:39 GMT
< Cache-Control: max-age=0, no-cache, no-store
< Pragma: no-cache
< Date: Tue, 12 Mar 2019 11:13:39 GMT
< Connection: keep-alive
< 
{
  "keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org/docs/staging-environment/"
  },
  "newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert",
  "sWTvBLuFkgY": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417"
* Connection #0 to host acme-staging-v02.api.letsencrypt.org left intact
8

Any chance there’s a hardcoded IP address for acme-v02.api.letsencrypt.org in /etc/hosts?

9

Why yes there is… Fantastic spot, thank you.

23.36.209.29 acme-v02.api.letsencrypt.org

By removing it, I successfully renewed my cert :smiley:

I suspect it was a dirty fix to get around the firewall. Is there a list of IPs I could add to the permit list on my firewall as it doesn’t do DNS?

10

Not that I’m aware of. :slightly_frowning_face:

Let’s Encrypt uses Akamai’s CDN. Last I looked, Akamai’s position on IP whitelisting was “please don’t”. The Let’s Encrypt API might have thousands of IPs and they can probably change at any time.

11

Dang!
Nevermind then. Will probably have to build a proxy,
Thanks for your help though it’s really appreciated.

12

Was an error --preferred-challenges is correct.

13

Ahhh, the perils of copy/paste - Thanks :slight_smile: