Dry Run works - Renewal Fails - Bad handshake

sarlacpit · March 12, 2019, 10:55am

Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. https://crt.sh/?q=example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.

My domain is: https://sch-nagios.nynet.co.uk

I ran this command: sudo certbot renew

It produced this output:

certbot renew
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Processing /etc/letsencrypt/renewal/nycc-nagios.nynet.co.uk.conf

Cert is due for renewal, auto-renewing…
Plugins selected: Authenticator apache, Installer apache
Attempting to renew cert (nycc-nagios.nynet.co.uk) from /etc/letsencrypt/renewal/nycc-nagios.nynet.co.uk.conf produced an unexpected error: (“bad handshake: Error([(‘SSL routines’, ‘SSL23_GET_SERVER_HELLO’, ‘tlsv1 alert internal error’)],)”,). Skipping.
All renewal attempts failed. The following certs could not be renewed:
/etc/letsencrypt/live/nycc-nagios.nynet.co.uk/fullchain.pem (failure)

My web server is (include version): Apache2.4.18-2ubuntu3.9

The operating system my web server runs on is (include version): Ubuntu 16.04.6 LTS

My hosting provider, if applicable, is: N/A

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

The version of my client is (e.g. output of certbot --version or certbot-auto --version if you’re using Certbot): certbot 0.28.0

Additional: If i ssue the same command with --dry-run it works so I am rather confused.
Any help would be great.

Thanks

mnordhoff · March 12, 2019, 10:57am

Can you post the traceback from /var/log/letsencrypt/letsencrypt.log and the contents of /etc/letsencrypt/renewal/nycc-nagios.nynet.co.uk.conf?

sarlacpit · March 12, 2019, 11:03am

2019-03-12 10:56:20,635:DEBUG:certbot.main:certbot version: 0.28.0
2019-03-12 10:56:20,635:DEBUG:certbot.main:Arguments: []
2019-03-12 10:56:20,636:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2019-03-12 10:56:20,642:DEBUG:certbot.log:Root logging level set at 20
2019-03-12 10:56:20,643:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2019-03-12 10:56:20,650:DEBUG:certbot.plugins.selection:Requested authenticator <certbot.cli._Default object at 0x7fa4d6064128> and installer <certbot.cli._Default object at 0x7fa4d6064128>
2019-03-12 10:56:20,656:DEBUG:certbot.storage:Should renew, less than 30 days before certificate expiry 2019-03-17 13:18:59 UTC.
2019-03-12 10:56:20,657:INFO:certbot.renewal:Cert is due for renewal, auto-renewing...
2019-03-12 10:56:20,657:DEBUG:certbot.plugins.selection:Requested authenticator apache and installer apache
2019-03-12 10:56:20,739:DEBUG:certbot_apache.configurator:Apache version is 2.4.18
2019-03-12 10:56:21,060:DEBUG:certbot.plugins.selection:Single candidate plugin: * apache
Description: Apache Web Server plugin - Beta
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT
Initialized: <certbot_apache.override_debian.DebianConfigurator object at 0x7fa4d603feb8>
Prep: True
2019-03-12 10:56:21,062:DEBUG:certbot.plugins.selection:Single candidate plugin: * apache
Description: Apache Web Server plugin - Beta
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: apache = certbot_apache.entrypoint:ENTRYPOINT
Initialized: <certbot_apache.override_debian.DebianConfigurator object at 0x7fa4d603feb8>
Prep: True
2019-03-12 10:56:21,062:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_apache.override_debian.DebianConfigurator object at 0x7fa4d603feb8> and installer <certbot_apache.override_debian.DebianCon
figurator object at 0x7fa4d603feb8>
2019-03-12 10:56:21,063:INFO:certbot.plugins.selection:Plugins selected: Authenticator apache, Installer apache
2019-03-12 10:56:21,065:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, status=None, agreement=None, only_return_existing=None, terms_of_service_agreed=None, contact=
()), terms_of_service=None, new_authzr_uri=None, uri='https://acme-v02.api.letsencrypt.org/acme/acct/47848428'), 618ae3da65b7b8becfa1a73b50c10f92, Meta(creation_dt=datetime.datetime(2018, 12, 17, 16, 15, 59, tzi
nfo=<UTC>), creation_host='nagios-sch.nynet.co.uk'))>
2019-03-12 10:56:21,066:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2019-03-12 10:56:21,069:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org
2019-03-12 10:56:21,134:WARNING:certbot.renewal:Attempting to renew cert (nycc-nagios.nynet.co.uk) from /etc/letsencrypt/renewal/nycc-nagios.nynet.co.uk.conf produced an unexpected error: ("bad handshake: Error(
[('SSL routines', 'SSL23_GET_SERVER_HELLO', 'tlsv1 alert internal error')],)",). Skipping.
2019-03-12 10:56:21,137:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 438, in wrap_socket
    cnx.do_handshake()
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1716, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/usr/lib/python3/dist-packages/OpenSSL/SSL.py", line 1456, in _raise_ssl_error
    _raise_current_error()
  File "/usr/lib/python3/dist-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'SSL23_GET_SERVER_HELLO', 'tlsv1 alert internal error')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 846, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 326, in connect
    ssl_context=context)
  File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 325, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3/dist-packages/urllib3/contrib/pyopenssl.py", line 445, in wrap_socket
    raise ssl.SSLError('bad handshake: %r' % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'tlsv1 alert internal error')],)",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 630, in urlopen
    raise SSLError(e)
urllib3.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'tlsv1 alert internal error')],)",)

During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 430, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1166, in renew_cert
    le_client = _init_le_client(config, auth, installer)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 611, in _init_le_client
    return client.Client(config, acc, authenticator, installer, acme=acme)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 248, in __init__
    acme = acme_from_config_key(config, self.account.key, self.account.regr)
  File "/usr/lib/python3/dist-packages/certbot/client.py", line 51, in acme_from_config_key
    return acme_client.BackwardsCompatibleClientV2(net, key, config.server)
  File "/usr/lib/python3/dist-packages/acme/client.py", line 763, in __init__
    directory = messages.Directory.from_json(net.get(server).json())
  File "/usr/lib/python3/dist-packages/acme/client.py", line 1097, in get
    self._send_request('GET', url, **kwargs), content_type=content_type)
  File "/usr/lib/python3/dist-packages/acme/client.py", line 1046, in _send_request
    response = self.session.request(method, url, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 502, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 612, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'tlsv1 alert internal error')],)",)

2019-03-12 10:56:21,138:ERROR:certbot.renewal:All renewal attempts failed. The following certs could not be renewed:
2019-03-12 10:56:21,138:ERROR:certbot.renewal:  /etc/letsencrypt/live/nycc-nagios.nynet.co.uk/fullchain.pem (failure)
2019-03-12 10:56:21,139:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.28.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1340, in main
    return config.func(config, plugins)
  File "/usr/lib/python3/dist-packages/certbot/main.py", line 1247, in renew
    renewal.handle_renewal_request(config)
  File "/usr/lib/python3/dist-packages/certbot/renewal.py", line 455, in handle_renewal_request
    len(renew_failures), len(parse_failures)))
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)

And the conf…

cat /etc/letsencrypt/renewal/nycc-nagios.nynet.co.uk.conf
# renew_before_expiry = 30 days
version = 0.28.0
archive_dir = /etc/letsencrypt/archive/nycc-nagios.nynet.co.uk
cert = /etc/letsencrypt/live/nycc-nagios.nynet.co.uk/cert.pem
privkey = /etc/letsencrypt/live/nycc-nagios.nynet.co.uk/privkey.pem
chain = /etc/letsencrypt/live/nycc-nagios.nynet.co.uk/chain.pem
fullchain = /etc/letsencrypt/live/nycc-nagios.nynet.co.uk/fullchain.pem

# Options used in the renewal process
[renewalparams]
server = https://acme-v02.api.letsencrypt.org/directory
authenticator = apache
installer = apache
account = 618ae3da65b7b8becfa1a73b50c10f92

JuergenAuer · March 12, 2019, 11:04am

Hi @sarlacpit

looks like you have used tls-sni-01 validation. That is deprecated. But --dry-run uses http-01 validation.

Your port 80 is open, that looks good.

So try

certbot renew --preferrend-challenges http

to switch your challenge method.

sarlacpit · March 12, 2019, 11:09am

Thanks, interestingly, that doesn’t work either:

certbot renew --preferrend-challenges http
usage: 
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate. 
certbot: error: unrecognized arguments: --preferrend-challenges http

mnordhoff · March 12, 2019, 11:13am

What happens if you run:

curl -v https://acme-v02.api.letsencrypt.org/directory

curl -v https://acme-staging-v02.api.letsencrypt.org/directory

sarlacpit · March 12, 2019, 11:16am

Thanks, the first option gives an error…

curl -v https://acme-v02.api.letsencrypt.org/directory
*   Trying 23.36.209.29...
* Connected to acme-v02.api.letsencrypt.org (23.36.209.29) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 594 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* gnutls_handshake() failed: Internal error
* Closing connection 0
curl: (35) gnutls_handshake() failed: Internal error

The second curl seems to pull more information:

curl -v https://acme-staging-v02.api.letsencrypt.org/directory
*   Trying 184.87.187.237...
* Connected to acme-staging-v02.api.letsencrypt.org (184.87.187.237) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 594 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: acme-v02.api.letsencrypt.org (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: CN=acme-v02.api.letsencrypt.org
* 	 start date: Fri, 01 Mar 2019 04:24:29 GMT
* 	 expire date: Thu, 30 May 2019 04:24:29 GMT
* 	 issuer: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
* 	 compression: NULL
* ALPN, server accepted to use http/1.1
> GET /directory HTTP/1.1
> Host: acme-staging-v02.api.letsencrypt.org
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Content-Type: application/json
< Content-Length: 724
< X-Frame-Options: DENY
< Strict-Transport-Security: max-age=604800
< Expires: Tue, 12 Mar 2019 11:13:39 GMT
< Cache-Control: max-age=0, no-cache, no-store
< Pragma: no-cache
< Date: Tue, 12 Mar 2019 11:13:39 GMT
< Connection: keep-alive
< 
{
  "keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org/docs/staging-environment/"
  },
  "newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert",
  "sWTvBLuFkgY": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417"
* Connection #0 to host acme-staging-v02.api.letsencrypt.org left intact

mnordhoff · March 12, 2019, 11:22am

Any chance there’s a hardcoded IP address for acme-v02.api.letsencrypt.org in /etc/hosts?

sarlacpit · March 12, 2019, 11:27am

Why yes there is… Fantastic spot, thank you.

23.36.209.29 acme-v02.api.letsencrypt.org

By removing it, I successfully renewed my cert

I suspect it was a dirty fix to get around the firewall. Is there a list of IPs I could add to the permit list on my firewall as it doesn’t do DNS?

mnordhoff · March 12, 2019, 11:31am

Not that I’m aware of.

Let’s Encrypt uses Akamai’s CDN. Last I looked, Akamai’s position on IP whitelisting was “please don’t”. The Let’s Encrypt API might have thousands of IPs and they can probably change at any time.

sarlacpit · March 12, 2019, 11:42am

Dang!
Nevermind then. Will probably have to build a proxy,
Thanks for your help though it’s really appreciated.

JuergenAuer · March 12, 2019, 1:45pm

Was an error --preferred-challenges is correct.

sarlacpit · March 13, 2019, 4:41pm

Ahhh, the perils of copy/paste - Thanks

system · April 12, 2019, 4:41pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Issue with renewals after TLS-SNI-01 Help	26	2609	March 30, 2019
Error Renewing Certbot Certificates Help	15	3552	October 1, 2018
The client lacks sufficient authorization :: Invalid response from http://erp.versus-alternative.ch/.well-known/acme-challenge/ Help	12	2259	April 13, 2019
Dry-run cert renewal works, live cert renewal fails on 400 error Help	9	1890	July 11, 2019
Problem with renewal: Incorrect validation certificate for tls-sni-01 challenge Help	6	2812	February 1, 2018

Dry Run works - Renewal Fails - Bad handshake

Related topics