Problems with sentry and letsencrypt

I keep getting URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)> errors from raven (sentry client). It seems to belong to the python requests library, but import requests;requests.get("https://mysentryserver") works without any problems.

Backtrace:

Sending a test message... DEBUG 2016-09-17 22:58:26,849 base 15122 140081331529472 Sending message of length 3239 to https://sentryserver
Event ID was 'blabla'
ERROR 2016-09-17 22:58:26,875 base 15122 140081255401216 Sentry responded with an error: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)> (url: https://sentryserver/api/3/store/)
Traceback (most recent call last):
  File "/venv/local/lib/python2.7/site-packages/raven/transport/threaded.py", line 174, in send_sync
	super(ThreadedHTTPTransport, self).send(data, headers)
  File "/venv/local/lib/python2.7/site-packages/raven/transport/http.py", line 47, in send
	ca_certs=self.ca_certs,
  File "/venv/local/lib/python2.7/site-packages/raven/utils/http.py", line 66, in urlopen
	return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
	response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
	'_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
	result = func(*args)
  File "/venv/local/lib/python2.7/site-packages/raven/utils/http.py", line 46, in https_open
	return self.do_open(ValidHTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1197, in do_open
	raise URLError(err)
URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>
ERROR 2016-09-17 22:58:26,875 base 15122 140081255401216 [u'This is a test message generated using ``raven test``']

sentry has a cacert.pem and requests has one, but both seem to include the IdenTrust Root-Cert, which signed the letsencrypt root. And it works with requests in the python shell itself.

This issue seems to indicate that raven does not support SNI. If you use multiple certificates on the same IP address, SNI is required for the server to know which certificate to use; if the client does not support SNI, the server picks the default certificate, which might be one for a different domain (hence the failing verification).

You can use SSL Labs to check whether your server currently requires SNI support (there’s a message at the top of the results if that’s the case).

There are a number of solutions available:

  • If you can get additional IP addresses for your server, you could run Sentry on a dedicated IP address.
  • Depending on your setup, getting a SAN certificate that covers all domains on that server would also work. If you use certbot, just pass multiple -d example.com arguments to the client to do this.
  • It might also be enough to set the certificate for sentry as the default certificate, typically by setting a default vhost or server block. In nginx, this would be something like listen 443 default_server ssl;.
2 Likes

Indeed. Its this bug:

And this is the solution:

from raven.transport.requests import RequestsHTTPTransport
RAVEN_CONFIG = {
    'dsn': '...',
    'transport': RequestsHTTPTransport,
}

More transports (which may or may not use SNI): https://raven.readthedocs.io/en/stable/transports.html

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.