[ssl:error] causing potential SIGTERM and crashing server?

Help
1

Hey all!

I’m running into an issue with a server setup that contains about 10 sites. Every single day between 8-9am EST and 8-9pm EST the sites start to show intermittent downtime (reported by Pingdom). Last night and this morning, the entire server blacked out during those time periods and all sites were unreachable.

Checking the apache logs for the downtime this morning, I see this:

Mon Jul 16 12:11:10.636673 2018] [ssl:error] [pid 27895] AH02032: Hostname 138.197.20.176 provided via SNI and hostname www.asdonline.com provided via HTTP have no compatible SSL setup
[Mon Jul 16 12:11:10.642131 2018] [ssl:error] [pid 27969] AH02032: Hostname 138.197.20.176 provided via SNI and hostname asdonline.com provided via HTTP have no compatible SSL setup
[Mon Jul 16 12:11:21.870380 2018] [ssl:error] [pid 27942] [client 195.154.184.122:41198] AH02261: Re-negotiation handshake failed
[Mon Jul 16 12:11:21.870453 2018] [ssl:error] [pid 27942] SSL Library Error: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error (SSL alert number 80)
[Mon Jul 16 12:11:22.305958 2018] [ssl:error] [pid 27951] [client 195.154.184.122:50689] AH02261: Re-negotiation handshake failed
**[Mon Jul 16 12:11:22.306040 2018] [ssl:error] [pid 27951] SSL Library Error: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error (SSL alert number 80)**
**[Mon Jul 16 12:19:01.894035 2018] [mpm_prefork:notice] [pid 1738] AH00169: caught SIGTERM, shutting down**
**[Mon Jul 16 12:19:09.919141 2018] [mpm_prefork:notice] [pid 28408] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations**
**[Mon Jul 16 12:19:09.919228 2018] [core:notice] [pid 28408] AH00094: Command line: '/usr/sbin/apache2'**
**[Mon Jul 16 12:19:10.156781 2018] [mpm_prefork:notice] [pid 28408] AH00169: caught SIGTERM, shutting down**
**[Mon Jul 16 12:25:57.567484 2018] [mpm_prefork:notice] [pid 1750] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations**
[Mon Jul 16 12:25:57.568246 2018] [core:notice] [pid 1750] AH00094: Command line: '/usr/sbin/apache2'
[Mon Jul 16 12:39:12.330997 2018] [ssl:error] [pid 2537] [client 180.76.15.136:51687] AH02225: Re-negotiation request failed
[Mon Jul 16 12:39:12.331065 2018] [ssl:error] [pid 2537] SSL Library Error: error:14080152:SSL routines:ssl3_accept:unsafe legacy renegotiation disabled

You can see those SIGTERM lines. They coincide with downtime. The second one took the server down for 8 minutes.

It was preceded with this error:

[Mon Jul 16 12:11:22.306040 2018] [ssl:error] [pid 27951] SSL Library Error: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error (SSL alert number 80)

I feel like there is something wrong with our OpenSSL/LetsEncrypt config, but I can’t figure it out.

My domain is: Multiple domains on same server, all affected. Example: www.nynow.com

My web server is (include version): Ubuntu LAMP on 16.04

The operating system my web server runs on is (include version): Ubuntu LAMP on 16.04

My hosting provider, if applicable, is: Digital Ocean

I can login to a root shell on my machine (yes or no, or I don’t know): Yes

I’m using a control panel to manage my site (no, or provide the name and version of the control panel): No

2

Hi,

Can you try to check the Apache ssl protocol in each vHost files? It seems that you enabled tls1.0 (you should disable it) (and maybe insexuee negotiation)

Thank you

3

Hi @blancast

see there:

Perhaps you may update your OpenSSL

4

Unfortunately alert 80 is the TLS fallback error code, it doesn’t have any particular meaning. The significant part is from Apache:

Re-negotiation handshake failed

If you can get a port 443 pcap at the time where the server crashes, seeing the TLS handshake would help to figure out what’s happening. You can use tshark or tcpdump to do this.

You can also increase the debugging level in Apache’s SSL module, which will help see the renegotiation patterns (and perhaps reveal if it is a specific client causing the crash):

LogLevel ssl:debug

I believe that the “no compatible SSL setup” is not the cause of the crashes, at least, I can’t crash your server with such a request. Nonetheless, it would be helpful to see your actual Apache configuration for your virtual hosts.

5

Hey all,

Thanks for the suggestions!

I’ve tracked this down to an issue with the auto-renew cron that runs every 12 hours. For some reason, it appears that it is struggling every single time. I have yet to figure out where the discrepancy is.

6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.