Having two sites at different hosting providers but almost the same servers got significantly different SSL handshake times. The difference is huge and stable from day to day, independently of daytime - 30 and 250 ms.
Well. It might be different networks (the ISPs that transit the requests...)
Also, You have a really OUTDATED php, Nginx... Not sure about other services. But those outdated services would affect loading (such as h1 / h2 and efficiency)
To better compare the server differences, you should use two machines on the same network/location with exact specifications. (and host exactly the same website with the same configurations)
But all we could do is guess, you'll need to resolve all things by yourself.
I’ve made one additional case and found that the difference depends of the server location.
If to ping Russian server from Russia - the SSL handshake takes 60 ms, but the same server if to ping from Europe - 250 ms. I’m not clever in HTTPS technique, but if CLIENT machine receiving the certificate - this mean RU-RU ping should be longer, if SERVER receives the certificate it should be no difference.
So I’m a little bit confusing.
By the way, it the delay is due to outdated PHP/Apache, this might bring the FIRST machine to be slower while it is not.