One thing that could cause this is if Apache wasn't restarted correctly after the certificate renewal: strangely enough, it's possible for some of the Apache worker processes on the server to have a different certificate from others. This is a bug in Apache.
You could try making sure that Apache is completely stopped on the server and that no copies are running at all, then restart your Apache.
This can also happen due to browser caching but, if so, the problem should not really last very long or be very widespread.
I have done two full server reboots thinking that was the problem (having seen it in other posts, gotta do my homework) as well as checked command certbot certificates and everything looks properly setup.
I agree with that concern, but I think the error produced should be different, since although the certificate presented for the www name is mismatched, it isn't expired.
We have setup forwarding rules to push www to non-www via htaccess, same with http pushing to https via apache virtual domains. But maybe that's insufficient?
That's really weird; I don't think I've ever seen this problem before with that combination of circumstances.
Can you get any user who has this problem to send you the result of a DNS lookup for your site, or a copy of the expired certificate as seen by the browser? Can the users also try a different browser for comparison?
That's also very plausible, but it looks like this is hosted directly on a Linode VPS.
You might not have covered the case where someone types in the URL: https://www.therathink.com/
That URL would require a cert just to serve the redirection to the non-www name.
Probably the bug @schoen mentioned with Apache childs not wanting to stop or something like that. Depending on how many good and how many bad child processes are active, it might be possible to reproduce this by hammering the server with lots of connections and automatically checking the certificate where you ignore all the valid certificates.
Correct, my review on crt.sh also points to these dates which is confusing. It makes it seem like a caching issue but I'm so confused how something like that could take place for a certificate?
Fascinating. So if we hit the maximum possible connections (as defined in apache2 settings) to push other users off the system, hypothetically their next visit would force cert review*?
I have no idea. Also, I have no idea how you got this idea from my text
I was merely saying you might see the erroneous certificate if you tried hard enough, i.e., made enough connections. But as it might be just a single buggy Apache child among perhaps hundereds (depending on your server of course) good Apache childs, you might need to make a lot of connections before you see a single erroneous certificate. Hence the "hammering".
When the service starts it loads the current cert info.
And uses that until the next reload/restart.
But if any processes fail to stop on restart...
Then you might have some new and some old processes running.
Some with new cert and some with old cert.
This would mostly go unnoticed if the old and new are the same cert.
Like if the cert lifetime was for a year - you would not notice an old process and it would act just like a new one.
Only during a period of cert replacement would you come to realize that your web service is wonky.
I would stop the service and see if all of it has actually stopped.
If not, problem found - kill any remaining processes and restart.
If all did stop, then just restart and wait for any more complaints.
