Some help needed

Wouldn't it be simpler to just get all new certs?

4 Likes

Will it let me do that, because I plan to kill the centos 7 box and everything on it?

1 Like

It should.

3 Likes

I'll have a go tomorrow. I've had it for today, I LOVE this weather, 3 days of sun. Topped 40 degrees yesterday, but more bloody rain is on it's way.
In South Africa we used to dance in the streets when it started to rain after winter, but since I came back to Europe it rains every bloody day!

2 Likes

So that means ...
A whole lot of dancing ! ! !
LOL

4 Likes

Whole lotta CRYING, you mean. I want my Vitamin D and not out of a bottle!

2 Likes

Here we go. A problem of my own making!

I have a small bit of software that searches my logs and looks for aholes that are trying to hack that fail2ban doesn't catch.

I think I've blocked one or more of lets encrypt's IP addresses, so I can't authenticate.

Certbot failed to authenticate some domains (authenticator: apache). The Certificate Authority reported these problems:
Domain: server1.MyDomainFQDN
Type: connection
Detail: 79.132.230.61: Fetching http://server1.MyDomainFQDN/.well-known/acme-challenge/tCXQKXZwwYhmNSXlD_ltjp9Hsr7pfxU3OHtrOC8Eg0g: Timeout during connect (likely firewall problem)

Do you have a list of IP's that I can whitelist?

1 Like

No.
You should consider accepting all HTTP and then handle the HTTP authentication requests there and redirect everything else to HTTPS [where you can ban whatever IPs you like]
Also see:

3 Likes

When I'd finished setting up the new PC, I had to go out, but I'd already entered it into the DNS. 27 hours later and I had a list of almost 400 IP addeesses that had tried to break into the server (because one reason, I'd forgotten to change Port 22), but I had page after page of logs.

I'm allergic to hackers. If I were the ruler, I'd bring in the death penalty (public burning at the stake) for hackers and removal of BOTH hands for spammers.

I'm sick to death of 100% bandwidth being taken up by these morons.

1 Like

I hear ya. I run a block list with over 4M hacker IPs on it - LOL

3 Likes

Do you publish it for other users like spamhaus? OR, can I get a copy, save me a lotta work! I add around 50 daily.

1 Like

Yes.

No, it works mainly via DNS.

I add around 1K a day - LOL

4 Likes

You said it should. - It doesn't.
After 2 days of battling, I gave up. I moved everything back to my old CentOS 7 server.
Nothing these days works like it's supposed to do. I guess one day before I die, I'll get used to that.

1 Like

We are here to help with such problems.
We failed you ... without being given a chance :frowning:

3 Likes

For complex stuff like this I'd really recommend doing a dry run migration - there is no need to kill your old server and jump to a new one without practising the migration first, unless you crave the adrenalin rush of fixing stuff in real-time. You've had your first practice attempt now, so you can start to anticipate which things are going to be a problem.

Copy services one at a time to a new server (or multiple new VMs, one service/website at a time is often usually best), edit your desktop hosts file to point the domain to the new server and try out the website and config at your leisure, then when you want to cut over you stop the old service, migrate the latest databases etc to the new server and re-point DNS.

From then on things like certificate renewals can use the new server. Point a test subdomain like staging.www.yoursite.com to the same site and get that up and running with a cert to prove your cert requests/renewals are working from that machine.

I'm a fan of moving important services to the latest server OS fairly regularly otherwise you start to worry about how hard it might be to move them and start to risk missing security updates, and from your initial summary it sounds like you have stuff ranging from 17 yrs old to about 10 years old, and pretty old hardware. It's possible nothing you're running is within a current support lifecycle from any vendor and realistically the hardware you're running your VMs on could fail any day. So while you're looking at migration you could also take the opportunity to consider if these VMs could really be cloud hosted (Azure etc).

Depending on what your sites actually do there are also alternatives to running your own server such as cloud web app hosting (so you don't maintain the OS) worth looking at. Often these services handle certs for you because they do all the actual web serving.

So, take your time to plan and practice your migration more than once, identify pain points before you need to do the real migration, otherwise you'll be trawling forums like this at 2am.

If your business is pretty serious (I'm assuming it is) it's normal to have a disaster recovery plan for each service which would allow a competent IT professional who has never used the system to recover it from backup and get it running. Use this opportunity to write that plan for each service, it only has to be a page or two if that covers that needs to be done in enough detail. If your services don't warrant documenting a plan for recovery, just shut them down.

5 Likes

I'd also suggest looking at whether you can use (automated) DNS validation instead of http validation, as this affords you the ability to request certs from any machine. Again, running your dns using Azure DNS, AWS Route 53, Good Cloud DNS etc will provide lots of automation options and free up your time and resources.

5 Likes

No, I haven't given up. I want to try something else. I still have the Alma server. I just took it out of the DNS, too many complaints that forums were down, so they are back up.

1 Like

It isn't something blocked on the firewall. I'm using OPNsense and they have the worst community support I have ever had the misfortune to come across.
I lease a /29 subnet from EDPNet.
I have xxx.xxx.xxx.57, 58, 59, 60, 61 and 62. 57 is listed as a gateway.
I have the old CentOS 7.3 box on 58, my mail server on 59 and I put the Alma on 60.
I copied all the relevant files (same for Centos 7.3 and centos 8
I tested), /etc/httpd/conf.d and the two forums in www/server1 and server2, name-based virtual webservers.
I cloned the Alias for Server and ports from the Centos ones, just changed the details to match Alma.
Cloned the Virtual IP, the NAT--> Port Forwarding and the rules, so apart from names and IP addreses. everything good, right?
I have GUI on the Alma box so I now head off to my DNS servers (in-house) and add alma with the .60 IP address. Then I change to IP for the two Forums from 192.168.0.203 to 213.
My in-house site is a sub-dimain of my main domain, hosted in Canada. The IP addressing for my sub-domain is handled by Dynu. I changed the DNS entries from xxx.xxx.xxx.58 to 60.
Wait 24 hours, down the Centos machine and go into Alma, load firefox, enter server1.mydomainfqdn and get NOTHING
Enter http://xxx.xxx.xxx.60 and get NOTHING
Check all Firewall rules and post a question about this on OPNsense forums. NO REPLY after 2 days!
I fire up the CentOS server and after lunch go back to Alma. OMG, it's working. http://server1.mydomainfqdn and it works!
http://xxx.xxx.xxx.60 NOTHING
Ask one of my customers to try to login. NOTHING (times out)
Bring down the CentOS server and now http://server1.mydomainfqdn doesn't work any longer.
Check DNS firewalls, No obvious problem.
I ask on Alma forum. No response. Great support!!!!
That's when I gave up.
I'm still looking for the cause of the problem but until I find it, at least everything works again. I still have a nice clean Veeam backup of Alma.

1 Like

I trust 'the Cloud' as much as I trust the CIA or a killer on Death row. Fortunately we are only a small business and I'm happy to host everything in our office where I have total control.

1 Like

I sense you're looking for moral support rather than technical support :slight_smile: all of the things you have mentioned here are to do with routing traffic to your new server and nothing to do with certificates.

Get your new server running and working, then practice migration to it a few times (with test domains) before committing.

5 Likes