Blue Green server set up

Context

My website is on DigitalOcean servers and I have them configured in a Blue-Green configuration with a Floating IP to point to whichever is live. The website uses Nginx.

The website is live but I am making regular updates to it and so switch between the Blue and the Green server often (about once every two weeks). I use the non-live server as a staging server and then switch that to live once I've tested my changes. I believe this is quite a common set up.

Problem

I used Certbot to install SSL on one of the servers and this worked very well. I now need to know how to install on the other server and retain the ability to switch between the two at will.

I have read a few posts but most of them seem to assume that switching between live and standby will be a rare event and that downtime is acceptable. In my case I regularly switch between the two servers and the whole point of the Blue Green configuration is to allow this change to happen seamlessly and I am keen to preserve this. The driver behind this is that I am not an expert in server admin and like to be 100% sure that a server is working before I switch to it.

Any advice welcome.

Are you also hosting your domain's DNS on Digital Ocean DNS? If not, where? DNS validation is one of the more straightforward solutions you could pursue.

Thanks for your reply.

I am hosting DNS on Fasthosts. I didn't know there was a DNS route to doing this! How does that work?

Ah, that's too bad. I don't know if there are any ACME clients which support it.

The DNS-01 challenge. An ACME client like Certbot is asked to create a TXT record (as opposed to an HTTP resource, like in your current setup), as the method to perform domain validation in order to obtain a certificate.

This has the benefit that the servers themselves do not need to respond to challenges directly, all they need to do is be able to create records in the DNS zone.

Had you been using Digital Ocean or another supported DNS host, you could follow instructions like these or these.

Some other approaches you can try:

  • You could move your domain to a commonly supported provider like Digital Ocean or Cloudflare or whatever.
    • (If you can do this, it's the simplest way by far imho).
  • If the 2 servers can be given authorization to write files to each other (via SSH or something), or have some kind of shared storage volume for the acme-challenge directory, you could use a Certbot --manual-auth-hook to automatically copy the required HTTP challenge to the other server.
  • You could try acme.sh stateless mode for nginx and duplicate the Let's Encrypt account across both servers.
  • You could try one of the other tricks with DNS like acme-dns or acme.sh alias mode.

I would go for whatever seems the simplest for you to achieve. Unfortunately most of those, besides changing DNS providers, are pretty tricky to pull off.

Note that you can also move your DNS nameserver from Fasthosts to Cloudflare etc while still keeping your services, domain renewals and packages etc, I do exactly this (with Fasthosts).

1 Like

Thanks for this reply, it's really helpful. I'll need to have a think.

I chose not to register my domain on DigitalOcean simply because I wasn't sure whether I would be web hosting with them long term and didn't want to get too tied in, but truthfully if it's going to make life easier technically then I can go down that route.

I'll have a peruse and may come back with more questions but thanks for your help so far :slight_smile:

Ok great - that's another option which may be more attractive than moving my domain to DO. Thanks :slight_smile:

1 Like

Yes, as @webprofusion mentioned, you would only need to switch to their DNS hosting (changing the nameservers). You can keep your domain registration (the billing side of things) unchanged. It's not necessary to achieve your goal here.

Ok thanks - that's helpful as I had thought there was a way to have DNS hosting with a different host from that which the domain is registered, but when I took over the domain I couldn't get it to work, so must have not read up on it properly! I'll have another look at this. I was unsure which records need to be in which host!

Thanks.

Yes, it is quite normal for the domain registrar and the DNS provider to be different. Though most domain registrars don't make it terribly obvious because they want to keep you in their ecosystem. There is usually a per-domain configuration option at the registrar to configure nameservers for the domain. Most have an option for "default nameservers" versus "custom nameservers". Here's a rough outline of the process to move just the DNS somewhere else:

  • Setup your domain/zone on the other provider
  • Copy all the records other than NS and SOA from the old provider to the new provider so you basically have two duplicate copies of the zone.
  • The new provider should tell you what custom nameservers to use with your registrar. They may also have a tool that checks the state of those records and whether they've been changed properly.
  • Change the config at your registrar to use the custom nameservers you got from the new provider.
  • Wait for the changes to propagate.

Done correctly, this should result in zero downtime for your domain. If you need to make record changes while you still have duplicate copies of the zone, just make sure to make them in both places. But it's usually better to try and avoid normal record changes until everything is moved over successfully.

P.S. If you're already on Digital Ocean for the servers, using them for DNS is super easy and also free.

3 Likes

Thanks for this. It's shown me that I need to understand a bit more about how DNS works, rather than basically use trial and error.

Currently at my registrar (Fasthosts) I have NS records pointing to Digital Ocean so that's consistent with your steps.

I also have 2xA records at the registrar, one for the root domain and one for www.(root), pointing to my DO floating IP address. I got to this by trial and error and I don't really know why I need that. I thought if I had the name servers set up to DO the requests would get routed there and the DO servers would do the rest, but I'm obviously missing something.

Any more info received gratefully but I'll make a point about learning how this works properly now :slight_smile:

Normally when you change your nameservers at the registrar, they either delete or remove access to the DNS tools on the registrar side because they are no longer "authoritative" (responsible for) for DNS in that domain. The two A records you mentioned now only need to exist on the DO side. But DO doesn't create them for you because it has no way to know what you plan on using the floating IP for. So you have to create those records. The DO DNS tools are in Networking - Domains.

Instead of creating two A records pointing to the same floating IP, it is also common to create a single A record for the domain root pointing to the IP and use a CNAME record for "www" that points to the domain root. DO's DNS interface uses the @ character to refer to the root of a zone.

This way if you ever change your floating IP, you only have to update 1 record instead of two. But two A records will work exactly the same if you prefer that.

Did you set your nameserver for the domain to digital ocean ones (in fasthosts it's under your domain > Manage Namservers) or did you just create NS records in your fasthosts DNS?

To move DNS to DO for your whole domain you'd need to use their nameservers unless you just want to delegate to them for a subdomain (e.g. example.com could technically have DNS hosted on fasthosts and have subdomain.example.com DNS hosted on DO, but I'm not sure that's what you're trying to do).

Thanks both for recent replies, and apologies for delay in replying.

So now, on Fasthosts, I just have name servers pointing to DO, no other records (although Fasthosts hasn't locked other records down).

On DO I have two A records, one for the root and one for the www subdomain. I like the idea of having an A record and a CNAME record to minimise changes. In fact in a previous incarnation of this configuration I did have that but for some reason I decided it wasn't best practice, so thanks for confirming that is ok. I'll change to that at some point.

One final question (I think). On DO there are three name server records, pointing to DO name servers. I am not sure whether I put those there or whether they were there by default when I set up the domain on DO. I'm not really sure why I need these. I'm assuming that by setting the name servers on Fasthosts to point to the DO name servers, then requests will be routed to DO, so why do I also need to set name servers on DO, pointing to itself?

So can I remove these records?

Thanks again for all help so far.

No. Don't remove those records. Though I'm not sure DO would let you if you tried.

The NS records for a domain always live in at least two places: the DNS provider hosting the domain and the parent domain (often called TLD or Top Level Domain). The TLD for example.com is com and the organization that runs DNS for com needs those NS records so it can tell clients where to find the nameservers for example.com when they ask. Your registrar, Fasthosts, is who provides those records to com.

This is also the reason why hosting my own copy of example.com doesn't work to steal your traffic. I don't have a registrar that will tell com to use my nameservers.

For a more detailed explanation as to why the two copies of the NS records need to exist, check out this ServerFault question:

Thanks! That does make sense.

You pointed the nameservers to DO using Fasthosts - Domain - Nameservers right? Not by adding NS records to Domain - Advanced DNS.

Generally you'd add all available nameservers, because it's free nameserver redundancy that's been given to you.

Yes that’s right. I used Fasthosts name server facility, not advanced DNS.

1 Like

Quick update.

I have now successfully set up SSL on my second server using Certbot with the DNS challenge approach as suggested here. I have flagged @_az's answer as the solution as everything that follows that is clarification of the detail, but still all very helpful.

Thanks for the help on this channel. Some very helpful and patient responses :slight_smile:

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.