I'm working to get automated renewals working on my EC2 instances. These instances are created by AWS CloudFormation (setup and teardown is automated/scripted such that scaling works). I'm running on Ubuntu 18.04.3 LTS, and my setup script looks something like this:
I know some of this is repetitive, but this was the result of lots of trial and error (months back) getting certbot to work with the DNS Route 53 renewal plugin.
Anyways, the certbot renew command works fine, so I can ssh into my hosts and manually renew. However, the systemctl timer looks for /usr/bin/certbot, which doesn't exist. The install above places certbot in /usr/local/bin/certbot (and /usr/local/bin is also in PATH, so it's unnecessary in general).
The quick fix would be to ln -s a symbolic link in my setup scripts to point /usr/bin/certbot to /usr/local/bin/certbot. However, I'm wondering if this is safe going forward, or if there's some combination of install steps that I'm messing up that leaves the systemctl timer referencing certbot in the wrong place.
Why would you install certbot through apt and through pip? That's asking for trouble I think.
The apt version of certbot should install /usr/bin/certbot and the systemd timer. I think it's the pip version of certbot ending up in /usr/local/bin and pip doesn't, as far as I know, have a systemd timer. So those two things bite each other.
Good question, and the answer is that this as the result of lots of trial and error in getting certbot/certbot-dns-route53/nginx installed together. The details are a little fuzzy because this was a while back, but pip is required for installing certbot-dns-route53 and the pip install of certbot was required to resolve the same issues encountered with this bug.
I don't actually remember why the apt-get install was also required, but maybe it was to get this to work with nginx.
I'm open to trying a different sequence of steps if you have a recommendation, and I can report back with the result.
I don't think I can help you much, as I could never figure out myself how to get both aws-cli and certbot's route53 plugin both working on the same EC2 system. (I was using Amazon Linux 2, though, not Ubuntu.) I think the approach needs to be to put each of them in separate python virtual environments, somehow. But all I tried ended up as a Python Superfund Site, so I'm following this thread closely hoping you have an answer I could point other people to. What I ended up doing eventually was just getting certificates through Lambda and then having them copied (along with the private key) to the server.
You also need to be wary of acquiring certificates in your server-creation script. I'm not completely sure if that's what you're doing, but if you are, then if you end up starting a lot of servers at once you run into rate limits, and if Let's Encrypt is down for whatever reason then you can't create new servers. I think the best practice in this sort of servers-as-cattle environment where servers are created and destroyed as needed is to have the certification creation in one place (doesn't need to be Lambda; that's just what I did) and then copied to the servers as they get created. (And of course also distributed to all existing servers upon certificate renewal.)
great points. I'm oversimplifying the server spin up process a bit only because I think my setup scripts mentioned above are the source of the problem. That said, the above install sequence works (and has been working since Feb). I actually have it updating CloudFront certs using the CLI as well.
That said, I didn't discover that auto-renewals weren't working until it actually came time to renew. I think a symbolic link will resolve the problem, but I reached out mainly in homes that there's a better way to install available at this point.
What I ended up doing eventually was just getting certificates 
So at most it works partially.