I’ll first show you my error and then describe what I performed for setup.
When running this command to get a new certificate:
user$ sudo certbot certonly -a manual -d gitlab.k8sbox.io --email jdoe@example.com
I’m getting this error:
Challenge failed for domain gitlab.k8sbox.io
http-01 challenge for gitlab.k8sbox.io
Cleaning up challenges
Some challenges have failed.
IMPORTANT NOTES:
- The following errors were reported by the server:
Domain: gitlab.k8sbox.io
Type: connection
Detail: Fetching
http://gitlab.k8sbox.io/.well-known/acme-challenge/j3jALHY9pxJTuDhKOfYZI9QUouQop8LlA1ifly6k6bo:
Timeout during connect (likely firewall problem)
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address. Additionally, please check that
your computer has a publicly routable IP address and that no
firewalls are preventing the server from communicating with the
client. If you're using the webroot plugin, you should also verify
that you are serving files from the webroot path you provided.
THE SETUP:
The webserver is implemented by GitLab software (not by me), and runs nginx. (Forgive me that I’m not a web administrator).
The webserver is in my house behind a Cable-Modem-Router. I have a static WAN IP-Address (i.e. my ISP provider never changes it).
DNS A-Records exist for both gitlab.k8sbox.io and www.gitlab.k8sbox.io, both of which point to said ISP static WAN IP-Address. Note: The GitLab web service will likely be down if you click the above. (I shut it down when not in use).
Next, within my Cable-Modem-Router, I have ports 80 and 443 port-forwarded to the home webserver (… again, that’s what runs GitLab / nginx). That home webserver, which runs Fedora-30, does not have firewall packages installed or enabled.
Following the interactive certbot(1) CLI instructions, I created the following on the home (GitLab / nginx) webserver before pressing enter:
And finally, using Firefox, Opera and Chrome, I can successfully browse to http://gitlab.k8sbox.io/.well-known/acme-challenge/j3jALHY9pxJTuDhKOfYZI9QUouQop8LlA1ifly6k6bo/index.html and see the ciphertext content I placed in it (and without logging into GitLab).
Did I miss something? Why could not certbot(1) complete the transaction?
This bit isn't working for me. I can't open a TCP connection on either 80 or 443 to your current IP.
The error from Let's Encrypt also reflects that.
Does your ISP block any ports? It seems that there's some reports that they do: rcn blocked ports at DuckDuckGo . In that case, it makes sense that you can access your own services from inside your LAN, but external visitors cannot.
Hello. I actually disabled port-forwarding after I posted this, so this is why you can’t get to it (sorry). I was, however, able to telnet gitlab.k8sbox.io 80 | 443 before turning it down. I should have mentioned that. For good measure, I’ll try again in a few moments. You can, too in a about a minute.
Try again now. I just re-enabled port-forwarding. Sorry. I haven’t locked things down (security wise), so I’m a little paranoid.
That said, maybe because I’m on the RCN network I can somehow get to it. I just tried from an outside cloud server and I can’t get to port 80; but can get to port 443. Can you confirm the same? Thank you!
Unfortunately it competes with nginx for port 443, so the above command stops nginx momentarily as the certificate is retrieved. There’s some ways to do this without downtime but they’re a bit complicated.
Qusetion: Do I need to provide the GitLab / nginxdocument-root to acme.sh; for example /var/opt/gitlab/nginx/www/.well-known/acme-challenge/ or even the entire path created for certbot?
Guys, thank you so much not only for the super rapidity in replying, but for debugging, for offering solutions and for writing answers that were so clear to follow (which makes a difference). I really appreciate it!
Using @_azacme.sh solution, I was able to generate certificates manually:
Now I have to determine what (from the manually generated above) and where to copy into GitLab’s, to get it to work with HTTPS. I suspect it’s here (below), but that’s a just stab-in-the-dark because it looks correct LoL:
user$ find /etc/gitlab/ssl/
/etc/gitlab/ssl/
/etc/gitlab/ssl/gitlab.k8sbox.io.crt <------- Here and
/etc/gitlab/ssl/gitlab.k8sbox.io.key <------- Here
/etc/gitlab/ssl/gitlab.k8sbox.io.key-staging
It will do that every time the certificate renews.
As long as you used the --[pre,post]-hook that I suggested in the original command, all of this should be totally automatic at every renewal (and assuming you don’t unforward your ports).
(Now that I check more carefully, I’m not sure whether Gitlab’s nginx can be controlled in that way exactly - depends how you installed it. But you probably get the idea).
BTW, I don’t mind stopping and starting GitLab when renewing certifications for now.
So cobbling everything together, can I repeatedly run acme.sh with the below options (noticing that I replaced --issue with --renew-all)? Meaning, after the original --issue, can I run the below for both subsequent cases – when the certifications haven’t yet expired as well as when they have? (One command for both cases – simpler).
Also, might there be a way to specify the notification eMail address (and preferably two of them) on the below command line? From what I’ve read (i.e. it’s manpage), the command wants to consult ACCOUNT_EMAIL= in ~/.acme.sh/account.conf, and even then I believe it only accepts one address; though I’m not sure about that last part.
# NOTE: This is run as root because it needs port 443.
#
DOMAIN='gitlab.k8sbox.io'
/home/user/.acme.sh/acme.sh \
--renew-all \ # --issue
--install-cert \
--alpn \
--domain ${DOMAIN} \
--key-file /etc/gitlab/ssl/${DOMAIN}.key \
--fullchain-file /etc/gitlab/ssl/${DOMAIN}.crt \
--webroot /tmp/www/
BTW @_az You caught a bug. I was copying to the destination as .cer not .crt (would have taken me forever to catch).
acme.sh deals with renewal automatically - it saves your --issue and --install-cert commands for later.
When you first installed acme.sh, it setup an automatic cron job on your server (check crontab -l of your root user).
The other issue is that you cannot combine --issue and --install-cert like that. They must be two separate commands, as they do different things.
So, imagining that you had a totally fresh Gitlab environment, you would:
Install acme.sh
Run the acme.sh --issue ... command with all the correct hooks to stop and start nginx.
Run the acme.sh --install-cert ... command.
Walk away. There’s no need to do anything else, acme.sh’s cronjob will deal with renewal for you (that’s the idea in --pre-hook, --post-hook, --reloadcmd - to be totally non-interactive and automated).
That’s what I thought (about the multiple eMails).
I get the end-to-end picture now (thank you). I will wipe my ~/.acme.sh/ directory and cron(1) job, and perform a fresh install as root (adding my eMail address along the way); because I initially did this as my own user, but root is needed for port 443. No big deal.
Then I’ll perform the steps you mentioned in sequence and call it a day – letting cron(1) do the rest.
Side Note: The reason I crafted a manual command above is because this server is actually a LXC O/S container nested within a bare-metal server (one of many LXC O/S containers), and won’t always be on. This is part of a larger setup in my personal R&D Lab. So I’ll likely have to manually renew the certificates anyway, but I can do that by simply running the command installed into cron(1) – as you mentioned.
I got so much accomplished today with this thread and your guidance. I really appreciate it. Thank you very much!
