Please fill out the fields below so we can help you better. Note: you must provide your domain name to get help. Domain names for issued certificates are all made public in Certificate Transparency logs (e.g. crt.sh | example.com), so withholding your domain name here does not increase secrecy, but only makes it harder for us to provide help.
My domain is: emyjakarta.tech
I ran this command: sudo certbot certonly --expand -d emyjakarta.tech -d www.emyjakarta.tech -d web-01.emyjakarta.tech -d web-02.emyjakarta.tech -d lb-01.emyjakarta.tech
It produced this output:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
How would you like to authenticate with the ACME CA?
1: Nginx Web Server plugin (nginx)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)
Select the appropriate number [1-3] then [enter] (press 'c' to cancel): 1
Plugins selected: Authenticator nginx, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for web-01.emyjakarta.tech
http-01 challenge for web-02.emyjakarta.tech
nginx: [error] invalid PID number "" in "/run/nginx.pid"
Waiting for verification...
Challenge failed for domain web-01.emyjakarta.tech
Challenge failed for domain web-02.emyjakarta.tech
http-01 challenge for web-01.emyjakarta.tech
http-01 challenge for web-02.emyjakarta.tech
Cleaning up challenges
Some challenges have failed.
To fix these errors, please make sure that your domain name was
entered correctly and the DNS A/AAAA record(s) for that domain
contain(s) the right IP address.
My web server is (include version): nginx/1.18.0 (Ubuntu)
The operating system my web server runs on is (include version): Ubuntu 22.04.4 LTS
My hosting provider, if applicable, is: Amazon.com
I can login to a root shell on my machine (yes or no, or I don't know): yes
I'm using a control panel to manage my site (no, or provide the name and version of the control panel): .tech domain
The version of my client is (e.g. output of certbot --version or certbot-auto --version if you're using Certbot): certbot 0.40.0
First, you should update that very old Certbot version. Ubuntu easily supports the recommended snap install. The latest is 2.10
Second, was nginx running before you tried the "expand" command? Because the "invalid PID" can be be caused by Certbot trying to start nginx for you. But, it does not do it in a way compatible with systemd systems. You should restart your server if nginx was not running before the expand command. There are other ways to clean the nginx system up but restarting is easiest if you can tolerate the outage.
I have updated my certbot version. Yet, it fails to expand the existing certificate. My domain name is emyjakarta.tech. i created the following subdomains, web-01.emyjakarta.tech, web-02.emyjakarta.tech, lb-01.emyjakarta.tech, www.emyjakarta.tech. At present, only https://www.emyjakarta.tech is secured. I want to expand my existing certificate.
How would you like to authenticate with the ACME CA?
1: Nginx Web Server plugin (nginx)
2: Runs an HTTP server locally which serves the necessary validation files under
the /.well-known/acme-challenge/ request path. Suitable if there is no HTTP
server already running. HTTP challenge only (wildcards not supported).
(standalone)
3: Saves the necessary validation files to a .well-known/acme-challenge/
directory within the nominated webroot path. A seperate HTTP server must be
running and serving files from the webroot path. HTTP challenge only (wildcards
not supported). (webroot)
Select the appropriate number [1-3] then [enter] (press 'c' to cancel): 1
An RSA certificate named www.emyjakarta.tech already exists. Do you want to
update its key type to ECDSA?
(U)pdate key type/(K)eep existing key type: U
Renewing an existing certificate for emyjakarta.tech and 4 more domains
Hint: The Certificate Authority failed to verify the temporary nginx configuration changes made by Certbot. Ensure the listed domains point to this nginx server and that it is accessible from the internet.
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
ubuntu@527465-lb-01:~$
Your web-01 and web-02 have a different IP than those other names.
The --nginx authenticator only works for domains handled by the nginx system running on the same server as Certbot.
Are these the same server?
dig +noall +answer web-01.emyjakarta.tech
web-01.emyjakarta.tech. 259 IN A 34.224.62.190
dig +noall +answer emyjakarta.tech
emyjakarta.tech. 240 IN A 52.91.135.17
Does it mean that I would have to generate separate SSL certificates for all my servers (individually)? I thought I could use one certificate for all since they belong to the same root domain, emyjakarta.tech.
Because if your LB is terminating SSL from the user-agents (like a browser) then it needs a cert for each of the names it will handle.
The LB makes a separate connection to your "backend" servers. If these are on your same network you may not even need HTTPS and just use HTTP.
But, yes, if you want to use HTTPS between your LB and the backend servers then you must have a cert on each of those servers too. If you get a cert with all the names on the LB you could just copy the needed cert files from there to your "backend" servers. You wouldn't need to setup Certbot on each of them.
If that isn't clear then please explain what you are using as your load balancer (nginx, caddy, ...) and what for your backend. Show an example of your load balancer config that routes traffic to a backend domain.
EDIT: And I repeat, if you want your LB handling all your names the DNS for each name must point to that LB. That isn't where web-01 and web-02 point to right now.
I want my load balancer (lb-01) to point to my root domain, emyjakarta.tech. web-01 would point to a different IP (34.224.62.190) and web-02 will point to another different IP (34.239.253.87).
So, from your explanation, it means that I would generate separate certificates for my web-01 and web-02 servers, right?
Yes, you need to setup a cert for each of those separately on their own servers.
Your LB will need a cert to include the 3 names it does handle. I say 3 names because I assume that lb-01.emyjakarta.tech is your LB? You showed that name in your first post.
Another option is to get a wildcard cert on your LB and then distribute that to your "web" domain name servers. That needs a DNS Challenge which is often more difficult to setup. But might work better overall for you once it is working
That's because your LB is using a cert that has only that one name in it. And, that cert was created a couple days ago.
I see you got a fresh cert today with all 3 names. You now have to configure your LB to use it. Or, maybe you just need to reload/restart your LB to pickup the new cert.
What is Certbot managing on that LB server. Run this there
Certificates don't become "active" on their own; there is no delay period after which the certificate would automatically start to be used.
Certbot and other clients can sometimes install a newly-obtained certificate in a local web server automatically (when you use --nginx Certbot attempts to modify the local nginx configuration to use the new certificate; when you use --apache Certbot attempts to modify the local Apache configuration to use it). If this doesn't happen, or if that server isn't actually the place where the new certificate needs to be used, the new certificate will just sit around not being used! In that case you need to take some kind of action to copy and configure the new certificate and private key into the relevant place where they can be used to secure incoming connections.
If you find a way to do that that can be scripted, Certbot can, for example, run a script specified with the --deploy-hook option whenever a new certificate is obtained. In that case that script can take whatever actions are necessary to deploy the new certificate.
Do you use SSL Termination or SSL Pass-Thru there?
What kind of server handles your base domain name, www, and lb-01? Does that run on the same machine as haproxy?
When I asked these questions earlier I should have insisted you answer them. It is okay if English is not your first language. Just explain as best you can. It is hard to give advice about more complex server configurations without much info.
Thanks a lot for the guide. I have been able to fix the problem.
I had to adjust the path to the certificates and private key in my haproxy.cfg file to reflect the updated path generated by certbot.
It's now working as expected.