I don’t see what kind of security I have, if I’m not creating the public and private keypair myself?
If I’m not generating the keypair myself, and exposing only the public key to be signed with a certificate - the obligation to keep the private key actually private is not entirely my responsibility in this case. Why do I have to use a 3rd party program on a computer that’s not air gapped to generate the keys to get the public key certified - especially a program I don’t have the source to?
You’re creating a certificate request (CSR), that is signed by your private key (which you already generated on your computetr). So far, no external network needed.
You then send the CSR to the CA (here you need external network), and the CA then signs the CSR, which generates a certificate.
The certificate attests that your public key is trusted for the subject information that appears on the certificate (in the case of Let’s Encrypt, domain names).
Popular clients generate the keys for you, but you’re not obliged to use any of them if you don’t trust them.
The Certbot client software is quite happy to do all the rest of the work after being given a CSR (Certificate Signing Request) that you’ve made, perhaps on an air gapped computer, and even (if you have far too much time on your hands) calculated by hand with pencil and paper. For most users that’s a bunch of extra hassle to gain security they don’t understand, but you’re very welcome to this option.
Also you absolutely can get the source code to the Certbot, https://acme.sh/ and many other popular clients, in the case of Certbot the source is the only code, it’s written in ordinary Python, ready to run, likewise acme.sh is a shell script. If you’re using a third party client for which source code isn’t available, you’d have to take that up with the people who made your client.