Great, that's really useful to be clear about! Given that, it's super important to have a good plan for how you will update these in the field. That's a bigger problem than certificates and is particularly hard for small devices, where there's often not enough storage space to hold a full copy of the OS. But it's necessary nonetheless, since bugs and vulnerabilities inevitably crop up post-deployment.
Once you've got that figured out, I recommend making your own CA to issue certificates for the server. You can generate one very easily with GitHub - jsha/minica: minica is a small, simple CA intended for use in situations where the CA operator also operates each host where a certificate will be used.. You load the resulting minica.pem (the root certificate aka root certificate) onto the devices, and keep a copy for yourself to generate future certificates.
The reason for making your own CA: public CAs (like Let's Encrypt) are subject to various rules that may require us to revoke your certificates for things like key compromise or problems with the certificates themselves. If that caused your devices to no longer be able to phone home, it would be a problem for you - and one we wouldn't be able to help you get out of. By having your own CA, just for your one server, you control your own destiny.