@Touma, the Let’s Encrypt client currently defaults to saving PEM files.
All of the file formats that various server software might want for certificate data are just different representations of the same data. You can convert between them with the openssl command in a single step.