Hello,
i could not find any manual way to create an own certificate and use Let’s Encrypt as authority to certify my public key i have generated?
There are only this tools like certbot that will do “anything automatic”?
I am feeling to loose understanding and control what is happening in the background.
But it is not so complicated to create a self signed certificate.
Why i must use a tool for let’s encrypt to use this authority?
I’s sorry, I don’t understand fully. What do you want to achieve ?
(do you want to get a certificate for a web site ? if so, on what OS? what webserver ? )
You won’t get around reading the specs then. To read actual source code, the simplest client is possibly https://github.com/diafygi/acme-tiny/blob/master/acme_tiny.py
You must use a tool because the API is a lot of JSON, signing and base64 encoding. To recreate that “manually” is basically writing a new client.
I want to understand the process getting the certificate.
Thanks for this Tip.
Where can i find the specs?
I am not a Python programmer, so it is inconvenient to reengineer the code.
This description is already helpful but not detailled:
https://letsencrypt.org/how-it-works/
I’m not sure if you are a novice user, who just wants to get a certificate for their site they host on a shared server somewhere, of if you want to develop a client to obtain a certificate …
If you want detailed information on the process ( i.e. if you were developing your own client ) then I’d suggest reading the integration guide and the draft ACME spec
I'm a novice user, who just wants to use certificates on my own servers and want to understand what is happening in detail.
Up to now i was satisfied to use self sign certificates.
Specially for an Intranet this is useful, because no internet interaction is needed to check the validity of a certificate.
But now i run into an problem trying to run an XMPP server with secure connection:
https://developer.pidgin.im/ticket/17148
Obviously i am forced to use certificates that are not self signed!
So i have to understand what is going wrong here ...
(It seems that both the developer of the server and of the clients can't tell me why the handshake fails 
Maybe you as experts of certificates can give a hint?
That link seems to link to 3 other links and various issues you are having. I got a little confused following the link though as to what was the current issue.
Let me try a summary of how SSL (TLS ) certs work - forgive me if I’m saying things you already know.
After building a TCP connection, the SSL handshake is started by the client. The client (which can be a browser as well as any other program such as XMPP client in your case ) sends a number of specifications:
The server checks what the highest SSL/TLS version is that is supported by them both, picks a ciphersuite from one of the client’s options (if it supports one), and optionally picks a compression method.
After this the basic setup is done, the server sends its certificate. This certificate must be trusted by either the client itself or a party that the client trusts. For example if the client trusts Let’s Encrypt, then the client can trust the certificate from your domain, because Let’s Encrypt cryptographically signed your certificate.
If you use a “self signed” certificate, or one that hasn’t been signed by someone you trust, then the software will give a warning and not connect ( or ask you if you really want to trust this connection ).
Similarly, if the client and server can not find a cipher or compression method that they both agree on, the connection will not complete.
On one of the threads you were asking why a “self signed” cert wouldn’t work - hopefully the above has partially answered that. You can tell your xmpp client to trust it, and then it should work.
For an Intranet site, self signed certificates are fine, as you can 100% trust the person who signed them (you) - hence is usually the best option. I’m not sure why you say you are “forced to use certificates that are not self-signed”. Is this because you have external users connecting that don’t know to trust your certificate ?
serverco - thank you very much for your explanations!
[quote="serverco, post:9, topic:25763, full:true"]
That link seems to link to 3 other links and various issues you are having. I got a little confused following the link though as to what was the current issue.[/quote]
Yes - it's confusing, because i need some time to understand what is going wrong.
First i thought that the connection is encrypted, because the configuration of jabber2 was set to accept only encrypted connections.
(verify-mode 7 => Use 7 to require all clients to present valid certificates and
require-starttls 1 => If this attribute is set to any value, clients must do STARTTLS before they can authenticate.)
But then i tracked a session with wireshark and find out that there is no encryption!
It's only plain XML communication!
So i altered the configuration of jabber2 again.
I could test the connection with the following result:
$ openssl s_client -connect mydomain.de:5222 -starttls xmpp
CONNECTED(00000003)
depth=0 C = DE, ST = DE, L = nowhere, O = exterretorial, OU = myname, CN = mydomain.de, emailAddress = webmaster@mydomain.de
verify error:num=18:self signed certificate
verify return:1
depth=0 C = DE, ST = DE, L = nowhere, O = exterretorial, OU = myname, CN = mydomain.de, emailAddress = webmaster@mydomain.de
verify return:1
139704603223696:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:
---
Certificate chain
0 s:/C=DE/ST=DE/L=nowhere/O=exterretorial/OU=myname/CN=mydomain.de/emailAddress=webmaster@mydomain.de
i:/C=DE/ST=DE/L=nowhere/O=exterretorial/OU=myname/CN=mydomain.de/emailAddress=webmaster@mydomain.de
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIC/DCCAmWgAwIBAgIJAO5IX1p/l9w8MA0GCSqGSIb3DQEBCwUAMIGWMQswCQYD
...
...
0lEL+r6KclBbHE3escH0aFCz2j7KP4CUVAN3hBdHyTzXQd+m0sJXdtBG5BvFLuhe
-----END CERTIFICATE-----
subject=/C=DE/ST=DE/L=nowhere/O=exterretorial/OU=myname/CN=mydomain.de/emailAddress=webmaster@mydomain.de
issuer=/C=DE/ST=DE/L=nowhere/O=exterretorial/OU=myname/CN=mydomain.de/emailAddress=webmaster@mydomain.de
---
No client certificate CA names sent
---
SSL handshake has read 1698 bytes and written 138 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID:
Session-ID-ctx:
Master-Key: BC713753C8B298823E413B65A681450FA207765A9A7FA367E8EDCC61B1F1D81FAFA3A5A78EFF15A27563ABCDF3F8BA93
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1484665718
Timeout : 300 (sec)
Verify return code: 18 (self signed certificate)
---
Is this O.K. in your eyes?
But now no client could connect anymore. I tested Pidgin and PSI.
That was the point i opened the bug at Pidgin.
You can see in the debug output that the handshake fails:
(19:17:04) jabber: Sending (name@domain.de/PC): <starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'/>
(19:17:04) jabber: Recv (50): <proceed xmlns='urn:ietf:params:xml:ns:xmpp-tls'/>
(19:17:04) nss: Handshake failed (-5938)
(19:17:04) connection: Connection error on 0x7f07f0c20280 (reason: 5 description: SSL-Verhandlung fehlgeschlagen)
(19:17:04) account: Disconnecting account name@domain.de/PC (0x7f07efec7a00)
(19:17:04) connection: Disconnecting connection 0x7f07f0c20280
(19:17:04) connection: Destroying connection 0x7f07f0c20280
[quote="serverco, post:9, topic:25763, full:true"]
After building a TCP connection, the SSL handshake is started by the client. The client (which can be a browser as well as any other program such as XMPP client in your case ) sends a number of specifications:
which version of SSL/TLS it is running,
what ciphersuites it wants to use, and
what compression methods it wants to use.[/quote]
This seems to fail in Pidgin.
But the debug output don't say why?
My guess is that something is missing in the handshake or that i have to made the certificate in a different way?
Or a selfsigned certificate is simply not accepted any more.
That's still no problem for apache or exim so far.
I already tested to put the certificate in the local folder
/home//.purple/certificates/x509/tls_peers
without success.
Pidgin or PSI is not asking if i want to trust the certificate.
It's because i have here now some clients that obviously won't accept this type of certificates.
Of course i will try to use let's encrypt, but it would be interesting to understand why it is not working now.
For an internal server i want to be independent from internet.
It depends what you want to do.
All you have checked there, is that the certificate is provided - and yes, a self signed certificate s provided. So if that's what you wanted to show, then it looks OK.
There could be several reasons your clients cant' connect, the most common are 1) they don't trust a self signed cert. 2) they can't agree a cipher they can both use.
is your xmpp connection available on the internet to test ? or is it completely internal ?
I want to encrypt a connection to get privacy.
So i must do what the software needs to get this result.
In this case i don't really understand what the software needs to work?
When i ask the developers of jabber2 or Pidgin i get no answer for this questions.
But maybe the answer is to understand the mechanism and debug what is going wrong?
Then i can help to solve the problem.
Sorry - yesterday i did not find the time to answer to this point.
Yes - this can be tested online.
It seems not possible to send a private message here - how can i send you details for an account?
I have an additional generally question here:
In the configuration of jabber2 there is an option that is not explained:
cachain='/etc/jabberd2/client_ca_certs.pem
What are Client CA certificates and how is it possible to create them?
In the test above a certificate chain can be seen:
Certificate chain
0 s:/C=DE/ST=DE/L=nowhere/O=exterretorial/OU=myname/CN=mydomain.de/emailAddress=webmaster@mydomain.de
i:/C=DE/ST=DE/L=nowhere/O=exterretorial/OU=myname/CN=mydomain.de/emailAddress=webmaster@mydomain.de
How can this be interpreted?
Is it possible to name my own certificate instance in an Intranet?
You should be able to PM by clicking on a persons name, then “message”. I’ll send you a PM.
The “Client CA cert” or the “chain cert” are certificates provided by the CA (Certificate Authority ) as part of the chain of trust
It is possible for you to be your own “CA”, yes - as long as you provide your “chain certs” to the people who want to to to trust you ( fine for an Intranet, but not generally accepted on the Internet of course )
The question would be who the users of this service will be. If they're all people you know and you can get them to accept your self-issued certificate, that's great (and in some ways safer than using a public CA). This is often the case within an organization or sometimes a social group. You can also do something similar when writing specialized client software for a service, because the software can include the expected cryptographic keys or certificates for that service.
But if people you don't know are going to be using the service and not using your software to access it, or if you can't easily get your users to change their software or settings, you normally need a certificate from a public CA which is trusted by software that they already have (like a web browser). Let's Encrypt can help with this case.
Yes - this is what i want to do primarily now.
I agree to use public CA when i want to drive a public server.
It's of course the better way.
So i am discussing a special case here.
And it is very special, because the CA problem is specific to the clients of an XMPP server.
Besides - there was a little reaction for my ticket at pidgin: https://developer.pidgin.im/ticket/17148
As i have written there is no problem runing an apache or exim server with self signed certificates.
Proudly i can tell you that i have produced a lets encrypt certificate manually now.
I have done it following the description at https://github.com/diafygi/acme-tiny
All is working perfect - the goal of this thread is reached!
Now i have all this files:
-rw-r--r-- 1 root root 3,2K Jan 20 12:52 account.key
-rw-r--r-- 1 root root 9,0K Jan 20 12:51 acme_tiny.py
-rw-r--r-- 1 root root 3,8K Jan 20 12:57 chained.pem
-rw-r--r-- 1 root root 772 Jan 20 12:50 conv.py
-rw-r--r-- 1 root root 1,7K Jan 20 12:52 domain.csr
-rw------- 1 root root 3,2K Jan 20 12:52 domain.key
-rw-r--r-- 1 root root 1,7K Jan 20 12:56 intermediate.pem
-rw-r--r-- 1 root root 2,2K Jan 20 12:53 signed.crt
It was a little surprising that i must have a running webserver to create the signed.crt!
What’s about you want to have a certificate for an email server only? 
The clear process is to generate two RSA keys (domain.key + account.key) and the account.key is certified by let’s encrypt. The result is signed.crt
O.K. Now i tried to use this certificate in jabber2.
But this still does not work!
I tried many different combinations to give the different keys and certs to jabber2.
The server always started, but there is still no TLS encryption.
But this is a side aspect.
Of course it would be fine to understand what keys or files are needed for the TLS encryption.
If you don't have, and don't want to have, a web server running even for the 30 seconds or so that each issuance/renewal would take, you can use the DNS challenge instead.
What's the DNS challenge?
That's new for me.
The DNS challenge allows you to validate control over the hostname in question by creating DNS records, rather than by serving up particular files over HTTP. It's available in most of the clients (including certbot), but really only a practical option if your DNS host has some kind of API that would allow automated changes to your DNS records.