Create SSL certificates for CouchDB

Lanhild · March 5, 2022, 3:55am

Hello,
I am using CouchDB and trying to issue a certificate for my domain.
NOTE: my domain isn't being hit by the rate limit, I verified it. (I'll refer to my domain as "the.actual.domain") I'm also using the --test-cert flag, to not create useless production certs.

The issue is that I can't seem to make it work. When I read the CouchDB docs about SSL, I understand that the service has it's own webserver, so it can handle certificates. I tried to generate Let's encrypt certificates for my domain, which I then specified the files path into my CouchDB config (as specified in their docs). Restarted, then tried to go to my machine
I ran this command:

certbot certonly --test-cert --webroot -w /var/www/html --config-dir /opt/couchdb/letsencrypt --logs-dir /opt/couchdb/var/log -d the.actual.domain

Output

Certificate not yet due for renewal

You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /opt/couchdb/letsencrypt/renewal/the.actual.domain.conf)

What would you like to do?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Keep the existing certificate for now
2: Renew & replace the certificate (may be subject to CA rate limits)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Renewing an existing certificate for the.actual.domain

Successfully received certificate.
Certificate is saved at: /opt/couchdb/letsencrypt/live/the.actual.domain/fullchain.pem
Key is saved at:         /opt/couchdb/letsencrypt/live/the.actual.domain/privkey.pem
This certificate expires on 2022-06-03.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

Using lighttpd v1.4.64 as webserver

The server runs Ubuntu 20.04

The server machine is hosted at DigitalOcean

I can login as root.

Using certbot v1.24.0

//

Could someone point out some guidelines on how I can setup an SSL certificates for a CouchDB server?

Any help is appreciated, and don't hesitate to ask for more information.

schoen · March 5, 2022, 4:01am

Hi @Lanhild,

So, after generating a non-test certificate from Let's Encrypt, you did the part they describe like

[ssl]
enable = true
cert_file = /etc/couchdb/cert/couchdb.pem
key_file = /etc/couchdb/cert/privkey.pem

where /etc/couchdb/cert/couchdb.pem is replaced by the Let's Encrypt fullchain.pem and /etc/couchdb/cert/privkey.pem is replaced by the Let's Encrypt privkey.pem? And then restarted or reloaded CouchDB?

And what kind of error or behavior do you see after that when trying to connect?

It seems in the CouchDB documentation that they were envisioning that you use CouchDB as its own web server, not an external one, or did I misunderstand that?

Lanhild · March 5, 2022, 4:18am

Oh my, thanks for the fast reply.
Indeed, I did all the steps as described in their docs (just like you described in your message). I also tried with and without lighttpd, and both methods didn't work.

Without lighttpd: I removed the flag --webroot -w /var/www/html to have this command;

certbot certonly --test-cert --config-dir /opt/couchdb/letsencrypt --logs-dir /opt/couchdb/var/log -d the.actual.domain

The output:

Saving debug log to /opt/couchdb/var/log/letsencrypt.log

How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Spin up a temporary webserver (standalone)
2: Place files in webroot directory (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 1
Certificate not yet due for renewal

You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /opt/couchdb/letsencrypt/renewal/the.actual.domain.conf)

What would you like to do?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Keep the existing certificate for now
2: Renew & replace the certificate (may be subject to CA rate limits)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Renewing an existing certificate for the.actual.domain

Successfully received certificate.
Certificate is saved at: /opt/couchdb/letsencrypt/live/the.actual.domain/fullchain.pem
Key is saved at:         /opt/couchdb/letsencrypt/live/the.actual.domain/privkey.pem
This certificate expires on 2022-06-03.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.

When I try to connect, I get an SSL_ERROR_SYSCALL, without more information.

The thing is, I don't really know what the precise steps are to setup a certificate for CouchDB, whether I'm using it's own web server or another.

P.S
I also tried a self signed certificate, and it works, but that is of no use in a production environment.

_az · March 5, 2022, 4:48am

It could be a permission error?

Looking at the linked documentation, this stands out to me:

chown couchdb privkey.pem couchdb.pem

So, I am guessing that CouchDB doesn't run as root (or run as root and drop privileges, like nginx and Apache do).

You might have to write a --deploy-hook to copy + chown the certificates.

schoen · March 5, 2022, 4:48am

With which client software? Can you also try with curl or openssl s_client?

Did you remove the --test-cert part at least once in order to get a genuine publicly-trusted certificate to configure on your service?

Lanhild · March 5, 2022, 3:25pm

Verified, and CouchDB has all the permissions over the certs files.

I'm uncertain about how to use that --deploy-hook flag, isn't that used for certificates renewal?

Lanhild · March 5, 2022, 4:02pm

The certificate doesn't seem to be the problem here, but maybe my config?

Tried with lighttpd and without (using simply couchdb) and both methods give me the same SSL_ERROR_SYSCALL.

Lanhild · March 5, 2022, 4:22pm

Maybe I am not generating the certificates the proper way?

The last method I tried is generating a Let's Encrypt standalone cert, and as described in CouchDB docs, specify the path to the files in the config. I also verified, and couchdb has full permissions over these files.

The command I used:

certbot certonly --test-cert --config-dir /opt/couchdb/letsencrypt --logs-dir /var/log/couchdb -d the.actual.domain

the output:

How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Spin up a temporary webserver (standalone)
2: Place files in webroot directory (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 1

My config for [ssl] in CouchDB

[ssl]
enable = true
cert_file = /opt/couchdb/letsencrypt/live/the.actual.domain/fullchain.pem
key_file = /opt/couchdb/letsencrypt/live/the.actual.domain/privkey.pem

Lanhild · March 5, 2022, 4:51pm

Managed to make it work by using nginx.
Now I don't have anymore issues with certificate issuance. I just need to find path to my webroot for nginx, as if I go to https://the.actual.domain/_utils, I get a 404. (./_utils being the homepage of CouchDB)

9peppe · March 5, 2022, 4:58pm

You cannot use fullchain.pem it looks like.

You have to use cert.pem and chain.pem separately.

The option you're missing is cacert_file

https://docs.couchdb.org/en/3.2.0/config/http.html#https-ssl-tls-options

Lanhild · March 5, 2022, 5:13pm

The certificate now works. The issue I have is that when I go to my url https://the.actual.domain, I arrive on the normal landing page of CouchDB;

Which describes some details of your installation. Up to there, nothing extraordinary. Now, if I try to go to https://the.actual.domain/_utils (_utils being the dashboard of CouchDB), I land on a 404 from nginx. Which makes me assume that I put the wrong web root path in the nginx config?

My config;

...
root /opt/couchdb/share/www;
...

Contents of /opt/couchdb/share/www;

dashboard.assets    docs    index.html

What should I do to resolve that 404 error?

9peppe · March 5, 2022, 5:17pm

You're making it difficult by not sharing your domain.

Is Nginx behaving like a reverse proxy? With what config?

Are you sure you don't want to configure couchdb to use the certificate directly? Try with this config:

[ssl]
enable = true
cert_file = /opt/couchdb/letsencrypt/live/the.actual.domain/cert.pem
cacert_file = /opt/couchdb/letsencrypt/live/the.actual.domain/chain.pem
key_file = /opt/couchdb/letsencrypt/live/the.actual.domain/privkey.pem

Lanhild · March 5, 2022, 5:28pm

When directly using CouchDB to use the certificate, it simply doesn't work. (gives the SSL_ERROR_SYSCALL)

My nginx config:

server {
        listen 6984 ssl default_server;
        listen [::]:6984 ssl default_server;

        root /opt/couchdb/share/www;

        index index.html index.htm index.nginx-debian.html;

        server_name the.actual.domain;

        location / {
                proxy_pass http://localhost:5984;
                proxy_redirect off;
                proxy_set_header Host $host;
                proxy_buffering off;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Ssl on;
                try_files $uri $uri/ =404;
        }
}

ssl_certificate /opt/couchdb/letsencrypt/live/the.actual.domain/fullchain.pem;
ssl_certificate_key /opt/couchdb/letsencrypt/live/the.actual.domain/privkey.pem;
ssl_protocols SSLv3;
ssl_session_cache shared:SSL:1m;

port 6984 being the HTTPS port for CouchDB. I tried with 443 as well in the nginx config but it doesn't change anything.

I just don't know why this 404 is happening. Any clues about the value I should put as the web root path?

9peppe · March 5, 2022, 5:45pm

What the f*cking f*ck. This probably has something to do with it.

Use

    ssl_protocols TLSv1.2 TLSv1.3;

or

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;

if you really have to.

9peppe · March 5, 2022, 5:48pm

None, you should not have a root there. You already have a proxy_pass directive, I am not even sure about the try_files directive.

My similar config looks like this:

upstream lampone {
        server 10.9.1.5 max_fails=0; # wireguard
        server 10.9.0.5 max_fails=0 backup; # tinc
}

server {
        server_name lampone.example.com;
        include snippets/listen;
        include snippets/ssl_conf;
        include snippets/ssl_cert_lampone;
        location / {
                root /var/www/html;
                index index.html index.htm index.nginx-debian.html;
                try_files $uri $uri/ @lampone; # <- this is location lampone
        }
        location @lampone {
                proxy_pass http://lampone; # <- this is upstream lampone
                include snippets/proxy_conf;
        }
}

Lanhild · March 5, 2022, 5:50pm

Lol. Sorry if I look like a noob, but it's because I'm pretty new to issuing certs.

I modified to

ssl_protocols TLSv1.2 TLSv1.3;

as you said.

9peppe · March 5, 2022, 5:58pm

I really think you should try using the certs in couchdb directly, as I said before. Unless you have some reason to use nginx in front of it, that is.

Giuseppe C.:

Try with this config:

[ssl]
enable = true
cert_file = /opt/couchdb/letsencrypt/live/the.actual.domain/cert.pem
cacert_file = /opt/couchdb/letsencrypt/live/the.actual.domain/chain.pem
key_file = /opt/couchdb/letsencrypt/live/the.actual.domain/privkey.pem

Lanhild · March 5, 2022, 9:36pm

Still get the same SSL_ERROR_SYSCALL, even when using the certs directly in the CouchDB config.

9peppe · March 5, 2022, 9:46pm

You should really give us the actual domain name.

_az · March 5, 2022, 10:30pm

Are you sure? How did you achieve this?

When I just tried to install CouchDB and point it to my Certbot certificate, it fails when I try and connect on the SSL port. Tracing what CouchDB is doing, the issue is indeed due to permissions:

root@british-guest:~# strace -e trace=openat -ff -p $(pidof beam.smp) 2>&1
strace: Process 4002 attached with 35 threads
[pid  4065] openat(AT_FDCWD, "/etc/letsencrypt/live/british-guest.bnr.la/fullchain.pem", O_RDONLY) = -1 EACCES (Permission denied)
[pid  4065] openat(AT_FDCWD, "/etc/letsencrypt/live/british-guest.bnr.la/fullchain.pem", O_RDONLY) = -1 EACCES (Permission denied)

and I got the same error as you when trying to access CouchDB's SSL port in a browser.

Then as soon as I used ACLs to grant access:

root@british-guest:~# setfacl -Rdm u:couchdb:rx /etc/letsencrypt/{live,archive}
root@british-guest:~# systemctl restart couchdb

everything started working.

It's both for renewal and for the initial certificate issuance. You can point to a shell script which could perform various tasks such as:

Copying the certificate files to /opt/couchdb/etc
Giving the certificate files appropriate permissions
Reloading/restarting CouchDB (if necessary to reload the certificate)

Topic		Replies	Views
CouchDB not working over https after certificate renewal Help	2	1070	October 3, 2019
Help me I have reached certs creation rate :( Server	4	984	March 9, 2018
Automate SSL cert generation and renewal using REST API Help	10	3693	June 16, 2021
Adding an SAN in Ubuntu - newbie, apologies in advance Help	23	3589	July 26, 2016
So i accedently "reached my limit" some how Server	13	2644	February 27, 2016

Create SSL certificates for CouchDB

Related topics