Observation and Question about "--must-staple" and nginx's "ssl_trusted_certificate" r.h.s. value

Server
1

Using "certbot-auto" I issued a new certificate specifying the "--must staple" flag.

sudo /etc/certbot-auto/certbot-auto certonly --authenticator webroot --installer nginx --must-staple --staple-ocsp -d ...

As part of the process the following nginx directive r.h.s.'s in the nginx configuration files were automatically updated:

ssl_certificate /etc/letsencrypt/live/go-oio.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/go-oio.com/privkey.pem; # managed by Certbot

However the following nginx r.h.s.'s value was not updated:

ssl_trusted certificate (old r.h.s. from a non-letsencrypt certificate)

Nevertheless, even with the (old r.h.s. from a non-letsencrypt certificate) value,
the website still functioned.

This letsencrypt comment by jsha recommends setting the r.h.s. value of "ssl_trusted certificate" to the same value as the r.h.s. value of ssl_certificate. I followed that jsha recommendation and of it course it still worked.

Then I removed the "ssl_trusted certificate" entirely:

# ssl_trusted certificate ... (commented out)

restarted nginx

sudo service nginx restart

and performed the qualys ssl test from a browser:

SSL Server Test (Powered by Qualys SSL Labs)

Even without the "ssl_trusted certificate" directive, the results showed
that both "OCSP must staple" and "OCSP stapling" were enabled, and the score was still A+.

This result doesn't surprise me, because jsha's comment mentions that

"Also bear in mind that Nginx lazy-loads OCSP responses. So the first request will not have a stapled response, but subsequent requests will."

which I interpret as meaning that on restart (or reload) nginx queries for an up-to-date OCSP response, and caches that value in memory, at least for some period of time. And all that is required to make a normal OCSP query is the certificate chain given as the r.h.s. value of "ssl_certificate". So it would seem the value of "ssl_trusted_certificate" is redundant.

My questions are:

  • is the directive "ssl_trusted certificate" now deprecated?
  • are there some cases (perhaps for other CA's than letsencrypt) where "ssl_trusted certificate" is still necessary?
  • was there a historical reason for the directive "ssl_trusted certificate", such as holding a text cache of the actual "OCSP response", which is now no longer used?
  • assuming nginx caches the OCSP response in memory, is there a nginx directive specifying the max time until a new response is obtained to refresh the cache?

Note: X.509 Internet Public Key Infrastructure specification for contents of OCSP response section 2.2.

nginx -V
nginx version: nginx/1.10.3 (Ubuntu)
built with OpenSSL 1.0.2g 1 Mar 2016

2

The ssl_trusted_certificate directive is not deprecated, you can read the docs at:
http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_trusted_certificate

Most likely you have ssl_stapling_verify off (see above docs page), which would allow things to work without ssl_trusted_certificate. I would recommend enabling both to avoid serving untrusted bytes to your visitors.

1 Like
3

Hi,

The answer is no, it's still functioning.
Heres the definition of this directive.

Specifies a file with trusted CA certificates in the PEM format used to verify client certificates and OCSP responses if ssl_stapling is enabled.

In contrast to the certificate set by ssl_client_certificate, the list of these certificates will not be sent to clients.
From:
Module ngx_http_ssl_module

Its not necessary, only if you enabled ocsp staple.

Only chrome doesn't actually check OCSP. Firefox still checks (which will reduce ocsp response time)

You can enabled expire time by specifying in resolver

resolver 8.8.8.8 8.8.4.4 valid=300s;

From https://raymii.org/s/tutorials/OCSP_Stapling_on_nginx.html

Thanks

1 Like
4

It does when stapled, I assume.

1 Like
5

I think actually it doesn’t. In theory it might some day do so for Must Staple certs, but AFAIK Chrome doesn’t yet implement Must Staple.

However, Firefox does check OCSP for stapled and non stapled handshakes.

1 Like
6

Not exactly the same value. The recommendation for Let's Encrypt OCSP stapling is to set:

ssl_certificate /etc/letsencrypt/live/go-oio.com/fullchain.pem;
ssl_trusted_certificate /etc/letsencrypt/live/go-oio.com/chain.pem;

fullchain.pem contains a superset of the information in chain.pem.

1 Like
7

ssl_stapling_verify is set to on. Complete setup, with ssl_trusted_certificate omitted, is below.

I guess that ssl_trusted_certificate is only required and referenced if NGINX can not find the top level certificate. Decoding the intermediate certificate (there is only one) shows:

sudo openssl x509 -in /etc/letsencrypt/live/go-oio.com/chain.pem -text -noout
...
Issuer: O=Digital Signature Trust Co., CN=DST Root CA X3
...

Checking in the cert directory:

ls /etc/ssl/certs | grep DST
DST_ACES_CA_X6.pem
DST_Root_CA_X3.pem

it is there - DST_Root_CA_X3.pem - the required top level certificate. It is reasonable to assume that NGINX knows to look in /etc/ssl/certs on my setup. The only alternative would be that it was failing to verify which is unlikely.

Another question is whether certbot-auto informed NGINX where to find the top level certificate as part of the certificate installation. Apparently it did not do so via ssl_trusted_certificate, but maybe there are other ways.

EDIT: After reading mnordhoff's reply, I recalled that in general practice OCSP is only used between the bottom and the first intermediate. If that is also the case the action performed when "ssl_stapling_verify on" is set, then a simpler explanation is that NGINX found the first intermediate as the second certificate of _fullchain.pem", assumed that was trusted, and didn't need to look any further.

--- complete setup ---

Here is the complete ssl configuration used:

    ssl on;
    ssl_certificate /etc/letsencrypt/live/go-oio.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/go-oio.com/privkey.pem; # managed by Certbot

    #ssl_trusted_certificate /etc/letsencrypt/live/go-oio.com/fullchain.pem; 

    ssl_stapling on;
    ssl_stapling_verify on;

    resolver_timeout 10s;

    ssl_session_cache shared:SSL:32m;
    ssl_buffer_size 8k;
    ssl_session_timeout 180m;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
    ssl_prefer_server_ciphers on;
    ssl_dhparam /etc/nginx/ssl/dhparam.pem;

    add_header Strict-Transport-Security max-age=15768000;

As you can see, ssl_stapling_verify is on

and ssl_trusted_certificate is comment out with a #

Of course I did not forget

sudo service nginx restart

before running the ssl test from a browser on a different network from the VPS:

which reported:

OCSP Must Staple Supported
OCSP stapling Yes

Path #1: Trusted
1 Sent by server go-oio.com
Fingerprint SHA256: 0e2c273d1dc3f9661dfe7739f5550ccc372013ce6c39b620481199db4ab2bf20
Pin SHA256: Yb8lO2XNnya2Raups1RGpfYX0QfcD/DZogANo4ACcn4=
RSA 2048 bits (e 65537) / SHA256withRSA

2 Sent by server Let's Encrypt Authority X3
Fingerprint SHA256: 25847d668eb4f04fdd40b12b6b0740c567da7d024308eb6c2c96fe41d9de218d
Pin SHA256: YLh1dUR9y6Kja30RrAn7JKnbQG/uEtLMkBgFF2Fuihg=
RSA 2048 bits (e 65537) / SHA256withRSA

3 In trust store DST Root CA X3 Self-signed
Fingerprint SHA256: 0687260331a72403d909f105e69bcf0d32e1bd2493ffc6d9206d11bcd6770739
Pin SHA256: Vjs8r4z+80wjNcr1YKepWQboSIRi63WsWXhIMN+eWys=
RSA 2048 bits (e 65537) / SHA1withRSA
Weak or insecure signature, but no impact on root certificate

8

I do recall reading somewhere that OCSP is, in general practical use, only used to confirm the revocation status of bottom certificate relative to the first intermediate. And not in further levels up to the root certificate.

So yes, declaring the first intermediate (chain.pem) as trusted matches the trust which the client is likely to put in that first intermediate.

However, only the root certificate is sufficiently well-known to be trusted at face value by all clients everywhere - meaning it is on every browsers list of trusted root certificates.

Since NGINX is only checking once and delivering many times, it seems it could afford to iterate OCSP checking all the way to root level. Does it do that if ssl_trusted_certificate is not specified (and the root can be found in a known certiificate respository directory like /etc/ssl/certs)? Probably not, I guess (although in my reply to jsha just a few minutes ago I guessed the opposite).

Stapling OCSP responses for all levels would mitigate the scenario of a revoked intermediate certificate being used with malicious intent.

Sorry to be talking about what is, what could be, and what should be- all together. Not clean.

9

That's the cache time for the hostname -> ipaddress lookup.

I was talking about the cache time for the OSCP response.

10

Chrome also doesn’t check CRL lists - only a central limited size list kept by Chrome - for highly visited websites only. I googled for a number of “revoked certificate test” websites and Chrome failed most while Firefox passed all. That’s when I switched to Firefox.

11

The widely deployed Certificate Status Request extension only allows one OCSP response to be stapled. Providing revocation information for the end-entity certificate and all intermediate(s) just isn't technically possible.

The newer Multiple Certificate Status Request extension fixes that limitation, but it isn't widely used yet. Or used at all? I don't know.

On the plus side, if an intermediate does get compromised or something, it will certainly be added to browsers' high priority revocation lists, like Chrome's CRLSets or Mozilla's OneCRL.

So there's not a large impetus to deploy RFC 6961.

2 Likes
12

It does not indeed. I myself opened the feature request eons ago, but...:

This is a feature request that is not high on any of the priority lists for the next few quarters.

:confused:

4 Likes
13

Thanks for making the request, @Osiris.

I find their response unfortunate in light of their also getting rid of HPKP support. While the exact overlap between these two mechanisms in terms of threat modeling isn’t all that extensive, it seems like Google’s philosophy has become extremely reliant on the notion of after-the-fact detection of misissuance via CT, at the expense of giving site operators other ways to set security policies that the browser will enforce. (And yet they haven’t even finished rolling out their mandatory CT enforcement for all DV CAs!)

CT is a great system that’s really improving security and CA accountability, but it used to be possible to say in talks about TLS that CT was only one of several mechanisms by which the CA ecosystem was improving (in terms of reducing CAs’ historically rather unaccountable power). But in Google’s deployment—as the developer of the most popular user-agent—it almost feels like it’s becoming the only newly supported mechanism. There’s no client-side CAA verification—by design according to the CAA specification. There’s no organized effort to audit issuance against CAA policies. Browser extensions to notify users about certificate changes have pretty much fallen out of use, maybe in part because of CDNs and load balancing and in part because of CAs like us with shorter certificate validity. Google is deprecating HPKP. No major browser has implemented any other pinning mechanism by default. And as you’ve said, we don’t have a must-staple implementation in Chrome. Only Tor onion sites have a mechanism to put cryptographic key information in the hostname (proposals to do this in regular DNS have also not been implemented in browsers). While we don’t really have evidence of an epidemic of undetected misissuance, this situation still makes me sad because I was able to tell people about a wider range of “upcoming” options just a couple of years ago.

Maybe we can reach out to some of the Chrome security folks to find out more about how they’re thinking about these mechanisms.

2 Likes
14

Since web servers almost universally implement OCSP stapling unreliably, I predict must-staple deployment will be exciting.

2 Likes
15

It does sound important for the servers to get at least most of those things right. I guess they still don’t, even a couple of years after Ryan Sleevi wrote up that list?

16

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.