Got Warning on OCSP SSL verification after upgrading MongoID

Help
1

Hello, I wonder if someone can shed me light on this issue; after getting Ruby Gem dependencies for MongoDB upgraded for Mongoid from 7.1.5 to 7.3.1, I am starting seeing the error as shown below:

MONGODB | TLS certificate of '{server_name}' could not be definitively verified via OCSP: For responders http://r3.o.lencr.org with a timeout of 5 seconds: OCSP response from http://r3.o.lencr.org is 6: unauthorized

Below is the configuration used which had never logged any warning prior to upgrade:

...
production:
  clients:
    default:
      uri: <%= ENV['MONGODB_URL'] %>
      options:
        ssl: true
        ssl_verify: true
        max_pool_size: <%= ENV['MONGODB_MAX_POOL_SIZE'].to_i %>

I am not sure if this is related to the Mongoid component it self or if I should upgrade other dependencies. Thanks in advance!

2

How many times did you get this error? Just a few, or always?

It can be lots of reasons, but we need to know what certificate is giving you this problem, and for what address.

2 Likes
3

This always happen whenever the job kicks off, which is scheduled for every 10 mins.

This is a TLS/SSL certificate used by mongoDB when option ssl-verify is set to true and one of it is pointing to composeDB, i.e. : {our_server_domain}.composedb.com.

Again, this was never logged prior to the upgrade. The app functions without any issue, but what worries me is the unauthorized concern.

4

You are querying the ocsp responder every 10 minutes instead of every few days? You're supposed to save the response and the response itself will tell you when to fetch a new one.

The responder's firewall might have blocked you. @lestaff?

3 Likes
5

Timestamps would be very helpful in trying to debug this from our side. (Timestamps and hostnames would be even better.)

The most common reason for Unauthorized responses tends to be improperly-formatted requests. I can imagine (and have long ago seen) bugs where OCSP stapling doesn't get the intermediate correct on the first try, which would present like this, but I mean this is just throwing darts.

3 Likes
6

The issue was started this morning: 02/21 07:30am EDT (US Eastern).

This warning started showing up after we deployed our changes with updated components of: mongoid from 7.1.5 to 7.3.1 along with others: rack (7.6.4 to 7.7.0), puma (5.0.4 to 5.6.2).

7

If you are able to DM me specific host names and timestamps, that would help. Overall view of that hour looks good without much error noise, but obviously there’s something weird.

Thanks!

3 Likes
8

Can I reach you via slack or what would be the best way to reach you at? I got some findings from MongoDB changeset here. Do let me know the best way to DM you. Thanks.

1 Like
9

Based on the change history of the impacted component, Mongodb: mongo-ruby-driver/lib/mongo/socket/ocsp_verifier.rb at 6ac4655b5be36c1fd283136a1ad890a33629b28b · mongodb/mongo-ruby-driver · GitHub, it introduces the warning when the call is timed out, which could be already there (the timing out) within the previous version of the component but it never got logged.

Is there anyway to check whether the ssl verification is timing out?

10

The standard way to debug problems like this is deploying a patch to the driver that adds extra logging and sets up some debugger breakpoints. Based on the results of that, I would probably file a ticket with MongoDB to at-least clarify those messages.

I'm not suggesting this problem isn't due to LetsEncrypt or some sort of network conditions - but Mongo has never been the most reliable software project/company, and you're dealing with something that is most likely a longstanding issue in the client (or your configuration) that was only surfaced by an improvement to their logging practices.

Some of that logging stuff was even merged into a PR without full testing!
( see comment in https://github.com/mongodb/mongo-ruby-driver/pull/2062 )

This situation just looks to me like it is most-likely caused by a mix of (i) bugs, (ii) lack of documentation, and (iii) lack of developer hooks in the MongoDB driver.

2 Likes
11

You can DM directly inside Discourse here -- just click on jcjones' profile picture and then click the blue "Message" button.

3 Likes
12

Not entirely sure, but I think newly registered users can't send DMs..

4 Likes
13

Good point, @Osiris. You're correct (although maybe there's an exception for PMing admins and moderators?).

At any rate, I've bumped @tsirait's trust level to 1 so they should be able to PM now.

6 Likes
14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.