Querying CRLite for WebPKI Revocations
Published 2020-11-26
Firefox Nightly is now using CRLite to determine if websites’ certificates are revoked — e.g., if the Certificate Authority published that web browsers shouldn’t trust that website certificate. Telemetry shows that querying the local CRLite dataset is much faster than making a network connection for OCSP, which makes intuitive sense. It also avoids sending the website’s certificate information in cleartext over the network to check the revocation status: solving one of the remaining cleartext browsing data leakages in Firefox.
Mozilla is currently publishing CRLite data to Remote Settings four times per day, keeping a very fresh set of revocation information for the public Web. I’ve provided some direct details on how to get at that data from the CRLite FAQ, and I want to introduce one of my command-line tools I’ve used to analyze and play with the dataset: moz_crlite_query. I’ll introduce crlite_status in a later post.
Querying CRLite Status at the CLI
moz_crlite_query is a Python tool that more-or-less implements the same algorithm Firefox does to check a website’s certificate against CRLite. It maintains a local database of the CRLite dataset in your ~/.crlite_db/ directory, which for me right now looks like:
0 Nov 25 09:24 .last_updated
5381898 Nov 25 09:24 2020-11-24T00:08:12+00:00Z-full
27363 Nov 25 09:24 2020-11-24T06:08:12+00:00Z-diff
59196 Nov 25 09:24 2020-11-24T12:08:14+00:00Z-diff
56121 Nov 25 09:24 2020-11-24T18:08:15+00:00Z-diff
204863 Nov 25 09:24 2020-11-25T00:08:23+00:00Z-diff
25826 Nov 25 09:24 2020-11-25T06:08:05+00:00Z-diff
66626 Nov 25 09:24 2020-11-25T12:08:22+00:00Z-diff
25826 Nov 25 09:24 2020-11-25T06:08:05+00:00Z-diff
66626 Nov 25 09:24 2020-11-25T12:08:22+00:00Z-diff
53783 Nov 25 22:19 2020-11-25T18:08:11+00:00Z-diff
190353 Nov 25 22:19 2020-11-26T00:08:11+00:00Z-diff
1118208 Nov 25 22:19 intermediates.sqlite
You can see here that the last full CRLite filter update happened at the midnight-UTC update on 24 November 2020, and since then there have been eight delta updates (e.g., “stashes”).
moz_crlite_query is distributed as a Python package that can be installed via an invocation of pip (Note that this tool requires Python 3.7 or newer because of SQLite features.):
pip install moz_crlite_query
On invocation, the tool will try and update to the latest CRLite data, unless it already updated in the last several hours.
You can query the CRLite status for either a certificate you already have, or the tool can connect to one or more websites and examine the certificates presented there.
Experimenting with the Query Tool
Let’s download a few certificates into /tmp from crt.sh: 77575263, 1988442812, 1485147627, and 2680822568. These simply are representative – right now – of various states in the Web PKI.
○ → for id in 77575263 1988442812 1485147627 2680822568; do
> curl --silent https://crt.sh/?d=${id} > /tmp/${id}.pem
> done
Now, run them all through CRLite:
○ → moz_crlite_query /tmp/*.pem
INFO:crlite_query:CRLite Update: Syncing CRLite filters.
100% (9 of 9) |######################################################| Elapsed Time: 0:00:00 Time: 0:00:00
INFO:query_cli:Status: 2457 Intermediates, Current filter: 2020-11-24T00:08:12+00:00Z-full
with 27 layers and 43053008 bit-count, 8 stash files with 43022 stashed revocations,
up-to-date as of 2020-11-26 00:08:11+00:00 (5:11:15.467645 ago).
/tmp/1485147627.pem Issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US
Enrolled in CRLite: ❌
Result: ❔ Not Enrolled ❔
/tmp/1988442812.pem Issuer: CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US
Enrolled in CRLite: ✅
Revoked via CRLite filter: 2020-11-24T00:08:12+00:00Z-full
Result: ⛔️ Revoked ⛔️
/tmp/2680822568.pem Issuer: CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US
Enrolled in CRLite: ✅
Result: 👍 Valid 👍
/tmp/77575263.pem Issuer: CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US
Enrolled in CRLite: ✅
Result: ⏰ Expired ⏰
We can see 1485147627 is issued by a CA not participating in CRLite: Let’s Encrypt doesn’t publish CRLs. 1988442812 is for revoked.badssl.com, and is revoked. 2680822568 is currently a certificate used for many Mozilla properties, like firefox.com and taskcluster.net. Finally, 77575263 is an expired certificate for blog.mozilla.com (and some others).
We can also supply hosts, either on the command line, as an input file, or both:
○ → cat >/tmp/top4.txt <<EOF
apple.com
youtube.com
www.google.com:443
# This is definitely half of my top 8 spaces
www.blogger.com
EOF
○ → moz_crlite_query --hosts mozilla.com firefox.com --hosts getfirefox.net --hosts-file /tmp/top4.txt
INFO:query_cli:Database was updated at 2020-11-25 22:19:26.467085, skipping.
INFO:query_cli:Status: 2457 Intermediates, Current filter: 2020-11-24T00:08:12+00:00Z-full
with 27 layers and 43053008 bit-count, 8 stash files with 43022 stashed revocations,
up-to-date as of 2020-11-26 00:08:11+00:00 (5:21:25.126138 ago).
mozilla.com:443 Issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US
Enrolled in CRLite: ❌
Result: ❔ Not Enrolled ❔
firefox.com:443 Issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US
Enrolled in CRLite: ❌
Result: ❔ Not Enrolled ❔
getfirefox.net:443 Issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US
Enrolled in CRLite: ❌
Result: ❔ Not Enrolled ❔
apple.com:443 Issuer: CN=DigiCert SHA2 Extended Validation Server CA-3,OU=www.digicert.com,O=DigiCert\, Inc.,C=US
Enrolled in CRLite: ✅
Result: 👍 Valid 👍
youtube.com:443 Issuer: CN=GTS CA 1O1,O=Google Trust Services,C=US
Enrolled in CRLite: ✅
Result: 👍 Valid 👍
www.google.com:443 Issuer: CN=GTS CA 1O1,O=Google Trust Services,C=US
Enrolled in CRLite: ✅
Result: 👍 Valid 👍
www.blogger.com:443 Issuer: CN=GTS CA 1O1,O=Google Trust Services,C=US
Enrolled in CRLite: ✅
Result: 👍 Valid 👍
This is all in good fun, but there’s more possibility here than just a command-line gimmick.
Embedding CRLite into Other Tools?
Many tools use the Mozilla Root CA program’s list of trust anchors for their trust decisions, particularly on Linux or BSD operating systems. For many such tools, certificate revocations are sufficiently unlikely and the consequences sufficiently low that checking revocation status is simply unimportant. For those tools that do check, leaking the status check in cleartext to network observers is likely to simply be evidence of doing a software update. However, I can imagine that some tools might benefit from CRLite’s infrastructure-as-a-service, and make good use of the privacy benefit.
As it’s not particularly difficult to develop a CRLite client, perhaps if this tooling makes it to Firefox Release, we’ll see some libraries for non-Firefox users of CRLite.
While Mozilla certainly isn’t intending to have numerous users of the data beyond Firefox — at least they aren’t accounted for in the budget — the data is there, and there to be useful.