Docs/examples that use Linux-specific system CA bundle path fail on Windows

[sunjayBhatia]

Description:
Envoy docs/examples mention some common Linux-distro specific filepaths, especially for the system CA bundle /etc/ssl/certs/ca-certificates.crt. These paths are of course not relevant on Windows. This showed up explicitly in #13534 when trying to test/validate all configs from docs. Right now validation of a few examples is disabled on Windows.

What makes this a little complicated on Windows is that the trusted certificate store is not entirely represented with the filesystem, you can get some info, but to get a PEM encoded x509 back out of Windows you need to do some registry querying.

From a quick grep, we use the system CA bundle path a few times in docs examples:

$ pwd
/c/workspace/envoy
$ grep -r ca-certificates.crt .
./docs/BUILD:#       dns-cache-circuit-breaker: "Error: unable to read file: /etc/ssl/certs/ca-certificates.crt"
./docs/root/configuration/http/http_filters/_include/dns-cache-circuit-breaker.yaml:            trusted_ca: {filename: /etc/ssl/certs/ca-certificates.crt}
./docs/root/intro/arch_overview/security/ssl.rst:*/etc/ssl/certs/ca-certificates.crt* is the default path for the system CA bundle on Debian systems.
./docs/root/intro/arch_overview/security/ssl.rst:* /etc/ssl/certs/ca-certificates.crt (Debian/Ubuntu/Gentoo etc.)
./docs/root/intro/arch_overview/security/_include/ssl.yaml:              filename: /etc/ssl/certs/ca-certificates.crt

Should we simply use a cert/filepath that we generate with Bazel in the example and offer a suggestion via comment in the YAML embedded in the docs that mentions the typical Debian/Ubuntu cert path? Template the config files in question and insert a different path on Windows, use a Linux/Windows tab structure in docs to display the differences? Another solution?

Another question is around how we would want to load system trusted certs into Envoy (which should possibly be a separate issue, let me know what you think and I will make one). Adding a new DataSource type could be possible, maybe we could maybe accept a cert store path like powershell (e.g. Cert:\LocalMachine\Root) to load from but detecting changes like we do with files to reload configuration may not be similarly possible (haven't looked into it). Do we want to instead force Windows users to extract certificates and write them to disk as PEM encoded x509 certs? (this is what they would have to do today)

Relevant Links:

• Example of a docs page that uses the typical Debian/Ubuntu/etc. system CA bundle: https://www.envoyproxy.io/docs/envoy/v1.16.0/intro/arch_overview/security/ssl#example-configuration
• Original PR that brought this issue up: docs: Unexclude remaining configs from validation #13534
  • See this diff: https://github.com/envoyproxy/envoy/pull/13534/files#diff-92ae908436b530a281eddfb50321c5a055ad4fa90e0194f0651b58ac39341800R20-R21

[sunjayBhatia]

cc @envoyproxy/windows-dev

[phlax]

i think in the docs we want to keep the current path for the linked config - ie /etc/ssl/certs/ca-certificates.crt as that works out of the box for mac/linux

which then leaves the question of how to test this on windows and how to document

potentially we can mangle or provide an alternate config file and include the required cert in the bazel test - that would at least provide a way of testing

not sure about docs - i guess one question i have is whether there is an "authoritative" place to find this ca cert on a win system

[sunjayBhatia]

I believe the authoritative root CAs can be queried with https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-certopensystemstorea, passing "Root" to the second argument (or https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-certopenstore to query the LocalMachine not CurrentUser), but this doesn't represent a file path which is why I had the idea of adding to the DataSource or similar types in the Envoy API to add the ability to fetch values from the system cert store (add to TLS validation context: https://www.envoyproxy.io/docs/envoy/v1.16.0/api-v3/extensions/transport_sockets/tls/v3/common.proto#extensions-transport-sockets-tls-v3-certificatevalidationcontext ?)

If youre curious as well, you can get system CAs in Powershell specifically:

PS C:\workspace\envoy> ls Cert:\LocalMachine\Root


   PSParentPath: Microsoft.PowerShell.Security\Certificate::LocalMachine\Root

Thumbprint                                Subject
----------                                -------
CDD4EEAE6000AC7F40C3802C171E30148030C072  CN=Microsoft Root Certificate Authority, DC=microsoft, DC=com
BE36A4562FB2EE05DBB3D32323ADF445084ED656  CN=Thawte Timestamping CA, OU=Thawte Certification, O=Thawte, L=Durbanville, S=Western Cape, C=ZA
A43489159A520F0D93D032CCAF37E7FE20A8B419  CN=Microsoft Root Authority, OU=Microsoft Corporation, OU=Copyright (c) 1997 Microsoft Corp.
92B46C76E13054E104F230517E6E504D43AB10B5  CN=Symantec Enterprise Mobile Root for Microsoft, O=Symantec Corporation, C=US
8F43288AD272F3103B6FB1428485EA3014C0BCFE  CN=Microsoft Root Certificate Authority 2011, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
7F88CD7223F3C813818C994614A89C99FA3B5247  CN=Microsoft Authenticode(tm) Root Authority, O=MSFT, C=US
3B1EFD3A66EA28B16697394703A72CA340A05BD5  CN=Microsoft Root Certificate Authority 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
31F9FC8BA3805986B721EA7295C65B3A44534274  CN=Microsoft ECC TS Root Certificate Authority 2018, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
245C97DF7514E7CF2DF8BE72AE957B9E04741E85  OU=Copyright (c) 1997 Microsoft Corp., OU=Microsoft Time Stamping Service Root, OU=Microsoft Corporation, O=Microsoft Trust Network
18F7C1FCC3090203FD5BAA2F861A754976C8DD25  OU="NO LIABILITY ACCEPTED, (c)97 VeriSign, Inc.", OU=VeriSign Time Stamping Service Root, OU="VeriSign, Inc.", O=VeriSign Trust Network
06F1AA330B927B753A40E68CDF22E34BCBEF3352  CN=Microsoft ECC Product Root Certificate Authority 2018, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
DF3C24F9BFD666761B268073FE06D1CC8D4F82A4  CN=DigiCert Global Root G2, OU=www.digicert.com, O=DigiCert Inc, C=US
DAC9024F54D8F6DF94935FB1732638CA6AD77C13  CN=DST Root CA X3, O=Digital Signature Trust Co.
D4DE20D05E66FC53FE1A50882C78DB2852CAE474  CN=Baltimore CyberTrust Root, OU=CyberTrust, O=Baltimore, C=IE
AD7E1C28B064EF8F6003402014C3D0E3370EB58A  OU=Starfield Class 2 Certification Authority, O="Starfield Technologies, Inc.", C=US
A8985D3A65E5E5C4B2D7D66D40C6DD2FB19C5436  CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
75E0ABB6138512271C04F85FDDDE38E4B7242EFE  CN=GlobalSign, O=GlobalSign, OU=GlobalSign Root CA - R2
742C3192E607E424EB4549542BE1BBC53E6174E2  OU=Class 3 Public Primary Certification Authority, O="VeriSign, Inc.", C=US
5FB7EE0633E259DBAD0C4C9AE6D38F1A61C7DC25  CN=DigiCert High Assurance EV Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
0563B8630D62D75ABBC8AB1E4BDFB5A899B24D43  CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US

And use some additional features of the Powershell Certificate Provider which gives you a logical drive to manipulate: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.security/about/about_certificate_provider?view=powershell-7

These objects can be grabbed through the registry as well via paths like: HKLM:\Software\Microsoft\SystemCertificates\Root\Certificates\

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[wrowe]

This is an aspect of #13281 to be resolved

[davinci26]

I am looking into this and I have the following approaches in mind:

1. Expand the DataSource in base.proto to take a known_location specifier which would be windows_certificate_store. The value of this property would be a string indicating the store.

The downside of this approach is that this proto is consumed by various source where the the certificate store does not make much sense.

2. Add an additional property to certificatevalidationcontext like trusted_store and implement this feature only on this specific level.

The downside of this approach is that we add an additional property for the trusted trusted_ca which makes the documentation feel a bit awkward.

On both cases I think it will be clear to user that we won't support filewatcher and certificate reloading when the trusted certificate is on the Windows store.

In terms of implementation we would need to change the default_validator

envoy/source/extensions/transport_sockets/tls/cert_validator/default_validator.cc

Line 74 in 54c7445

bssl::UniquePtr<STACK_OF(X509_INFO)> list(

to get the certificates from the windows store.

Refs:
certificatevalidationcontext
Cert Store
Enum Certs in Store
certcontrolstore

[davinci26]

@envoyproxy/windows-dev @mattklein123 I am in favor of implementation (2) but I would like to get a high level consensus before I move into the implementation.

[mattklein123]

From a quick glance (2) seems reasonable but perhaps you could put up a draft API review only and we can discuss with a bit more context?

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.