Support for PEM certificates when using ssl

[kriebb]

Is your feature request related to a problem? Please describe.
When running dotnet test on a builderserver using systemtests with Wiremock ssl custom certificates, a pfx can be specified, but gives problems when loading the pfx on a build server:

  Error Message:
   System.AggregateException : One or more errors occurred. (Service start failed with error: One or more errors occurred. (The specified network password is not correct.)) (The following constructor parameters did not have matching fixture data: ServersSetupFixture fixture)
---- WireMock.Exceptions.WireMockException : Service start failed with error: One or more errors occurred. (The specified network password is not correct.)
-------- System.AggregateException : One or more errors occurred. (The specified network password is not correct.)
------------ Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : The specified network password is not correct.

I can replay this on a buildserver creating a test just running this code

          if (!string.IsNullOrEmpty(filePath) && !string.IsNullOrEmpty(password))
           {
               return new X509Certificate2(filePath, password);
           }

However when using the PEM format, this works.

Describe the solution you'd like
In the CertificateLoader you have the code

            if (!string.IsNullOrEmpty(filePath) && !string.IsNullOrEmpty(password))
            {
                return new X509Certificate2(filePath, password);
            }

You can modify the if statements to look at the extension. If(filepath.Endswith(pem)
return X509.LoadFromPem(filepath,password) where the password is the filepath to the key

Other solution can be to use a IX509Certificate2Factory that you can supply using the services or to the add it to the settings and when that is available, use the factory

... any suggestions?

Describe alternatives you've considered
Try to persude the devops people to give access to the certification store
google, and try to convert the pfx to other supported pfx`s formats like DER.

Is your feature request supported by WireMock (java version)? Please provide details.
PEM doesnt seem to be supported. Only jwks on wiremock.org

Additional context

[StefH]

The easiest is to use If(filepath.Endswith(pem).

Please note that this is only supported for netcoreapp3.1 ; .NET 5.0 and higher.

You can try preview version
1.5.3-ci-16350

[kriebb]

The easiest is to use If(filepath.Endswith(pem).

Please note that this is only supported for netcoreapp3.1 ; .NET 5.0 and higher.

You can try preview version 1.5.3-ci-16350

Thx for your fast response!

tried it, but somehow got this message:

Error Message:
   System.AggregateException : One or more errors occurred. (Service start failed with error: One or more errors occurred. (The server mode SSL must use a certificate with the associated private key.)) (The following constructor parameters did not have matching fixture data: ServersSetupFixture fixture)
---- WireMock.Exceptions.WireMockException : Service start failed with error: One or more errors occurred. (The server mode SSL must use a certificate with the associated private key.)
-------- System.AggregateException : One or more errors occurred. (The server mode SSL must use a certificate with the associated private key.)
------------ System.NotSupportedException : The server mode SSL must use a certificate with the associated private key.

Use it like this:

  settings.CertificateSettings = new WireMockCertificateSettings()
            {
                X509CertificateFilePath = @".\Assets\somefolder\mycer.nokey.pem",
                X509CertificatePassword = @".\Assets\someFolder\mycer.traditional-key.pem",
            };

this is also more helpfull

----- Inner Stack Trace -----
   at System.Net.Security.SslStreamCertificateContext.Create(X509Certificate2 target, X509Certificate2Collection additionalCertificates, Boolean offline, SslCertificateTrust trust)
   at Microsoft.AspNetCore.Server.Kestrel.Https.Internal.HttpsConnectionMiddleware..ctor(ConnectionDelegate next, HttpsConnectionAdapterOptions options, ILoggerFactory loggerFactory)
   at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.<>c__DisplayClass12_0.<UseHttps>b__0(ConnectionDelegate next)
   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.Build()
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass30_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.AnyIPListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IEnumerable`1 listenOptions, AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Hosting.WebHost.StartAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token)
----- Inner Stack Trace #2 (Xunit.Sdk.TestClassException) -----

Seems like something is not working with Kestrel en Pem x509Certificate2.

According to this post: https://stackoverflow.com/questions/67147703/get-the-server-mode-ssl-must-use-a-certificate-with-the-associated-private-key

The pfx needs to be created inmemory from the pem file and exported with a random key then.

Something to tryout tough. sigh

Don't know if you want o include something like this, so I can try it out? (copy paste from the stackoverflow tough.
If however, you'd rather have me test it in a personal project to mimic it, let me know.

string pass = Guid.NewGuid().ToString();
var certificate = X509Certificate2.CreateFromPemFile(certificatePath, privateKeyPath);
return new X509Certificate2(certificate.Export(X509ContentType.Pfx, pass), pass);

[StefH]

Can you try
1.5.3-ci-16353

[kriebb]

Yes,

Same result:

  Error Message:
   WireMock.Exceptions.WireMockException : Service start failed with error: One or more errors occurred. (The server mode SSL must use a certificate with the associated private key.)
---- System.AggregateException : One or more errors occurred. (The server mode SSL must use a certificate with the associated private key.)
-------- System.NotSupportedException : The server mode SSL must use a certificate with the associated private key.
  Stack Trace:
     at WireMock.Server.WireMockServer..ctor(WireMockServerSettings settings)
   at WireMock.Server.WireMockServer.Start(WireMockServerSettings settings)

settings are used like this:

            var keyPem = System.IO.File.ReadAllText(@".\Assets\folderName\cert.traditional-key.pem");
            settings.CertificateSettings = new WireMockCertificateSettings()
            {
                
                X509CertificateFilePath = @".\Assets\folderName\cert.nokey.pem",
                X509CertificatePassword = keyPem ,
            };

other stacktraces that be usefull

----- Inner Stack Trace -----
   at System.Net.Security.SslStreamCertificateContext.Create(X509Certificate2 target, X509Certificate2Collection additionalCertificates, Boolean offline, SslCertificateTrust trust)
   at Microsoft.AspNetCore.Server.Kestrel.Https.Internal.HttpsConnectionMiddleware..ctor(ConnectionDelegate next, HttpsConnectionAdapterOptions options, ILoggerFactory loggerFactory)
   at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.<>c__DisplayClass12_0.<UseHttps>b__0(ConnectionDelegate next)
   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.Build()
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass30_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.AnyIPListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IEnumerable`1 listenOptions, AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Hosting.WebHost.StartAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token)
----- Inner Stack Trace #2 (Xunit.Sdk.TestClassException) -----

I did see you, you didnt do the "trick" with the line certificate.Export(X509ContentType.Pfx, pass), pass wich I can understand. I guess it will also try to have some rights again for accessing the cert store. But I didnt try it out yet.

[StefH]

Code updated with:
return new X509Certificate2(certificate.Export(X509ContentType.Pfx, pass), pass);

New preview will be available within few minutes.

[kriebb]

blush what is the preview? cant seem to find it on actions ?

will try it tomorrow :)

[StefH]

@kriebb
See this wiki:
https://github.com/WireMock-Net/WireMock.Net/wiki/MyGet-preview-versions

Also if you can send me test PEM = password (or tell me how to generate it), I can also test on my machine.

[StefH]

Latest preview (1.5.3-ci-16357) should work fine.

I based my example + logic on https://www.scottbrady91.com/c-sharp/pem-loading-in-dotnet-core-and-dotnet

See https://github.com/WireMock-Net/WireMock.Net/pull/787/files#diff-721297a29d1cb2c8d0ca5463eea9738306dd5416115e04711d2c7bf20576d71b

[kriebb]

Sorry, the same behavior occurs.

 Error Message:
   System.AggregateException : One or more errors occurred. (Service start failed with error: One or more errors occurred. (The server mode SSL must use a certificate with the associated private key.)) (The following constructor parameters did not have matching fixture data: Auth0UserAgentClassFixture fixture1)
---- WireMock.Exceptions.WireMockException : Service start failed with error: One or more errors occurred. (The server mode SSL must use a certificate with the associated private key.)
-------- System.AggregateException : One or more errors occurred. (The server mode SSL must use a certificate with the associated private key.)
------------ System.NotSupportedException : The server mode SSL must use a certificate with the associated private key.

----- Inner Stack Trace -----
   at System.Net.Security.SslStreamCertificateContext.Create(X509Certificate2 target, X509Certificate2Collection additionalCertificates, Boolean offline, SslCertificateTrust trust)
   at Microsoft.AspNetCore.Server.Kestrel.Https.Internal.HttpsConnectionMiddleware..ctor(ConnectionDelegate next, HttpsConnectionAdapterOptions options, ILoggerFactory loggerFactory)
   at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.<>c__DisplayClass12_0.<UseHttps>b__0(ConnectionDelegate next)
   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.Build()
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass30_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.AnyIPListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IEnumerable`1 listenOptions, AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Hosting.WebHost.StartAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token)
----- Inner Stack Trace #2 (Xunit.Sdk.TestClassException) -----

However, intresting read that you mention.

When I read at the end:

So you cant pass an x509Certificate to Kestrel, because it fucks something up in windows when opening an SSLStream ( as designed it seems)

System.ComponentModel.Win32Exception (0x8009030E): No credentials are available in the security package
This is because the private key is being loaded into memory (like the ephemeral keyset flag), **but Windows needs the key to be in the system key set. This is again discussed in the .NET Design Review.**

The CI pipeline at the company dont give any access to the system keyset. So the current solution that is built in ( works fine on my machine tough)

However, when I read further more on:

ASP.NET Core works around this in the Kestrel configuration loader, which means if you define your endpoints in config like so, you can use PEM files in Kestrel for HTTPS. Path needs to be the PEM file that contains your certificate and KeyPath needs to be the PEM file containing the corresponding private key.

{
  "Kestrel": {
    "Endpoints": {
      "HttpsFromPem": {
        "Url": "https://localhost:5001",
        "Certificate": {
          "Path": "../cert.pem",
          "KeyPath": "../ecc.pem"
        }
      }
    }
  }
}

So just passing filepaths to the KestrelEndPointOptions should be able to work.

Are you in for a final try? after then, I give up. But that article that you mentioned, really confirms what I was thinking, You dont need access to the store using PEM ( or am I misreading it?)

You asked on how I was creating the PEM

Using the following:

https://www.cryptool.org/en/cto/openssl

openssl genrsa -aes-256-cbc -passout pass:inmemory -out privkey.pem 2048
openssl req -new -x509 -key privkey.pem -out openssl_crt.pem -outform pem
	Enter pass phrase for privkey.pem:inmemory

	Country Name (2 letter code) [AU]:BE
	State or Province Name (full name) [Some-State]:Brussels
	Locality Name (eg, city) []:Brussels
	Organization Name (eg, company) [Internet Widgits Pty Ltd]:Some
	Organizational Unit Name (eg, section) []:Something
	Common Name (e.g. server FQDN or YOUR name) []:localhost
	Email Address []:
openssl rsa -traditional -in private-key.pem -out private-keypkcs1.pem #Needed for .net core

openssl pkcs12 -export -out certificate.pfx -inkey privkey.pem -in openssl_crt.pem
	Enter pass phrase for privkey.pem:inmemory
	Enter Export Password:inmemory
	Verifying - Enter Export Password:inmemory 

openssl rsa -pubout -in privkey.pem -passin pass:inmemory -outform PEM -out pubkey.pem

When I want to convert an existing PFX,  I use the following:

https://www.cryptool.org/en/cto/openssl

openssl pkcs12 -in some2048.pfx -out some2048.nokey.pem -nokeys

Enter Import Password:some

openssl pkcs12 -in some2048.pfx -out some2048.withkey.pem

Enter Import Password:some

Enter PEM pass phrase:somepem

Verifying - Enter PEM pass phrase:somepem

openssl rsa -traditional -in some2048.withkey.pem -out file-traditional.key

Enter pass phrase for some2048.withkey.pem:somepem

writing RSA key

[StefH]

1
BTW : Yesterday I did quickly test creating an RSA Certificate using that website, however I found that only a EC Certificate did work in my example app.
So can you also try that?

2
The Kestrel options you mention:

{
  "Kestrel": {
    "Endpoints": {
      "HttpsFromPem": {
        "Url": "https://localhost:5001",
        "Certificate": {
          "Path": "../cert.pem",
          "KeyPath": "../ecc.pem"
        }
      }
    }
  }
}

This is already supported by WireMock.Net, see https://github.com/WireMock-Net/WireMock.Net/wiki/KestrelServerOptions

[StefH]

@kriebb RSA does also work when I follow https://www.scottbrady91.com/openssl/creating-rsa-keys-using-openssl

See my PR for details.

[kriebb]

Seems to work 👍

What also a good way of having a valid certificate (if you work with local host is using the dotnet dev-certs https -v -ep $(HOME).aspnet\https --format pem"

[StefH]

OK, I'll merge the code to master and close this issue.