SaveUnmatchedRequests stopped working

[LevYas]

Hi! Glad to see the project is thriving! I noticed a little side-effect though :)

Describe the bug

Unmatched requests are no longer saved

Expected behavior:

They should be

Test to reproduce

• Set this option to true
• Try sending the wrong request
• It will be present in logs, but no file is created and LocalFileSystemHandler.WriteUnmatchedRequest() is not even called

Other related info

This doesn't work on the latest version 1.5.35, but I see many changes published on Aug 3. I tried to downgrade to version 1.5.32 and it works in 1.5.32, but doesn't work in 1.5.34 and 1.5.35.

The function WriteUnmatchedRequest is not even called in the latest versions. I tested this using my file handler:

public class MyLocalFileSystemHandler : LocalFileSystemHandler
{
    private readonly string _dataFolder;
    public MyLocalFileSystemHandler(string dataFolder) => _dataFolder = dataFolder;

    public override string GetMappingFolder() => _dataFolder;
    public override string GetUnmatchedRequestsFolder() => _dataFolder;

    public override void WriteUnmatchedRequest(string filename, string text)
    {
        string unmatchedRequestsFolder = GetUnmatchedRequestsFolder(); // We never stop at the breakpoint here
        if (!FolderExists(unmatchedRequestsFolder))
            CreateFolder(unmatchedRequestsFolder);
        File.WriteAllText(Path.Combine(unmatchedRequestsFolder, filename), text);
    }
}

If I downgrade to 1.5.32 the breakpoint does hit and the file is successfully saved.

Maybe something has changed in the way the condition is calculated here

WireMock.Net/src/WireMock.Net/Owin/WireMockMiddleware.cs

Lines 190 to 191 in 59aab9e

	if (_options.SaveUnmatchedRequests == true && result.Match?.RequestMatchResult.IsPerfectMatch != true)
	{

[StefH]

#1002

[StefH]

@LevYas
I have added an extra unit test and changed the code you suggested.

Can you please test preview version 1.5.35-ci-17786 ?

[LevYas]

Thank you for such a quick reaction!

I installed this version via MyGet feed, but the error is still reproducible. I'm not sure that the problem is exactly in the line I mentioned - I meant that the match structure/logic might be changed, but I'm not sure.

Does your unit test pass without the modification to the condition line? I don't see the logic changes, for me, it does the same.

Is it possible that an exception is thrown in the request logging? Json serialization in this method can throw.

Not in this line, but if I start debugging with "just my code" disabled, I get JSON serialization error:

Newtonsoft.Json.JsonReaderException: 'Unexpected character encountered while parsing value: P. Path '', line 0, position 0.'

In that moment, current string in the json parser is

"POST https://uat.portal.suppliesnet.net/invoices/invoices.asmx\r\nContent-Type: text/xml; charset=utf-8\r\n\r\n<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/\">\r\n \r\n <GetInvoices xmlns="http://portal.suppliesnet.net\">\r\n \r\n <dmi:InvoiceRequest xmlns:dmi="http://portal.suppliesnet.net\">\r\n dmi:RequesterISA1065133D2D</dmi:RequesterISA>\r\n dmi:InvoiceCreateDate2023-05-25</dmi:InvoiceCreateDate>\r\n </dmi:InvoiceRequest>\r\n \r\n \r\n \r\n\r\n\r\n\r\n\r\n

This happens at the initialization stage:

Newtonsoft.Json.dll!Newtonsoft.Json.JsonTextReader.ParseValue() Line 1817 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonTextReader.Read() Line 429 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonReader.ReadAndMoveToContent() Line 1240 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonReader.ReadForType(Newtonsoft.Json.Serialization.JsonContract contract, bool hasConverter) Line 1235 C#
Newtonsoft.Json.dll!Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(Newtonsoft.Json.JsonReader reader, System.Type objectType, bool checkAdditionalContent) Line 155 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonSerializer.DeserializeInternal(Newtonsoft.Json.JsonReader reader, System.Type objectType) Line 904 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonSerializer.Deserialize(Newtonsoft.Json.JsonReader reader, System.Type objectType) Line 883 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonConvert.DeserializeObject(string value, System.Type type, Newtonsoft.Json.JsonSerializerSettings settings) Line 831 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonConvert.DeserializeObject(string value, Newtonsoft.Json.JsonSerializerSettings settings) Line 696 C#
WireMock.Net.dll!WireMock.Util.JsonUtils.DeserializeObject(string json) Line 54 C#
WireMock.Net.dll!WireMock.Server.WireMockServer.DeserializeJsonToArray<WireMock.Admin.Mappings.MappingModel>(string value) Line 795 C#
WireMock.Net.dll!WireMock.Server.WireMockServer.ReadStaticMappingAndAddOrUpdate(string path) Line 275 C#
WireMock.Net.dll!WireMock.Server.WireMockServer.ReadStaticMappings(string folder) Line 225 C#
WireMock.Net.dll!WireMock.Server.WireMockServer.InitSettings(WireMock.Settings.WireMockServerSettings settings) Line 1507 C#
WireMock.Net.dll!WireMock.Server.WireMockServer.WireMockServer(WireMock.Settings.WireMockServerSettings settings) Line 1342 C#
WireMock.Net.dll!WireMock.Server.WireMockServer.Start(WireMock.Settings.WireMockServerSettings settings) Line 1234 C#

My mapping

{
  "Guid": "77fc9efa-8410-46c7-a7f3-b8e05a76ffdb",
  "UpdatedAt": "2023-09-08T18:33:18.5661474Z",
  "Title": "Proxy Mapping for POST /invoices/invoices.asmx",
  "Description": "Proxy Mapping for POST /invoices/invoices.asmx",
  "Priority": 100,
  "Request": {
    "Path": {
      "Matchers": [
        {
          "Name": "WildcardMatcher",
          "Pattern": "/invoices/invoices.asmx",
          "IgnoreCase": false
        }
      ]
    },
    "Methods": [
      "POST"
    ],
    "Headers": [
      {
        "Name": "Connection",
        "Matchers": [
          {
            "Name": "WildcardMatcher",
            "Pattern": "keep-alive",
            "IgnoreCase": true
          }
        ]
      },
      {
        "Name": "Accept-Encoding",
        "Matchers": [
          {
            "Name": "WildcardMatcher",
            "Pattern": "gzip, deflate",
            "IgnoreCase": true
          }
        ]
      },
      {
        "Name": "Content-Type",
        "Matchers": [
          {
            "Name": "WildcardMatcher",
            "Pattern": "text/xml; charset=utf-8",
            "IgnoreCase": true
          }
        ]
      },
    ],
    "Body": {
      "Matcher": {
        "Name": "WildcardMatcher",
        "Pattern": "<?xml version=\"1.0\" encoding=\"utf-8\"?><Envelope xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns=\"http://schemas.xmlsoap.org/soap/envelope/\"><Body><GetInvoices xmlns=\"http://portal.suppliesnet.net\"><xmlRequest><InvoiceRequest xmlns:dmi=\"http://portal.suppliesnet.net\"><dmi:RequesterISAa>*</dmi:RequesterISA><dmi:InvoiceCreateDate>*</dmi:InvoiceCreateDate></InvoiceRequest></xmlRequest></GetInvoices></Body></Envelope>",
        "IgnoreCase": true
      }
    }
  },
  "Response": {
    "StatusCode": 200,
    "Body": "<?xml version=\"1.0\" encoding=\"utf-8\"?><soap:Envelope xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\"><soap:Body><GetInvoicesResponse xmlns=\"http://portal.suppliesnet.net\"><GetInvoicesResult><Invoices xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns=\"http://portal.suppliesnet.net\"></Invoices></GetInvoicesResult></GetInvoicesResponse></soap:Body></soap:Envelope>",
    "Headers": {
      "Content-Type": "text/xml; charset=utf-8",
      "Cache-Control": "max-age=0, private",
      "Vary": "Accept-Encoding",
      "X-Frame-Options": "SAMEORIGIN",
      "X-Content-Type-Options": "nosniff",
      "Date": "Fri, 08 Sep 2023 18:33:18 GMT"
    }
  }
}

[LevYas]

I think I got it. The JSON serialization exception is thrown here

WireMock.Net/src/WireMock.Net/Owin/WireMockMiddleware.cs

Line 193 in 59aab9e

_options.FileSystemHandler?.WriteUnmatchedRequest(filename, Util.JsonUtils.Serialize(log));

Stack trace

Newtonsoft.Json.dll!Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(Newtonsoft.Json.JsonWriter writer, object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) Line 499 C#
Newtonsoft.Json.dll!Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(Newtonsoft.Json.JsonWriter writer, object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) Line 489 C#
Newtonsoft.Json.dll!Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(Newtonsoft.Json.JsonWriter writer, object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) Line 489 C#
Newtonsoft.Json.dll!Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(Newtonsoft.Json.JsonWriter writer, object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) Line 489 C#
Newtonsoft.Json.dll!Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(Newtonsoft.Json.JsonWriter writer, object value, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract collectionContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty) Line 489 C#
Newtonsoft.Json.dll!Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(Newtonsoft.Json.JsonWriter jsonWriter, object value, System.Type objectType) Line 81 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonSerializer.SerializeInternal(Newtonsoft.Json.JsonWriter jsonWriter, object value, System.Type objectType) Line 1148 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonSerializer.Serialize(Newtonsoft.Json.JsonWriter jsonWriter, object value, System.Type objectType) Line 1047 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonConvert.SerializeObjectInternal(object value, System.Type type, Newtonsoft.Json.JsonSerializer jsonSerializer) Line 665 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonConvert.SerializeObject(object value, System.Type type, Newtonsoft.Json.JsonSerializerSettings settings) Line 614 C#
Newtonsoft.Json.dll!Newtonsoft.Json.JsonConvert.SerializeObject(object value, Newtonsoft.Json.JsonSerializerSettings settings) Line 592 C#
WireMock.Net.dll!WireMock.Util.JsonUtils.Serialize(object value) Line 38 C#
WireMock.Net.dll!WireMock.Owin.WireMockMiddleware.InvokeInternalAsync(Microsoft.AspNetCore.Http.HttpContext ctx) Line 190 C#
WireMock.Net.dll!WireMock.Owin.WireMockMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext ctx) Line 59 C#
WireMock.Net.dll!WireMock.Owin.GlobalExceptionMiddleware.InvokeInternalAsync(Microsoft.AspNetCore.Http.HttpContext ctx) Line 38 C#
WireMock.Net.dll!WireMock.Owin.GlobalExceptionMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext ctx) Line 29 C#

That is why we never reach the WriteUnmatchedRequest.

The exceptions itself is quite strange:

Newtonsoft.Json.JsonSerializationException
Error getting value from 'ReadTimeout' on 'MimeKit.IO.MemoryBlockStream'. Timeouts are not supported on this stream.

Probably that happened when the serializer tried to serialize BodyAsMimeMessage.

[StefH]

Thank you for the analysis.

MimeMessage support is fairly new. However you are not using any Multiparty Mine, correct?

I'll try to use your mapping and debug the code.

---

Btw thank you for the sponsoring.

[LevYas]

UR welcome, happy to help.
I don't use anything like that. My main use case is to write some request-response pairs in proxy mode, disable proxy mode, and use the recordings both as snapshot tests for request factories and tests for response handlers and many other parts of the code that require interaction with third-party API. I work with JSON and XML (SOAP) APIs. Thanks to the speed and convenience of WMN, I don't use mocked unit tests for API-related code.

My pleasure!

[StefH]

@LevYas
Can you try preview 1.5.35-ci-17788?

[LevYas]

That worked, thank you! WriteUnmatchedRequest was called and the file with failed matching is recorded successfully.

[StefH]

PR is merged and I'll create a new NuGet version.