Using google.protobuf.Empty as response results in a bad gRPC response #1144

ArcaneTSGK · 2024-07-24T10:50:00Z

When writing a response builder with google.protobuf.Empty the mock gRPC client will return StatusCode=Unimplemented.

Expected behavior:

It should be possible to use googles 'WellKnown' types as the protobuf body in the response builder.

Test to reproduce

Proto file greet.proto

syntax = "proto3"

import "google/protobuf/empty.proto";

package organization.greet.api.v1;

message HelloRequest {
  string name = 1;
}

service Greeter {
  rpc SayHello (HelloRequest) returns (google.protobuf.Empty);
}

The test

[Fact]
public async Task SayHello_ShouldReturnEmptyResponse_WhenInvoked()
{
  using var server = WireMockServer.Start(new WireMockServerSettings
  {
    Urls = ["grpcs://localhost:5000"],
    UseHttp2 = true
  });

  var protoFile = await File.ReadAllTextAsync(@"C:\projects\GrpcIntegrationTests\GrpcIntegrationTests\Protos\greet.proto");

  var protoDefinitionid = "GrpcGreet";

  server.AddProtoDefinition(protoDefinitionId, protoFile)
    .Given(Request.Create()
      .UsingPost()
      .WithHttpVersion("2")
      .WithPath("/organization.greet.api.v1.Greeter/SayHello")
      .WithBodyAsProtoBud("organization.greet.api.v1.HelloRequest"))
    .WithProtoDefinition(protoDefinitionId)
    .RespondWith(Response.Create()
      .WithHeader("Content-Type", "application/grpc")
      .WithTrailingHeader("grpc-status", "0")
      .WithBodyAsProtoBuf("google.protobuf.Empty", new {})
      .WithTransformer());

  var channel = GrpcChannel.ForAddress(server.Url!);

  var client = new Greeter.GreeterClient(channel);

  var reply = await client.SayHelloAsync(new HelloRequest { Name = "Test" });

  reply.Should.BeOfType<Empty>();

  server.Stop();
}

I have tried with and without the .WithTransformer(), I have also tried with new Empty() new Empty {} and new() but none of those match.

Other related info

I am able to get the tests to work if the response is a type that I create in my proto, for example lets say that the HelloReply is in the proto file with a message HelloReply { string reply_message = 1; }

.WithBodyAsProtoBuf("organization.greet.api.v1.HelloReply", new { reply_message = "Hello" })

I have also tried creating my own message EmptyResponse { } and this also did not work

.WithBodyAsProtoBuf("organization.greet.api.v1.EmptyResponse", new {})

It only works when the response contains something like

I have also tried creating my own message EmptyResponse { string status = 1; }

.WithBodyAsProtoBuf("organization.greet.api.v1.EmptyResponse", new { status = "accepted" })

Please can someone either point me in the direction of what I am doing wrong to match an empty response, or confirm that this is a bug

Thanks

The text was updated successfully, but these errors were encountered:

StefH · 2024-07-24T11:53:19Z

@ArcaneTSGK Thanks for noticing this. This is a new scenario I never tested.
Probably I need to fix this here: StefH/ProtoBufJsonConverter#15

I'll keep you informed on the progress...

StefH · 2024-07-29T05:57:30Z

@ArcaneTSGK
I was thinking, are there any other predefined types in google?

And what should actually be the behavior of WireMock.Net if a google.protobuf.Empty is a return value?
Is this just an empty response?

ArcaneTSGK · 2024-07-29T08:23:59Z

@StefH There are a number of them yes,

For Empty return value it's Empty {} it's because protobuf does not allow you to request/response with null / no body like a typical REST API would.

Converting Empty to JSON would represent an empty JSON object {}

Here are all of Googles Well Known types:

https://protobuf.dev/reference/protobuf/google.protobuf/

https://protobuf.dev/reference/protobuf/google.protobuf/#empty

Returning any of the scalar types in protobuf within a custom response that maps to a C# type is fine except for 'Enum'. I've not tried the 'Any' well-known type as my application doesn't have use for it, but I imagine that one might cause some issues, also when working with nullable types you use 'OneOf', again I haven't had to use these so I do not know if they'll work, but this is what a proto file would look like using those:

Usage of 'Any'

syntax = "proto3";

package tutorial;

import "include/google/protobuf/any.proto"

message Animal {
	string name=1;
	int32 age=2;
	google.protobuf.Any care_giver=3;
}

message Owner {
	int32 id=1;
	string first_name=2;
	string last_name=3;
}

message Foster {
	int32 id=1;
	string address=2;
}

In that example a care give can be any pre-defined message in the proto, which could be Owner or the Foster, so that might be a test case for Wiremock to see if Any maps back, and lastly OneOf for nullables where you'd need to be able to allow a null return in the response:

And this is OneOf

import "google/protobuf/struct.proto";

package tutorial;

option csharp_namespace = "MyExample.Dog";

message Dog {
	google.protobuf.Int32Value id=1;
	string name=2;
	NullableField profile_picture=3;
}

message NullableField {
	oneof kind {
		google.protobuf.NullValue null=1;
		google.protobuf.StringValue value=2;
	}
}

StefH · 2024-07-30T17:46:44Z

Currently I only have an easy way to support:

google.protobuf.Empty
google.protobuf.Duration
google.protobuf.Timestamp

StefH · 2024-08-04T12:33:04Z

StefH/ProtoBufJsonConverter#18

StefH · 2024-08-04T18:11:52Z

@ArcaneTSGK
If you have time, you can use this preview version: 1.5.62-ci-19066

StefH · 2024-08-15T07:59:17Z

@ArcaneTSGK
Can you please test preview 1.5.62-ci-19067 , this version supports:

google.protobuf.Empty
google.protobuf.Duration
google.protobuf.Timestamp
google/protobuf/wrappers
google/protobuf/any
google/protobuf/struct
Enum
oneof

ArcaneTSGK · 2024-08-15T08:55:47Z

@ArcaneTSGK Can you please test preview 1.5.62-ci-19067 , this version supports:

google.protobuf.Empty

google.protobuf.Duration

google.protobuf.Timestamp

google/protobuf/wrappers

google/protobuf/any

google/protobuf/struct

Enum

oneof

I will have some time tomorrow to take a look, I'll let you know my results, thanks @StefH

StefH · 2024-08-22T18:36:24Z

@ArcaneTSGK
Did you have some time to check?

StefH · 2024-08-27T09:53:21Z

@ArcaneTSGK
Did you have some time to check?

ArcaneTSGK · 2024-08-31T13:11:25Z

Hi @StefH sorry for the delay,

Unfortunately, I can't install your prerelease version 1.5.62-ci-19067 as it does not resolve, are your pre-releases publicly available?

I only have the option (with prereleases enabled) to install 1.5.62 or 1.60 / 1.61

StefH · 2024-08-31T13:15:29Z

Preview versions are defined on MyGet.
See https://github.com/WireMock-Net/WireMock.Net/wiki/MyGet-preview-versions

But it could be that that specific version is automatically deleted because only x versions are kept on MyGet.

I will take a look and maybe I need to build a new preview.

ArcaneTSGK · 2024-08-31T13:21:43Z

Preview versions are defined on MyGet. See https://github.com/WireMock-Net/WireMock.Net/wiki/MyGet-preview-versions

But it could be that that specific version is automatically deleted because only x versions are kept on MyGet.

I will take a look and maybe I need to build a new preview.

No worries, I've added MyGet as a feed and will await the specific version with the gRPC fix to test

StefH · 2024-08-31T15:21:39Z

@ArcaneTSGK
Preview version on MyGet for this fix should be : 1.6.1-ci-19109

ArcaneTSGK · 2024-09-02T15:42:58Z

@StefH

So I attempted to use this build and I get the following exception:

(Status(StatusCode="Internal", Detail="Failed to deserialize response message.")

The proto file looks like this

syntax = "proto3";

import "google/protobuf/empty.proto";

package communication.api.v1;

message SendEmailRequest {
  string email_address = 1;
}

service CommunicationService {
  rpc SendEmail(SendEmailRequest) returns (google.protobuf.Empty);
}

The test is setup as follows:

public void SetupSendEmail()
{
    _server!.AddProtoDefinition(ProtoDefinitionId, CommunicationProtoFile)
        .Given(Request.Create()
            .UsingPost()
            .WithHttpVersion("2")
            .WithPath("/communication.api.v1.CommunicationService/SendEmail")
            .WithBodyAsProtoBuf("communication.api.v1.SendEmailRequest"))
        .WithProtoDefinition(ProtoDefinitionId)
        .RespondWith(Response.Create()
            .WithHeader("Content-Type", "application/grpc")
            .WithTrailingHeader("grpc-status", "0")
            .WithBodyAsProtoBuf(
                "google.protobuf.Empty",
                new { })
            .WithTransformer());
}

I did try Duration and Timestamp aswell but those were returning unimplemented for me when I was using google.protobuf.Timestamp and google.protobuf.Duration respectively as return types

Let me know if I'm missing anything

I also tried the fully qualified name of Google.Protobuf.WellKnownTypes.Empty as the message type to no avail.

Thanks

StefH · 2024-09-02T16:26:39Z

Strange, I did add a unit test https://github.com/WireMock-Net/WireMock.Net/blob/bug/1144-protobuf/test/WireMock.Net.Tests/Grpc/WireMockServerTests.Grpc.cs#L104

proto

private const string ProtoDefinitionWithWellKnownTypes = @"
syntax = ""proto3"";

import ""google/protobuf/empty.proto"";
import ""google/protobuf/timestamp.proto"";
import ""google/protobuf/duration.proto"";

service Greeter {
    rpc SayNothing (google.protobuf.Empty) returns (google.protobuf.Empty);
}

message MyMessageTimestamp {
    google.protobuf.Timestamp ts = 1;
}

message MyMessageDuration {
    google.protobuf.Duration du = 1;
}
";

Unit test code:

// Arrange
        var bytes = Convert.FromBase64String("CgRzdGVm");

        using var server = WireMockServer.Start();

        server
            .Given(Request.Create()
                .UsingPost()
                .WithPath("/grpc/Greeter/SayNothing")
                .WithBody(new NotNullOrEmptyMatcher())
            )
            .RespondWith(Response.Create()
                .WithBodyAsProtoBuf(ProtoDefinitionWithWellKnownTypes, "google.protobuf.Empty",
                    new { }
                )
                .WithTrailingHeader("grpc-status", "0")
                .WithTransformer()
            );

        // Act
        var protoBuf = new ByteArrayContent(bytes);
        protoBuf.Headers.ContentType = new MediaTypeHeaderValue("application/grpc-web");

        var client = server.CreateClient();
        var response = await client.PostAsync("/grpc/Greeter/SayNothing", protoBuf);

        // Assert
        response.StatusCode.Should().Be(HttpStatusCode.OK);
        var responseBytes = await response.Content.ReadAsByteArrayAsync();

        Convert.ToBase64String(responseBytes).Should().Be("");

        server.Stop();

What is the difference?

StefH · 2024-09-02T16:33:15Z

Also added another test: https://github.com/WireMock-Net/WireMock.Net/blob/bug/1144-protobuf/test/WireMock.Net.Tests/Grpc/WireMockServerTests.Grpc.cs#L144

ArcaneTSGK · 2024-09-03T16:47:42Z

I will have to provide a minimal reproduction solution when I have time, I've tried a few different things and for .Empty I always get the unable to deserialize response, and for duration/timestamp I'm getting the Unimplemented error.

I'll provide a link to the repository and throw something together by the end of the week

StefH · 2024-09-05T05:21:55Z

#1153

StefH · 2024-09-05T05:43:17Z

Using https://protobuf.heyenrath.nl/ with your message and google.protobuf.Empty does return an empty byte array

StefH · 2024-09-11T16:13:31Z

Hello @ArcaneTSGK, could you make an example project to demonstrate this issue?

StefH · 2024-09-17T18:16:48Z

Hello @ArcaneTSGK, could you make an example project to demonstrate this issue?

carlos-pilao-deltatre · 2024-09-19T13:43:26Z

Hello,
Just confirmed the reported issue (using the latest version: 1.6.3):

If you have import google/protobuf/empty.proto in your proto file and you are not referencing Empty anywhere the response will be always 404 Unimplemented
If you have import google/protobuf/empty.proto in your proto file and you are referencing Empty in the contracts then it works
If you have import google/protobuf/struct.proto in your proto file then the response will be always 404 Unimplemented
cc @StefH

StefH · 2024-09-20T11:36:36Z

@carlos-pilao-deltatre
Can you please test preview version 1.6.2-ci-19188?

carlos-pilao-deltatre · 2024-09-20T14:27:11Z

@StefH
Cannot find that release anymore. Tried with the latest preview (1.6.3-ci-19196) and cant compile my solution with that one:

Getting ambiguous reference on Google.Protobuf.WellKnownTypes

StefH · 2024-09-20T14:32:33Z

You did check MyGet?

carlos-pilao-deltatre · 2024-09-20T15:23:46Z

yes

StefH · 2024-09-22T08:44:12Z

@carlos-pilao-deltatre

try this
1.6.3-ci-19198

carlos-pilao-deltatre · 2024-09-23T13:42:36Z

Getting the same issue...
@StefH

StefH · 2024-09-23T17:14:35Z

1. If you have _import google/protobuf/empty.proto_ in your proto file and you are **not** referencing _Empty_ anywhere the response will be always _404 Unimplemented_

2. If you have _import google/protobuf/empty.proto_ in your proto file and you are referencing _Empty_ in the contracts then it works

3. If you have _import google/protobuf/struct.proto_ in your proto file then the response will be always _404 Unimplemented_

1 and 3 are fixed. See preview version:
1.6.3-ci-19205

StefH · 2024-09-26T06:32:42Z

Hello @carlos-pilao-deltatre / @ArcaneTSGK ; did you have time to validate that latest preview version?

samir-babachikj-deltatre · 2024-10-09T10:59:45Z

Hello @StefH ,
The preview version '1.6.3-ci-19205' is currently unavailable.

StefH · 2024-10-09T17:44:32Z

@samir-babachikj-deltatre
@ArcaneTSGK
@carlos-pilao-deltatre

You can use version 1.6.6-ci-19355.

Preview versions are defined on MyGet.
See https://github.com/WireMock-Net/WireMock.Net/wiki/MyGet-preview-versions

samir-babachikj-deltatre · 2024-10-10T07:53:50Z

@StefH I only have these versions available:

StefH · 2024-10-12T09:35:19Z

@samir-babachikj-deltatre

Use this version
1.6.6-ci-19363

samir-babachikj-deltatre · 2024-10-12T13:41:38Z

@StefH The version you mentioned depends on: WireMock.Net.Abstractions 1.6.6-ci-19363, WireMock.Net.OpenApiParser 1.6.6-ci-19363 and WireMock.Org.Abstractions 1.6.6-ci-19363.
Due this reason, I am getting the NU1603 warning (ref. https://learn.microsoft.com/en-us/nuget/reference/errors-and-warnings/nu1603), thus it was resolving it to 1.6.6 stable. I instructed my project not to treat this warning as error, which would then result with an exception with the following message: "System.TypeLoadException : Could not load type 'WireMock.Models.IdOrTexts' from assembly 'WireMock.Net.Abstractions, Version=1.6.6.0, Culture=neutral, PublicKeyToken=c8d65537854e1f03'." Making me unable to properly test it.

StefH · 2024-10-12T14:12:29Z

I deleted all packages from MyGet and ran a new build.

Now only 1.6.6-ci-19365 should be there.

samir-babachikj-deltatre · 2024-10-14T08:57:15Z

@StefH Tried now, and at least for
3. If you have _import google/protobuf/struct.proto_ in your proto file then the response will be always _404 Unimplemented_
, I can confirm that the problem still persists.

StefH · 2024-10-14T09:06:59Z

That's very strange.
Using
import "google/protobuf/empty.proto";
or
import "google/protobuf/struct.proto";

Will actually be the same logic, both are well defined google elements.

Using this proto:

syntax = "proto3";

package greet;

import "google/protobuf/empty.proto";
import "google/protobuf/struct.proto";

service Greeter {
  rpc Nothing (google.protobuf.Empty) returns (google.protobuf.Empty);
  
  rpc SayHello (HelloRequest) returns (HelloReply);

  rpc SayOther (Other) returns (HelloReply);
}

message HelloRequest {
  string name = 1;
  NullValue x = 2;
}

message Other {
  string name = 1;
  NullValue x = 2;
}

message HelloReply {
  string message = 1;
}

message Person {
  string name = 1;
  int32 id = 2;
  string email = 3;
}

In https://protobuf.heyenrath.nl/
Will just work fine:

Can you please provide your .proto file? which has an issue ?

samir-babachikj-deltatre · 2024-10-14T09:13:49Z

@StefH The problem is the use of the Value type defined inside the struct.proto (ref. https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/struct.proto).
I see you are using NullValue, but it is an enum, so if you enrich your message with the use of google.protobuf.Value type, then the issue appears.

StefH · 2024-10-14T20:45:52Z

Can you try 1.6.6-ci-19366 ?

samir-babachikj-deltatre · 2024-10-15T19:20:59Z

@StefH It works now. Thank you!

samir-babachikj-deltatre · 2024-10-16T10:58:36Z

Hi @StefH ,
currently I am encountering a different issue.
It is not throwing exceptions now, but I am not able to mock the struct.Value field.
I am trying to mock it to ExpandoObject, but I am not having success with it, as it returns null always.
Can you please check that?

StefH · 2024-10-16T13:36:05Z

Yes, I think I made a mistake in that Value or Struct. I'll take a look.

StefH · 2024-10-17T09:45:45Z

@samir-babachikj-deltatre
if possible, can you provide the code / protobuf you are using?

StefH · 2024-10-17T11:47:14Z

@samir-babachikj-deltatre
Can you try 1.6.6-ci-19385 ?

note that you need to use a Value this:

samir-babachikj-deltatre · 2024-10-17T16:11:02Z

@StefH It is working as expected now! Thank you🙏

StefH · 2024-10-17T16:21:04Z

#1198

carlos-pilao-deltatre · 2024-11-17T09:59:18Z

@StefH the original issue of this thread still happening: if you have a service returning an empty response you cannot mock it. Tried several approaches like @ArcaneTSGK did without success. The only way I found so far is to create a custom response like VoidResponse and I need to have a field at least on it because if I return a structure without fields it will not work either :(
@ArcaneTSGK did you find any solution for this?

StefH · 2024-11-17T10:26:34Z

It should work correctly.

Can you please create a new issue with example code?

ArcaneTSGK added the bug label Jul 24, 2024

StefH self-assigned this Jul 24, 2024

StefH changed the title ~~google.protobuf.Empty results in a Bad gRPC response.~~ Using google.protobuf.Empty as response results in a bad gRPC response Jul 24, 2024

StefH mentioned this issue Aug 8, 2024

Grpc support for multiple proto files #1153

Closed

StefH mentioned this issue Aug 22, 2024

Use latest ProtoBufJsonConverter to support WellKnownTypes #1161

Merged

StefH closed this as completed Oct 15, 2024

StefH reopened this Oct 16, 2024

StefH closed this as completed Oct 17, 2024

Using google.protobuf.Empty as response results in a bad gRPC response #1144

Using google.protobuf.Empty as response results in a bad gRPC response #1144

Comments

ArcaneTSGK commented Jul 24, 2024 • edited by StefH Loading

Expected behavior:

Test to reproduce

Other related info

StefH commented Jul 24, 2024

StefH commented Jul 29, 2024

ArcaneTSGK commented Jul 29, 2024 • edited by StefH Loading

StefH commented Jul 30, 2024 • edited Loading

StefH commented Aug 4, 2024

StefH commented Aug 4, 2024

StefH commented Aug 15, 2024

ArcaneTSGK commented Aug 15, 2024

StefH commented Aug 22, 2024

StefH commented Aug 27, 2024

ArcaneTSGK commented Aug 31, 2024

StefH commented Aug 31, 2024

ArcaneTSGK commented Aug 31, 2024

StefH commented Aug 31, 2024

ArcaneTSGK commented Sep 2, 2024

StefH commented Sep 2, 2024

StefH commented Sep 2, 2024

ArcaneTSGK commented Sep 3, 2024

StefH commented Sep 5, 2024

StefH commented Sep 5, 2024

StefH commented Sep 11, 2024

StefH commented Sep 17, 2024

carlos-pilao-deltatre commented Sep 19, 2024 • edited Loading

StefH commented Sep 20, 2024

carlos-pilao-deltatre commented Sep 20, 2024

StefH commented Sep 20, 2024

carlos-pilao-deltatre commented Sep 20, 2024

StefH commented Sep 22, 2024

carlos-pilao-deltatre commented Sep 23, 2024

StefH commented Sep 23, 2024 • edited Loading

StefH commented Sep 26, 2024

samir-babachikj-deltatre commented Oct 9, 2024 • edited Loading

StefH commented Oct 9, 2024

samir-babachikj-deltatre commented Oct 10, 2024

StefH commented Oct 12, 2024

samir-babachikj-deltatre commented Oct 12, 2024

StefH commented Oct 12, 2024

samir-babachikj-deltatre commented Oct 14, 2024

StefH commented Oct 14, 2024

samir-babachikj-deltatre commented Oct 14, 2024

StefH commented Oct 14, 2024

samir-babachikj-deltatre commented Oct 15, 2024

samir-babachikj-deltatre commented Oct 16, 2024

StefH commented Oct 16, 2024 • edited Loading

StefH commented Oct 17, 2024

StefH commented Oct 17, 2024

samir-babachikj-deltatre commented Oct 17, 2024

StefH commented Oct 17, 2024

carlos-pilao-deltatre commented Nov 17, 2024 • edited Loading

StefH commented Nov 17, 2024

ArcaneTSGK commented Jul 24, 2024 •

edited by StefH

Loading

ArcaneTSGK commented Jul 29, 2024 •

edited by StefH

Loading

StefH commented Jul 30, 2024 •

edited

Loading

carlos-pilao-deltatre commented Sep 19, 2024 •

edited

Loading

StefH commented Sep 23, 2024 •

edited

Loading

samir-babachikj-deltatre commented Oct 9, 2024 •

edited

Loading

StefH commented Oct 16, 2024 •

edited

Loading

carlos-pilao-deltatre commented Nov 17, 2024 •

edited

Loading