Performance implications of using the grpc-gateway for a REST API

[cyantarek]

Hi, I'm using the grpc-gateway "runtime" package to expose REST endpoint from proto buffers.
When I deployed to production, I observed very poor performance, something like 400 req/s.

So I started benchmarking in localhost using "bombardier" as benchmarking tool. Here's the result:

Bombarding http://localhost:5400/v1/link/ping with 1000000 request(s) using 100 connection(s)
 1000000 / 1000000 [===============] 100.00% 58s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     17102.51    1810.68   25905.78
  Latency        5.85ms     1.00ms    59.33ms
  HTTP codes:
    1xx - 0, 2xx - 1000000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     6.31MB/s

No database or I/O calls, just ping response by calling the service layer function. Still the throughput is very low.

So I wrote my custom HTTP endpoint that will call the same service layer functions that the gRPC function is using. And now look at the benchmark:

Bombarding http://localhost:5400/v1/link/ping with 1000000 request(s) using 100 connection(s)
 1000000 / 1000000 [=============] 100.00% 7s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec    147464.89   22618.44  190638.47
  Latency      677.75us   551.27us    48.33ms
  HTTP codes:
    1xx - 0, 2xx - 1000000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    35.51MB/s%

Boom! 6x faster than using the grpc-gateway "runtime" package.

I went further down into the rabbit hole. the grpc gateway runtime package has a http mux frontend that receives request from HTTP client and then calls the grpc server as a client.

So I try to simulate it manually. In my code, instead of calling the service layer function from the HTTP endpoint handler, I call the grpc server as a client.

And now the benchmark looks like this:

Bombarding http://localhost:5400/v1/link/ping with 1000000 request(s) using 100 connection(s)
 1000000 / 1000000 [==================] 100.00% 28s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     35193.17    3242.29   46530.99
  Latency        2.84ms   459.67us    53.38ms
  HTTP codes:
    1xx - 0, 2xx - 1000000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     8.59MB/s%

Still not great, but its 2x better (35k req/s) than grpc gateway runtime (17k req/s)

So, gRPC gateway uses this flow:

HTTP Request -> REST Handler gRPC Client -> Call gRPC Server -> Returns response to the REST Handler gRPC Client -> Returns to the original caller

It looks like this is not the most efficient way to put a REST frontend before the gRPC server.

Anyway, I think you guys should take a look at this issue. Or maybe I'm doing something wrong

[johanbrandhorst]

Hi Cyan, thanks for raising this issue! Performance measurements aren't something we've been great at, I think the assumption has mostly been that using well known patterns will give us the best performance, but if you'd like to dig into why you're seeing these numbers we'd be happy to make any changes you propose.

Have you tried benchmarking the RegisterXXXHandlerFromService interface? That skips the gRPC roundtrip altogether and should be higher throughput, though it comes with other compromises.

[cyantarek]

Where can I find 'RegisterXXXHandlerFromService" this interface?

It doesn't exist in the *.pb.gw.go file

[johanbrandhorst]

Sorry, it will be something like this:

func RegisterGreeterHandlerClient(ctx context.Context, mux *runtime.ServeMux, client GreeterClient) error {

[cyantarek]

client GreeterClient

So it needs client, which means dialing to the gRPC server to initiate a gRPC client? If yes, then it's using the network layer again. So how does it skip the RoundTripping?

Can you please look at the grpcweb-proxy implementation? They don't use network layer to proxy pass the HTTP request to gRPC request

[johanbrandhorst]

No, it depends on your generated types what it's called, but it expects the same interface that you use to register a server with gRPC.

[cyantarek]

I've changed the code as you said:

func (h *HttpRestTransport) Register(client link.LinkClient) error {
	return link.RegisterLinkHandlerClient(context.Background(), h.gMux, client)
}

No change, still the same poor performance:

Bombarding http://localhost:5400/v1/link/ping with 100000 request(s) using 100 connection(s)
 100000 / 100000 [===============================] 100.00% 6s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     16994.47    2095.98   26177.33
  Latency        5.89ms     1.86ms    42.72ms
  HTTP codes:
    1xx - 0, 2xx - 100000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     6.01MB/s

[johanbrandhorst]

Sorry, this is what I meant:

https://github.com/grpc-ecosystem/grpc-gateway/blob/master/examples/internal/helloworld/helloworld.pb.gw.go#L758

RegisterXXXHandlerServer

[cyantarek]

Perfect! Now it's upto the mark. I think the README or doc should explicitly mention this issues

Bombarding http://localhost:5400/v1/link/ping with 1000000 request(s) using 250 connection(s)
 1000000 / 1000000 [=====================================] 100.00% 7s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec    132807.69   15299.24  161072.03
  Latency        1.87ms     1.63ms    88.94ms
  HTTP codes:
    1xx - 0, 2xx - 1000000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    41.50MB/s

[cyantarek]

Heck, no gRPC interceptors support with this method? :( @johanbrandhorst

[johanbrandhorst]

There's a comment warning of this when you use this method. It also doesn't support streaming right now. The reason there's no interceptor support is because it's not using gRPC at all. If you want interceptor functionality, you have to wrap the grpc-gateway mux in HTTP middleware. Eventually we hope to be able to support a grpc-in-process transport, which should help this sort of situation immensely, but that doesn't exist yet, see grpc/grpc-go#906. You can use third-party solutions mentioned in that thread though.

[cyantarek]

Yeah I've seen the comment on code. And it's understandable too why it is like that.

[seriouspoop]

If you want interceptor functionality, you have to wrap the grpc-gateway mux in HTTP middleware.

@johanbrandhorst Do we have interceptor functionality available now? If not can you please provide an example of the above method.

[johanbrandhorst]

See https://www.alexedwards.net/blog/making-and-using-middleware for the general Go HTTP middleware pattern. Because the runtime.ServeMux is a http.Handler, it can be wrapped as in this example.