Replace prototool with buf

[johanbrandhorst]

Changes

Replace prototool generation and linting with buf

Verification

Reran generation and lint locally. Need to see what CI thinks about the changes.

[johanbrandhorst]

These ignores are based on the existing failures in this file. These are the warnings:

$ buf lint
integration_test/proto/improbable/grpcweb/test/test.proto:3:1:Package name "improbable.grpcweb.test" should be suffixed with a correctly formed version, such as "improbable.grpcweb.test.v1".
integration_test/proto/improbable/grpcweb/test/test.proto:10:5:Enum value name "NONE" should be prefixed with "FAILURE_TYPE_".
integration_test/proto/improbable/grpcweb/test/test.proto:10:5:Enum zero value name "NONE" should be suffixed with "_UNSPECIFIED".
integration_test/proto/improbable/grpcweb/test/test.proto:11:5:Enum value name "CODE" should be prefixed with "FAILURE_TYPE_".
integration_test/proto/improbable/grpcweb/test/test.proto:13:5:Enum value name "CODE_UNICODE" should be prefixed with "FAILURE_TYPE_".
integration_test/proto/improbable/grpcweb/test/test.proto:28:10:Field name "Value" should be lower_snake_case, such as "value".
integration_test/proto/improbable/grpcweb/test/test.proto:39:3:".google.protobuf.Empty" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:39:3:".improbable.grpcweb.test.PingResponse" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:39:17:RPC request type "Empty" should be named "PingEmptyRequest" or "TestServicePingEmptyRequest".
integration_test/proto/improbable/grpcweb/test/test.proto:39:49:RPC response type "PingResponse" should be named "PingEmptyResponse" or "TestServicePingEmptyResponse".
integration_test/proto/improbable/grpcweb/test/test.proto:41:3:".improbable.grpcweb.test.PingRequest" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:41:3:".improbable.grpcweb.test.PingResponse" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:43:3:".google.protobuf.Empty" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:43:3:".improbable.grpcweb.test.PingRequest" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:43:17:RPC request type "PingRequest" should be named "PingErrorRequest" or "TestServicePingErrorRequest".
integration_test/proto/improbable/grpcweb/test/test.proto:43:39:RPC response type "Empty" should be named "PingErrorResponse" or "TestServicePingErrorResponse".
integration_test/proto/improbable/grpcweb/test/test.proto:45:3:".improbable.grpcweb.test.PingRequest" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:45:3:".improbable.grpcweb.test.PingResponse" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:45:16:RPC request type "PingRequest" should be named "PingListRequest" or "TestServicePingListRequest".
integration_test/proto/improbable/grpcweb/test/test.proto:45:45:RPC response type "PingResponse" should be named "PingListResponse" or "TestServicePingListResponse".
integration_test/proto/improbable/grpcweb/test/test.proto:47:3:".improbable.grpcweb.test.PingRequest" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:47:3:".improbable.grpcweb.test.PingResponse" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:47:27:RPC request type "PingRequest" should be named "PingPongBidiRequest" or "TestServicePingPongBidiRequest".
integration_test/proto/improbable/grpcweb/test/test.proto:47:56:RPC response type "PingResponse" should be named "PingPongBidiResponse" or "TestServicePingPongBidiResponse".
integration_test/proto/improbable/grpcweb/test/test.proto:49:3:".improbable.grpcweb.test.PingRequest" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:49:3:".improbable.grpcweb.test.PingResponse" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:49:25:RPC request type "PingRequest" should be named "PingStreamRequest" or "TestServicePingStreamRequest".
integration_test/proto/improbable/grpcweb/test/test.proto:49:47:RPC response type "PingResponse" should be named "PingStreamResponse" or "TestServicePingStreamResponse".
integration_test/proto/improbable/grpcweb/test/test.proto:51:3:RPC "Echo" has the same type ".improbable.grpcweb.test.TextMessage" for the request and response.
integration_test/proto/improbable/grpcweb/test/test.proto:51:12:RPC request type "TextMessage" should be named "EchoRequest" or "TestServiceEchoRequest".
integration_test/proto/improbable/grpcweb/test/test.proto:51:34:RPC response type "TextMessage" should be named "EchoResponse" or "TestServiceEchoResponse".
integration_test/proto/improbable/grpcweb/test/test.proto:67:3:".google.protobuf.Empty" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:67:54:RPC response type "Empty" should be named "ContinueStreamResponse" or "TestUtilServiceContinueStreamResponse".
integration_test/proto/improbable/grpcweb/test/test.proto:73:3:".improbable.grpcweb.test.PingRequest" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:73:3:".improbable.grpcweb.test.PingResponse" is used as the request or response type for multiple RPCs.
integration_test/proto/improbable/grpcweb/test/test.proto:73:19:RPC request type "PingRequest" should be named "NonExistantRequest" or "FailServiceNonExistantRequest".
integration_test/proto/improbable/grpcweb/test/test.proto:73:41:RPC response type "PingResponse" should be named "NonExistantResponse" or "FailServiceNonExistantResponse".

[johanbrandhorst]

I didn't want to fix any warnings in this PR to make the impact as small as possible.

[johanbrandhorst]

Note that this doesn't use uname -s since protoc uses lowercase names. I figured since this is for CI it doesn't matter, but I can make probably do a lowercase transformation somehow if necessary.

[johanbrandhorst]

Tests that failed are giving me an npm error:

/tmp/.bash_env-601a8141e5331f7178d9a11c-0-build: line 2: /home/circleci/.nvm/nvm.sh: No such file or directory
/bin/bash: npm: command not found

No idea what's going on there, any ideas @MarcusLongmuir?

[johanbrandhorst]

Rerunning the tests now.

[MarcusLongmuir]

1. Do we still need to install these?

- run: go install ./vendor/github.com/golang/protobuf/protoc-gen-go
- run: export PATH=/go/src/github.com/improbable-eng/grpc-web/protobuf/bin:$PATH

2. What's the relationship between buf and protoc in this configuration? From my understanding buf doesn't require protoc, but it's optional.

[johanbrandhorst]

Do we still need to install these?

• run: go install ./vendor/github.com/golang/protobuf/protoc-gen-go
• run: export PATH=/go/src/github.com/improbable-eng/grpc-web/protobuf/bin:$PATH

We still have to install the protobuf plugins, as buf doesn't handle language specific generation. This is consistent with protoc when not looking at the built-in generators.

What's the relationship between buf and protoc in this configuration? From my understanding buf doesn't require protoc, but it's optional.

Indeed, buf doesn't depend on protoc, however, since we're generating using --js_out we need the JavaScript generator plugin, which is built into protoc, so that's the only reason we still have to download protoc.

[johanbrandhorst]

Bump

[johanbrandhorst]

Rebased on master and removed some old dep references I missed in the modules migration, please take a look when you can @MarcusLongmuir.

[MarcusLongmuir]

Thanks for updating this 👍

[johanbrandhorst]

Hm, not sure about the test errors, could it be flakes? The logic shouldn't have changed and all the other tests passing is suspicious.

[MarcusLongmuir]

Looks like the testserver is panicing with panic: rpc error: code = Unavailable desc = transport is closing. This is unrelated so it's safe to merge this.

[MarcusLongmuir]

You should be able to retry just the failing tests and merge when they pass, but comment if that doesn't work.

[johanbrandhorst]

Thanks for the quick review!