Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure .proto file name is globally unique #125

Merged
merged 1 commit into from
May 21, 2023

Conversation

ejweber
Copy link
Contributor

@ejweber ejweber commented May 17, 2023

In service of longhorn/longhorn#5595.

Longhorn-manager has a BackingImageManagerService client, and thus imports backing-image-manager. After upgrading the Kubernetes dependency of longhorn-manager to v1.27.1, I came across this error when executing longhorn-manager:

panic: proto: file "rpc.proto" is already registered
See https://protobuf.dev/reference/go/faq#namespace-conflict


goroutine 1 [running]:
google.golang.org/protobuf/reflect/protoregistry.glob..func1({0x3ba6860?, 0xc000111bc0?}, {0x3ba6860?, 0xc000111bc0})
        /go/pkg/mod/google.golang.org/protobuf@v1.30.0/reflect/protoregistry/registry.go:56 +0x266
google.golang.org/protobuf/reflect/protoregistry.(*Files).RegisterFile(0xc0001aa8a0, {0x3bdf0d8?, 0xc000452540?})
        /go/pkg/mod/google.golang.org/protobuf@v1.30.0/reflect/protoregistry/registry.go:130 +0x502
google.golang.org/protobuf/internal/filedesc.Builder.Build({{0x0, 0x0}, {0xc000803c00, 0xa8d, 0xc00}, 0x0, 0xd, 0x0, 0x1, {0x3bade58, ...}, ...})
        /go/pkg/mod/google.golang.org/protobuf@v1.30.0/internal/filedesc/build.go:112 +0x26b
github.com/golang/protobuf/proto.RegisterFile({0x36688d9, 0x9}, {0x4f8be80, 0x34d, 0x34d})
        /go/pkg/mod/github.com/golang/protobuf@v1.5.3/proto/registry.go:48 +0x187
github.com/longhorn/backing-image-manager/pkg/rpc.init.1()
        /go/pkg/mod/github.com/longhorn/backing-image-manager@v0.0.0-20220609065820-a08f7f47442f/pkg/rpc/rpc.pb.go:680 +0x50
FAIL    github.com/longhorn/longhorn-manager/controller 0.106s

After reading the referenced web page and doing some additional digging, I came across golang/protobuf#1122. For reasons debated in that issue, the name that feeds into the proto.RegisterFile function in any .pb.go file must be globally unique in a Go program. Unfortunately, the Kubernetes version update introduced a new transitive dependency on go.etcd.io/etcd/api/v3, which also has an rpc.proto file that conflicts with backing-image-manager's rpc.proto file. Since we can't easily change that one, it makes sense to more fully specify ours.

The entire purpose of this PR is to change this:
proto.RegisterFile("rpc.proto", fileDescriptor_77a6da22d6a3feb1),
to this:
proto.RegisterFile("github.com/longhorn/backing-image-manager/pkg/rpc/rpc.proto", fileDescriptor_cdf73a3350f27fd7).

Sadly, protoc makes this a bit annoying to do, resulting in the file copying I did in generate_grpc.sh.

Signed-off-by: Eric Weber <eric.weber@suse.com>
TMP_DIR="${TMP_DIR_BASE}/github.com/longhorn/backing-image-manager/pkg/rpc"
mkdir -p "${TMP_DIR}"
cp "${PKG_DIR}/rpc.proto" "${TMP_DIR}/rpc.proto"
protoc -I "${TMP_DIR_BASE}" -I "proto/vendor/protobuf/src/" "${TMP_DIR}/rpc.proto" --go_out=plugins=grpc:"${TMP_DIR_BASE}"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question. Is it ok to combine the import path like:

protoc -I "${TMP_DIR_BASE}" "proto/vendor/protobuf/src/" "${TMP_DIR}/rpc.proto" --go_out=plugins=grpc:"${TMP_DIR_BASE}"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like no

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spoke with @PhanLe1010 offline and we are agreed. I tried to find a good primary source for protoc (not protoc-gen-go) flags, but struggled. The protoc --help output provides the following direction:

  -IPATH, --proto_path=PATH   Specify the directory in which to search for
                              imports.  May be specified multiple times;
                              directories will be searched in order.  If not
                              given, the current working directory is used.
                              If not found in any of the these directories,
                              the --descriptor_set_in descriptors will be
                              checked for required proto file.

Copy link

@PhanLe1010 PhanLe1010 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the PR

@innobead innobead self-requested a review May 21, 2023 15:18
Copy link
Member

@innobead innobead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Suggest applying the same way to other repos if needed.

cc @longhorn/dev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants