-
Notifications
You must be signed in to change notification settings - Fork 9.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
server: ignore raft messages if member id mismatch #17078
Conversation
Ignore Raft messages when the `To` field mismatches the local member ID. In cases where incorrect Raft messages are dispatched, potentially due to a malfunctioning switch, this proactive check prevents panics, such as "tocommit is out of range". Signed-off-by: Neil Shen <overvenus@gmail.com>
Hi @overvenus. Thanks for your PR. I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
High level the change looks good.
I also did some digging ito Process
validation.
It was introduced in #1343 (comment) with promised followups. Don't think they were realised though.
CCing the people involved with the original discussion to get their opinion:
cc @yichengq @jonboulle @xiang90
cc @ahrtr |
server/etcdserver/server_test.go
Outdated
// Mock a mad switch dispatching messages to wrong node. | ||
m := raftpb.Message{ | ||
Type: raftpb.MsgHeartbeat, | ||
To: 2, // Wrong ID. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It took me some time to get what's current member's ID. You do not explicitly specify a member ID for current member, so it's 0. Please let's explicitly set a member ID for current member to make it clearer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 9f82390, actually I want the server member ID to be 1.
Signed-off-by: Neil Shen <overvenus@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
We also need to backport the fix to both 3.4 and 3.5
Followup:
@overvenus could you please raise a ticket to track the todo items? |
Certainly! I've created ticket #17081. Feel free to review and comments. |
@overvenus @ahrtr |
Thanks for the reminder. Contribution is welcome for #17081 |
Hey @ahrtr I can take this backporting task :) |
…#17078 Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
…d-io#17078 Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
…d-io#17078 Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
Ignore Raft messages when the
To
field mismatches the local member ID. In cases where incorrect Raft messages are dispatched, potentially due to a malfunctioning switch, this proactive check prevents panics, such as "tocommit is out of range".Fix #16220
Fix #17081
Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.