NPE in raft.removeNode(mcc.msg.NodeID)

[tbg]

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x8 pc=0x4671397]

goroutine 4154 [running]:
github.com/coreos/etcd/raft.(*multiNode).run(0xc820463020)
        /Users/tschottdorf/go/src/github.com/coreos/etcd/raft/multinode.go:253 +0x14e7
created by github.com/coreos/etcd/raft.StartMultiNode
        /Users/tschottdorf/go/src/github.com/coreos/etcd/raft/multinode.go:56 +0x30d

offending code:

            case pb.ConfChangeRemoveNode:
                group.raft.removeNode(mcc.msg.NodeID)

in (not raft-pertinent) WIP freshly rebased on top of master at time of writing. Can't reproduce.

[mrtracy]

Ben opened a few pull requests against etcd last week for similar NPEs:

etcd-io/etcd#3396
etcd-io/etcd#3404

These all related to configuration changes.

[bdarnell]

Probably too late now, but do you have any more information? Which test case was this panic in? Do you have any more of the logs?

I suspect it's a race involving Store.RemoveReplica (in TestReplicateAddAndRemove), which might put this failure in the same category as #1911.

[bdarnell]

Just went back to the logs in #1911; this is an exact dupe.