Introduce Replication unit tests using real shards #18930
Conversation
This commit introduce unit testing infrastructure to test replication operations using real index shards. This is infra is complimentary to the full integration tests and unit testing of ReplicationOperation we already have. The new ESIndexLevelReplicationTestCase base makes it easier to test and simulate failure mode that require real shards and but do not need the full blow stack of a complete node. The commit also add a simple "nothing is wrong" test plus a test that checks we don't drop docs during the various stages of recovery. For now, only single doc indexing is supported but this can be easily extended in the future.
| boolean executeOnReplicas, boolean checkWriteConsistency, | ||
| Replicas<ReplicaRequest> replicas, | ||
| Supplier<ClusterState> clusterStateSupplier, ESLogger logger, String opType) { | ||
| public ReplicationOperation(Request request, Primary<Request, ReplicaRequest, PrimaryResultT> primary, |
There was a problem hiding this comment.
instead of making it public maybe we move this entire package into core/src/test/java/org/elasticsearch/index/replication/ or even move everything into core/src/test/java/org/elasticsearch/index/
There was a problem hiding this comment.
we could, problem is that ReplicationOperation is currently part of the universe of actions and is also used by TransportReplicationAction (which is subclassed by TransportIndexAction, for example) - so moving it will cause problems with those actions. IMO it's not a big deal to make this public?
There was a problem hiding this comment.
sure - I just think we can move all of them in one package :)
|
I left a bunch of comments, but hey lets push this it's really such an improvement over what we have, we should not wait! LGTM |
|
@s1monw I pushed another commit |
|
looks good - get it in! |
This commit introduce unit testing infrastructure to test replication operations using real index shards. This is infra is complementary to the full integration tests and unit testing of ReplicationOperation we already have. The new ESIndexLevelReplicationTestCase base makes it easier to test and simulate failure mode that require real shards and but do not need the full blow stack of a complete node.
The commit also add a simple "nothing is wrong" test plus a test that checks we don't drop docs during the various stages of recovery.
For now, only single doc indexing is supported but this can be easily extended in the future.
This is still rough around the edges but I think it's in good enough shape to get feedback. @s1monw @ywelsch @jasontedor your input is welcome