schedule pendding online chunkserver #252

cw123 · 2021-02-03T03:30:16Z

What is changed and how it works?

What's Changed:
schedule out copyset from online and pendding chunkserver

How it Works:
schedule out copyset from online and pendding chunkserver

Side effects(Breaking backward compatibility? Performance regression?): no

Check List

Relevant documentation/comments is changed or added
I acknowledge that all my contributions will be made under the project's license

cw123 · 2021-02-04T11:22:06Z

recheck

cw123 · 2021-02-04T11:32:14Z

recheck

ilixiaocui · 2021-03-02T08:37:09Z

src/mds/schedule/copySetScheduler.cpp

@@ -38,27 +38,84 @@ int CopySetScheduler::Schedule() {
    for (auto lid : topo_->GetLogicalpools()) {
        res = DoCopySetSchedule(lid);
    }
+
+    LOG(INFO) << "copysetScheduler end, generate operator " << res;


format of commit message and title should include module name:
chunkserver: schedule pendding online chunkserver

ilixiaocui · 2021-03-02T08:46:01Z

test/mds/schedule/schedulerPOC/scheduler_poc.cpp

+}
+
+TEST_F(CopysetSchedulerPOC, test_scatterwith_after_copysetRebalance_5) { //NOLINT
+    // 测试两个online的chunkserver 标记为pendding，copyset迁移走


ilixiaocui · 2021-03-02T09:05:29Z

src/mds/schedule/copySetScheduler.cpp

-    if (distribute.empty()) {
-        LOG(WARNING) << "no not-retired chunkserver in topology";
-        return UNINTIALIZE_ID;
+int CopySetScheduler::PenddingCopySetSchedule(const std::map<ChunkServerIdType,


Is it more reasonable to schedule chunkserver in pendding status by recoverSchedule?

RecorveSchedule handle the copyset which aready lost at least one copy. CopysetScheduler handle the copyset which has all copys.

ilixiaocui · 2021-03-02T09:07:55Z

src/mds/schedule/copySetScheduler.cpp

+    auto chunkserverList = topo_->GetChunkServersInLogicalPool(lid);
+
+    std::map<ChunkServerIdType, std::vector<CopySetInfo>> penddingDistribute;
+    SchedulerHelper::CopySetDistributionInOnlinePenddingChunkServer(


chunkserver in pendding status do not need sort?

It has no necessary to sort in this situation. The purpose is to change peer from pendding chunkserver to online and no pendding chunkserver, the sequence of change peer operator is less important.

ilixiaocui · 2021-03-02T09:27:05Z

src/mds/schedule/scheduler_helper.cpp

@@ -379,6 +379,69 @@ void SchedulerHelper::CopySetDistributionInOnlineChunkServer(
        }
    }
 }
+
+void SchedulerHelper::CopySetDistributionInOnlinePenddingChunkServer(


CopySetDistributionInOnlinePenddingChunkServer is similar with CopySetDistributionInOnlineNormalChunkServer, It is recommended to distinguish status by parameter. Otherwise, it is easy to forget to modify later

void SchedulerHelper::CopySetDistributionChunkServer(
const Status status,
const std::vector &copysetList,
const std::vector &chunkserverList,
std::map<ChunkServerIdType, std::vector> *out) {

ilixiaocui · 2021-03-03T02:19:51Z

src/mds/schedule/leaderScheduler.cpp

+        if (csInfo.IsPendding()) {
+            continue;
+        }
+


if (csInfo.IsOffline() || csInfo.IsPendding()) {
continue;
}

ilixiaocui · 2021-03-03T02:20:50Z

src/mds/schedule/leaderScheduler.cpp

+            // can not transfer to pendding chunkserver
+            if (csInfo.IsPendding()) {
+                continue;
+            }


if (csInfo.IsOffline() || csInfo.IsPendding()) {
continue;
}

The case between offline and pendding is different. If the chunk is offline, it break this while. If the chunk is pendding, it skip this round with continue.

ilixiaocui · 2021-03-03T02:25:29Z

src/mds/schedule/scheduler.cpp

@@ -289,9 +290,16 @@ ChunkServerIdType Scheduler::SelectRedundantReplicaToRemove(
                         << cs << " from " << copySetInfo.CopySetInfoStr();
            return cs;
        }
+
+        // chunkserver is not pendding


chunkserver is pendding？

ilixiaocui · 2021-03-03T02:39:17Z

test/mds/schedule/schedulerPOC/scheduler_poc.cpp

-        ApplyOperatorsInOpController(std::set<ChunkServerIdType>{removeOne});
-    } while (removeOne > 0);
+        operatorCount = copySetScheduler_->Schedule();
+        ApplyOperatorsInOpController(csSet);


This node is not necessarily removed

The copySetScheduler_->Schedule() returns the operator count rather then the chunkserver want to be removed. So the ApplyOperatorsInOpController use all chunkserver list instead of the removeOne.

ilixiaocui · 2021-03-03T02:39:36Z

test/mds/schedule/schedulerPOC/scheduler_poc.cpp

@@ -1217,6 +1229,61 @@ TEST_F(CopysetSchedulerPOC, test_scatterwith_after_copysetRebalance_3) { //NOLIN
    // 均值：100, 方差：1, 标准差： 1, 最大值： 101, 最小值：91
 }

+TEST_F(CopysetSchedulerPOC, test_scatterwith_after_copysetRebalance_4) { //NOLINT
+    // 测试一个online的chunkserver 标记为pendding，copyset迁移走


xu-chaojie · 2021-05-26T06:50:44Z

src/mds/schedule/scheduler_helper.cpp

@@ -379,6 +379,43 @@ void SchedulerHelper::CopySetDistributionInOnlineChunkServer(
        }
    }
 }
+
+void SchedulerHelper::CopySetDistributionInOnlineChunkServer(


Function naming is inappropriate, use verbs to name it

ilixiaocui · 2021-05-31T06:19:18Z

src/mds/schedule/copySetScheduler.cpp

+            continue;
+        }
+
+        // find one copy set to migrate out from source chunkserver


ilixiaocui · 2021-05-31T06:29:05Z

src/mds/schedule/copySetScheduler.cpp

+
+            if (AddOperatorAndCreateCopyset(op, info, target)) {
+                oneRoundGenOp++;
+                break;


why break? One record at a time for a chunkserver? It Should be based on concurrency.

ilixiaocui · 2021-05-31T06:37:26Z

src/mds/schedule/scheduler_helper.cpp

+            continue;
+        }
+
+        if (item.status != status) {


Find the chunkserver with the same status, this judgment condition is enough.

cw123 · 2021-06-07T03:08:54Z

recheck

cw123 · 2021-06-08T06:05:44Z

recheck

cw123 · 2021-06-08T06:51:51Z

recheck

ilixiaocui · 2021-06-08T09:17:09Z

src/mds/schedule/scheduler_helper.cpp

+            distributions->erase(item.info.id);
+        }
+
+        if (item.status == ChunkServerStatus::PENDDING


do not need

Signed-off-by: Fabian Deutsch <fabiand@fedoraproject.org>

ilixiaocui reviewed Mar 3, 2021

View reviewed changes

cw123 force-pushed the schedule_out_date branch 2 times, most recently from 082fd16 to 09a3763 Compare March 16, 2021 07:49

xu-chaojie reviewed May 26, 2021

View reviewed changes

ilixiaocui reviewed May 31, 2021

View reviewed changes

cw123 force-pushed the schedule_out_date branch 2 times, most recently from dd772ff to 390055a Compare June 3, 2021 07:36

cw123 force-pushed the schedule_out_date branch from 390055a to bedb694 Compare June 8, 2021 02:34

schedule pendding online chunkserver

a9270a8

cw123 force-pushed the schedule_out_date branch from bedb694 to a9270a8 Compare June 8, 2021 09:48

ilixiaocui reviewed Jun 10, 2021

View reviewed changes

ilixiaocui approved these changes Jun 10, 2021

View reviewed changes

ilixiaocui merged commit 563c137 into opencurve:master Jun 10, 2021

cw123 deleted the schedule_out_date branch June 10, 2021 02:36

ilixiaocui pushed a commit to ilixiaocui/curve that referenced this pull request Feb 6, 2023

kubevirt: Fix indentation (opencurve#252)

e4d2d3e

Signed-off-by: Fabian Deutsch <fabiand@fedoraproject.org>

schedule pendding online chunkserver #252

schedule pendding online chunkserver #252

Conversation

cw123 commented Feb 3, 2021

What is changed and how it works?

Check List

cw123 commented Feb 4, 2021

cw123 commented Feb 4, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cw123 commented Jun 7, 2021

cw123 commented Jun 8, 2021

cw123 commented Jun 8, 2021

Choose a reason for hiding this comment