filter: add label for filterCounter #3320

Yisaer · 2021-01-04T08:51:54Z

What problem does this PR solve?

What is changed and how it works?

Add label for filterCounter. If the Filter filter the target by comparing origin store and target store, we can record the origin store id into metrics.

Then we can query the metrics like this:

By this way, we can easily judge whether the filter is reasonable.

Check List

Tests

Unit test

Release note

No release note

Signed-off-by: Song Gao <disxiaofei@163.com>

codecov · 2021-01-04T09:35:14Z

Codecov Report

Merging #3320 (f6c42ce) into master (8eabcca) will increase coverage by 0.20%.
The diff coverage is 87.09%.

@@            Coverage Diff             @@
##           master    #3320      +/-   ##
==========================================
+ Coverage   74.76%   74.96%   +0.20%     
==========================================
  Files         243      243              
  Lines       23223    23246      +23     
==========================================
+ Hits        17363    17427      +64     
+ Misses       4283     4257      -26     
+ Partials     1577     1562      -15

Flag	Coverage Δ
unittests	`74.96% <87.09%> (+0.20%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
server/schedule/filter/metrics.go	`100.00% <ø> (ø)`
server/schedule/filter/filters.go	`90.61% <87.09%> (-0.38%)`	⬇️
pkg/etcdutil/etcdutil.go	`84.70% <0.00%> (-3.53%)`	⬇️
server/member/member.go	`67.93% <0.00%> (-0.55%)`	⬇️
server/tso/allocator_manager.go	`76.88% <0.00%> (+0.35%)`	⬆️
client/client.go	`65.67% <0.00%> (+0.37%)`	⬆️
server/cluster/cluster.go	`83.50% <0.00%> (+0.50%)`	⬆️
server/grpc_service.go	`57.42% <0.00%> (+0.83%)`	⬆️
server/server.go	`73.41% <0.00%> (+0.94%)`	⬆️
pkg/btree/btree.go	`86.26% <0.00%> (+0.96%)`	⬆️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8eabcca...f6c42ce. Read the comment docs.

Yisaer · 2021-01-05T04:36:34Z

@rleungx PTAL

lhy1024 · 2021-01-05T07:35:16Z

LGTM

Signed-off-by: Song Gao <disxiaofei@163.com>

rleungx · 2021-01-13T04:09:58Z

server/schedule/filter/filters.go

+type ComparingFilter interface {
+	Filter
+	// GetOldStoreID returns the source store when comparing.
+	GetOldStoreID() uint64


Why not source store

Signed-off-by: Song Gao <disxiaofei@163.com>

rleungx · 2021-01-13T04:55:20Z

server/schedule/filter/filters.go

@@ -166,6 +194,7 @@ type distinctScoreFilter struct {
 	stores    []*core.StoreInfo
 	policy    string
 	safeScore float64
+	oldStore  uint64


Suggested change

oldStore uint64

srcStore uint64

Signed-off-by: Song Gao <disxiaofei@163.com>

rleungx · 2021-01-13T09:32:32Z

BTW, I think we can add the metrics but hide it

lhy1024 · 2021-01-13T10:11:56Z

BTW, I think we can add the metrics but hide it

In some cases, we use tools to export metrics, which may not be convenient for later troubleshooting after the hidden monitoring

Yisaer · 2021-01-13T10:27:39Z

BTW, I think we can add the metrics but hide it

In some cases, we use tools to export metrics, which may not be convenient for later troubleshooting after the hidden monitoring

We can manually display the hidden panel, then expose the metrics.

Signed-off-by: Song Gao <disxiaofei@163.com>

metrics/grafana/pd.json

server/schedule/filter/filters.go

Signed-off-by: Song Gao <disxiaofei@163.com>

lhy1024 · 2021-01-14T06:46:19Z

/merge

ti-chi-bot · 2021-01-14T06:46:20Z

@lhy1024: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/ti-community-prow repository.

ti-chi-bot · 2021-01-14T06:46:23Z

This pull request has been accepted and is ready to merge.

Commit hash: 0876018

ti-chi-bot · 2021-01-14T06:47:17Z

Merge canceled because a new commit is pushed.

ti-chi-bot · 2021-01-14T06:47:19Z

@Yisaer: Your PR has out-of-dated, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/ti-community-prow repository.

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

ti-srebot · 2021-01-14T07:26:17Z

cherry pick to release-3.0 in PR #3358

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

ti-srebot · 2021-01-14T07:26:47Z

cherry pick to release-4.0 in PR #3359

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

ti-srebot · 2021-01-14T07:27:14Z

cherry pick to release-5.0-rc in PR #3360

* cherry pick #3320 to release-4.0 Signed-off-by: ti-srebot <ti-srebot@pingcap.com> * solve conflict Signed-off-by: Song Gao <disxiaofei@163.com> * fix lint Signed-off-by: Song Gao <disxiaofei@163.com> * fix error Signed-off-by: Song Gao <disxiaofei@163.com> * fix error Signed-off-by: Song Gao <disxiaofei@163.com> Co-authored-by: Song Gao <disxiaofei@163.com> Co-authored-by: ShuNing <nolouch@gmail.com>

Signed-off-by: ti-srebot <ti-srebot@pingcap.com> Co-authored-by: Song Gao <disxiaofei@163.com> Co-authored-by: Ti Chi Robot <71242396+ti-chi-bot@users.noreply.github.com>

Yisaer added 2 commits January 4, 2021 15:52

add comparing filter

71f745f

Signed-off-by: Song Gao <disxiaofei@163.com>

fix label

7e0ad16

Signed-off-by: Song Gao <disxiaofei@163.com>

ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 4, 2021

ti-chi-bot requested a review from rleungx January 4, 2021 08:52

fix metrics

7355ead

Signed-off-by: Song Gao <disxiaofei@163.com>

Yisaer changed the title ~~filter: support more metrics for filters~~ filter: add label for filterCounter Jan 4, 2021

support filters

f4dd0f7

Signed-off-by: Song Gao <disxiaofei@163.com>

Yisaer marked this pull request as ready for review January 5, 2021 10:55

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 5, 2021

Yisaer added needs-cherry-pick-release-3.0 The PR needs to cherry pick to release-3.0 branch. needs-cherry-pick-release-4.0 The PR needs to cherry pick to release-4.0 branch. needs-cherry-pick-5.0-rc component/metrics Metrics. labels Jan 5, 2021

address the comment

24e9bf7

Signed-off-by: Song Gao <disxiaofei@163.com>

Yisaer marked this pull request as draft January 11, 2021 02:08

ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 11, 2021

Yisaer mentioned this pull request Jan 12, 2021

scheduler, operator: support operatorLimitCounter for region-schedule-limit #3351

Merged

Yisaer marked this pull request as ready for review January 12, 2021 09:10

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 12, 2021

rleungx reviewed Jan 13, 2021

View reviewed changes

address the comment

276ac2b

Signed-off-by: Song Gao <disxiaofei@163.com>

rleungx reviewed Jan 13, 2021

View reviewed changes

address the comment

44c97bd

Signed-off-by: Song Gao <disxiaofei@163.com>

address the comment

063d699

Signed-off-by: Song Gao <disxiaofei@163.com>

rleungx reviewed Jan 14, 2021

View reviewed changes

metrics/grafana/pd.json Outdated Show resolved Hide resolved

rleungx reviewed Jan 14, 2021

View reviewed changes

server/schedule/filter/filters.go Outdated Show resolved Hide resolved

Yisaer added 3 commits January 14, 2021 12:28

address the comment

01cc54d

Signed-off-by: Song Gao <disxiaofei@163.com>

address the comment

493f504

Signed-off-by: Song Gao <disxiaofei@163.com>

address the comment

0876018

Signed-off-by: Song Gao <disxiaofei@163.com>

rleungx approved these changes Jan 14, 2021

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jan 14, 2021

lhy1024 approved these changes Jan 14, 2021

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jan 14, 2021

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jan 14, 2021

Merge branch 'master' into extend_filter

f6c42ce

ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Jan 14, 2021

Yisaer merged commit 5f447ae into tikv:master Jan 14, 2021

ti-srebot pushed a commit to ti-srebot/pd that referenced this pull request Jan 14, 2021

cherry pick tikv#3320 to release-3.0

e02b113

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

ti-srebot mentioned this pull request Jan 14, 2021

filter: add label for filterCounter (#3320) #3358

Closed

ti-srebot pushed a commit to ti-srebot/pd that referenced this pull request Jan 14, 2021

cherry pick tikv#3320 to release-4.0

effbac0

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

ti-srebot mentioned this pull request Jan 14, 2021

filter: add label for filterCounter (#3320) #3359

Merged

ti-srebot pushed a commit to ti-srebot/pd that referenced this pull request Jan 14, 2021

cherry pick tikv#3320 to release-5.0-rc

b4f242d

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>

ti-srebot mentioned this pull request Jan 14, 2021

filter: add label for filterCounter (#3320) #3360

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

filter: add label for filterCounter #3320

filter: add label for filterCounter #3320

Yisaer commented Jan 4, 2021 •

edited

Loading

codecov bot commented Jan 4, 2021 •

edited

Loading

Yisaer commented Jan 5, 2021

lhy1024 commented Jan 5, 2021

rleungx Jan 13, 2021

Yisaer Jan 13, 2021

rleungx Jan 13, 2021

Yisaer Jan 13, 2021

rleungx commented Jan 13, 2021

lhy1024 commented Jan 13, 2021

Yisaer commented Jan 13, 2021

lhy1024 commented Jan 14, 2021

ti-chi-bot commented Jan 14, 2021

ti-chi-bot commented Jan 14, 2021

ti-chi-bot commented Jan 14, 2021

ti-chi-bot commented Jan 14, 2021

ti-srebot commented Jan 14, 2021

ti-srebot commented Jan 14, 2021

ti-srebot commented Jan 14, 2021

filter: add label for filterCounter #3320

filter: add label for filterCounter #3320

Conversation

Yisaer commented Jan 4, 2021 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

codecov bot commented Jan 4, 2021 • edited Loading

Codecov Report

Yisaer commented Jan 5, 2021

lhy1024 commented Jan 5, 2021

rleungx Jan 13, 2021

Choose a reason for hiding this comment

Yisaer Jan 13, 2021

Choose a reason for hiding this comment

rleungx Jan 13, 2021

Choose a reason for hiding this comment

Yisaer Jan 13, 2021

Choose a reason for hiding this comment

rleungx commented Jan 13, 2021

lhy1024 commented Jan 13, 2021

Yisaer commented Jan 13, 2021

lhy1024 commented Jan 14, 2021

ti-chi-bot commented Jan 14, 2021

ti-chi-bot commented Jan 14, 2021

ti-chi-bot commented Jan 14, 2021

ti-chi-bot commented Jan 14, 2021

ti-srebot commented Jan 14, 2021

ti-srebot commented Jan 14, 2021

ti-srebot commented Jan 14, 2021

Yisaer commented Jan 4, 2021 •

edited

Loading

codecov bot commented Jan 4, 2021 •

edited

Loading