Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve heartbeat of storage. #1173

Closed
critical27 opened this issue Oct 30, 2019 · 3 comments · Fixed by #1236
Closed

Improve heartbeat of storage. #1173

critical27 opened this issue Oct 30, 2019 · 3 comments · Fixed by #1236
Assignees
@zhangguoqing @critical27
Labels
type/enhancement Type: make the code neat or more efficient

Comments

@critical27
Copy link
Contributor

We can add more info in heartbeat from storage to meta, such as leader/follower/learner count and so on. So we can avoid block in meta service.

After #1086 merged in, I will change the interface.
@critical27 critical27 added the type/enhancement Type: make the code neat or more efficient label Oct 30, 2019
@critical27 critical27 self-assigned this Oct 30, 2019
@zhangguoqing
Copy link
Contributor

good idea!

@dangleptr
Copy link
Contributor

Close #1060

@dangleptr
Copy link
Contributor

Besides that, we could optimize the load data logic inside meta client.

@zhangguoqing zhangguoqing mentioned this issue Nov 7, 2019
Closed
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Nov 8, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
f388b17 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class.

close vesoft-inc#1173
close vesoft-inc#1060
@zhangguoqing zhangguoqing mentioned this issue Nov 8, 2019
Merged
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Nov 11, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
9484ec5 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class.

close vesoft-inc#1173
close vesoft-inc#1060
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Nov 12, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
9ea9c31 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   isStoraged_ to indicate whether it is a metaclient in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Nov 13, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
4745138 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Nov 14, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
e708df7 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Nov 19, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
5b343c0 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Nov 22, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
7a30d44 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Nov 22, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
d43549b 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Nov 22, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
a93774d 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 4, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
da7bea6 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 6, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
7fd5db8 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 12, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
2805c4b 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 13, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
cfdc90b 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 16, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
cf45825 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 17, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
5e554d6 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 19, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
a516234 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 20, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
f38c412 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 23, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
bb4406c 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 25, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
1b77b7c 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 27, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
1affef5 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
zhangguoqing added a commit to zhangguoqing/nebula that referenced this issue Dec 27, 2019
@zhangguoqing
Improve heartbeat between metad and storaged by MetaClient
1f39faa 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284
dangleptr added a commit that referenced this issue Dec 27, 2019
@zhangguoqing @dangleptr
Improve heartbeat between metad and storaged by MetaClient (#1236)
527c6cd 
1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close #1173
close #1060
close #284

Co-authored-by: dangleptr <37216992+dangleptr@users.noreply.github.com>
tong-hao pushed a commit to tong-hao/nebula that referenced this issue Jun 1, 2021
@zhangguoqing @dangleptr
Improve heartbeat between metad and storaged by MetaClient (vesoft-in…
8992fa4 
…c#1236)

1. Add the LastUpdateTimeMan to record the latest metadata change(insert, update, delete)
   time on the MetaServer side.
2. Enrich the information(lastUpdateTime, partitions leaderDist) of `heartbeat` exchange
   between MetaServer and MetaClient at storage node.
3. It could optimize the load data logic inside meta client by lastUpdateTime, which makes
   metadata synchronization smarter and more timely.
4. Remove sendHeartBeat_ and load_data_interval_secs from MetaClient class, and add the
   bool inStoraged_ to indicate whether a metaclient is in storaged.

close vesoft-inc#1173
close vesoft-inc#1060
close vesoft-inc#284

Co-authored-by: dangleptr <37216992+dangleptr@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zhangguoqing zhangguoqing

@critical27 critical27

Labels
type/enhancement Type: make the code neat or more efficient
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Improve heartbeat between metad and storaged by MetaClient zhangguoqing/nebula
3 participants
@zhangguoqing @critical27 @dangleptr