[WIP][feat]curvefs/client: warmup manager #2234
Conversation
1bc41e5 to
ed64f2e
Compare
676a3d1 to
70cbc2f
Compare
|
cicheck |
.clang-format
Outdated
| @@ -1,5 +1,6 @@ | |||
| --- | |||
| Language: Cpp | |||
| BasedOnStyle: Google | |||
There was a problem hiding this comment.
suggestions
---
Language: Cpp
BasedOnStyle: Google
AccessModifierOffset: -3
ContinuationIndentWidth: 8
DerivePointerAlignment: false
IndentWidth: 4
...
There was a problem hiding this comment.
suggestions
--- Language: Cpp BasedOnStyle: Google AccessModifierOffset: -3 ContinuationIndentWidth: 8 DerivePointerAlignment: false IndentWidth: 4 ...
fix
89364c6 to
dc8f4a6
Compare
|
cicheck |
|
cichck |
ff4e7b2 to
fd4e2f3
Compare
| LOG(ERROR) << "not support warmup type, only support single/list"; | ||
| ret = ERANGE; | ||
| switch (type) { | ||
| case curvefs::client::common::WarmupType::kWarmupTypeList: |
There was a problem hiding this comment.
indenet？
that's ok
| @@ -307,9 +325,10 @@ int Warmup(const std::string& name, const std::string& value) { | |||
| int ret = 0; | |||
|
|
||
| void WarmupManagerS3Impl::AddWarmupFilelist(fuse_ino_t key) { | ||
| if (!mounted_.load(std::memory_order_acquire)) { | ||
| LOG(ERROR) << "not mounted"; |
There was a problem hiding this comment.
maybe need wait util mounted？
There was a problem hiding this comment.
maybe need wait util mounted？
fix
There was a problem hiding this comment.
maybe need wait util mounted？
fix
|
|
||
| void WarmupManagerS3Impl::AddWarmupFile(fuse_ino_t key) { | ||
| if (!mounted_.load(std::memory_order_acquire)) { | ||
| LOG(ERROR) << "not mounted"; |
| } | ||
| // add warmup Progress | ||
| if (!AddWarmupProcess(key)) { | ||
| return; |
There was a problem hiding this comment.
need logs？ and do we need notice the users？
There was a problem hiding this comment.
need logs？ and do we need notice the users？
Unnecessary, failure indicates that the warm-up task has been joined and not completed.
There was a problem hiding this comment.
need logs？ and do we need notice the users？
fix
| if (ret != CURVEFS_ERROR::OK) { | ||
| LOG(ERROR) << "inodeManager get inode fail, ret = " << ret | ||
| << ", inodeid = " << key; | ||
| return; |
There was a problem hiding this comment.
maybe need delete the value added upside？and also do we need notice the users？
There was a problem hiding this comment.
maybe need delete the value added upside？and also do we need notice the users？
fix
There was a problem hiding this comment.
maybe need delete the value added upside？and also do we need notice the users？
fix
| } | ||
| // add warmup Progress | ||
| if (!AddWarmupProcess(key)) { | ||
| return; |
| std::function<void()> task) { | ||
| VLOG(9) << "add fetchDentry task: " << key; | ||
| std::unique_ptr<ThreadPool> tp = absl::make_unique<ThreadPool>(); | ||
| tp->Start(warmupThreadsNum_); |
There was a problem hiding this comment.
Each inode has a thread pool. What if there are many files downloaded?
There was a problem hiding this comment.
Each inode has a thread pool. What if there are many files downloaded?
A warmup task has a thread pool, and if there are many files, share the thread pool
There was a problem hiding this comment.
Each inode has a thread pool. What if there are many files downloaded?
A warmup task has a thread pool, and if there are many files, share the thread pool
There was a problem hiding this comment.
ok, need limit the task nums(thread nums)
There was a problem hiding this comment.
Each inode has a thread pool. What if there are many files downloaded?
A warmup task has a thread pool, and if there are many files, share the thread pool
threads will be created even same key
There was a problem hiding this comment.
threads will be created even same key
that is todo work
dcae9c7 to
126803f
Compare
1. add WarmupManager 2. add WarmupManagerS3Impl 3. add query warmup progress in tools-v2 Signed-off-by: Cyber-SiKu <Cyber-SiKu@outlook.com>
ffef139 to
caa5677
Compare
|
cicheck |
Signed-off-by: Cyber-SiKu Cyber-SiKu@outlook.com
What problem does this PR solve?
Issue Number: #2039
Problem Summary:
What is changed and how it works?
What's Changed:
How it Works:
Side effects(Breaking backward compatibility? Performance regression?):
Check List