Remove disks endpoint using IntraBrokerDiskCapacityGoal #12
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… of the same broker (linkedin#7)" This reverts commit 8b203be.
- Extend IntraBrokerDiskCapacityGoal to allow moving all replicas from a 0 capacity disk - Added new endpoint that removes disks by setting 0 capacity on them and optimizing the IntraBrokerDiskCapacityGoal
|
@ilievladiulian do we also have a PR with the upstream project? |
This PR addresses the comments in the upstream PR. Will also update the PR there. |
5 tasks
alex-necula
approved these changes
Apr 20, 2023
2 tasks
- refactored for better readability - throw exception if goal could not move all replicas from 0 capacity disks
|
Added exception in case the goal could not move all replicas from deleted disks so that the task ends with CompletedWithError status. |
This was referenced May 2, 2023
| @@ -244,12 +262,17 @@ public String name() { | |||
|
|
|||
| /** | |||
| * Check whether the combined replica utilization is above the given disk capacity limits. | |||
| * If _shouldEmptyZeroCapacityDisks is true, the disk utilization is over limit only if it greater than 0. | |||
There was a problem hiding this comment.
Suggested change
| * If _shouldEmptyZeroCapacityDisks is true, the disk utilization is over limit only if it greater than 0. | |
| * If _shouldEmptyZeroCapacityDisks is true, the disk utilization is over limit only if it is greater than 0. |
There was a problem hiding this comment.
Fixed typo in latest commit.
| @@ -43,6 +43,7 @@ public class IntraBrokerDiskCapacityGoal extends AbstractGoal { | |||
| private static final Logger LOG = LoggerFactory.getLogger(IntraBrokerDiskCapacityGoal.class); | |||
| private static final int MIN_NUM_VALID_WINDOWS = 1; | |||
| private static final Resource RESOURCE = Resource.DISK; | |||
| private boolean _shouldEmptyZeroCapacityDisks = false; | |||
There was a problem hiding this comment.
Is this naming format specific to CC? Is it ok not to be thread safe?
There was a problem hiding this comment.
This seems to be the naming convention in CC. I changed the field to be final.
|
|
||
| public RemoveDisksRunnable(KafkaCruiseControl kafkaCruiseControl, | ||
| OperationFuture future, | ||
| RemoveDisksParameters parameters, | ||
| String uuid) { | ||
| super(kafkaCruiseControl, future, parameters, parameters.dryRun(), parameters.stopOngoingExecution(), parameters.skipHardGoalCheck(), | ||
| super(kafkaCruiseControl, future, parameters, parameters.dryRun(), parameters.stopOngoingExecution(), false, |
There was a problem hiding this comment.
Why is the param removal needed?
There was a problem hiding this comment.
This was a parameter that was previously exposed in the API, although it was never used (and should not be needed for disk removal). There is no sense in skipping goal checking as this endpoint only uses one goal, the IntraBrokerDiskCapacityGoal.
cristianpetrache
approved these changes
May 3, 2023
amuraru
approved these changes
May 3, 2023
amuraru
pushed a commit
that referenced
this pull request
May 14, 2023
* Revert "Add endpoint to move all replicas from a disk to another disk of the same broker (#7)" This reverts commit 8b203be. * Added new endpoint to remove disks: - Extend IntraBrokerDiskCapacityGoal to allow moving all replicas from a 0 capacity disk - Added new endpoint that removes disks by setting 0 capacity on them and optimizing the IntraBrokerDiskCapacityGoal * Additional fixes: - refactored for better readability - throw exception if goal could not move all replicas from 0 capacity disks * Fix typo * Changed class field to final
amuraru
pushed a commit
that referenced
this pull request
May 14, 2023
* Revert "Add endpoint to move all replicas from a disk to another disk of the same broker (#7)" This reverts commit 8b203be. * Added new endpoint to remove disks: - Extend IntraBrokerDiskCapacityGoal to allow moving all replicas from a 0 capacity disk - Added new endpoint that removes disks by setting 0 capacity on them and optimizing the IntraBrokerDiskCapacityGoal * Additional fixes: - refactored for better readability - throw exception if goal could not move all replicas from 0 capacity disks * Fix typo * Changed class field to final
amuraru
pushed a commit
that referenced
this pull request
May 19, 2023
* Revert "Add endpoint to move all replicas from a disk to another disk of the same broker (#7)" This reverts commit 8b203be. * Added new endpoint to remove disks: - Extend IntraBrokerDiskCapacityGoal to allow moving all replicas from a 0 capacity disk - Added new endpoint that removes disks by setting 0 capacity on them and optimizing the IntraBrokerDiskCapacityGoal * Additional fixes: - refactored for better readability - throw exception if goal could not move all replicas from 0 capacity disks * Fix typo * Changed class field to final
amuraru
pushed a commit
that referenced
this pull request
Jun 24, 2023
* Revert "Add endpoint to move all replicas from a disk to another disk of the same broker (#7)" This reverts commit 8b203be. * Added new endpoint to remove disks: - Extend IntraBrokerDiskCapacityGoal to allow moving all replicas from a 0 capacity disk - Added new endpoint that removes disks by setting 0 capacity on them and optimizing the IntraBrokerDiskCapacityGoal * Additional fixes: - refactored for better readability - throw exception if goal could not move all replicas from 0 capacity disks * Fix typo * Changed class field to final
amuraru
pushed a commit
that referenced
this pull request
Jul 5, 2023
* Revert "Add endpoint to move all replicas from a disk to another disk of the same broker (#7)" This reverts commit 8b203be. * Added new endpoint to remove disks: - Extend IntraBrokerDiskCapacityGoal to allow moving all replicas from a 0 capacity disk - Added new endpoint that removes disks by setting 0 capacity on them and optimizing the IntraBrokerDiskCapacityGoal * Additional fixes: - refactored for better readability - throw exception if goal could not move all replicas from 0 capacity disks * Fix typo * Changed class field to final
amuraru
pushed a commit
that referenced
this pull request
Jul 30, 2023
* Revert "Add endpoint to move all replicas from a disk to another disk of the same broker (#7)" This reverts commit 8b203be. * Added new endpoint to remove disks: - Extend IntraBrokerDiskCapacityGoal to allow moving all replicas from a 0 capacity disk - Added new endpoint that removes disks by setting 0 capacity on them and optimizing the IntraBrokerDiskCapacityGoal * Additional fixes: - refactored for better readability - throw exception if goal could not move all replicas from 0 capacity disks * Fix typo * Changed class field to final
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR changes the implementation of the remove disks endpoint to set disks to 0 capacity and to use IntraBrokerDiskCapacityGoal to move all replicas from a disk with 0 capacity.