Skip to content

Commit

Permalink
update rfc
Browse files Browse the repository at this point in the history
  • Loading branch information
hongyunyan committed Aug 31, 2022
1 parent 0bcd8a8 commit bd2e12b
Showing 1 changed file with 17 additions and 10 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,29 @@ This RFC enables tables to support both setting placement rules and setting tifl

## Motivation or Background

Currently, there is a conflict between setting tiflash replica and setting placement rules, i.e., if the table has a tiflash replica, it is not supported to set placement rules for this table. Similarly, if a table is set with some placement rules, it is not allowed to set tiflash replica for this table. The conflict between tiflash replicas and placement rules has a negative impact on user experience. Thus, this RFC will resolve this conflict and support setting tiflash replica and placement rules on tables at the same time.
Currently, there is a conflict between setting tiflash replicas and setting placement rules. For example, if a table has a tiflash replica, it is not supported to set placement rules for this table. Similarly, if a table has set some placement rules, it is not allowed to set tiflash replica for this table. The conflict between tiflash replicas and placement rules has a negative impact on user experience. Thus, this RFC will resolve this conflict, to support setting tiflash replicas and placement rules on tables at the same time.

The conflict between tiflash replicas and placement rules setting is because both setting tiflash replicas and setting placement rules will generate placement configuration in PD. In the configuration generated by setting placement rules, ```group_index = 40, group_override = true```; while in the configuration generated by tiflash replica, ```group_index = 0```, ```group_override = false```. According to the rules in PD, if ```group_override``` is true , the configuration will override the configuration of other groups whose ```group_index``` is smaller than its own; if ```group_override``` is false, other configurations will not be overridden. Therefore, placement rules will overwrite the setting of tiflash replicas. In order to avoid this problem, the coexistence of placement rules and tiflash replicas was prohibited in the original development.
The conflict between tiflash replicas and placement rules setting is because setting tiflash replicas and placement rules will both generate placement configuration in PD.

- In the configuration generated by placement rules, `group_index = 40, group_override = true`.
- In the configuration generated by tiflash replicas, `group_index = 0`, `group_override = false`.

According to the rules in PD, if `group_override` is true , the configuration will override the configuration of other groups whose `group_index` is smaller; While if `group_override` is false, other configurations will not be overridden. Therefore, placement rules will overwrite the setting of tiflash replicas. To avoid this problem, the coexistence of placement rules and tiflash replicas was prohibited in the original development.

## Detailed Design

Because placement rules and tiflash replica do not affect each other in their own rules and work independently, we can give a larger value to group_index (```RuleIndexTiFlash = 120```, larger than all other placement configurations) in the placement configuration generated by setting tiflash replicas. In this way, other placement rules will not override the settings of tiflash replicas, and because ```group_override = false``` in tiflash replicas' configuration, tiflash replicas configuration will not override other placement rules. This allows the tables to support both placement rules and tiflash replicas settings.
Because placement rules and tiflash replica do not affect each other in their own rules and work independently, we can give a largest value to group_index for the configuration generated by tiflash replicas. We set `RuleIndexTiFlash = 120`, which is larger than all other placement configurations.

In this way, other placement rules can not override the settings of tiflash replicas. Besides, due to `group_override = false` in tiflash replicas' configuration, tiflash replicas configuration will not override other placement rules. This allows the tables to support both placement rules and tiflash replicas settings.

Therefore, before setting tiflash replicas (```POST /pd/api/v1/config/rule```), we call ```POST /pd/api/v1/config/rule-group``` to modify the group_index value of the tiflash group as RuleIndexTiFlash.
Therefore, before setting tiflash replicas(`POST /pd/api/v1/config/rule`), we call `POST /pd/api/v1/config/rule-group` to modify the group_index value of the tiflash group as RuleIndexTiFlash.

For the already existing placement configuration of the tiflash replica setting in the old version, we will call ```POST /pd/api/v1/config/rule-group``` in the background task of PollTiFlashRoutine to update the group index of the placement configuration of the existing tiflash replica to ensure normal settings for subsequent placement rules.
For the already existing placement configuration of the tiflash replica setting in the old version, we will call `POST /pd/api/v1/config/rule-group` in the background task of `PollTiFlashRoutine` to update the group index of the placement configuration generated by the existing tiflash replicas. In this way, we can ensure normal settings for subsequent placement rules.

## Investigation & Alternatives

1. Add setting group_index in the api ```POST /pd/api/v1/config/rule``` called by the tiflash replica settings, so that the group_index is RuleIndexTiFlash when the tiflash replica is set.
+ This practice changes the semantics of ```/pd/api/v1/config/rule```, which is inappropriate.
2. tiflash uses the ```POST /pd/api/v1/config/placement-rule``` API to replace the original API.
+ This API will require setting the full number of rules in the group. All the settings of tiflash are in a group, which will lead to a large package of http requests when there are many tiflash tables.
+ Compared with the original code, this change is relatively large, and the introduction of concurrency ddl may cause concurrent coverage problems.
1. Add setting group_index in the api `POST /pd/api/v1/config/rule` called by the tiflash replica settings, so that the group_index is `RuleIndexTiFlash` when the tiflash replica is set.
- This practice changes the semantics of `/pd/api/v1/config/rule`, which is inappropriate.
2. tiflash uses the `POST /pd/api/v1/config/placement-rule` API to replace the original API.
- This API will require setting the full number of rules in the group. All the settings of tiflash are in a group, which will lead to a large package of http requests when there are many tiflash tables.
- Compared with the original code, this change is relatively large, and the introduction of concurrency ddl may cause concurrent coverage problems.

0 comments on commit bd2e12b

Please sign in to comment.