Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

br: Add docs for br debug #12263

Merged
merged 9 commits into from
Feb 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 44 additions & 1 deletion br-usage-backup-for-maintain.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,49 @@ br backup full\

BR supports server-side encryption (SSE) when backing up data to S3. In this scenario, you can use AWS KMS keys you have created to encrypt data. For details, see [BR S3 server-side encryption](/encryption-at-rest.md#br-s3-server-side-encryption).

## Validate backup data

After you back up data using BR, you can validate the backup data, including checking its integrity and viewing the metadata (such as TSO) by decoding the `backupmeta` file.

### Check the integrity of backup data

To check the integrity of backup data, you can run the `tiup br debug checksum` command to calculate the checksum of the backup data.

Example: Calculate the checksum of the backup data in the `${prefix}` directory in the `backup-data` bucket on Amazon S3.

```shell
br debug checksum \
--storage 's3://backup-data/${prefix}' \
--s3.endpoint '${S3-endpoint-URL}' \
--log-file checksum.log
```

### Decode `backupmeta` to a readable JSON file

After a backup is complete, you can run the `tiup br debug decode` command to decode the `backupmeta` file into a readable JSON file, through which you can view the metadata (such as TSO) of the snapshot.

Example: Decode the `backupmeta` file in the `${prefix}` directory in the `backup-data` bucket on Amazon S3 into a JSON file `backupmeta.json`. The decoded file is stored in `s3://backup-data/${prefix}/backupmeta.json`.

```shell
br debug decode \
--storage 's3://backup-data/${prefix}' \
--s3.endpoint '${S3-endpoint-URL}' \
--log-file decode-backupmeta.log
```

Open the `backupmeta.json` file and search for `end_version` to view the TSO of the snapshot.

If necessary, you can also encode the JSON format `backupmeta` file back to the original state. Specifically, run the `tiup br debug encode` command to generate the file named `backupmeta_from_json`.

Example: Encode the `backupmeta.json` file in the `${prefix}` directory in the `backup-data` bucket on Amazon S3 into a `backupmeta` file. The encoded file is stored in `s3://backup-data/${prefix}/backupmeta_from_json`.

```shell
br debug encode \
--storage 's3://backup-data/${prefix}' \
--s3.endpoint '${S3-endpoint-URL}' \
--log-file encode-backupmeta.log
```

## Backup performance and impact

The backup feature has some impact on cluster performance (transaction latency and QPS). However, you can mitigate the impact by adjusting the number of backup threads [`backup.num-threads`](/tikv-configuration-file.md#num-threads-1) or by adding more clusters.
Expand All @@ -211,6 +254,6 @@ You can mitigate impact on cluster performance by reducing the number of backup

> **Note:**
>
> The impact and speed of backup depends much on cluser configuration, deployment, and running services. The preceding test conclusions, based on simulation tests in many scenarios and verified in some customer sites, are worthy of reference. However, the exact impact and performance cap may vary depending on the scenarios. Therefore, you should always run the test and verify the test results.
> The impact and speed of backup depends much on cluster configuration, deployment, and running services. The preceding test conclusions, based on simulation tests in many scenarios and verified in some customer sites, are worthy of reference. However, the exact impact and performance cap may vary depending on the scenarios. Therefore, you should always run the test and verify the test results.

Since v5.3.0, BR introduces the auto tunning feature (enabled by default) to adjust the number of backup threads. It can maintain the CPU utilization of the cluster below 80% during backup tasks. For details, see [BR Auto-Tune](/br/br-auto-tune.md).
45 changes: 44 additions & 1 deletion br/br-usage-backup.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,49 @@ br backup full\

BR supports server-side encryption (SSE) when backing up data to S3. In this scenario, you can use AWS KMS keys you have created to encrypt data. For details, see [BR S3 server-side encryption](/encryption-at-rest.md#br-s3-server-side-encryption).

## Validate backup data

After you back up data using BR, you can validate the backup data, including checking its integrity and viewing the metadata (such as TSO) by decoding the `backupmeta` file.

### Check the integrity of backup data

To check the integrity of backup data, you can run the `tiup br debug checksum` command to calculate the checksum of the backup data.

Example: Calculate the checksum of the backup data in the `${prefix}` directory in the `backup-data` bucket on Amazon S3.

```shell
br debug checksum \
--storage 's3://backup-data/${prefix}' \
--s3.endpoint '${S3-endpoint-URL}' \
--log-file checksum.log
```

### Decode `backupmeta` to a readable JSON file

After a backup is complete, you can run the `tiup br debug decode` command to decode the `backupmeta` file into a readable JSON file, through which you can view the metadata (such as TSO) of the snapshot.

Example: Decode the `backupmeta` file in the `${prefix}` directory in the `backup-data` bucket on Amazon S3 into a JSON file `backupmeta.json`. The decoded file is stored in `s3://backup-data/${prefix}/backupmeta.json`.

```shell
br debug decode \
--storage 's3://backup-data/${prefix}' \
--s3.endpoint '${S3-endpoint-URL}' \
--log-file decode-backupmeta.log
```

Open the `backupmeta.json` file and search for `end_version` to view the TSO of the snapshot.

If necessary, you can also encode the JSON format `backupmeta` file back to the original state. Specifically, run the `tiup br debug encode` command to generate the file named `backupmeta_from_json`.

Example: Encode the `backupmeta.json` file in the `${prefix}` directory in the `backup-data` bucket on Amazon S3 into a `backupmeta` file. The encoded file is stored in `s3://backup-data/${prefix}/backupmeta_from_json`.

```shell
br debug encode \
--storage 's3://backup-data/${prefix}' \
--s3.endpoint '${S3-endpoint-URL}' \
--log-file encode-backupmeta.log
```

## Backup performance and impact

The backup feature has some impact on cluster performance (transaction latency and QPS). However, you can mitigate the impact by adjusting the number of backup threads [`backup.num-threads`](/tikv-configuration-file.md#num-threads-1) or by adding more clusters.
Expand All @@ -211,6 +254,6 @@ You can mitigate impact on cluster performance by reducing the number of backup

> **Note:**
>
> The impact and speed of backup depends much on cluser configuration, deployment, and running services. The preceding test conclusions, based on simulation tests in many scenarios and verified in some customer sites, are worthy of reference. However, the exact impact and performance cap may vary depending on the scenarios. Therefore, you should always run the test and verify the test results.
> The impact and speed of backup depends much on cluster configuration, deployment, and running services. The preceding test conclusions, based on simulation tests in many scenarios and verified in some customer sites, are worthy of reference. However, the exact impact and performance cap may vary depending on the scenarios. Therefore, you should always run the test and verify the test results.

Since v5.3.0, BR introduces the auto tunning feature (enabled by default) to adjust the number of backup threads. It can maintain the CPU utilization of the cluster below 80% during backup tasks. For details, see [BR Auto-Tune](/br/br-auto-tune.md).