ticdc: add description about ticdc new architecture #19765

lidezhu · 2025-02-10T09:11:44Z

First-time contributors' checklist

I've signed Contributor License Agreement that's required for repo owners to accept my contribution.

What is changed, added or deleted? (Required)

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions (in Chinese).

What is the related PR or file link(s)?

This PR is translated from:
Other reference link(s):

Do your changes match any of the following descriptions?

Delete files
Change aliases
Need modification after applied to another branch
Might cause conflicts after applied to another branch

ti-chi-bot · 2025-02-11T02:47:50Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from qiancai, ensuring that each of them provides their approval before proceeding. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ticdc/ticdc-new-arch.md

Co-authored-by: Flowyi <flowbehappy@gmail.com>

ticdc/ticdc-new-arch.md

ti-chi-bot · 2025-02-14T08:33:47Z

@wlwilliamx: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

wlwilliamx

/CC @flowbehappy

ticdc/ticdc-server-config.md

ti-chi-bot · 2025-02-14T08:42:17Z

@wlwilliamx: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

/CC @flowbehappy

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Co-authored-by: wlwilliamx <53336371+wlwilliamx@users.noreply.github.com>

…ocs-cn into ldz/add-ticdc-new-arch

ticdc/ticdc-new-arch.md

Co-authored-by: Flowyi <flowbehappy@gmail.com>

ti-chi-bot · 2025-02-18T04:09:55Z

@flowbehappy: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ti-chi-bot · 2025-02-18T04:11:46Z

@lidezhu: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-verify	`23a82ec`	link	true	`/test pull-verify`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

wlwilliamx

@flowbehappy PTAL

wlwilliamx · 2025-02-18T08:24:37Z

ticdc/ticdc-new-arch.md

+
+![TiCDC New Architecture](/media/ticdc/ticdc-new-arch-2.jpg)
+
+## 使用指南


Suggested change

## 使用指南

## 新架构部署指南

“部署指南” 感觉应该足够了，不用再加 “新架构”

wlwilliamx · 2025-02-18T09:43:37Z

ticdc/ticdc-new-arch.md

+```
+
+3. 参考[恢复同步任务](/ticdc/ticdc-manage-changefeed.md#恢复同步任务)恢复所有的 Changefeed 同步任务；
+


Suggested change

## 新架构命令使用指南

在部署完新架构的节点之后，即可使用新架构的命令来操作新架构的 TiCDC 节点了。

新架构的使用命令与老架构采用的是同一套命令逻辑，需要注意的是要在命令中加上一个标志或者设置一个环境变量来让新架构的 TiCDC 节点执行命令。下面是使用方式的具体说明：

### 方式一：使用 `--newarch` 或 `-x` 标志

如果想让新架构的 TiCDC 节点执行某个命令，可以在命令中添加 `--newarch`（或 `-x`）标志。例如：

```

cdc --newarch cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"

```

或使用 `-x` 的简写形式：

```

cdc -x cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"

```

这样便可在新架构的 TiCDC 节点中创建 Changefeed。

如果不加 `--newarch` 或 `-x` 标志，将默认让老架构的 TiCDC 节点执行命令：

```

# 默认使用老架构

cdc cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"

```

这样依旧是在老架构的 TiCDC 节点中创建 Changefeed。

### 方式二：使用 `TICDC_NEWARCH` 环境变量

若需要经常手动执行命令操作 TiCDC 新架构，可设置环境变量 `TICDC_NEWARCH=true`，这样当前终端会话中的所有 TiCDC 命令都会采用新架构。例如：

```

export TICDC_NEWARCH=true # 在当前终端会话中，之后所有的 TiCDC 命令都会采用新架构

# 现在默认使用的是新架构，在新架构 TiCDC 节点中创建 Changefeed

cdc cli changefeed create cdc cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"

```

如果需要切换回老架构，可以取消该环境变量：

```

unset TICDC_NEWARCH

# 重新默认使用老架构，在老架构 TiCDC 节点中创建 Changefeed

cdc cli changefeed create cdc cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"

```

### 常见命令示例

以下是一些 TiCDC 相关命令在新架构下的常见使用示例。

1. **在新架构的 TiCDC 节点中创建 Changefeed**

* 添加 `--newarch` 或 `-x` 标志：

```

cdc -x cli changefeed create cdc cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"

```

* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：

```

export TICDC_NEWARCH=true

cdc cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"

```

2. **查看 Changefeed 列表**

* 添加 `--newarch` 或 `-x` 标志：

```

cdc -x cli changefeed list --server=http://10.0.10.25:8300

```

* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：

```

export TICDC_NEWARCH=true

cdc cli changefeed list --server=http://10.0.10.25:8300

```

3. **暂停 Changefeed**

* 添加 `--newarch` 或 `-x` 标志：

```

cdc -x cli changefeed pause --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task

```

* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：

```

export TICDC_NEWARCH=true

cdc cli changefeed pause --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task

```

4. **恢复 Changefeed**

* 添加 `--newarch` 或 `-x` 标志：

```

cdc -x cli changefeed resume --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task

```

* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：

```

export TICDC_NEWARCH=true

cdc cli changefeed resume --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task

```

5. **删除 Changefeed**

* 添加 `--newarch` 或 `-x` 标志：

```

cdc -x cli changefeed remove --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task

```

* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：

```

export TICDC_NEWARCH=true

cdc cli changefeed remove --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task

```

6. **查询特定 Changefeed 信息**

* 添加 `--newarch` 或 `-x` 标志：

```

cdc -x cli changefeed query -s --server=http://10.0.10.25:8300 --changefeed-id=simple-replication-task

```

* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：

```

export TICDC_NEWARCH=true

cdc cli changefeed query -s --server=http://10.0.10.25:8300 --changefeed-id=simple-replication-task

```

7. **查看可用命令**

由于新架构仍在逐步完善，部分命令可能尚未支持。可以使用以下命令查看当前架构下的可用命令：

* 查看新架构当前支持的命令：

```

cdc --newarch help # 或者使用 -x

```

* 对比查看老架构当前支持的命令：

```

cdc help

```

在未来版本中，我们将逐步增加对 TiCDC 新架构更多命令的支持，请关注后续版本更新，以获取最新功能支持。

”常见命令示例” 这一部分太冗长了，建议直接链接到老架构的操作指南的页面，再简单说明一下即可。

ti-chi-bot · 2025-02-18T09:44:43Z

@wlwilliamx: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

@flowbehappy PTAL

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

lidezhu added 3 commits February 10, 2025 16:38

alpha commit

48978b2

more desc

396c144

small fix

50ac3c0

ti-chi-bot bot added missing-translation-status This PR does not have translation status info. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 10, 2025

lidezhu changed the title ~~ticdc: add new description about ticdc new architecture~~ ticdc: add description about ticdc new architecture Feb 10, 2025

qiancai self-requested a review February 10, 2025 09:18

qiancai added area/ticdc Indicates that the Issue or PR belongs to the area of TiCDC. v9.0 labels Feb 10, 2025

qiancai self-assigned this Feb 10, 2025

qiancai added translation/doing This PR’s assignee is translating this PR. and removed missing-translation-status This PR does not have translation status info. labels Feb 10, 2025

small fix

bd99757

flowbehappy reviewed Feb 11, 2025

View reviewed changes

ticdc/ticdc-new-arch.md Outdated Show resolved Hide resolved

lidezhu and others added 2 commits February 11, 2025 11:14

fix

b939596

Update ticdc/ticdc-new-arch.md

543d3b2

Co-authored-by: Flowyi <flowbehappy@gmail.com>

flowbehappy reviewed Feb 11, 2025

View reviewed changes

ticdc/ticdc-new-arch.md Outdated Show resolved Hide resolved

wlwilliamx requested changes Feb 14, 2025

View reviewed changes

ticdc/ticdc-new-arch.md Show resolved Hide resolved

ti-chi-bot bot requested a review from flowbehappy February 14, 2025 08:35

wlwilliamx requested changes Feb 14, 2025

View reviewed changes

ticdc/ticdc-server-config.md Outdated Show resolved Hide resolved

flowbehappy and others added 7 commits February 17, 2025 14:52

Update ticdc/ticdc-new-arch.md

24cfe76

Co-authored-by: wlwilliamx <53336371+wlwilliamx@users.noreply.github.com>

add more desc

ce844a1

Update ticdc/ticdc-server-config.md

bf6fd88

Co-authored-by: wlwilliamx <53336371+wlwilliamx@users.noreply.github.com>

small fix

7b4ec72

add more desc

ba7e3fa

Merge branch 'ldz/add-ticdc-new-arch' of https://github.com/pingcap/d…

c3c7914

…ocs-cn into ldz/add-ticdc-new-arch

format

4a0d176

lidezhu force-pushed the ldz/add-ticdc-new-arch branch from 6de9f00 to 4a0d176 Compare February 17, 2025 13:11

flowbehappy reviewed Feb 18, 2025

View reviewed changes

ticdc/ticdc-new-arch.md Outdated Show resolved Hide resolved

flowbehappy reviewed Feb 18, 2025

View reviewed changes

ticdc/ticdc-new-arch.md Outdated Show resolved Hide resolved

Apply suggestions from code review

23a82ec

Co-authored-by: Flowyi <flowbehappy@gmail.com>

flowbehappy approved these changes Feb 18, 2025

View reviewed changes

wlwilliamx requested changes Feb 18, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ticdc: add description about ticdc new architecture #19765

ticdc: add description about ticdc new architecture #19765

lidezhu commented Feb 10, 2025

ti-chi-bot bot commented Feb 11, 2025

ti-chi-bot bot commented Feb 14, 2025

wlwilliamx left a comment

ti-chi-bot bot commented Feb 14, 2025

ti-chi-bot bot commented Feb 18, 2025

ti-chi-bot bot commented Feb 18, 2025

wlwilliamx left a comment

wlwilliamx Feb 18, 2025

lidezhu Feb 20, 2025

wlwilliamx Feb 18, 2025

lidezhu Feb 20, 2025

ti-chi-bot bot commented Feb 18, 2025


		![TiCDC New Architecture](/media/ticdc/ticdc-new-arch-2.jpg)

		## 使用指南

		```

		3. 参考[恢复同步任务](/ticdc/ticdc-manage-changefeed.md#恢复同步任务)恢复所有的 Changefeed 同步任务；

+## 新架构命令使用指南
+在部署完新架构的节点之后，即可使用新架构的命令来操作新架构的 TiCDC 节点了。
+新架构的使用命令与老架构采用的是同一套命令逻辑，需要注意的是要在命令中加上一个标志或者设置一个环境变量来让新架构的 TiCDC 节点执行命令。下面是使用方式的具体说明：
+### 方式一：使用 `--newarch` 或 `-x` 标志
+如果想让新架构的 TiCDC 节点执行某个命令，可以在命令中添加 `--newarch`（或 `-x`）标志。例如：
+```
+cdc --newarch cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"
+```
+或使用 `-x` 的简写形式：
+```
+cdc -x cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"
+```
+这样便可在新架构的 TiCDC 节点中创建 Changefeed。
+如果不加 `--newarch` 或 `-x` 标志，将默认让老架构的 TiCDC 节点执行命令：
+```
+# 默认使用老架构
+cdc cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"
+```
+这样依旧是在老架构的 TiCDC 节点中创建 Changefeed。
+### 方式二：使用 `TICDC_NEWARCH` 环境变量
+若需要经常手动执行命令操作 TiCDC 新架构，可设置环境变量 `TICDC_NEWARCH=true`，这样当前终端会话中的所有 TiCDC 命令都会采用新架构。例如：
+```
+export TICDC_NEWARCH=true # 在当前终端会话中，之后所有的 TiCDC 命令都会采用新架构
+# 现在默认使用的是新架构，在新架构 TiCDC 节点中创建 Changefeed
+cdc cli changefeed create cdc cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"
+```
+如果需要切换回老架构，可以取消该环境变量：
+```
+unset TICDC_NEWARCH
+# 重新默认使用老架构，在老架构 TiCDC 节点中创建 Changefeed
+cdc cli changefeed create cdc cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"
+```
+### 常见命令示例
+以下是一些 TiCDC 相关命令在新架构下的常见使用示例。
+. **在新架构的 TiCDC 节点中创建 Changefeed**
+* 添加 `--newarch` 或 `-x` 标志：
+    ```
+    cdc -x cli changefeed create cdc cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"
+    ```
+* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：
+    ```
+    export TICDC_NEWARCH=true
+    cdc cli changefeed create --server=http://10.0.10.25:8300 --sink-uri="mysql://root:123456@127.0.0.1:3306/" --changefeed-id="simple-replication-task"
+    ```
+. **查看 Changefeed 列表**
+  * 添加 `--newarch` 或 `-x` 标志：
+    ```
+    cdc -x cli changefeed list --server=http://10.0.10.25:8300
+    ```
+* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：
+    ```
+    export TICDC_NEWARCH=true
+    cdc cli changefeed list --server=http://10.0.10.25:8300
+    ```
+. **暂停 Changefeed**
+* 添加 `--newarch` 或 `-x` 标志：
+    ```
+    cdc -x cli changefeed pause --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task
+    ```
+* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：
+    ```
+    export TICDC_NEWARCH=true
+    cdc cli changefeed pause --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task
+    ```
+. **恢复 Changefeed**
+* 添加 `--newarch` 或 `-x` 标志：
+    ```
+    cdc -x cli changefeed resume --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task
+    ```
+* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：
+    ```
+    export TICDC_NEWARCH=true
+    cdc cli changefeed resume --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task
+    ```
+. **删除 Changefeed**
+* 添加 `--newarch` 或 `-x` 标志：
+    ```
+    cdc -x cli changefeed remove --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task
+    ```
+* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：
+    ```
+    export TICDC_NEWARCH=true
+    cdc cli changefeed remove --server=http://10.0.10.25:8300 --changefeed-id simple-replication-task
+    ```
+. **查询特定 Changefeed 信息**
+* 添加 `--newarch` 或 `-x` 标志：
+    ```
+    cdc -x cli changefeed query -s --server=http://10.0.10.25:8300 --changefeed-id=simple-replication-task
+    ```
+* 设置 `TICDC_NEWARCH` 环境变量，后续该终端会话下的所有 TiCDC 命令都无需额外设置即可使用新架构：
+    ```
+    export TICDC_NEWARCH=true
+    cdc cli changefeed query -s --server=http://10.0.10.25:8300 --changefeed-id=simple-replication-task
+    ```
+. **查看可用命令**
+    由于新架构仍在逐步完善，部分命令可能尚未支持。可以使用以下命令查看当前架构下的可用命令：
+* 查看新架构当前支持的命令：
+    ```
+    cdc --newarch help # 或者使用 -x
+    ```
+* 对比查看老架构当前支持的命令：
+    ```
+    cdc help
+    ```
+    在未来版本中，我们将逐步增加对 TiCDC 新架构更多命令的支持，请关注后续版本更新，以获取最新功能支持。

ticdc: add description about ticdc new architecture #19765

Are you sure you want to change the base?

ticdc: add description about ticdc new architecture #19765

Conversation

lidezhu commented Feb 10, 2025

First-time contributors' checklist

What is changed, added or deleted? (Required)

Which TiDB version(s) do your changes apply to? (Required)

What is the related PR or file link(s)?

Do your changes match any of the following descriptions?

ti-chi-bot bot commented Feb 11, 2025

ti-chi-bot bot commented Feb 14, 2025

wlwilliamx left a comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Feb 14, 2025

ti-chi-bot bot commented Feb 18, 2025

ti-chi-bot bot commented Feb 18, 2025

wlwilliamx left a comment

Choose a reason for hiding this comment

wlwilliamx Feb 18, 2025

Choose a reason for hiding this comment

lidezhu Feb 20, 2025

Choose a reason for hiding this comment

wlwilliamx Feb 18, 2025

Choose a reason for hiding this comment

lidezhu Feb 20, 2025

Choose a reason for hiding this comment

ti-chi-bot bot commented Feb 18, 2025