From a939e324358185f663dd90dcf113c279f6b7d2a5 Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 20 Jul 2023 11:41:29 +0800 Subject: [PATCH 01/72] Add temp.md --- temp.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 temp.md diff --git a/temp.md b/temp.md new file mode 100644 index 0000000000000..af27ff4986a7b --- /dev/null +++ b/temp.md @@ -0,0 +1 @@ +This is a test file. \ No newline at end of file From f2583659825d4975036fad0a156e3b7891ee4219 Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 20 Jul 2023 11:41:33 +0800 Subject: [PATCH 02/72] Delete temp.md --- temp.md | 1 - 1 file changed, 1 deletion(-) delete mode 100644 temp.md diff --git a/temp.md b/temp.md deleted file mode 100644 index af27ff4986a7b..0000000000000 --- a/temp.md +++ /dev/null @@ -1 +0,0 @@ -This is a test file. \ No newline at end of file From 78aa92d9250a327793c9c3fa47c1a9486089e08e Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 20 Jul 2023 13:12:40 +0800 Subject: [PATCH 03/72] add template --- releases/release-7.3.0.md | 196 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 196 insertions(+) create mode 100644 releases/release-7.3.0.md diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md new file mode 100644 index 0000000000000..04f8e8f80a4fe --- /dev/null +++ b/releases/release-7.3.0.md @@ -0,0 +1,196 @@ +--- +title: TiDB 7.3.0 Release Notes +summary: Learn about the new features, compatibility changes, improvements, and bug fixes in TiDB 7.3.0. +--- + +# TiDB 7.3.0 Release Notes + +Release date: xx xx, 2023 + +TiDB version: 7.3.0 + +Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with-tidb) | [Installation packages](https://www.pingcap.com/download/?version=v7.3.0#version-list) + +7.3.0 introduces the following key features and improvements: + + + +## Feature details + +### Performance + +- note 1 + +- note 2 + +### Reliability + +- note 1 + +- note 2 + +### SQL + +- note 1 + +- note 2 + +### DB operations + +- note 1 + +- note 2 + +### Observability + +- note 1 + +- note 2 + +### Data migration + +- note 1 + +- note 2 + +## Compatibility changes + +> **注意：** +> +> 以下为从 v7.1.0 升级至当前版本 (v7.2.0) 所需兼容性变更信息。如果从 v7.0.0 或之前版本升级到当前版本，可能也需要考虑和查看中间版本 release notes 中提到的兼容性变更信息。 + +### Behavior changes + + + +* 兼容性 1 + +* 兼容性 2 + +### System variables + +| 变量名 | 修改类型（包括新增/修改/删除） | 描述 | +|--------|------------------------------|------| +| [`tidb_remove_orderby_in_subquery`](/system-variables.md#tidb_remove_orderby_in_subquery-从-v610-版本开始引入) | 修改 | 经进一步的测试后，该变量默认值从 `OFF` 修改为 `ON`，即优化器改写会移除子查询中的 `ORDER BY` 子句。 | +| | | | +| | | | +| | | | + +### Configuration file parameters + +| 配置文件 | 配置项 | 修改类型 | 描述 | +| -------- | -------- | -------- | -------- | +| | | | | +| | | | | +| TiDB Lightning | `send-kv-pairs` | 废弃 | 从 7.2 版本开始 TiDB Lightning 配置文件的参数 "send-kv-pairs" 不再生效，由新的参数 "send-kv-size" 代替。该新参数用于指定 KV 键值对的大小阈值，单位为 KiB 或 MiB，默认值为 "16 KiB"。当 KV 键值对的大小达到设定的阈值时，它们将立即发送到 TiKV，避免在导入大宽表等一些场景因为 Lightning 节点内存积累键值对过多导致 OOM 的问题。**tw@hfxsd** | +| TiDB Lightning | `send-kv-size` | 新增 | 从 7.2 版本开始在 Lightning 配置文件 "[tikv-importer]" 这个 Session 中引入 `send-kv-size` 参数，用于设置发单次送到 TiKV 的 KV pairs 的大小。当 KV 键值对的大小达到设定的阈值时，它们将被 Lightning 立即发送到 TiKV，避免在导入大宽表的时候 Lightning 节点因为内存积累键值对过多导致 OOM 的问题。通过调整 "send-kv-size" 参数，你可以在内存使用和导入速度之间找到平衡，提高导入过程的稳定性和效率。**tw@hfxsd** | +| Data Migration | `strict-optimistic-shard-mode` | 新增 | 用于兼容历史版本 2.0 分库分表同步 DDL 的行为。当用户选择乐观模式时，可以启用该参数，开启后，乐观模式下，同步任务遇到二类 DDL 时，整个任务会中断，在多个表的 DDL变更有依赖关系的场景，可以及时中断，用户手动处理完各表的 DDL 后，再继续同步数据，保障上下游数据的一致性。 **tw@ran-huang** | + +## Deprecated features + +- note [#issue](链接) @[贡献者 GitHub ID](链接) + +## Improvements + ++ TiDB + + - 优化构造索引扫描范围的逻辑，支持将一些复杂条件转化为索引扫描范围 [#41572](https://github.com/pingcap/tidb/issues/41572) [#44389](https://github.com/pingcap/tidb/issues/44389) @xuyifangreeneyes + - 支持分区表下推 TopN 算子，减少该情况下 indexLookUp 和 IndexMerge 算子回表次数，提升相关 SQL 执行性能 [#26166](https://github.com/pingcap/tidb/issues/26166) [#41028](https://github.com/pingcap/tidb/issues/41028) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ++ TiKV + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ++ PD + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ++ TiFlash + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ++ Tools + + + Backup & Restore (BR) + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiCDC + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiDB Data Migration (DM) + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiDB Lightning + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiUP + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + +## Bug fixes + ++ TiDB + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ++ TiKV + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ++ PD + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ++ TiFlash + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + ++ Tools + + + Backup & Restore (BR) + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiCDC + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiDB Data Migration (DM) + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiDB Lightning + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + + + TiUP + + - note [#issue](链接) @[贡献者 GitHub ID](链接) + - note [#issue](链接) @[贡献者 GitHub ID](链接) + +## Contributors + +We would like to thank the following contributors from the TiDB community: + +- [Contributor GitHub ID](链接) \ No newline at end of file From 80e090966960a1dbac1a931b5eb3af1a4931ee0d Mon Sep 17 00:00:00 2001 From: Sam Dillard Date: Mon, 24 Jul 2023 15:10:24 -0400 Subject: [PATCH 04/72] add release highlights table and new one-off description for it. --- releases/release-7.3.0.md | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 04f8e8f80a4fe..718842a6a96ba 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -11,9 +11,26 @@ TiDB version: 7.3.0 Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with-tidb) | [Installation packages](https://www.pingcap.com/download/?version=v7.3.0#version-list) -7.3.0 introduces the following key features and improvements: - - +7.3.0 introduces the following major as generally available. The rest of the release (detailed in the Details section) was a series of enhancements to query stability in TiDB server and TiFlash. These are more miscellaneous in nature and not user-facing so they are not included in this section dedicated to release highlights: + + + + + + + + + + + + + + + + +
CategoryFeatureDescription
Scalability and PerformancePartitioned Raft KV GA. Learn more. Every key region will store its key-value data in its own isolated LSM tree (RocksDB). +This drastically improves write performance, reduces I/O amplication, speeds up scale-in/-out operations, and is a huge step toward TiDB handling beyond PB-scale workloads per cluster. +
## Feature details From f7f7d00dbc34cb49aa45b63b2fa825e4f479f755 Mon Sep 17 00:00:00 2001 From: Sam Dillard Date: Mon, 24 Jul 2023 15:12:43 -0400 Subject: [PATCH 05/72] add a word --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 718842a6a96ba..d048343dbe77f 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -11,7 +11,7 @@ TiDB version: 7.3.0 Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with-tidb) | [Installation packages](https://www.pingcap.com/download/?version=v7.3.0#version-list) -7.3.0 introduces the following major as generally available. The rest of the release (detailed in the Details section) was a series of enhancements to query stability in TiDB server and TiFlash. These are more miscellaneous in nature and not user-facing so they are not included in this section dedicated to release highlights: +7.3.0 introduces the following major feature as generally available. The rest of the release (detailed in the Details section) was a series of enhancements to query stability in TiDB server and TiFlash. These are more miscellaneous in nature and not user-facing so they are not included in this section dedicated to release highlights: From 4445cb3b2ed05bbd98e48da5de62f47febdac544 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 26 Jul 2023 15:12:16 +0800 Subject: [PATCH 06/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index d048343dbe77f..5058b37cd193c 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -23,8 +23,8 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - - + + From fa92400135eac86e3979c3b9e5915448d00cfb0e Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 26 Jul 2023 15:12:44 +0800 Subject: [PATCH 07/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 5058b37cd193c..fe66ed10b66bd 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -24,7 +24,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - + From ba8c793d92560da25cf6e3594154c21489c50644 Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 31 Jul 2023 18:54:03 +0800 Subject: [PATCH 08/72] add new features Signed-off-by: Ran --- releases/release-7.3.0.md | 98 ++++++++++++++++++++++++++++++++------- 1 file changed, 82 insertions(+), 16 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index fe66ed10b66bd..0f0772f3f1b72 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -36,21 +36,58 @@ This drastically improves write performance, reduces I/O amplication, speeds up ### Performance -- note 1 -- note 2 +* TiFlash 支持副本选择策略 [#44106](https://github.com/pingcap/tidb/issues/44106) @[XuHuaiyu](https://github.com/XuHuaiyu) **tw@qiancai** + + 在 v7.3.0 之前，当 TiFlash 进行数据扫描和 MPP 计算时，会尽可能使用其所有节点的副本，以提供最强大的性能。从 v7.3.0 起，TiFlash 引入副本选择策略，该策略由系统变量 [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-从-v730-版本开始引入) 控制，可以根据节点的[区域属性](/schedule-replicas-by-topology-labels.md#设置-tidb-的-labels可选)选择特定的副本，调度部分节点进行数据扫描及 MPP 计算。 + + 当集群部署在多个机房且每个机房都拥有完整的 TiFlash 数据副本时，你可以设置该策略只选择使用当前机房的 TiFlash 副本，即在当前机房的 TiFlash 节点中进行数据扫描和 MPP 计算，从而避免大量跨机房的网络数据传输。 + + 更多信息，请参考[用户文档](/system-variables.md/system-variables.md#tiflash_replica_read-从-v730-版本开始引入)。 + +* TiFlash 支持节点内的 Runtime Filter [#40220](https://github.com/pingcap/tidb/issues/40220) @[elsa0520](https://github.com/elsa0520) **tw@ran-huang** + + Runtime Filter 是一种在查询规划时生成的动态取值的谓词。在表连接的过程中，这些动态谓词能够进一步过滤掉不满足条件的行，减少扫描时间和网络开销，提升表连接的效率。自 v7.3.0 起，TiFlash 支持节点内的 Runtime Filter，提升了数据分析类查询的整体性能，在部分 TPC-DS 查询中可达到 10% ~ 50% 的性能提升。 + + 该特性在 v7.3.0 默认关闭。要启用此功能，需将变量 [`tidb_runtime_filter_mode`](#tidb_runtime_filter_mode-从-v720-版本开始引入) 设置为 `LOCAL`。 + + 更多信息，请参考[用户文档](/runtime-filter.md)。 + +* TiFlash 支持执行公共表表达式 (CTE)（实验特性）[#43333](https://github.com/pingcap/tidb/issues/43333) @[winoros](https://github.com/winoros) **tw@ran-huang** + + 在 v7.3.0 版本之前，TiFlash 的 MPP 引擎默认无法执行包含 CTE 的查询，你需要通过系统变量 [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-从-v630-版本开始引入) 将 CTE inline 展开，达到让查询尽可能在 MPP 框架下执行的效果。在 v7.3.0 中，TiFlash MPP 引擎支持执行包含 CTE 的查询，无需将 CTE inline 展开也可以尽可能地在 MPP 框架中执行查询。在 TPC-DS 基准测试中，与 inline 的执行方式相比，该功能可以将包含 CTE 的查询的总执行速度提升 20%。 + + 该功能为实验特性，默认关闭，由变量 [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-从-v720-版本开始引入) 控制。 ### Reliability -- note 1 +* 新增部分优化器提示 [#45520](https://github.com/pingcap/tidb/issues/45520) @[qw4990](https://github.com/qw4990) **tw@ran-huang** -- note 2 + TiDB 在 v7.3.0 新增了几个优化器提示，用来控制表之间的连接方式，包括： + + - [`INDEX_JOIN()`](链接) 选择 Index Nested Loop Join，利用索引过滤并将结果集作为内表连接。 + - [`NO_HASH_JOIN()`](链接) 选择哈希连接以外的连接方式。 + - [`NO_INDEX_HASH_JOIN()`](链接) 选择除 [Index Nested Loop Hash Join](/optimizer-hints#inl_hash_join) 以外的连接方式。 + + 更多信息，请参考[用户文档](/optimizer-hints)。 + +* 手工标记资源使用超出预期的查询 (实验特性) [#43691](https://github.com/pingcap/tidb/issues/43691) @[Connor1996](https://github.com/Connor1996) @[CabinfeverB](https://github.com/CabinfeverB) **tw@hfxsd** + + 在 v7.2.0 中，TiDB 对资源使用超出预期的查询 (Runaway Queries) 实施自动管理，运行时间超过预期的查询能够被自动降级或取消。在实际运行时，只依靠规则无法筛覆盖所有情况。 因此，在 v7.3.0 中，TiDB 补充了手工标记查询的能力。 利用新增的命令 [`QUERY WATCH`]()，用户可以根据 SQL 的文本、SQL Digest、或者执行计划对查询进行标记，命中的查询可以被降级或取消。 + + 手工标记 Runaway Queries 的能力，为数据库中突发的性能问题提供了有效的干预手段。针对由查询引发的性能问题，在找到问题根本原因之前，能够快速缓解其对整体性能的影响，提升系统服务质量。 + + 更多信息，请参考[用户文档](/tidb-resource-control#管理资源消耗超出预期的查询-runaway-queries)。 ### SQL -- note 1 +* List 和 List COLUMNS 分区表支持默认分区 [#20679](https://github.com/pingcap/tidb/issues/20679) @[mjonss](https://github.com/mjonss) @[bb7133](https://github.com/bb7133) **tw@qiancai** -- note 2 + 当使用 `INSERT` 语句向 List 或 List COLUMNS 分区表插入数据时，这些数据需要满足分区表指定的分区条件。如果要插入的数据不匹配任何分区条件，该语句将执行失败或不符合分区条件的数据被忽略。 + + 在 v7.3.0 中，List 和 List COLUMNS 分区表支持默认分区功能。在创建默认分区后，如果要插入的数据不匹配任何分区条件，则数据将被写入默认分区。默认分区功能可以提升 List 分区和 List COLUMNS 分区的使用便捷性，避免不符合分区条件的数据导致 `INSERT` 语句执行失败或者数据被忽略。该功能默认关闭，可通过 [`tidb_enable_default_list_partition`](/system-variables.md#tidb_enable_default_list_partition-new-in-v730) 变量开启。 + + 更多信息，请参考[用户文档](/partitioned-table.md#list-分区)。 ### DB operations @@ -60,15 +97,39 @@ This drastically improves write performance, reduces I/O amplication, speeds up ### Observability -- note 1 +* 显示统计信息收集的进度 [#issue号](链接) @[hawkingrei](https://github.com/hawkingrei) **tw@Oreoxmt** -- note 2 + 对大表的统计信息收集经常会持续比较长的时间。在过去的版本里，用户无从得知统计信息收集的进度，进而没法预测完成时间。在 v7.3.0 中，TiDB 加入了对统计信息收集进度的信息展示，能够显示各个子任务的总体工作量、当前进度、以及对完成时间的预测。在大规模数据导入、SQL 性能优化等场景下，用户能够了解整体任务进展，提升用户体验。 + + 更多信息，请参考[用户文档](链接)。 + +* Plan Replayer 支持导出历史统计信息 [#45038](https://github.com/pingcap/tidb/issues/45038) @[time-and-fate](https://github.com/time-and-fate) **tw@ran-huang** + + 自 v7.3.0 起，通过新增的 [`dump with stats as of timestamp`](/sql-plan-replayer.md) 子句，Plan Replayer 能够导出指定 SQL 相关对象在指定时间点的统计信息。在执行计划问题的诊断过程中，通过对历史统计信息的准确抓取，能够更精确地分析出执行计划在问题发生的时间点是如何生成的，从而找到问题的根本原因，大大提升执行计划问题的诊断效率。 + + 更多信息，请参考[用户文档](/sql-plan-replayer.md)。 ### Data migration -- note 1 +* Lightning 引入新版冲突数据检测与处理的能力 [#41629](https://github.com/pingcap/tidb/issues/41629) @[lance6716](https://github.com/lance6716) **tw@hfxsd** -- note 2 + 之前的版本 Lightning 逻辑导入和物理导入模式都有各自的冲突检测和处理的方式，配置较为复杂且不利于用户理解。同时使用物理导入模式，冲突的数据无法通过 replace 和 ignore 策略来处理。新版的冲突检测和处理方式，逻辑导入和物理导入都是用同一套冲突检测和处理方式即遇到冲突数据报错，或者 replace 以及 ignore 掉冲突数据。同时还支持用户设置冲突记录的上限，如处理多少冲突记录后任务中断退出，用户也可以让程序记录哪些数据发生了冲突，方便用户排查。 + + 在明确所需导入数据有较多的冲突数据时，推荐使用新版的冲突检测和处理策略，会有更好的性能。注意新、旧版冲突策略互斥使用，会在未来废弃掉旧版冲突检测和处理策略。 + + 更多信息，请参考[用户文档](链接)。 + +* Lightning 支持 Partitioned Raft KV（实验特性） [#15069](https://github.com/tikv/tikv/pull/15069) @[GMHDBJD](https://github.com/GMHDBJD) **tw@hfxsd** + + 该版本 Lightning 支持了 Partitioned Raft KV ，当用户使用了 Partitioned Raft KV 特性后，能提升 Lightning 导入数据的性能。 + + 更多信息，请参考[用户文档](链接)。 + +* Lightning 引入新的参数"enable-diagnose-log" 用于打印更多的诊断日志，方便定位问题 [#45497](https://github.com/pingcap/tidb/issues/45497) @[D3Hunter](https://github.com/D3Hunter) **tw@hfxsd** + + 默认情况下，此功能未启用，只会打印包含 "lightning/main" 的日志。当启用时，将打印所有包（包括 "client-go" 和 "tidb"）的日志，以帮助诊断与 "client-go" 和 "tidb" 相关的问题。 + + 更多信息，请参考[用户文档](链接)。 ## Compatibility changes @@ -80,7 +141,10 @@ This drastically improves write performance, reduces I/O amplication, speeds up -* 兼容性 1 +* TiDB Lightning **tw@hfxsd** + + - 逻辑导入模式插入冲突数据时执行的操作，默认配置从 on-duplicate = "replace" 改为 on-duplicate = "error" 即遇到冲突数据即报错。 + - TiDB Lightning 停止迁移任务之前能容忍的最大非严重 (non-fatal errors) 错误数的参数 "max-error" 不再包含导入数据冲突的上限。而是由新的参数 "conflict.threshold" 来控制可容忍的最大冲突的记录数。 * 兼容性 2 @@ -111,8 +175,9 @@ This drastically improves write performance, reduces I/O amplication, speeds up + TiDB - - 优化构造索引扫描范围的逻辑，支持将一些复杂条件转化为索引扫描范围 [#41572](https://github.com/pingcap/tidb/issues/41572) [#44389](https://github.com/pingcap/tidb/issues/44389) @xuyifangreeneyes - - 支持分区表下推 TopN 算子，减少该情况下 indexLookUp 和 IndexMerge 算子回表次数，提升相关 SQL 执行性能 [#26166](https://github.com/pingcap/tidb/issues/26166) [#41028](https://github.com/pingcap/tidb/issues/41028) + - 游标 (Cursor) 结果过大时，写入 TiDB 临时磁盘空间从而避免OOM [#43233](https://github.com/pingcap/tidb/issues/43233) @[YangKeao](https://github.com/YangKeao) + - EXPLAIN 新增开关用以展示在优化期间被执行的子查询 [#22076](https://github.com/pingcap/tidb/issues/22076) @[winoros](https://github.com/winoros] **tw@Oreoxmt** + - 在启用 [`Global Kill`](/tidb-configuration-file#enable-global-kill-从-v610-版本开始引入) 的情况下，可以通过 Ctrl+C 终止当前会话。 [#8854](https://github.com/pingcap/tidb/issues/8854) @[pingyu](https://github.com/pingyu) - note [#issue](链接) @[贡献者 GitHub ID](链接) + TiKV @@ -127,6 +192,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up + TiFlash + - 支持新的 DTFile 格式版本，减少物理文件数量（实验特性） [#7595](https://github.com/pingcap/tiflash/issues/7595) @[hongyunyan](https://github.com/hongyunyan) **tw@qiancai** - note [#issue](链接) @[贡献者 GitHub ID](链接) - note [#issue](链接) @[贡献者 GitHub ID](链接) @@ -134,12 +200,12 @@ This drastically improves write performance, reduces I/O amplication, speeds up + Backup & Restore (BR) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + - 为外部存储 Azure Blob Storage 提供加密范围和加密密钥的支持 [#45025](https://github.com/pingcap/tidb/issues/45025) @[Leavrth](https://github.com/Leavrth) **tw@Oreoxmt** - note [#issue](链接) @[贡献者 GitHub ID](链接) + TiCDC - - note [#issue](链接) @[贡献者 GitHub ID](链接) + - Kafka Sink 支持在消息过大时只发送 Handle Key 数据，减少数据大小 [#9382](https://github.com/pingcap/tiflow/issues/9382) @[3AceShowHand](https://github.com/3AceShowHand) **tw@ran-huang** - note [#issue](链接) @[贡献者 GitHub ID](链接) + TiDB Data Migration (DM) @@ -149,7 +215,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up + TiDB Lightning - - note [#issue](链接) @[贡献者 GitHub ID](链接) + - 更新 TiDB Lightning 并行导入的参数名称从 "tikv-importer.incremental-import" 变更为 “tikv-importer.parallel-import” ，避免用户误认为是增量导入而误用该参数。 [#45501](https://github.com/pingcap/tidb/issues/45501) @[lyzx2001](https://github.com/lyzx2001) **tw@hfxsd** - note [#issue](链接) @[贡献者 GitHub ID](链接) + TiUP From 5e7e84bb165e3e070f11413b669ee5c721bf7610 Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 31 Jul 2023 19:12:18 +0800 Subject: [PATCH 09/72] add translation Signed-off-by: Ran --- releases/release-7.3.0.md | 37 +++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 0f0772f3f1b72..8487023dac6ec 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -36,7 +36,6 @@ This drastically improves write performance, reduces I/O amplication, speeds up ### Performance - * TiFlash 支持副本选择策略 [#44106](https://github.com/pingcap/tidb/issues/44106) @[XuHuaiyu](https://github.com/XuHuaiyu) **tw@qiancai** 在 v7.3.0 之前，当 TiFlash 进行数据扫描和 MPP 计算时，会尽可能使用其所有节点的副本，以提供最强大的性能。从 v7.3.0 起，TiFlash 引入副本选择策略，该策略由系统变量 [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-从-v730-版本开始引入) 控制，可以根据节点的[区域属性](/schedule-replicas-by-topology-labels.md#设置-tidb-的-labels可选)选择特定的副本，调度部分节点进行数据扫描及 MPP 计算。 @@ -45,31 +44,33 @@ This drastically improves write performance, reduces I/O amplication, speeds up 更多信息，请参考[用户文档](/system-variables.md/system-variables.md#tiflash_replica_read-从-v730-版本开始引入)。 -* TiFlash 支持节点内的 Runtime Filter [#40220](https://github.com/pingcap/tidb/issues/40220) @[elsa0520](https://github.com/elsa0520) **tw@ran-huang** +* TiFlash supports Runtime Filter within nodes [#40220](https://github.com/pingcap/tidb/issues/40220) @[elsa0520](https://github.com/elsa0520) **tw@ran-huang** + + Runtime Filter is a predicate that generates dynamic values during query planning. During the process of table joining, these dynamic predicates can further filter out rows that do not meet the conditions, reducing scan time and network overhead, and improving the efficiency of table joining. Starting from v7.3.0, TiFlash supports Runtime Filter within nodes, improving the overall performance of analytical queries with performance improvements ranging from 10% to 50% in some TPC-DS workloads. - Runtime Filter 是一种在查询规划时生成的动态取值的谓词。在表连接的过程中，这些动态谓词能够进一步过滤掉不满足条件的行，减少扫描时间和网络开销，提升表连接的效率。自 v7.3.0 起，TiFlash 支持节点内的 Runtime Filter，提升了数据分析类查询的整体性能，在部分 TPC-DS 查询中可达到 10% ~ 50% 的性能提升。 + This feature is disabled by default in v7.3.0. To enable this feature, set the system variable [`tidb_runtime_filter_mode`](/system-variables.md#tidb_runtime_filter_mode-new-in-v720) to `LOCAL`. - 该特性在 v7.3.0 默认关闭。要启用此功能，需将变量 [`tidb_runtime_filter_mode`](#tidb_runtime_filter_mode-从-v720-版本开始引入) 设置为 `LOCAL`。 + For more information, refer to [user documentation](/runtime-filter.md). - 更多信息，请参考[用户文档](/runtime-filter.md)。 +* TiFlash supports executing common table expressions (CTEs) (experimental) [#43333](https://github.com/pingcap/tidb/issues/43333) @[winoros](https://github.com/winoros) **tw@ran-huang** -* TiFlash 支持执行公共表表达式 (CTE)（实验特性）[#43333](https://github.com/pingcap/tidb/issues/43333) @[winoros](https://github.com/winoros) **tw@ran-huang** + Before v7.3.0, the MPP engine of TiFlash cannot execute queries that contain CTEs by default. To achieve the best execution performance within the MPP framework, you need to use the system variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-introduced-since-v630) to inline expand CTE. - 在 v7.3.0 版本之前，TiFlash 的 MPP 引擎默认无法执行包含 CTE 的查询，你需要通过系统变量 [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-从-v630-版本开始引入) 将 CTE inline 展开，达到让查询尽可能在 MPP 框架下执行的效果。在 v7.3.0 中，TiFlash MPP 引擎支持执行包含 CTE 的查询，无需将 CTE inline 展开也可以尽可能地在 MPP 框架中执行查询。在 TPC-DS 基准测试中，与 inline 的执行方式相比，该功能可以将包含 CTE 的查询的总执行速度提升 20%。 + Starting from v7.3.0, TiFlash's MPP engine supports executing queries with CTEs without inline expanding them, allowing for optimal query execution within the MPP framework. In TPC-DS benchmark tests, compared to using inline expansion, this feature has shown a 20% improvement in overall query execution speed for queries containing CTE. - 该功能为实验特性，默认关闭，由变量 [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-从-v720-版本开始引入) 控制。 + This feature is an experimental feature and is disabled by default. It is controlled by the system variable [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-new-in-v720). ### Reliability -* 新增部分优化器提示 [#45520](https://github.com/pingcap/tidb/issues/45520) @[qw4990](https://github.com/qw4990) **tw@ran-huang** +* Add new optimizer hints [#45520](https://github.com/pingcap/tidb/issues/45520) @[qw4990](https://github.com/qw4990) **tw@ran-huang** - TiDB 在 v7.3.0 新增了几个优化器提示，用来控制表之间的连接方式，包括： + In v7.3.0, TiDB introduces several new optimizer hints to control the join methods between tables, including: - - [`INDEX_JOIN()`](链接) 选择 Index Nested Loop Join，利用索引过滤并将结果集作为内表连接。 - - [`NO_HASH_JOIN()`](链接) 选择哈希连接以外的连接方式。 - - [`NO_INDEX_HASH_JOIN()`](链接) 选择除 [Index Nested Loop Hash Join](/optimizer-hints#inl_hash_join) 以外的连接方式。 + - [`INDEX_JOIN()`](link) selects index nested loop join, which uses indexes to filter and use the result set as the inner table to join. + - [`NO_HASH_JOIN()`](link) selects join methods other than hash join. + - [`NO_INDEX_HASH_JOIN()`](link) selects join methods other than [index nested loop join](/optimizer-hints.md#inl_hash_join). - 更多信息，请参考[用户文档](/optimizer-hints)。 + For more information, refer to [user documentation](/optimizer-hints). * 手工标记资源使用超出预期的查询 (实验特性) [#43691](https://github.com/pingcap/tidb/issues/43691) @[Connor1996](https://github.com/Connor1996) @[CabinfeverB](https://github.com/CabinfeverB) **tw@hfxsd** @@ -103,11 +104,11 @@ This drastically improves write performance, reduces I/O amplication, speeds up 更多信息，请参考[用户文档](链接)。 -* Plan Replayer 支持导出历史统计信息 [#45038](https://github.com/pingcap/tidb/issues/45038) @[time-and-fate](https://github.com/time-and-fate) **tw@ran-huang** +* Plan Replayer supports exporting historical statistics [#45038](https://github.com/pingcap/tidb/issues/45038) @[time-and-fate](https://github.com/time-and-fate) **tw@ran-huang** - 自 v7.3.0 起，通过新增的 [`dump with stats as of timestamp`](/sql-plan-replayer.md) 子句，Plan Replayer 能够导出指定 SQL 相关对象在指定时间点的统计信息。在执行计划问题的诊断过程中，通过对历史统计信息的准确抓取，能够更精确地分析出执行计划在问题发生的时间点是如何生成的，从而找到问题的根本原因，大大提升执行计划问题的诊断效率。 + Starting from v7.3.0, with the newly added [`dump with stats as of timestamp`](/sql-plan-replayer.md) clause, Plan Replayer can export the statistics of specified SQL-related objects at a specific point in time. During the diagnosis of execution plan issues, accurately capturing historical statistics can help analyze more precisely how the execution plan was generated at the time when the issue occurred. This helps identify the root cause of the problem and greatly improves efficiency in diagnosing execution plan issues. - 更多信息，请参考[用户文档](/sql-plan-replayer.md)。 + For more information, refer to [user documentation](/sql-plan-replayer.md). ### Data migration @@ -205,7 +206,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up + TiCDC - - Kafka Sink 支持在消息过大时只发送 Handle Key 数据，减少数据大小 [#9382](https://github.com/pingcap/tiflow/issues/9382) @[3AceShowHand](https://github.com/3AceShowHand) **tw@ran-huang** + - Kafka Sink supports sending only handle key data when the message is too large, reducing the size of the message [#9382](https://github.com/pingcap/tiflow/issues/9382) @[3AceShowHand](https://github.com/3AceShowHand) **tw@ran-huang** - note [#issue](链接) @[贡献者 GitHub ID](链接) + TiDB Data Migration (DM) From ae2607132e0eee48854dc4cb4b1c62fd3aa250cc Mon Sep 17 00:00:00 2001 From: qiancai Date: Tue, 1 Aug 2023 14:15:19 +0800 Subject: [PATCH 10/72] add feature translation --- releases/release-7.3.0.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 8487023dac6ec..f004196ea6dd5 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -36,13 +36,13 @@ This drastically improves write performance, reduces I/O amplication, speeds up ### Performance -* TiFlash 支持副本选择策略 [#44106](https://github.com/pingcap/tidb/issues/44106) @[XuHuaiyu](https://github.com/XuHuaiyu) **tw@qiancai** +* TiFlash supports the replica selection strategy [#44106](https://github.com/pingcap/tidb/issues/44106) @[XuHuaiyu](https://github.com/XuHuaiyu) **tw@qiancai** - 在 v7.3.0 之前，当 TiFlash 进行数据扫描和 MPP 计算时，会尽可能使用其所有节点的副本，以提供最强大的性能。从 v7.3.0 起，TiFlash 引入副本选择策略，该策略由系统变量 [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-从-v730-版本开始引入) 控制，可以根据节点的[区域属性](/schedule-replicas-by-topology-labels.md#设置-tidb-的-labels可选)选择特定的副本，调度部分节点进行数据扫描及 MPP 计算。 + Before v7.3.0, TiFlash uses replicas from all its nodes for data scanning and MPP calculations to maximize performance. Starting from v7.3.0, TiFlash introduces the replica selection strategy and lets you configure it using the [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-new-in-v730) system variable. This strategy supports selecting specific replicas based on the [zone attributes](/schedule-replicas-by-topology-labels.md#optional-configure-labels-for-tidb) of nodes and scheduling specific nodes for data scanning and MPP calculations. - 当集群部署在多个机房且每个机房都拥有完整的 TiFlash 数据副本时，你可以设置该策略只选择使用当前机房的 TiFlash 副本，即在当前机房的 TiFlash 节点中进行数据扫描和 MPP 计算，从而避免大量跨机房的网络数据传输。 + For a cluster that is deployed in multiple data centers and each data center has complete TiFlash data replicas, you can configure this strategy to only select TiFlash replicas from the current data center. This means data scanning and MPP calculations are performed only on TiFlash nodes in the current data center, which avoids excessive network data transmission across data centers. - 更多信息，请参考[用户文档](/system-variables.md/system-variables.md#tiflash_replica_read-从-v730-版本开始引入)。 + For more information，see [documentation](/system-variables.md/system-variables.md#tiflash_replica_read-new-in-v730). * TiFlash supports Runtime Filter within nodes [#40220](https://github.com/pingcap/tidb/issues/40220) @[elsa0520](https://github.com/elsa0520) **tw@ran-huang** @@ -82,13 +82,15 @@ This drastically improves write performance, reduces I/O amplication, speeds up ### SQL -* List 和 List COLUMNS 分区表支持默认分区 [#20679](https://github.com/pingcap/tidb/issues/20679) @[mjonss](https://github.com/mjonss) @[bb7133](https://github.com/bb7133) **tw@qiancai** +* List and List COLUMNS partitioned tables support default partitions [#20679](https://github.com/pingcap/tidb/issues/20679) @[mjonss](https://github.com/mjonss) @[bb7133](https://github.com/bb7133) **tw@qiancai** - 当使用 `INSERT` 语句向 List 或 List COLUMNS 分区表插入数据时，这些数据需要满足分区表指定的分区条件。如果要插入的数据不匹配任何分区条件，该语句将执行失败或不符合分区条件的数据被忽略。 + When you use the `INSERT` statement to insert data into List or List COLUMNS partitioned tables, the data needs to meet the specified partitioning conditions of the table. If the data to be inserted does not meet any partitioning condition, the execution of the statement will fail, or the data that does not meet any partitioning condition will be ignored. - 在 v7.3.0 中，List 和 List COLUMNS 分区表支持默认分区功能。在创建默认分区后，如果要插入的数据不匹配任何分区条件，则数据将被写入默认分区。默认分区功能可以提升 List 分区和 List COLUMNS 分区的使用便捷性，避免不符合分区条件的数据导致 `INSERT` 语句执行失败或者数据被忽略。该功能默认关闭，可通过 [`tidb_enable_default_list_partition`](/system-variables.md#tidb_enable_default_list_partition-new-in-v730) 变量开启。 + Starting from v7.3.0, List and List COLUMNS partitioned tables support default partitions. After a default partition is created, if the data to be inserted does not meet any partitioning condition, it will be written to the default partition. The default partition feature improves the usability of List and List COLUMNS partitioning, avoiding the execution failure of the `INSERT` statement or data being ignored due to data that does not meet partitioning conditions. - 更多信息，请参考[用户文档](/partitioned-table.md#list-分区)。 + Note that this feature is a TiDB extension to MySQL syntax. For a partitioned table with a default partition, the data in the table cannot be directly replicated to MySQL. + + For more information, see [documentation](/partitioned-table.md#list-partition). ### DB operations @@ -193,7 +195,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up + TiFlash - - 支持新的 DTFile 格式版本，减少物理文件数量（实验特性） [#7595](https://github.com/pingcap/tiflash/issues/7595) @[hongyunyan](https://github.com/hongyunyan) **tw@qiancai** + - Support a new DTFile format version to reduce the number of physical files (experimental) [#7595](https://github.com/pingcap/tiflash/issues/7595) @[hongyunyan](https://github.com/hongyunyan) **tw@qiancai** - note [#issue](链接) @[贡献者 GitHub ID](链接) - note [#issue](链接) @[贡献者 GitHub ID](链接) From add592841bb7a5bfbdac32dfa9ddf858edb805b9 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 1 Aug 2023 15:07:41 +0800 Subject: [PATCH 11/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index f004196ea6dd5..041d6be726d68 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -168,6 +168,8 @@ This drastically improves write performance, reduces I/O amplication, speeds up | | | | | | TiDB Lightning | `send-kv-pairs` | 废弃 | 从 7.2 版本开始 TiDB Lightning 配置文件的参数 "send-kv-pairs" 不再生效，由新的参数 "send-kv-size" 代替。该新参数用于指定 KV 键值对的大小阈值，单位为 KiB 或 MiB，默认值为 "16 KiB"。当 KV 键值对的大小达到设定的阈值时，它们将立即发送到 TiKV，避免在导入大宽表等一些场景因为 Lightning 节点内存积累键值对过多导致 OOM 的问题。**tw@hfxsd** | | TiDB Lightning | `send-kv-size` | 新增 | 从 7.2 版本开始在 Lightning 配置文件 "[tikv-importer]" 这个 Session 中引入 `send-kv-size` 参数，用于设置发单次送到 TiKV 的 KV pairs 的大小。当 KV 键值对的大小达到设定的阈值时，它们将被 Lightning 立即发送到 TiKV，避免在导入大宽表的时候 Lightning 节点因为内存积累键值对过多导致 OOM 的问题。通过调整 "send-kv-size" 参数，你可以在内存使用和导入速度之间找到平衡，提高导入过程的稳定性和效率。**tw@hfxsd** | +|TiDB Lightning | `tikv-importer.parallel-import` | 新增 | TiDB Lightning 并行导入参数。用于替代原有的 `tikv-importer.incremental-import` 参数，因为原有的参数容易被误认为是增量导入的参数而导致误用。 **tw:qiancai** | +|TiDB Lightning | tikv-importer.incremental-import | 删除 | TiDB Lightning 并行导入参数的旧名称，因为该参数名会被误认为增量导入的参数而误用，因此用新的参数名 tikv-importer.parallel-import 代替，且如果用户传入旧的参数会被自动转成新的参数名| | Data Migration | `strict-optimistic-shard-mode` | 新增 | 用于兼容历史版本 2.0 分库分表同步 DDL 的行为。当用户选择乐观模式时，可以启用该参数，开启后，乐观模式下，同步任务遇到二类 DDL 时，整个任务会中断，在多个表的 DDL变更有依赖关系的场景，可以及时中断，用户手动处理完各表的 DDL 后，再继续同步数据，保障上下游数据的一致性。 **tw@ran-huang** | ## Deprecated features From f38cd48a03c81ebc630f64a141756a0c7f9f917a Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 1 Aug 2023 16:18:39 +0800 Subject: [PATCH 12/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 041d6be726d68..23c4302b69339 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -168,8 +168,8 @@ This drastically improves write performance, reduces I/O amplication, speeds up | | | | | | TiDB Lightning | `send-kv-pairs` | 废弃 | 从 7.2 版本开始 TiDB Lightning 配置文件的参数 "send-kv-pairs" 不再生效，由新的参数 "send-kv-size" 代替。该新参数用于指定 KV 键值对的大小阈值，单位为 KiB 或 MiB，默认值为 "16 KiB"。当 KV 键值对的大小达到设定的阈值时，它们将立即发送到 TiKV，避免在导入大宽表等一些场景因为 Lightning 节点内存积累键值对过多导致 OOM 的问题。**tw@hfxsd** | | TiDB Lightning | `send-kv-size` | 新增 | 从 7.2 版本开始在 Lightning 配置文件 "[tikv-importer]" 这个 Session 中引入 `send-kv-size` 参数，用于设置发单次送到 TiKV 的 KV pairs 的大小。当 KV 键值对的大小达到设定的阈值时，它们将被 Lightning 立即发送到 TiKV，避免在导入大宽表的时候 Lightning 节点因为内存积累键值对过多导致 OOM 的问题。通过调整 "send-kv-size" 参数，你可以在内存使用和导入速度之间找到平衡，提高导入过程的稳定性和效率。**tw@hfxsd** | -|TiDB Lightning | `tikv-importer.parallel-import` | 新增 | TiDB Lightning 并行导入参数。用于替代原有的 `tikv-importer.incremental-import` 参数，因为原有的参数容易被误认为是增量导入的参数而导致误用。 **tw:qiancai** | -|TiDB Lightning | tikv-importer.incremental-import | 删除 | TiDB Lightning 并行导入参数的旧名称，因为该参数名会被误认为增量导入的参数而误用，因此用新的参数名 tikv-importer.parallel-import 代替，且如果用户传入旧的参数会被自动转成新的参数名| +|TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. **tw:qiancai** | +|TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | | Data Migration | `strict-optimistic-shard-mode` | 新增 | 用于兼容历史版本 2.0 分库分表同步 DDL 的行为。当用户选择乐观模式时，可以启用该参数，开启后，乐观模式下，同步任务遇到二类 DDL 时，整个任务会中断，在多个表的 DDL变更有依赖关系的场景，可以及时中断，用户手动处理完各表的 DDL 后，再继续同步数据，保障上下游数据的一致性。 **tw@ran-huang** | ## Deprecated features From 4a8d4176d78d1f5e9d76e92ead1257cd82289f65 Mon Sep 17 00:00:00 2001 From: Ran Date: Tue, 1 Aug 2023 18:23:41 +0800 Subject: [PATCH 13/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 23c4302b69339..08c081f7a3b20 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -171,6 +171,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up |TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. **tw:qiancai** | |TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | | Data Migration | `strict-optimistic-shard-mode` | 新增 | 用于兼容历史版本 2.0 分库分表同步 DDL 的行为。当用户选择乐观模式时，可以启用该参数，开启后，乐观模式下，同步任务遇到二类 DDL 时，整个任务会中断，在多个表的 DDL变更有依赖关系的场景，可以及时中断，用户手动处理完各表的 DDL 后，再继续同步数据，保障上下游数据的一致性。 **tw@ran-huang** | +| TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | ## Deprecated features From e8f60a5a9f7a5fa4b2b2b445ce01b986bb04c90e Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 2 Aug 2023 11:28:05 +0800 Subject: [PATCH 14/72] add feature translation --- releases/release-7.3.0.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 08c081f7a3b20..5db87b67f8c62 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -100,11 +100,11 @@ This drastically improves write performance, reduces I/O amplication, speeds up ### Observability -* 显示统计信息收集的进度 [#issue号](链接) @[hawkingrei](https://github.com/hawkingrei) **tw@Oreoxmt** +* Show the progress of collecting statistics [#issue号](链接) @[hawkingrei](https://github.com/hawkingrei) **tw@Oreoxmt** - 对大表的统计信息收集经常会持续比较长的时间。在过去的版本里，用户无从得知统计信息收集的进度，进而没法预测完成时间。在 v7.3.0 中，TiDB 加入了对统计信息收集进度的信息展示，能够显示各个子任务的总体工作量、当前进度、以及对完成时间的预测。在大规模数据导入、SQL 性能优化等场景下，用户能够了解整体任务进展，提升用户体验。 + Collecting statistics for large tables often takes a long time. In previous versions, you cannot see the progress of collecting statistics, and therefore cannot predict the completion time. TiDB v7.3.0 introduces a feature to show the progress of collecting statistics. You can view the overall workload, current progress, and estimated completion time for each subtask using the system table `mysql.analyze_jobs` or `SHOW ANALYZE STATUS`. In scenarios such as large-scale data import and SQL performance optimization, this feature helps you understand the overall task progress and improves the user experience. - 更多信息，请参考[用户文档](链接)。 + For more information, see [documentation](/sql-statements/sql-statement-show-analyze-status.md). * Plan Replayer supports exporting historical statistics [#45038](https://github.com/pingcap/tidb/issues/45038) @[time-and-fate](https://github.com/time-and-fate) **tw@ran-huang** @@ -182,8 +182,8 @@ This drastically improves write performance, reduces I/O amplication, speeds up + TiDB - 游标 (Cursor) 结果过大时，写入 TiDB 临时磁盘空间从而避免OOM [#43233](https://github.com/pingcap/tidb/issues/43233) @[YangKeao](https://github.com/YangKeao) - - EXPLAIN 新增开关用以展示在优化期间被执行的子查询 [#22076](https://github.com/pingcap/tidb/issues/22076) @[winoros](https://github.com/winoros] **tw@Oreoxmt** - - 在启用 [`Global Kill`](/tidb-configuration-file#enable-global-kill-从-v610-版本开始引入) 的情况下，可以通过 Ctrl+C 终止当前会话。 [#8854](https://github.com/pingcap/tidb/issues/8854) @[pingyu](https://github.com/pingyu) + - Introduce a new system variable `tidb_opt_enable_non_eval_scalar_subquery` to control whether the `EXPLAIN` statement executes subqueries in advance during the optimization phase [#22076](https://github.com/pingcap/tidb/issues/22076) @[winoros](https://github.com/winoros) **tw@Oreoxmt** + - When [Global Kill](/tidb-configuration-file#enable-global-kill-new-in-v610) is enabled, you can terminate the current session by pressing Control+C [#8854](https://github.com/pingcap/tidb/issues/8854) @[pingyu](https://github.com/pingyu) **tw@Oreoxmt** - note [#issue](链接) @[贡献者 GitHub ID](链接) + TiKV @@ -206,7 +206,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up + Backup & Restore (BR) - - 为外部存储 Azure Blob Storage 提供加密范围和加密密钥的支持 [#45025](https://github.com/pingcap/tidb/issues/45025) @[Leavrth](https://github.com/Leavrth) **tw@Oreoxmt** + - When backing up data to Azure Blob Storage using BR, you can specify either an encryption scope or an encryption key for server-side encryption [#45025](https://github.com/pingcap/tidb/issues/45025) @[Leavrth](https://github.com/Leavrth) **tw@Oreoxmt** - note [#issue](链接) @[贡献者 GitHub ID](链接) + TiCDC From 3b88712d6411462ee3cdcbd371a87f45839575bf Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 2 Aug 2023 13:56:37 +0800 Subject: [PATCH 15/72] Apply suggestions from code review --- releases/release-7.3.0.md | 49 ++++++++++++++++++--------------------- 1 file changed, 22 insertions(+), 27 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 5db87b67f8c62..afa2b0ffc4fb3 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -72,13 +72,13 @@ This drastically improves write performance, reduces I/O amplication, speeds up For more information, refer to [user documentation](/optimizer-hints). -* 手工标记资源使用超出预期的查询 (实验特性) [#43691](https://github.com/pingcap/tidb/issues/43691) @[Connor1996](https://github.com/Connor1996) @[CabinfeverB](https://github.com/CabinfeverB) **tw@hfxsd** +* Manually mark queries that use resources more than expected (experimental) [#43691](https://github.com/pingcap/tidb/issues/43691) @[Connor1996](https://github.com/Connor1996) @[CabinfeverB](https://github.com/CabinfeverB) **tw@hfxsd** - 在 v7.2.0 中，TiDB 对资源使用超出预期的查询 (Runaway Queries) 实施自动管理，运行时间超过预期的查询能够被自动降级或取消。在实际运行时，只依靠规则无法筛覆盖所有情况。 因此，在 v7.3.0 中，TiDB 补充了手工标记查询的能力。 利用新增的命令 [`QUERY WATCH`]()，用户可以根据 SQL 的文本、SQL Digest、或者执行计划对查询进行标记，命中的查询可以被降级或取消。 + In v7.2.0, TiDB automatically manages runaway queries, where queries that take longer than expected can be automatically demoted or canceled. In practice, it is not possible to cover all cases by relying on rules alone. Therefore, in v7.3.0, TiDB adds the ability to manually mark queries. With the new command [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md), you can mark queries based on SQL text, SQL Digest, or execution plan. Queries that are marked can be downgraded or canceled. - 手工标记 Runaway Queries 的能力，为数据库中突发的性能问题提供了有效的干预手段。针对由查询引发的性能问题，在找到问题根本原因之前，能够快速缓解其对整体性能的影响，提升系统服务质量。 + The ability of manually marking runaway queries provides an effective means of intervening in unexpected performance problems in the database. For query-induced performance problems, the impact on overall performance can be quickly mitigated before the root cause of the problem is found, improving system service quality. - 更多信息，请参考[用户文档](/tidb-resource-control#管理资源消耗超出预期的查询-runaway-queries)。 + For more information, refer to [user documentation](/tidb-resource-control.md#query-watch-parameters) ### SQL @@ -114,25 +114,19 @@ This drastically improves write performance, reduces I/O amplication, speeds up ### Data migration -* Lightning 引入新版冲突数据检测与处理的能力 [#41629](https://github.com/pingcap/tidb/issues/41629) @[lance6716](https://github.com/lance6716) **tw@hfxsd** +* TiDB Lightning introduces a new version of conflict data detection and processing capabilities [#41629](https://github.com/pingcap/tidb/issues/41629) @[lance6716](https://github.com/lance6716) **tw@hfxsd** + + Previous versions of TiDB Lightning used different conflict detection and handling methods for logical and physical import modes, which were complicated to configure and not easy for users to understand. In addition, when you use physical import mode, conflicting data cannot be handled by the `replace` and `ignore` policies. In the new version of conflict detection and handling, both logical import mode and physical import mode use the same set of conflict detection and handling, that is, reporting error (`error`), replacing (`replace`) or ignoring (`ignore`) conflicting data when encountering conflicting data. It also allows you to set an upper limit on the number of conflict records, such as how many conflict records should be processed before the task is interrupted and exits. You can also let the program record the data in conflict for easy troubleshooting. - 之前的版本 Lightning 逻辑导入和物理导入模式都有各自的冲突检测和处理的方式，配置较为复杂且不利于用户理解。同时使用物理导入模式，冲突的数据无法通过 replace 和 ignore 策略来处理。新版的冲突检测和处理方式，逻辑导入和物理导入都是用同一套冲突检测和处理方式即遇到冲突数据报错，或者 replace 以及 ignore 掉冲突数据。同时还支持用户设置冲突记录的上限，如处理多少冲突记录后任务中断退出，用户也可以让程序记录哪些数据发生了冲突，方便用户排查。 + When it is clear that the import data has a high amount of conflicting data, it is recommended to use the new version of the conflict detection and handling strategy for better performance. Note that the new version and the old version of the conflict policy are mutually exclusive, and cannot be used at the same time. The old conflict detection and handling policy will be deprecated in the future. + + For more information, refer to [user documentation](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#conflict-detection). - 在明确所需导入数据有较多的冲突数据时，推荐使用新版的冲突检测和处理策略，会有更好的性能。注意新、旧版冲突策略互斥使用，会在未来废弃掉旧版冲突检测和处理策略。 - - 更多信息，请参考[用户文档](链接)。 - -* Lightning 支持 Partitioned Raft KV（实验特性） [#15069](https://github.com/tikv/tikv/pull/15069) @[GMHDBJD](https://github.com/GMHDBJD) **tw@hfxsd** - - 该版本 Lightning 支持了 Partitioned Raft KV ，当用户使用了 Partitioned Raft KV 特性后，能提升 Lightning 导入数据的性能。 - - 更多信息，请参考[用户文档](链接)。 - -* Lightning 引入新的参数"enable-diagnose-log" 用于打印更多的诊断日志，方便定位问题 [#45497](https://github.com/pingcap/tidb/issues/45497) @[D3Hunter](https://github.com/D3Hunter) **tw@hfxsd** - - 默认情况下，此功能未启用，只会打印包含 "lightning/main" 的日志。当启用时，将打印所有包（包括 "client-go" 和 "tidb"）的日志，以帮助诊断与 "client-go" 和 "tidb" 相关的问题。 - - 更多信息，请参考[用户文档](链接)。 +* TiDB Lightning introduces a new parameter `enable-diagnose-log` to print more diagnostic logs and make it easier to pinpoint problems [#45497](https://github.com/pingcap/tidb/issues/45497) @[D3Hunter](https://github.com/D3Hunter) **tw@hfxsd** + + By default, this feature is not enabled and only prints logs containing `lightning/main`. When enabled, it prints logs for all packages (including `client-go` and `tidb`) to help diagnose problems related to `client-go` and `tidb`. + + For more information, refer to [user documentation](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-global). ## Compatibility changes @@ -146,8 +140,8 @@ This drastically improves write performance, reduces I/O amplication, speeds up * TiDB Lightning **tw@hfxsd** - - 逻辑导入模式插入冲突数据时执行的操作，默认配置从 on-duplicate = "replace" 改为 on-duplicate = "error" 即遇到冲突数据即报错。 - - TiDB Lightning 停止迁移任务之前能容忍的最大非严重 (non-fatal errors) 错误数的参数 "max-error" 不再包含导入数据冲突的上限。而是由新的参数 "conflict.threshold" 来控制可容忍的最大冲突的记录数。 + - `tikv-importer.on-duplicate` is deprecated and replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). + - The `max-error` parameter for the maximum number of non-fatal errors that can be tolerated before TiDB Lightning stops the migration task no longer contains an upper limit for import data conflicts. A new parameter [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) controls the maximum number of conflicting records that can be tolerated. * 兼容性 2 @@ -166,10 +160,11 @@ This drastically improves write performance, reduces I/O amplication, speeds up | -------- | -------- | -------- | -------- | | | | | | | | | | | -| TiDB Lightning | `send-kv-pairs` | 废弃 | 从 7.2 版本开始 TiDB Lightning 配置文件的参数 "send-kv-pairs" 不再生效，由新的参数 "send-kv-size" 代替。该新参数用于指定 KV 键值对的大小阈值，单位为 KiB 或 MiB，默认值为 "16 KiB"。当 KV 键值对的大小达到设定的阈值时，它们将立即发送到 TiKV，避免在导入大宽表等一些场景因为 Lightning 节点内存积累键值对过多导致 OOM 的问题。**tw@hfxsd** | -| TiDB Lightning | `send-kv-size` | 新增 | 从 7.2 版本开始在 Lightning 配置文件 "[tikv-importer]" 这个 Session 中引入 `send-kv-size` 参数，用于设置发单次送到 TiKV 的 KV pairs 的大小。当 KV 键值对的大小达到设定的阈值时，它们将被 Lightning 立即发送到 TiKV，避免在导入大宽表的时候 Lightning 节点因为内存积累键值对过多导致 OOM 的问题。通过调整 "send-kv-size" 参数，你可以在内存使用和导入速度之间找到平衡，提高导入过程的稳定性和效率。**tw@hfxsd** | -|TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. **tw:qiancai** | -|TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | +|TiDB Lightning | `conflict.max-record-rows` | Newly added | The new version of strategy to handle conflicting data. It controls the maximum number of rows in the `conflict_records` table. The default value is 100. | +|TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | +|TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace”` or `conflict.strategy=“ignore"`, you can set it as a maxint. | +|TiDB Lightning | `enable-diagnose-logs` | Newly added | Controls whether to enable the diagnostic logs. The default value is `false`, that is, only the logs related to the import are output, and the logs of other dependent components are not output. When you set it to `true`, logs from both the import process and other dependent components are output, and GRPC debugging is enabled, which can be used for diagnosis. | +|TiDB Lightning | `tikv-importer.on-duplicate` | Deprecated | Controls action to do when trying to insert a conflicting record in the logical import mode. Startign from v7.3.0, this parameter is replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). | | Data Migration | `strict-optimistic-shard-mode` | 新增 | 用于兼容历史版本 2.0 分库分表同步 DDL 的行为。当用户选择乐观模式时，可以启用该参数，开启后，乐观模式下，同步任务遇到二类 DDL 时，整个任务会中断，在多个表的 DDL变更有依赖关系的场景，可以及时中断，用户手动处理完各表的 DDL 后，再继续同步数据，保障上下游数据的一致性。 **tw@ran-huang** | | TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | From 7711ec5af67d6b25e6eaf4c7428dae058f539cac Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 2 Aug 2023 15:58:15 +0800 Subject: [PATCH 16/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index afa2b0ffc4fb3..3afd77b19e51b 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -150,7 +150,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up | 变量名 | 修改类型（包括新增/修改/删除） | 描述 | |--------|------------------------------|------| | [`tidb_remove_orderby_in_subquery`](/system-variables.md#tidb_remove_orderby_in_subquery-从-v610-版本开始引入) | 修改 | 经进一步的测试后，该变量默认值从 `OFF` 修改为 `ON`，即优化器改写会移除子查询中的 `ORDER BY` 子句。 | -| | | | +| [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-new-in-v730) | Newly added | Controls the strategy for selecting TiFlash replicas when a query requires the TiFlash engine. | | | | | | | | | From 9095fb5baee32f53c9e797b82bf488ede353142c Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 2 Aug 2023 16:14:08 +0800 Subject: [PATCH 17/72] added `tikv-importer.parallel-import` --- releases/release-7.3.0.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 3afd77b19e51b..dfebfefba70d3 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -165,6 +165,8 @@ This drastically improves write performance, reduces I/O amplication, speeds up |TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace”` or `conflict.strategy=“ignore"`, you can set it as a maxint. | |TiDB Lightning | `enable-diagnose-logs` | Newly added | Controls whether to enable the diagnostic logs. The default value is `false`, that is, only the logs related to the import are output, and the logs of other dependent components are not output. When you set it to `true`, logs from both the import process and other dependent components are output, and GRPC debugging is enabled, which can be used for diagnosis. | |TiDB Lightning | `tikv-importer.on-duplicate` | Deprecated | Controls action to do when trying to insert a conflicting record in the logical import mode. Startign from v7.3.0, this parameter is replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). | +|TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | +|TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. **tw:qiancai** | | Data Migration | `strict-optimistic-shard-mode` | 新增 | 用于兼容历史版本 2.0 分库分表同步 DDL 的行为。当用户选择乐观模式时，可以启用该参数，开启后，乐观模式下，同步任务遇到二类 DDL 时，整个任务会中断，在多个表的 DDL变更有依赖关系的场景，可以及时中断，用户手动处理完各表的 DDL 后，再继续同步数据，保障上下游数据的一致性。 **tw@ran-huang** | | TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | From 79a5696bce96017a309bfbe014746560908f6737 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 2 Aug 2023 16:18:43 +0800 Subject: [PATCH 18/72] Apply suggestions from code review Co-authored-by: Ran --- releases/release-7.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index dfebfefba70d3..174f05530b28f 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -84,9 +84,9 @@ This drastically improves write performance, reduces I/O amplication, speeds up * List and List COLUMNS partitioned tables support default partitions [#20679](https://github.com/pingcap/tidb/issues/20679) @[mjonss](https://github.com/mjonss) @[bb7133](https://github.com/bb7133) **tw@qiancai** - When you use the `INSERT` statement to insert data into List or List COLUMNS partitioned tables, the data needs to meet the specified partitioning conditions of the table. If the data to be inserted does not meet any partitioning condition, the execution of the statement will fail, or the data that does not meet any partitioning condition will be ignored. + Before v7.3.0, when you use the `INSERT` statement to insert data into List or List COLUMNS partitioned tables, the data needs to meet the specified partitioning conditions of the table. If the data to be inserted does not meet any of these conditions, either the execution of the statement will fail or the non-compliant data will be ignored. - Starting from v7.3.0, List and List COLUMNS partitioned tables support default partitions. After a default partition is created, if the data to be inserted does not meet any partitioning condition, it will be written to the default partition. The default partition feature improves the usability of List and List COLUMNS partitioning, avoiding the execution failure of the `INSERT` statement or data being ignored due to data that does not meet partitioning conditions. + Starting from v7.3.0, List and List COLUMNS partitioned tables support default partitions. After a default partition is created, if the data to be inserted does not meet any partitioning condition, it will be written to the default partition. This feature improves the usability of List and List COLUMNS partitioning, avoiding the execution failure of the `INSERT` statement or data being ignored due to data that does not meet partitioning conditions. Note that this feature is a TiDB extension to MySQL syntax. For a partitioned table with a default partition, the data in the table cannot be directly replicated to MySQL. From aceb48cd7956010aac87eeec4389813138389c87 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 2 Aug 2023 16:33:58 +0800 Subject: [PATCH 19/72] Apply suggestions from code review Co-authored-by: Aolin --- releases/release-7.3.0.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 174f05530b28f..33ac6e5400bda 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -74,11 +74,11 @@ This drastically improves write performance, reduces I/O amplication, speeds up * Manually mark queries that use resources more than expected (experimental) [#43691](https://github.com/pingcap/tidb/issues/43691) @[Connor1996](https://github.com/Connor1996) @[CabinfeverB](https://github.com/CabinfeverB) **tw@hfxsd** - In v7.2.0, TiDB automatically manages runaway queries, where queries that take longer than expected can be automatically demoted or canceled. In practice, it is not possible to cover all cases by relying on rules alone. Therefore, in v7.3.0, TiDB adds the ability to manually mark queries. With the new command [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md), you can mark queries based on SQL text, SQL Digest, or execution plan. Queries that are marked can be downgraded or canceled. + In v7.2.0, TiDB automatically manages queries that use resources more than expected (Runaway Query) by automatically downgrading or canceling runaway queries. In actual practice, rules alone cannot cover all cases. Therefore, TiDB v7.3.0 introduces the ability to manually mark runaway queries. With the new command [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md), you can mark runaway queries based on SQL text, SQL Digest, or execution plan, and the marked runaway queries can be downgraded or cancelled. - The ability of manually marking runaway queries provides an effective means of intervening in unexpected performance problems in the database. For query-induced performance problems, the impact on overall performance can be quickly mitigated before the root cause of the problem is found, improving system service quality. + This feature provides an effective intervention method for sudden performance issues in the database. For performance issues caused by queries, before identifying the root cause, this feature can quickly alleviate its impact on overall performance, thereby improving system service quality. - For more information, refer to [user documentation](/tidb-resource-control.md#query-watch-parameters) + For more information, see [documentation](/tidb-resource-control.md#query-watch-parameters). ### SQL @@ -114,19 +114,19 @@ This drastically improves write performance, reduces I/O amplication, speeds up ### Data migration -* TiDB Lightning introduces a new version of conflict data detection and processing capabilities [#41629](https://github.com/pingcap/tidb/issues/41629) @[lance6716](https://github.com/lance6716) **tw@hfxsd** - - Previous versions of TiDB Lightning used different conflict detection and handling methods for logical and physical import modes, which were complicated to configure and not easy for users to understand. In addition, when you use physical import mode, conflicting data cannot be handled by the `replace` and `ignore` policies. In the new version of conflict detection and handling, both logical import mode and physical import mode use the same set of conflict detection and handling, that is, reporting error (`error`), replacing (`replace`) or ignoring (`ignore`) conflicting data when encountering conflicting data. It also allows you to set an upper limit on the number of conflict records, such as how many conflict records should be processed before the task is interrupted and exits. You can also let the program record the data in conflict for easy troubleshooting. +* TiDB Lightning introduces a new version of conflict data detection and handling strategy [#41629](https://github.com/pingcap/tidb/issues/41629) @[lance6716](https://github.com/lance6716) **tw@hfxsd** - When it is clear that the import data has a high amount of conflicting data, it is recommended to use the new version of the conflict detection and handling strategy for better performance. Note that the new version and the old version of the conflict policy are mutually exclusive, and cannot be used at the same time. The old conflict detection and handling policy will be deprecated in the future. - - For more information, refer to [user documentation](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#conflict-detection). + In previous versions, TiDB Lightning uses different conflict detection and handling methods for Logical Import Mode and Physical Import Mode, which are complex to configure and not easy for users to understand. In addition, Physical Import Mode cannot handle conflicts using the `replace` or `ignore` policy. Starting from v7.3.0, TiDB Lightning introduces a unified conflict detection and handling strategy for both Logical Import Mode and Physical Import Mode. You can choose to report an error (`error`), replace (`replace`) or ignore (`ignore`) conflicting data when encountering conflicts. You can limit the number of conflict records, such as the task is interrupted and terminated after processing a specified number of conflict records. Furthermore, the system can record conflicting data for troubleshooting. -* TiDB Lightning introduces a new parameter `enable-diagnose-log` to print more diagnostic logs and make it easier to pinpoint problems [#45497](https://github.com/pingcap/tidb/issues/45497) @[D3Hunter](https://github.com/D3Hunter) **tw@hfxsd** - - By default, this feature is not enabled and only prints logs containing `lightning/main`. When enabled, it prints logs for all packages (including `client-go` and `tidb`) to help diagnose problems related to `client-go` and `tidb`. - - For more information, refer to [user documentation](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-global). + For import data with many conflicts, it is recommended to use the new version of the conflict detection and handling strategy for better performance. Note that the new version and the old version of the conflict strategy cannot be used at the same time. The old conflict detection and handling strategy will be deprecated in the future. + + For more information, see [documentation](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#conflict-detection). + +* TiDB Lightning introduces a new parameter `enable-diagnose-log` to enhance troubleshooting by printing more diagnostic logs [#45497](https://github.com/pingcap/tidb/issues/45497) @[D3Hunter](https://github.com/D3Hunter) **tw@hfxsd** + + By default, this feature is disabled and TiDB Lightning only prints logs containing `lightning/main`. When enabled, TiDB Lightning prints logs for all packages (including `client-go` and `tidb`) to help diagnose issues related to `client-go` and `tidb`. + + For more information, see [documentation](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-global). ## Compatibility changes @@ -141,7 +141,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up * TiDB Lightning **tw@hfxsd** - `tikv-importer.on-duplicate` is deprecated and replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). - - The `max-error` parameter for the maximum number of non-fatal errors that can be tolerated before TiDB Lightning stops the migration task no longer contains an upper limit for import data conflicts. A new parameter [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) controls the maximum number of conflicting records that can be tolerated. + - The `max-error` parameter, which controls the maximum number of non-fatal errors that TiDB Lightning can tolerate before stopping the migration task, no longer limits import data conflicts. The [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) parameter now controls the maximum number of conflicting records that can be tolerated. * 兼容性 2 From 32e627b4c9082d15aa000c5a75f240737a018c14 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 2 Aug 2023 16:44:14 +0800 Subject: [PATCH 20/72] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 33ac6e5400bda..646007146c27a 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -46,7 +46,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up * TiFlash supports Runtime Filter within nodes [#40220](https://github.com/pingcap/tidb/issues/40220) @[elsa0520](https://github.com/elsa0520) **tw@ran-huang** - Runtime Filter is a predicate that generates dynamic values during query planning. During the process of table joining, these dynamic predicates can further filter out rows that do not meet the conditions, reducing scan time and network overhead, and improving the efficiency of table joining. Starting from v7.3.0, TiFlash supports Runtime Filter within nodes, improving the overall performance of analytical queries with performance improvements ranging from 10% to 50% in some TPC-DS workloads. + Runtime Filter is a **dynamic predicate** generated during the query planning phase. In the process of table joining, these dynamic predicates can effectively filter out rows that do not meet the join conditions, reducing scan time and network overhead, and improving the efficiency of table joining. Starting from v7.3.0, TiFlash supports Runtime Filter within nodes, improving the overall performance of analytical queries. In some TPC-DS workloads, the performance can be improved by 10% to 50%. This feature is disabled by default in v7.3.0. To enable this feature, set the system variable [`tidb_runtime_filter_mode`](/system-variables.md#tidb_runtime_filter_mode-new-in-v720) to `LOCAL`. @@ -58,7 +58,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up Starting from v7.3.0, TiFlash's MPP engine supports executing queries with CTEs without inline expanding them, allowing for optimal query execution within the MPP framework. In TPC-DS benchmark tests, compared to using inline expansion, this feature has shown a 20% improvement in overall query execution speed for queries containing CTE. - This feature is an experimental feature and is disabled by default. It is controlled by the system variable [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-new-in-v720). + This feature is experimental and is disabled by default. It is controlled by the system variable [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-new-in-v720). ### Reliability @@ -66,9 +66,9 @@ This drastically improves write performance, reduces I/O amplication, speeds up In v7.3.0, TiDB introduces several new optimizer hints to control the join methods between tables, including: - - [`INDEX_JOIN()`](link) selects index nested loop join, which uses indexes to filter and use the result set as the inner table to join. - - [`NO_HASH_JOIN()`](link) selects join methods other than hash join. - - [`NO_INDEX_HASH_JOIN()`](link) selects join methods other than [index nested loop join](/optimizer-hints.md#inl_hash_join). + - [`INDEX_JOIN()`](link) selects index nested loop join, which uses indexes to filter and uses the filter result as the inner table to join. + - [`NO_HASH_JOIN()`](/optimizer-hints.md#no_hash_joint1_name--tl_name-) selects join methods other than hash join. + - [`NO_INDEX_HASH_JOIN()`](/optimizer-hints.md#no_index_hash_joint1_name--tl_name-) selects join methods other than [index nested loop join](/optimizer-hints.md#inl_hash_join). For more information, refer to [user documentation](/optimizer-hints). From dfec6fd1a5419c27624a7d389ebd9d726dedb80f Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 2 Aug 2023 16:52:34 +0800 Subject: [PATCH 21/72] Update releases/release-7.3.0.md Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 646007146c27a..cfa1f02590e04 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -108,7 +108,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up * Plan Replayer supports exporting historical statistics [#45038](https://github.com/pingcap/tidb/issues/45038) @[time-and-fate](https://github.com/time-and-fate) **tw@ran-huang** - Starting from v7.3.0, with the newly added [`dump with stats as of timestamp`](/sql-plan-replayer.md) clause, Plan Replayer can export the statistics of specified SQL-related objects at a specific point in time. During the diagnosis of execution plan issues, accurately capturing historical statistics can help analyze more precisely how the execution plan was generated at the time when the issue occurred. This helps identify the root cause of the problem and greatly improves efficiency in diagnosing execution plan issues. + Starting from v7.3.0, with the newly added [`dump with stats as of timestamp`](/sql-plan-replayer.md) clause, Plan Replayer can export the statistics of specified SQL-related objects at a specific point in time. During the diagnosis of execution plan issues, accurately capturing historical statistics can help analyze more precisely how the execution plan was generated at the time when the issue occurred. This helps identify the root cause of the issue and greatly improves efficiency in diagnosing execution plan issues. For more information, refer to [user documentation](/sql-plan-replayer.md). From e1e8ca58711b38469149a7b52b67c071899479f6 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 2 Aug 2023 17:51:12 +0800 Subject: [PATCH 22/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index cfa1f02590e04..53f65a0bab294 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -143,7 +143,10 @@ This drastically improves write performance, reduces I/O amplication, speeds up - `tikv-importer.on-duplicate` is deprecated and replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). - The `max-error` parameter, which controls the maximum number of non-fatal errors that TiDB Lightning can tolerate before stopping the migration task, no longer limits import data conflicts. The [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) parameter now controls the maximum number of conflicting records that can be tolerated. -* 兼容性 2 +* TiCDC **tw@ran-huang** + + - When Kafka sink uses Avro protocol, if the `force-replicate` parameter is set to `true`, TiCDC reports an error when creating a changefeed. + - Due to incompatibility between `delete-only-output-handle-key-columns` and `force-replicate` parameters, when both parameters are enabled, TiCDC reports an error when creating a changefeed. ### System variables From 33787796cce98ce243c49f83b52b9664ba809a0c Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 2 Aug 2023 18:22:38 +0800 Subject: [PATCH 23/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 53f65a0bab294..3284ca6877c06 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -54,7 +54,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up * TiFlash supports executing common table expressions (CTEs) (experimental) [#43333](https://github.com/pingcap/tidb/issues/43333) @[winoros](https://github.com/winoros) **tw@ran-huang** - Before v7.3.0, the MPP engine of TiFlash cannot execute queries that contain CTEs by default. To achieve the best execution performance within the MPP framework, you need to use the system variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-introduced-since-v630) to inline expand CTE. + Before v7.3.0, the MPP engine of TiFlash cannot execute queries that contain CTEs by default. To achieve the best execution performance within the MPP framework, you need to use the system variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-introduced-since-v630) to enforce inlining CTE. Starting from v7.3.0, TiFlash's MPP engine supports executing queries with CTEs without inline expanding them, allowing for optimal query execution within the MPP framework. In TPC-DS benchmark tests, compared to using inline expansion, this feature has shown a 20% improvement in overall query execution speed for queries containing CTE. From bc0c04e2452c150705d69b60c4dcbe0aa4e6d88d Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 2 Aug 2023 18:22:46 +0800 Subject: [PATCH 24/72] Update releases/release-7.3.0.md Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 3284ca6877c06..8c030ac6e6c3e 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -56,7 +56,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up Before v7.3.0, the MPP engine of TiFlash cannot execute queries that contain CTEs by default. To achieve the best execution performance within the MPP framework, you need to use the system variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-introduced-since-v630) to enforce inlining CTE. - Starting from v7.3.0, TiFlash's MPP engine supports executing queries with CTEs without inline expanding them, allowing for optimal query execution within the MPP framework. In TPC-DS benchmark tests, compared to using inline expansion, this feature has shown a 20% improvement in overall query execution speed for queries containing CTE. + Starting from v7.3.0, TiFlash's MPP engine supports executing queries with CTEs without inlining them, allowing for optimal query execution within the MPP framework. In TPC-DS benchmark tests, compared with inlining CTEs, this feature has shown a 20% improvement in overall query execution speed for queries containing CTE. This feature is experimental and is disabled by default. It is controlled by the system variable [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-new-in-v720). From 25f646b9e1a0ba501c76c108f00926f5afe600a2 Mon Sep 17 00:00:00 2001 From: Aolin Date: Thu, 3 Aug 2023 10:45:46 +0800 Subject: [PATCH 25/72] add compatibility changes Signed-off-by: Aolin --- releases/release-7.3.0.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 8c030ac6e6c3e..6cb9db0de1592 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -152,6 +152,8 @@ This drastically improves write performance, reduces I/O amplication, speeds up | 变量名 | 修改类型（包括新增/修改/删除） | 描述 | |--------|------------------------------|------| +| [`enable-32bits-connection-id`](/system-variables.md#enable-32bits-connection-id-new-in-v730) | Newly added | Controls whether to enable the 32-bit connection ID feature. | +| [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) | Newly added | Controls whether the `EXPLAIN` statement disables the execution of constant subqueries that can be expanded at the optimization stage. | | [`tidb_remove_orderby_in_subquery`](/system-variables.md#tidb_remove_orderby_in_subquery-从-v610-版本开始引入) | 修改 | 经进一步的测试后，该变量默认值从 `OFF` 修改为 `ON`，即优化器改写会移除子查询中的 `ORDER BY` 子句。 | | [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-new-in-v730) | Newly added | Controls the strategy for selecting TiFlash replicas when a query requires the TiFlash engine. | | | | | @@ -161,8 +163,15 @@ This drastically improves write performance, reduces I/O amplication, speeds up | 配置文件 | 配置项 | 修改类型 | 描述 | | -------- | -------- | -------- | -------- | -| | | | | -| | | | | +| TiKV | [rocksdb.\[defaultcf\|writecf\|lockcf\].compaction-guard-min-output-file-size](/tikv-configuration-file.md#compaction-guard-min-output-file-size) | Modified | Changes the default value from `"1MB"` to `"8MB"` to resolve the issue that compaction speed cannot keep up with the write speed during large data writes. | +| TiKV | [rocksdb.\[defaultcf\|writecf\|lockcf\].format-version](/tikv-configuration-file.md#format-version-new-in-v620) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), Ribbon filter is used. Therefore, TiKV changes the default value from `2` to `5`. | +| TiKV | [`raft-engine.format-version`](/tikv-configuration-file.md#format-version-new-in-v630) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), Ribbon filter is used. Therefore, TiKV changes the default value from `2` to `5`. | +| TiKV | [`raftdb.max-total-wal-size`](/tikv-configuration-file.md#max-total-wal-size-1) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), TiKV skips writing WAL. Therefore, TiKV changes the default value from `"4GB"` to `1`, meaning that WAL is disabled. | +| TiKV | [`rocksdb.max-total-wal-size`](/tikv-configuration-file.md#max-total-wal-size) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), TiKV skips writing WAL. Therefore, TiKV changes the default value from `"4GB"` to `1`, meaning that WAL is disabled. | +| TiKV | [`rocksdb.stats-dump-period`](/tikv-configuration-file.md#stats-dump-period) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to disable redundant log printing, changes the default value from `"10m"` to `"0"`. | +| TiKV | [`storage.block-cache.capacity`](/tikv-configuration-file.md#capacity) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to compensate for the memory overhead of memtables, TiKV changes the default value from 45% to 30% of the size of total system memory. | +| TiKV | [`rocksdb.write-buffer-limit`](/tikv-configuration-file.md#write-buffer-limit-new-in-v660) | Modified | To reduce the memory overhead of memtables, when `storage.engine="raft-kv"`, TiKV changes the default value from 25% of the memory of the machine to `0`, which means no limit. When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), TiKV changes the default value from 25% to 20% of the memory of the machine. | +| TiKV | [`rocksdb.lockcf.write-buffer-size`](/tikv-configuration-file.md#write-buffer-size) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to speed up compaction on lockcf, TiKV changes the default value from `"32MB"` to `"4MB"`. | |TiDB Lightning | `conflict.max-record-rows` | Newly added | The new version of strategy to handle conflicting data. It controls the maximum number of rows in the `conflict_records` table. The default value is 100. | |TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | |TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace”` or `conflict.strategy=“ignore"`, you can set it as a maxint. | From d8e878aff5890ee79b173664cc6444bae2009715 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 3 Aug 2023 11:02:30 +0800 Subject: [PATCH 26/72] Update releases/release-7.3.0.md Co-authored-by: Aolin --- releases/release-7.3.0.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 6cb9db0de1592..fef9fe1318a3b 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -122,6 +122,10 @@ This drastically improves write performance, reduces I/O amplication, speeds up For more information, see [documentation](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#conflict-detection). +* TiDB Lightning supports Partitioned Raft KV (experimental) [#14916](https://github.com/tikv/tikv/issues/14916) @[GMHDBJD](https://github.com/GMHDBJD) **tw@hfxsd** + + TiDB Lightning now supports Partitioned Raft KV. This feature helps improve the data import performance of TiDB Lightning. + * TiDB Lightning introduces a new parameter `enable-diagnose-log` to enhance troubleshooting by printing more diagnostic logs [#45497](https://github.com/pingcap/tidb/issues/45497) @[D3Hunter](https://github.com/D3Hunter) **tw@hfxsd** By default, this feature is disabled and TiDB Lightning only prints logs containing `lightning/main`. When enabled, TiDB Lightning prints logs for all packages (including `client-go` and `tidb`) to help diagnose issues related to `client-go` and `tidb`. From df68f11a1b6dfb38e87acdd3271bce7b99b0b953 Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 3 Aug 2023 12:35:45 +0800 Subject: [PATCH 27/72] Update releases/release-7.3.0.md Co-authored-by: Aolin --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index fef9fe1318a3b..6c690cb464823 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -100,7 +100,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up ### Observability -* Show the progress of collecting statistics [#issue号](链接) @[hawkingrei](https://github.com/hawkingrei) **tw@Oreoxmt** +* Show the progress of collecting statistics [#44033](https://github.com/pingcap/tidb/issues/44033) @[hawkingrei](https://github.com/hawkingrei) **tw@Oreoxmt** Collecting statistics for large tables often takes a long time. In previous versions, you cannot see the progress of collecting statistics, and therefore cannot predict the completion time. TiDB v7.3.0 introduces a feature to show the progress of collecting statistics. You can view the overall workload, current progress, and estimated completion time for each subtask using the system table `mysql.analyze_jobs` or `SHOW ANALYZE STATUS`. In scenarios such as large-scale data import and SQL performance optimization, this feature helps you understand the overall task progress and improves the user experience. From ddcce13d77f875d4e17ed64f6ff4b24da7ee3c2b Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 3 Aug 2023 17:14:31 +0800 Subject: [PATCH 28/72] add `sink.csv.binary-encoding-method` --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 6c690cb464823..4c370029a6334 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -185,7 +185,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up |TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. **tw:qiancai** | | Data Migration | `strict-optimistic-shard-mode` | 新增 | 用于兼容历史版本 2.0 分库分表同步 DDL 的行为。当用户选择乐观模式时，可以启用该参数，开启后，乐观模式下，同步任务遇到二类 DDL 时，整个任务会中断，在多个表的 DDL变更有依赖关系的场景，可以及时中断，用户手动处理完各表的 DDL 后，再继续同步数据，保障上下游数据的一致性。 **tw@ran-huang** | | TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | - +| TiCDC | [`sink.csv.binary-encoding-method`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | The encoding method of binary data, which can be `'base64'` or `'hex'`. The default value is `'base64'`. | ## Deprecated features - note [#issue](链接) @[贡献者 GitHub ID](链接) From a4e20bb4187710eed175880c371d60c91678628f Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 3 Aug 2023 17:44:02 +0800 Subject: [PATCH 29/72] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 4c370029a6334..c677cfd621292 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -66,9 +66,11 @@ This drastically improves write performance, reduces I/O amplication, speeds up In v7.3.0, TiDB introduces several new optimizer hints to control the join methods between tables, including: - - [`INDEX_JOIN()`](link) selects index nested loop join, which uses indexes to filter and uses the filter result as the inner table to join. + - [`NO_MERGE_JOIN()`](/optimizer-hints.md#no_merge_joint1_name--tl_name-) selects join methods other than merge join. + - [`NO_INDEX_JOIN()`](/optimizer-hints.md#no_index_joint1_name--tl_name-) selects join methods other than index nested loop join. + - [`NO_INDEX_MERGE_JOIN()`](/optimizer-hints.md#no_index_merge_joint1_name--tl_name-) selects join methods other than index nested loop merge join. - [`NO_HASH_JOIN()`](/optimizer-hints.md#no_hash_joint1_name--tl_name-) selects join methods other than hash join. - - [`NO_INDEX_HASH_JOIN()`](/optimizer-hints.md#no_index_hash_joint1_name--tl_name-) selects join methods other than [index nested loop join](/optimizer-hints.md#inl_hash_join). + - [`NO_INDEX_HASH_JOIN()`](/optimizer-hints.md#no_index_hash_joint1_name--tl_name-) selects join methods other than [index nested loop hash join](/optimizer-hints.md#inl_hash_join). For more information, refer to [user documentation](/optimizer-hints). From 62600d4491ad6c43e7b514576a05c681c7732926 Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 3 Aug 2023 22:05:14 +0800 Subject: [PATCH 30/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 1 + 1 file changed, 1 insertion(+) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index c677cfd621292..ea79c9f5f8c93 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -153,6 +153,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up - When Kafka sink uses Avro protocol, if the `force-replicate` parameter is set to `true`, TiCDC reports an error when creating a changefeed. - Due to incompatibility between `delete-only-output-handle-key-columns` and `force-replicate` parameters, when both parameters are enabled, TiCDC reports an error when creating a changefeed. + - When the output protocol is Open Protocol, the `UPDATE` events only output the changed columns. ### System variables From 56538f465f76813352c61470476b5e5e41b65079 Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 4 Aug 2023 09:56:38 +0800 Subject: [PATCH 31/72] Apply suggestions from code review --- releases/release-7.3.0.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index ea79c9f5f8c93..36d847332b0e6 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -198,7 +198,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up + TiDB - 游标 (Cursor) 结果过大时，写入 TiDB 临时磁盘空间从而避免OOM [#43233](https://github.com/pingcap/tidb/issues/43233) @[YangKeao](https://github.com/YangKeao) - - Introduce a new system variable `tidb_opt_enable_non_eval_scalar_subquery` to control whether the `EXPLAIN` statement executes subqueries in advance during the optimization phase [#22076](https://github.com/pingcap/tidb/issues/22076) @[winoros](https://github.com/winoros) **tw@Oreoxmt** + - Introduce a new system variable [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) to control whether the `EXPLAIN` statement executes subqueries in advance during the optimization phase [#22076](https://github.com/pingcap/tidb/issues/22076) @[winoros](https://github.com/winoros) **tw@Oreoxmt** - When [Global Kill](/tidb-configuration-file#enable-global-kill-new-in-v610) is enabled, you can terminate the current session by pressing Control+C [#8854](https://github.com/pingcap/tidb/issues/8854) @[pingyu](https://github.com/pingyu) **tw@Oreoxmt** - note [#issue](链接) @[贡献者 GitHub ID](链接) @@ -237,7 +237,6 @@ This drastically improves write performance, reduces I/O amplication, speeds up + TiDB Lightning - - 更新 TiDB Lightning 并行导入的参数名称从 "tikv-importer.incremental-import" 变更为 “tikv-importer.parallel-import” ，避免用户误认为是增量导入而误用该参数。 [#45501](https://github.com/pingcap/tidb/issues/45501) @[lyzx2001](https://github.com/lyzx2001) **tw@hfxsd** - note [#issue](链接) @[贡献者 GitHub ID](链接) + TiUP From 30601e91136319cbcbae2f54540ab8fdd2106dc5 Mon Sep 17 00:00:00 2001 From: Aolin Date: Fri, 4 Aug 2023 10:13:06 +0800 Subject: [PATCH 32/72] fix ci Signed-off-by: Aolin --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 36d847332b0e6..35c3503ce8528 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -159,7 +159,6 @@ This drastically improves write performance, reduces I/O amplication, speeds up | 变量名 | 修改类型（包括新增/修改/删除） | 描述 | |--------|------------------------------|------| -| [`enable-32bits-connection-id`](/system-variables.md#enable-32bits-connection-id-new-in-v730) | Newly added | Controls whether to enable the 32-bit connection ID feature. | | [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) | Newly added | Controls whether the `EXPLAIN` statement disables the execution of constant subqueries that can be expanded at the optimization stage. | | [`tidb_remove_orderby_in_subquery`](/system-variables.md#tidb_remove_orderby_in_subquery-从-v610-版本开始引入) | 修改 | 经进一步的测试后，该变量默认值从 `OFF` 修改为 `ON`，即优化器改写会移除子查询中的 `ORDER BY` 子句。 | | [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-new-in-v730) | Newly added | Controls the strategy for selecting TiFlash replicas when a query requires the TiFlash engine. | @@ -170,6 +169,7 @@ This drastically improves write performance, reduces I/O amplication, speeds up | 配置文件 | 配置项 | 修改类型 | 描述 | | -------- | -------- | -------- | -------- | +| TiDB | [`enable-32bits-connection-id`](/tidb-configuration-file.md#enable-32bits-connection-id-new-in-v730) | Newly added | Controls whether to enable the 32-bit connection ID feature. | | TiKV | [rocksdb.\[defaultcf\|writecf\|lockcf\].compaction-guard-min-output-file-size](/tikv-configuration-file.md#compaction-guard-min-output-file-size) | Modified | Changes the default value from `"1MB"` to `"8MB"` to resolve the issue that compaction speed cannot keep up with the write speed during large data writes. | | TiKV | [rocksdb.\[defaultcf\|writecf\|lockcf\].format-version](/tikv-configuration-file.md#format-version-new-in-v620) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), Ribbon filter is used. Therefore, TiKV changes the default value from `2` to `5`. | | TiKV | [`raft-engine.format-version`](/tikv-configuration-file.md#format-version-new-in-v630) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), Ribbon filter is used. Therefore, TiKV changes the default value from `2` to `5`. | From 771970bc5d220321a202e493b99102d5ca92ea7b Mon Sep 17 00:00:00 2001 From: Ran Date: Fri, 4 Aug 2023 11:43:03 +0800 Subject: [PATCH 33/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 35c3503ce8528..d1d3c91320e5c 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -189,6 +189,11 @@ This drastically improves write performance, reduces I/O amplication, speeds up | Data Migration | `strict-optimistic-shard-mode` | 新增 | 用于兼容历史版本 2.0 分库分表同步 DDL 的行为。当用户选择乐观模式时，可以启用该参数，开启后，乐观模式下，同步任务遇到二类 DDL 时，整个任务会中断，在多个表的 DDL变更有依赖关系的场景，可以及时中断，用户手动处理完各表的 DDL 后，再继续同步数据，保障上下游数据的一致性。 **tw@ran-huang** | | TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | | TiCDC | [`sink.csv.binary-encoding-method`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | The encoding method of binary data, which can be `'base64'` or `'hex'`. The default value is `'base64'`. | + +### 系统表 + +- 新增系统表 `mysql.tidb_timers` 用来存储系统内部定时器的元信息。 + ## Deprecated features - note [#issue](链接) @[贡献者 GitHub ID](链接) From 077debf0559444eaeb0dd9e0d22793bd519b6f8f Mon Sep 17 00:00:00 2001 From: Sam Dillard Date: Fri, 4 Aug 2023 16:58:22 -0700 Subject: [PATCH 34/72] Redo highlights table --- releases/release-7.3.0.md | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index d1d3c91320e5c..e06111ee39b59 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -23,15 +23,33 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - - - + + + + + + + + + + + + + + + + + + +
Scalability and PerformancePartitioned Raft KV GA. Learn more. Scalability and PerformancePartitioned Raft KV GA. Every key region will store its key-value data in its own isolated LSM tree (RocksDB). This drastically improves write performance, reduces I/O amplication, speeds up scale-in/-out operations, and is a huge step toward TiDB handling beyond PB-scale workloads per cluster.
Scalability and PerformancePartitioned Raft KV GA.Partitioned Raft KV GA Every key region will store its key-value data in its own isolated LSM tree (RocksDB). This drastically improves write performance, reduces I/O amplication, speeds up scale-in/-out operations, and is a huge step toward TiDB handling beyond PB-scale workloads per cluster.
Scalability and PerformancePartitioned Raft KV GAEvery key region will store its key-value data in its own isolated LSM tree (RocksDB). -This drastically improves write performance, reduces I/O amplication, speeds up scale-in/-out operations, and is a huge step toward TiDB handling beyond PB-scale workloads per cluster. + Scalability and PerformanceLightning supports Partitioned Raft KV Lightning import service now supports the new Partitioned Raft KV architecture, as part of the near-term GA of the architecture.
Reliability and AvailabilityAdd automatic conflict detection and resolution on importsIn the Lightning local backend data import scenario, support is provided for pre-checking conflict records and implementing the semantics of replace and insert ignore. It automatically handles conflicting data for users while improving the performance of conflict resolution.
Manual management of runaway queries (experimental)Query timeouts exist on a per-TiKV node basis but now Resource Groups can manage queries by global parallel execution time and either deprioritize or kill them. Allowing operators to configure this timeout and what do with the queries at a Resource Group level, gives them much more control over the impact unexpected large queries may have on the cluster.
SQLEnhanced operator control over query stability by adding more complex operator hints to the query plannerAdded hints: NO_HASH_JOIN(), INDEX_JOIN(), NO_INDEX_HASH_JOIN() +
DB Operations and ObservabilityAdded command to show progress of cluster stats collectionAdd SHOW ANALYZE STATUS to peer into the progress of a cluster ANALYZE job.
+ ## Feature details ### Performance From 558d043a5394d756cbbd9aa1960f5f86c5449c46 Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 7 Aug 2023 12:23:32 +0800 Subject: [PATCH 35/72] Apply suggestions from code review Co-authored-by: xixirangrang --- releases/release-7.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index e06111ee39b59..850bd62ff1d1f 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -24,8 +24,8 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- Scalability and Performance - Lightning supports Partitioned Raft KV - Lightning import service now supports the new Partitioned Raft KV architecture, as part of the near-term GA of the architecture. + TiDB Lightning supports Partitioned Raft KV + TiDB Lightning now supports the new Partitioned Raft KV architecture, as part of the near-term GA of the architecture. From b766293847b0b2946a478f489929a3f56bd049fa Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 7 Aug 2023 13:04:54 +0800 Subject: [PATCH 36/72] improvements and bug fixes: update TiDB and TiKV part Signed-off-by: Aolin --- releases/release-7.3.0.md | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 850bd62ff1d1f..5fd2ce05a7d39 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -220,15 +220,16 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiDB - - 游标 (Cursor) 结果过大时，写入 TiDB 临时磁盘空间从而避免OOM [#43233](https://github.com/pingcap/tidb/issues/43233) @[YangKeao](https://github.com/YangKeao) + - Introduce a new system variable [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) to control whether the `EXPLAIN` statement executes subqueries in advance during the optimization phase [#22076](https://github.com/pingcap/tidb/issues/22076) @[winoros](https://github.com/winoros) **tw@Oreoxmt** - When [Global Kill](/tidb-configuration-file#enable-global-kill-new-in-v610) is enabled, you can terminate the current session by pressing Control+C [#8854](https://github.com/pingcap/tidb/issues/8854) @[pingyu](https://github.com/pingyu) **tw@Oreoxmt** - - note [#issue](链接) @[贡献者 GitHub ID](链接) + - Support the `IS_USED_LOCK()` locking function [#44493](https://github.com/pingcap/tidb/issues/44493) @[dveeden](https://github.com/dveeden) + - Optimize the performance of reading the dumped chunks from disk [#45125](https://github.com/pingcap/tidb/issues/45125) @[YangKeao](https://github.com/YangKeao) + TiKV - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + + - Add the `Max gap of safe-ts` and `Min safe ts region` metrics and introduce the `tikv-ctl get_region_read_progress` command to better observe and diagnose the status of resolved-ts and safe-ts [#15082](https://github.com/tikv/tikv/issues/15082) [@ekexium](https://github.com/ekexium) + PD @@ -271,13 +272,19 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiDB - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + + - Fix the issue that when the MySQL Cursor Fetch protocol is used, the memory consumption of result sets might exceed the `tidb_mem_quota_query` limit and causes TiDB OOM. After the fix, TiDB will automatically write result sets to the disk to release memory [#43233](https://github.com/pingcap/tidb/issues/43233) @[YangKeao](https://github.com/YangKeao) + - Fix the TiDB panic issue caused by data race [#45561](https://github.com/pingcap/tidb/issues/45561) @[genliqi](https://github.com/gengliqi) + - Fix the hang-up issue that occurs when queries with `indexMerge` are killed [#45279](https://github.com/pingcap/tidb/issues/45279) @[xzhangxian1008](https://github.com/xzhangxian1008) + - Fix the issue that query results in MPP mode are incorrect when `tidb_enable_parallel_apply` is enabled [#45299](https://github.com/pingcap/tidb/issues/45299) @[windtalker](https://github.com/windtalker) + - Fix the issue that `resolve lock` might hang when there is a sudden change in PD time [#44822](https://github.com/pingcap/tidb/issues/44822) @[zyguan](https://github.com/zyguan) + - Fix the issue that the GC Resolve Locks step might miss some pessimistic locks [#45134](https://github.com/pingcap/tidb/issues/45134) @[MyonKeminta](https://github.com/MyonKeminta) + - Fix the issue that the query with `ORDER BY` returns incorrect results in dynamic pruning mode [#45007](https://github.com/pingcap/tidb/issues/45007) @[Defined2014](https://github.com/Defined2014) + TiKV - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + + - Fix the issue that in some rare cases, reading data during GC might cause TiKV panic [#15109](https://github.com/tikv/tikv/issues/15109) [@MyonKeminta](https://github.com/MyonKeminta) + PD From 56e25c13885b46a3657bcd79cfca039efdfba1fe Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 7 Aug 2023 13:23:43 +0800 Subject: [PATCH 37/72] Apply suggestions from code review Co-authored-by: xixirangrang Co-authored-by: Roger Song Co-authored-by: Aolin --- releases/release-7.3.0.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 5fd2ce05a7d39..82c7d52a1bb26 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -31,20 +31,20 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- Reliability and Availability Add automatic conflict detection and resolution on imports - In the Lightning local backend data import scenario, support is provided for pre-checking conflict records and implementing the semantics of replace and insert ignore. It automatically handles conflicting data for users while improving the performance of conflict resolution. + The TiDB Lightning Physical Import Mode supports the new version of conflict detection and implementing the semantics of replacing (`replace`) or ignoring (`ignore`) conflicting data when encountering conflicts. It automatically handles conflicting data for you while improving the performance of conflict resolution. Manual management of runaway queries (experimental) - Query timeouts exist on a per-TiKV node basis but now Resource Groups can manage queries by global parallel execution time and either deprioritize or kill them. Allowing operators to configure this timeout and what do with the queries at a Resource Group level, gives them much more control over the impact unexpected large queries may have on the cluster. + Query timeouts exist on a per-TiKV node basis but now Resource Groups can manage queries by global parallel execution time and either deprioritize or kill them. Allowing operators to mark target queries by SQL manually Text, SQL Digest or Plan Digest and what do with the queries at a Resource Group level, gives them much more control over the impact unexpected large queries may have on the cluster. SQL - Enhanced operator control over query stability by adding more complex operator hints to the query planner - Added hints: NO_HASH_JOIN(), INDEX_JOIN(), NO_INDEX_HASH_JOIN() + Enhanced operator control over query stability by adding more complex optimizer hints to the query planner + Added hints: NO_INDEX_JOIN(), NO_MERGE_JOIN(), NO_INDEX_MERGE_JOIN(), NO_HASH_JOIN(), NO_INDEX_HASH_JOIN() DB Operations and Observability - Added command to show progress of cluster stats collection - Add SHOW ANALYZE STATUS to peer into the progress of a cluster ANALYZE job. + Show the progress of statistics collection tasks + Support viewing the progress of ANALYZE tasks using SHOW ANALYZE STATUS or through the system table mysql.analyze_jobs. From 4bd93bf4c0f132dbb5cffeae591dd75d44cc07fd Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 7 Aug 2023 13:30:17 +0800 Subject: [PATCH 38/72] Apply suggestions from code review --- releases/release-7.3.0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 82c7d52a1bb26..4ea361675dc7c 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -23,27 +23,27 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Scalability and Performance + Scalability and Performance TiDB Lightning supports Partitioned Raft KV TiDB Lightning now supports the new Partitioned Raft KV architecture, as part of the near-term GA of the architecture. Reliability and Availability - Add automatic conflict detection and resolution on imports + Add automatic conflict detection and resolution on imports The TiDB Lightning Physical Import Mode supports the new version of conflict detection and implementing the semantics of replacing (`replace`) or ignoring (`ignore`) conflicting data when encountering conflicts. It automatically handles conflicting data for you while improving the performance of conflict resolution. - Manual management of runaway queries (experimental) + Manual management of runaway queries (experimental) Query timeouts exist on a per-TiKV node basis but now Resource Groups can manage queries by global parallel execution time and either deprioritize or kill them. Allowing operators to mark target queries by SQL manually Text, SQL Digest or Plan Digest and what do with the queries at a Resource Group level, gives them much more control over the impact unexpected large queries may have on the cluster. SQL - Enhanced operator control over query stability by adding more complex optimizer hints to the query planner + Enhanced operator control over query stability by adding more complex optimizer hints to the query planner Added hints: NO_INDEX_JOIN(), NO_MERGE_JOIN(), NO_INDEX_MERGE_JOIN(), NO_HASH_JOIN(), NO_INDEX_HASH_JOIN() DB Operations and Observability - Show the progress of statistics collection tasks + Show the progress of statistics collection tasks Support viewing the progress of ANALYZE tasks using SHOW ANALYZE STATUS or through the system table mysql.analyze_jobs. From 656ab621bd99796d3e3da9da3b1cf2099bed78b4 Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 7 Aug 2023 13:43:33 +0800 Subject: [PATCH 39/72] Apply suggestions from code review Co-authored-by: Grace Cai Co-authored-by: Aolin --- releases/release-7.3.0.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 4ea361675dc7c..c6c224095c69a 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -11,7 +11,7 @@ TiDB version: 7.3.0 Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with-tidb) | [Installation packages](https://www.pingcap.com/download/?version=v7.3.0#version-list) -7.3.0 introduces the following major feature as generally available. The rest of the release (detailed in the Details section) was a series of enhancements to query stability in TiDB server and TiFlash. These are more miscellaneous in nature and not user-facing so they are not included in this section dedicated to release highlights: +7.3.0 introduces the following major features. The rest of the release (detailed in the Details section) was a series of enhancements to query stability in TiDB server and TiFlash. These are more miscellaneous in nature and not user-facing so they are not included in this section dedicated to release highlights: @@ -41,6 +41,8 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + @@ -178,6 +180,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | 变量名 | 修改类型（包括新增/修改/删除） | 描述 | |--------|------------------------------|------| | [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) | Newly added | Controls whether the `EXPLAIN` statement disables the execution of constant subqueries that can be expanded at the optimization stage. | +| [`tidb_skip_missing_partition_stats`](/system-variables.md#tidb_skip_missing_partition_stats-new-in-v730) | Newly added | This variable controls the generation of GlobalStats when partition statistics are missing. | | [`tidb_remove_orderby_in_subquery`](/system-variables.md#tidb_remove_orderby_in_subquery-从-v610-版本开始引入) | 修改 | 经进一步的测试后，该变量默认值从 `OFF` 修改为 `ON`，即优化器改写会移除子查询中的 `ORDER BY` 子句。 | | [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-new-in-v730) | Newly added | Controls the strategy for selecting TiFlash replicas when a query requires the TiFlash engine. | | | | | @@ -188,6 +191,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | 配置文件 | 配置项 | 修改类型 | 描述 | | -------- | -------- | -------- | -------- | | TiDB | [`enable-32bits-connection-id`](/tidb-configuration-file.md#enable-32bits-connection-id-new-in-v730) | Newly added | Controls whether to enable the 32-bit connection ID feature. | +| TiKV | [`coprocessor.region-bucket-size`](/tikv-configuration-file.md#region-bucket-size-new-in-v610) | Modified | Changes the default value from `96MiB` to `50MiB`. | | TiKV | [rocksdb.\[defaultcf\|writecf\|lockcf\].compaction-guard-min-output-file-size](/tikv-configuration-file.md#compaction-guard-min-output-file-size) | Modified | Changes the default value from `"1MB"` to `"8MB"` to resolve the issue that compaction speed cannot keep up with the write speed during large data writes. | | TiKV | [rocksdb.\[defaultcf\|writecf\|lockcf\].format-version](/tikv-configuration-file.md#format-version-new-in-v620) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), Ribbon filter is used. Therefore, TiKV changes the default value from `2` to `5`. | | TiKV | [`raft-engine.format-version`](/tikv-configuration-file.md#format-version-new-in-v630) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), Ribbon filter is used. Therefore, TiKV changes the default value from `2` to `5`. | From a367e81f0e181f0ec391921fa96a514d9ac8a66e Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 7 Aug 2023 14:42:55 +0800 Subject: [PATCH 40/72] Update releases/release-7.3.0.md Co-authored-by: Aolin --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index c6c224095c69a..f9d787064e745 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -140,7 +140,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- In previous versions, TiDB Lightning uses different conflict detection and handling methods for Logical Import Mode and Physical Import Mode, which are complex to configure and not easy for users to understand. In addition, Physical Import Mode cannot handle conflicts using the `replace` or `ignore` policy. Starting from v7.3.0, TiDB Lightning introduces a unified conflict detection and handling strategy for both Logical Import Mode and Physical Import Mode. You can choose to report an error (`error`), replace (`replace`) or ignore (`ignore`) conflicting data when encountering conflicts. You can limit the number of conflict records, such as the task is interrupted and terminated after processing a specified number of conflict records. Furthermore, the system can record conflicting data for troubleshooting. - For import data with many conflicts, it is recommended to use the new version of the conflict detection and handling strategy for better performance. Note that the new version and the old version of the conflict strategy cannot be used at the same time. The old conflict detection and handling strategy will be deprecated in the future. + For import data with many conflicts, it is recommended to use the new version of the conflict detection and handling strategy for better performance. In the lab environment, the new version strategy can improve the performance of conflict detection and handling up to three times faster than the old version. This performance value is for reference only. The actual performance might vary depending on your configuration, table structure, and the percentage of conflicting data. Note that the new version and the old version of the conflict strategy cannot be used at the same time. The old conflict detection and handling strategy will be deprecated in the future. For more information, see [documentation](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#conflict-detection). From edad93d002156c134b686f5df6d6746899e97d34 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 7 Aug 2023 14:59:43 +0800 Subject: [PATCH 41/72] Update releases/release-7.3.0.md Co-authored-by: Aolin --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index f9d787064e745..e6e57ac3c531b 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -138,7 +138,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- * TiDB Lightning introduces a new version of conflict data detection and handling strategy [#41629](https://github.com/pingcap/tidb/issues/41629) @[lance6716](https://github.com/lance6716) **tw@hfxsd** - In previous versions, TiDB Lightning uses different conflict detection and handling methods for Logical Import Mode and Physical Import Mode, which are complex to configure and not easy for users to understand. In addition, Physical Import Mode cannot handle conflicts using the `replace` or `ignore` policy. Starting from v7.3.0, TiDB Lightning introduces a unified conflict detection and handling strategy for both Logical Import Mode and Physical Import Mode. You can choose to report an error (`error`), replace (`replace`) or ignore (`ignore`) conflicting data when encountering conflicts. You can limit the number of conflict records, such as the task is interrupted and terminated after processing a specified number of conflict records. Furthermore, the system can record conflicting data for troubleshooting. + In previous versions, TiDB Lightning uses different conflict detection and handling methods for Logical Import Mode and Physical Import Mode, which are complex to configure and not easy for users to understand. In addition, Physical Import Mode cannot handle conflicts using the `replace` or `ignore` strategy. Starting from v7.3.0, TiDB Lightning introduces a unified conflict detection and handling strategy for both Logical Import Mode and Physical Import Mode. You can choose to report an error (`error`), replace (`replace`) or ignore (`ignore`) conflicting data when encountering conflicts. You can limit the number of conflict records, such as the task is interrupted and terminated after processing a specified number of conflict records. Furthermore, the system can record conflicting data for troubleshooting. For import data with many conflicts, it is recommended to use the new version of the conflict detection and handling strategy for better performance. In the lab environment, the new version strategy can improve the performance of conflict detection and handling up to three times faster than the old version. This performance value is for reference only. The actual performance might vary depending on your configuration, table structure, and the percentage of conflicting data. Note that the new version and the old version of the conflict strategy cannot be used at the same time. The old conflict detection and handling strategy will be deprecated in the future. From 8625dc7e6d7b0c1e68c5509798c42b362f3cf01a Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 7 Aug 2023 15:12:00 +0800 Subject: [PATCH 42/72] support IS_FREE_LOCK --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index e6e57ac3c531b..4bee8c4b3033e 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -227,7 +227,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Introduce a new system variable [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) to control whether the `EXPLAIN` statement executes subqueries in advance during the optimization phase [#22076](https://github.com/pingcap/tidb/issues/22076) @[winoros](https://github.com/winoros) **tw@Oreoxmt** - When [Global Kill](/tidb-configuration-file#enable-global-kill-new-in-v610) is enabled, you can terminate the current session by pressing Control+C [#8854](https://github.com/pingcap/tidb/issues/8854) @[pingyu](https://github.com/pingyu) **tw@Oreoxmt** - - Support the `IS_USED_LOCK()` locking function [#44493](https://github.com/pingcap/tidb/issues/44493) @[dveeden](https://github.com/dveeden) + - Support the `IS_FREE_LOCK()` and `IS_USED_LOCK()` locking functions [#44493](https://github.com/pingcap/tidb/issues/44493) @[dveeden](https://github.com/dveeden) - Optimize the performance of reading the dumped chunks from disk [#45125](https://github.com/pingcap/tidb/issues/45125) @[YangKeao](https://github.com/YangKeao) + TiKV From 3d04b952ec246f17f2d9a797851e64460afb0fc6 Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 7 Aug 2023 16:04:43 +0800 Subject: [PATCH 43/72] Apply suggestions from code review --- releases/release-7.3.0.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 4bee8c4b3033e..60d891c522224 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -255,7 +255,10 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiCDC - - Kafka Sink supports sending only handle key data when the message is too large, reducing the size of the message [#9382](https://github.com/pingcap/tiflow/issues/9382) @[3AceShowHand](https://github.com/3AceShowHand) **tw@ran-huang** + + - Optimize the message size output using the Open Protocol to make it include only the updated column values when sending `UPDATE` type events [#9336](https://github.com/pingcap/tiflow/issues/9336) @[3AceShowHand](https://github.com/3AceShowHand) + - Storage Sink now supports hexadecimal encoding for HEX formatted data, making it compatible with AWS DMS format specifications [#9373](https://github.com/pingcap/tiflow/issues/9373) @[CharlesCheung96](https://github.com/CharlesCheung96) + - Kafka Sink supports [sending only handle key data](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) when the message is too large, reducing the size of the message [#9382](https://github.com/pingcap/tiflow/issues/9382) @[3AceShowHand](https://github.com/3AceShowHand) **tw@ran-huang** - note [#issue](链接) @[贡献者 GitHub ID](链接) + TiDB Data Migration (DM) @@ -309,8 +312,12 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiCDC - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + + - Fix the issue that changefeeds would fail due to the temporary unavailability of PD [#9294](https://github.com/pingcap/tiflow/issues/9294) @[asddongmen](https://github.com/asddongmen) + - Fix the data inconsistency issue that might occur when some TiCDC nodes were isolated from the network [#9344](https://github.com/pingcap/tiflow/issues/9344) @[CharlesCheung96](https://github.com/CharlesCheung96) + - Fix the issue that when Kafka Sink encounters errors it might indefinitely block changefeed progress [#9309](https://github.com/pingcap/tiflow/issues/9309) @[hicqu](https://github.com/hicqu) + - Fix the panic issue that might occur when the TiCDC node status changes [#9354](https://github.com/pingcap/tiflow/issues/9354) @[sdojjy](https://github.com/sdojjy) + - Fix the encoding error for the default `ENUM` values [#9259](https://github.com/pingcap/tiflow/issues/9259) @[3AceShowHand](https://github.com/3AceShowHand) + TiDB Data Migration (DM) @@ -319,8 +326,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiDB Lightning - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + - Fix the issue that executing checksum after TiDB Lightning completes import might encounter SSL errors [#45462](https://github.com/pingcap/tidb/issues/45462) @[D3Hunter](https://github.com/D3Hunter) + TiUP From 20f403e3a6a3fb5991a5dfe842578ae1bceb0738 Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 7 Aug 2023 16:05:31 +0800 Subject: [PATCH 44/72] add contributors Signed-off-by: Ran --- releases/release-7.3.0.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 60d891c522224..7449248e114e5 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -215,7 +215,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ### 系统表 - 新增系统表 `mysql.tidb_timers` 用来存储系统内部定时器的元信息。 - + ## Deprecated features - note [#issue](链接) @[贡献者 GitHub ID](链接) @@ -337,4 +337,16 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- We would like to thank the following contributors from the TiDB community: -- [Contributor GitHub ID](链接) \ No newline at end of file +- [charleszheng44](https://github.com/charleszheng44) +- [dhysum](https://github.com/dhysum) +- [haiyux](https://github.com/haiyux) +- [Jiang-Hua](https://github.com/Jiang-Hua) +- [Jille](https://github.com/Jille) +- [jiyfhust](https://github.com/jiyfhust) +- [krishnaduttPanchagnula](https://github.com/krishnaduttPanchagnula) +- [L-maple](https://github.com/L-maple) +- [pingandb](https://github.com/pingandb) +- [testwill](https://github.com/testwill) +- [tisonkun](https://github.com/tisonkun) +- [xuyifangreeneyes](https://github.com/xuyifangreeneyes) +- [yumchina](https://github.com/yumchina) From b375b12dcd6f66c57c3c774d1b2d812049f5a58f Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 8 Aug 2023 08:51:49 +0800 Subject: [PATCH 45/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 7449248e114e5..2d36ff1136c0c 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -287,7 +287,20 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Fix the issue that `resolve lock` might hang when there is a sudden change in PD time [#44822](https://github.com/pingcap/tidb/issues/44822) @[zyguan](https://github.com/zyguan) - Fix the issue that the GC Resolve Locks step might miss some pessimistic locks [#45134](https://github.com/pingcap/tidb/issues/45134) @[MyonKeminta](https://github.com/MyonKeminta) - Fix the issue that the query with `ORDER BY` returns incorrect results in dynamic pruning mode [#45007](https://github.com/pingcap/tidb/issues/45007) @[Defined2014](https://github.com/Defined2014) - + - Fix the issue that `AUTO_INCREMENT` and the default value cannot coexist in the column definition [#45136](https://github.com/pingcap/tidb/issues/45136) @[Defined2014](https://github.com/Defined2014) + - Fix the issue that in some scenarios, querying the system table `information_schema.TIKV_REGION_STATUS` returns incorrect results [#45531](https://github.com/pingcap/tidb/issues/45531) @[Defined2014](https://github.com/Defined2014) + - Fix the issue of incorrect partition pruning of partition table partitions in some scenarios [#42273](https://github.com/pingcap/tidb/issues/42273) @[jiyfhust](https://github.com/jiyfhust) + - Fix the issue that global indexes cannot be cleared when executing `TRUNCATE` to truncate a certain partition of a partitioned table [#42435](https://github.com/pingcap/tidb/issues/42435) @[L-maple](https://github.com/L-maple) + - Fix the issue that in some scenarios unfinished TTL tasks cannot be re-executed after TiDB restarts [#45022](https://github.com/pingcap/tidb/issues/45022) @[lcwangchao](https://github.com/lcwangchao) + - Fix the memory leak issue when TTL is running [#45510](https://github.com/pingcap/tidb/issues/45510) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue of inaccurate error messages when inserting data into partitioned tables [#44966](https://github.com/pingcap/tidb/issues/44966) @[lilinghai](https://github.com/lilinghai) + - Fix the read permission issue on the `information_schema.tiflash_replica` table [#7795](https://github.com/pingcap/tiflash/issues/7795) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + - Fix the issue that an error occurs when using the wrong partition table name [#44967](https://github.com/pingcap/tidb/issues/44967) @[River2000i](https://github.com/River2000i) + - Fix the issue that in some scenarios creating indexes gets stuck when `tidb_enable_dist_task` is enabled [#44440](https://github.com/pingcap/tidb/issues/44440) @[tangenta](https://github.com/tangenta) + - Fix the issue that using BR to recover a table with `AUTO_ID_CACHE = 1` causes the `Duplicate entry` error [#44716](https://github.com/pingcap/tidb/issues/44716) @[tiancaiamao](https://github.com/tiancaiamao) + - Fix the issue that the time consumed for executing `TRUNCATE TABLE` is not consistent with the task execution time shown in the `ADMIN SHOW DDL JOBS` result [#44785](https://github.com/pingcap/tidb/issues/44785) @[tangenta](https://github.com/tangenta) + - Fix the issue that upgrading TiDB gets stuck when reading metadata takes longer than one DDL lease [#45176](https://github.com/pingcap/tidb/issues/45176) @[zimulala](https://github.com/zimulala) + + TiKV From 56742b412d72001db3fd3cca4f59cd2842aa422c Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 8 Aug 2023 11:39:06 +0800 Subject: [PATCH 46/72] Apply suggestions from code review --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 2d36ff1136c0c..d4bae830f8b81 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -304,7 +304,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiKV - - Fix the issue that in some rare cases, reading data during GC might cause TiKV panic [#15109](https://github.com/tikv/tikv/issues/15109) [@MyonKeminta](https://github.com/MyonKeminta) + - Fix the issue that reading data during GC might cause TiKV panic in some rare cases [#15109](https://github.com/tikv/tikv/issues/15109) [@MyonKeminta](https://github.com/MyonKeminta) + PD From e5a2b61a4d60184bab46edfafc736840f4dfbd8d Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 8 Aug 2023 12:13:26 +0800 Subject: [PATCH 47/72] Update releases/release-7.3.0.md Co-authored-by: Aolin --- releases/release-7.3.0.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index d4bae830f8b81..d29e13d2268a8 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -287,18 +287,18 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Fix the issue that `resolve lock` might hang when there is a sudden change in PD time [#44822](https://github.com/pingcap/tidb/issues/44822) @[zyguan](https://github.com/zyguan) - Fix the issue that the GC Resolve Locks step might miss some pessimistic locks [#45134](https://github.com/pingcap/tidb/issues/45134) @[MyonKeminta](https://github.com/MyonKeminta) - Fix the issue that the query with `ORDER BY` returns incorrect results in dynamic pruning mode [#45007](https://github.com/pingcap/tidb/issues/45007) @[Defined2014](https://github.com/Defined2014) - - Fix the issue that `AUTO_INCREMENT` and the default value cannot coexist in the column definition [#45136](https://github.com/pingcap/tidb/issues/45136) @[Defined2014](https://github.com/Defined2014) - - Fix the issue that in some scenarios, querying the system table `information_schema.TIKV_REGION_STATUS` returns incorrect results [#45531](https://github.com/pingcap/tidb/issues/45531) @[Defined2014](https://github.com/Defined2014) - - Fix the issue of incorrect partition pruning of partition table partitions in some scenarios [#42273](https://github.com/pingcap/tidb/issues/42273) @[jiyfhust](https://github.com/jiyfhust) - - Fix the issue that global indexes cannot be cleared when executing `TRUNCATE` to truncate a certain partition of a partitioned table [#42435](https://github.com/pingcap/tidb/issues/42435) @[L-maple](https://github.com/L-maple) - - Fix the issue that in some scenarios unfinished TTL tasks cannot be re-executed after TiDB restarts [#45022](https://github.com/pingcap/tidb/issues/45022) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that `AUTO_INCREMENT` can be specified on the same column with the `DEFAULT` column value [#45136](https://github.com/pingcap/tidb/issues/45136) @[Defined2014](https://github.com/Defined2014) + - Fix the issue that querying the system table `INFORMATION_SCHEMA.TIKV_REGION_STATUS` returns incorrect results in some cases [#45531](https://github.com/pingcap/tidb/issues/45531) @[Defined2014](https://github.com/Defined2014) + - Fix the issue of incorrect partition table pruning in some cases [#42273](https://github.com/pingcap/tidb/issues/42273) @[jiyfhust](https://github.com/jiyfhust) + - Fix the issue that global indexes are not cleared when truncating partition of a partitioned table [#42435](https://github.com/pingcap/tidb/issues/42435) @[L-maple](https://github.com/L-maple) + - Fix the issue that other TiDB nodes do not take over TTL tasks after failures in one TiDB node [#45022](https://github.com/pingcap/tidb/issues/45022) @[lcwangchao](https://github.com/lcwangchao) - Fix the memory leak issue when TTL is running [#45510](https://github.com/pingcap/tidb/issues/45510) @[lcwangchao](https://github.com/lcwangchao) - Fix the issue of inaccurate error messages when inserting data into partitioned tables [#44966](https://github.com/pingcap/tidb/issues/44966) @[lilinghai](https://github.com/lilinghai) - - Fix the read permission issue on the `information_schema.tiflash_replica` table [#7795](https://github.com/pingcap/tiflash/issues/7795) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) - - Fix the issue that an error occurs when using the wrong partition table name [#44967](https://github.com/pingcap/tidb/issues/44967) @[River2000i](https://github.com/River2000i) - - Fix the issue that in some scenarios creating indexes gets stuck when `tidb_enable_dist_task` is enabled [#44440](https://github.com/pingcap/tidb/issues/44440) @[tangenta](https://github.com/tangenta) - - Fix the issue that using BR to recover a table with `AUTO_ID_CACHE = 1` causes the `Duplicate entry` error [#44716](https://github.com/pingcap/tidb/issues/44716) @[tiancaiamao](https://github.com/tiancaiamao) - - Fix the issue that the time consumed for executing `TRUNCATE TABLE` is not consistent with the task execution time shown in the `ADMIN SHOW DDL JOBS` result [#44785](https://github.com/pingcap/tidb/issues/44785) @[tangenta](https://github.com/tangenta) + - Fix the read permission issue on the `INFORMATION_SCHEMA.TIFLASH_REPLICA` table [#7795](https://github.com/pingcap/tiflash/issues/7795) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + - Fix the issue that an error occurs when using a wrong partition table name [#44967](https://github.com/pingcap/tidb/issues/44967) @[River2000i](https://github.com/River2000i) + - Fix the issue that creating indexes gets stuck when `tidb_enable_dist_task` is enabled in some cases [#44440](https://github.com/pingcap/tidb/issues/44440) @[tangenta](https://github.com/tangenta) + - Fix the `duplicate entry` error that occurs when restoring a table with `AUTO_ID_CACHE=1` using BR [#44716](https://github.com/pingcap/tidb/issues/44716) @[tiancaiamao](https://github.com/tiancaiamao) + - Fix the issue that the time consumed for executing `TRUNCATE TABLE` is inconsistent with the task execution time shown in `ADMIN SHOW DDL JOBS` [#44785](https://github.com/pingcap/tidb/issues/44785) @[tangenta](https://github.com/tangenta) - Fix the issue that upgrading TiDB gets stuck when reading metadata takes longer than one DDL lease [#45176](https://github.com/pingcap/tidb/issues/45176) @[zimulala](https://github.com/zimulala) + TiKV From e3b304cb7b4908758be1fba7cc3a214e18304f8e Mon Sep 17 00:00:00 2001 From: Ran Date: Tue, 8 Aug 2023 14:54:34 +0800 Subject: [PATCH 48/72] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index d29e13d2268a8..71dfcb373d6f7 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -11,7 +11,7 @@ TiDB version: 7.3.0 Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with-tidb) | [Installation packages](https://www.pingcap.com/download/?version=v7.3.0#version-list) -7.3.0 introduces the following major features. The rest of the release (detailed in the Details section) was a series of enhancements to query stability in TiDB server and TiFlash. These are more miscellaneous in nature and not user-facing so they are not included in this section dedicated to release highlights: +7.3.0 introduces the following major features. In addition to that, 7.3.0 also includes a series of enhancements (described in the [Feature details](#feature-details) section) to query stability in TiDB server and TiFlash. These enhancements are more miscellaneous in nature and not user-facing so they are not included in the following table.
SQL Enhanced operator control over query stability by adding more complex optimizer hints to the query planner Added hints: NO_INDEX_JOIN(), NO_MERGE_JOIN(), NO_INDEX_MERGE_JOIN(), NO_HASH_JOIN(), NO_INDEX_HASH_JOIN() +
DB Operations and Observability Show the progress of statistics collection tasks
@@ -25,28 +25,28 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - - - + + - - + + - + - +
Scalability and Performance TiDB Lightning supports Partitioned Raft KV TiDB Lightning now supports the new Partitioned Raft KV architecture, as part of the near-term GA of the architecture. + TiDB Lightning now supports the new [Partitioned Raft KV](https://docs.pingcap.com/tidb/v7.3/partitioned-raft-kv) architecture, as part of the near-term GA of the architecture.
Reliability and AvailabilityAdd automatic conflict detection and resolution on importsThe TiDB Lightning Physical Import Mode supports the new version of conflict detection and implementing the semantics of replacing (`replace`) or ignoring (`ignore`) conflicting data when encountering conflicts. It automatically handles conflicting data for you while improving the performance of conflict resolution.Add automatic conflict detection and resolution on importsThe TiDB Lightning Physical Import Mode supports a new version of conflict detection, which implements the semantics of replacing (`replace`) or ignoring (`ignore`) conflict data when encountering conflicts. It automatically handles conflict data for you while improving the performance of conflict resolution.
Manual management of runaway queries (experimental)Query timeouts exist on a per-TiKV node basis but now Resource Groups can manage queries by global parallel execution time and either deprioritize or kill them. Allowing operators to mark target queries by SQL manually Text, SQL Digest or Plan Digest and what do with the queries at a Resource Group level, gives them much more control over the impact unexpected large queries may have on the cluster.Manual management of runaway queries (experimental)Query timeouts might occur on any TiKV node but now resource groups can manage queries by global parallel execution time and either deprioritize or kill them. Allowing operators to mark target queries by exact SQL text, SQL digest, or plan digest and deal with the queries at a resource group level, this feature gives you much more control over the potential impact of unexpected large queries on a cluster.
SQLEnhanced operator control over query stability by adding more complex optimizer hints to the query plannerEnhance operator control over query stability by adding more optimizer hints to the query planner Added hints: NO_INDEX_JOIN(), NO_MERGE_JOIN(), NO_INDEX_MERGE_JOIN(), NO_HASH_JOIN(), NO_INDEX_HASH_JOIN()
DB Operations and Observability Show the progress of statistics collection tasksSupport viewing the progress of ANALYZE tasks using SHOW ANALYZE STATUS or through the system table mysql.analyze_jobs.Support viewing the progress of ANALYZE tasks using the SHOW ANALYZE STATUS statement or through the mysql.analyze_jobs system table.
From 91b041ddad132a4b67d3e1c5cc3af30a7c1357f3 Mon Sep 17 00:00:00 2001 From: qiancai Date: Tue, 8 Aug 2023 14:57:15 +0800 Subject: [PATCH 49/72] add translation for PD and TiFlash parts --- releases/release-7.3.0.md | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index d29e13d2268a8..142de80493aa3 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -237,14 +237,15 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + PD - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + + - Support blocking the Swagger API by default when the Swagger server is not enabled [#6786](https://github.com/tikv/pd/issues/6786) @[bufferflies](https://github.com/bufferflies) + - Improve the high availability of etcd [#6554](https://github.com/tikv/pd/issues/6554) [#6442](https://github.com/tikv/pd/issues/6442) @[lhy1024](https://github.com/lhy1024) + - Reduce the memory consumption of `GetRegions` requests [#6835](https://github.com/tikv/pd/issues/6835) @[lhy1024](https://github.com/lhy1024) + TiFlash - - Support a new DTFile format version to reduce the number of physical files (experimental) [#7595](https://github.com/pingcap/tiflash/issues/7595) @[hongyunyan](https://github.com/hongyunyan) **tw@qiancai** - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + + - Support a new DTFile format version to reduce the number of physical files (experimental) [#7595](https://github.com/pingcap/tiflash/issues/7595) @[hongyunyan](https://github.com/hongyunyan) + Tools @@ -308,13 +309,20 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + PD - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + + - Fix the issue that restarting PD might cause the `default` resource group to be reinitialized [#6787](https://github.com/tikv/pd/issues/6787) @[glorv](https://github.com/glorv) + - Fix the issue that when etcd is already started but the client has not yet connected to it, calling the client might cause PD to panic [#6860](https://github.com/tikv/pd/issues/6860) @[HuSharp](https://github.com/HuSharp) + - Fix the issue that the `health-check` output of a Region is inconsistent with the Region information returned by querying the Region ID [#6560](https://github.com/tikv/pd/issues/6560) @[JmPotato](https://github.com/JmPotato) + - Fix the issue that failed learner peers in `unsafe recovery` are ignored in `auto-detect` mode [#6690](https://github.com/tikv/pd/issues/6690) @[v01dstar](https://github.com/v01dstar) + - Fix the issue that Placement Rules select TiFlash learners that do not meet the rules [#6662](https://github.com/tikv/pd/issues/6662) @[rleungx](https://github.com/rleungx) + - Fix the issue that unhealthy peers cannot be removed when rule checker selects peers [#6559](https://github.com/tikv/pd/issues/6559) [nolouch](https://github.com/nolouch) + TiFlash - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) + + - Fix the issue that TiFlash cannot replicate partitioned tables successfully due to deadlocks [#7758](https://github.com/pingcap/tiflash/issues/7758) @[hongyunyan](https://github.com/hongyunyan) + - Fix the issue that the `INFORMATION_SCHEMA.TIFLASH_REPLICA` system table contains tables that users do not have privileges to access [#7795](https://github.com/pingcap/tiflash/issues/7795) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + - Fix the issue that when there are multiple HashAgg operators within the same MPP task, the compilation of the MPP task might take an excessively long time, severely affecting query performance [#7810](https://github.com/pingcap/tiflash/issues/7810) @[SeaRise](https://github.com/SeaRise) + Tools From e7e051a4d09efc721599e3420c535d306e6fa51c Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 8 Aug 2023 16:59:57 +0800 Subject: [PATCH 50/72] add storage.format_version --- releases/release-7.3.0.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index d98bddb2bc4ff..f1e7bbd9e3532 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -201,6 +201,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | TiKV | [`storage.block-cache.capacity`](/tikv-configuration-file.md#capacity) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to compensate for the memory overhead of memtables, TiKV changes the default value from 45% to 30% of the size of total system memory. | | TiKV | [`rocksdb.write-buffer-limit`](/tikv-configuration-file.md#write-buffer-limit-new-in-v660) | Modified | To reduce the memory overhead of memtables, when `storage.engine="raft-kv"`, TiKV changes the default value from 25% of the memory of the machine to `0`, which means no limit. When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), TiKV changes the default value from 25% to 20% of the memory of the machine. | | TiKV | [`rocksdb.lockcf.write-buffer-size`](/tikv-configuration-file.md#write-buffer-size) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to speed up compaction on lockcf, TiKV changes the default value from `"32MB"` to `"4MB"`. | +| TiFlash | [`storage.format_version`](/tiflash/tiflash-configuration.md) | Modified | Introduces a new DTFile format `format_version = 5` to reduce the number of physical files by merging smaller files. Note that this format is experimental and not enabled by default. |TiDB Lightning | `conflict.max-record-rows` | Newly added | The new version of strategy to handle conflicting data. It controls the maximum number of rows in the `conflict_records` table. The default value is 100. | |TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | |TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace”` or `conflict.strategy=“ignore"`, you can set it as a maxint. | @@ -245,7 +246,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiFlash - - Support a new DTFile format version to reduce the number of physical files (experimental) [#7595](https://github.com/pingcap/tiflash/issues/7595) @[hongyunyan](https://github.com/hongyunyan) + - Support a new DTFile format version [`storage.format_version = 5`](/tiflash/tiflash-configuration.md) to reduce the number of physical files (experimental) [#7595](https://github.com/pingcap/tiflash/issues/7595) @[hongyunyan](https://github.com/hongyunyan) + Tools From f524c7e3b59d13f96c2b0306011cc0c8ff736d17 Mon Sep 17 00:00:00 2001 From: Ran Date: Tue, 8 Aug 2023 17:13:05 +0800 Subject: [PATCH 51/72] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index f1e7bbd9e3532..d058328fc589d 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -25,7 +25,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- Scalability and Performance TiDB Lightning supports Partitioned Raft KV - TiDB Lightning now supports the new [Partitioned Raft KV](https://docs.pingcap.com/tidb/v7.3/partitioned-raft-kv) architecture, as part of the near-term GA of the architecture. + TiDB Lightning now supports the new Partitioned Raft KV architecture, as part of the near-term GA of the architecture. From 3a8dbd2d95f675c0a3e9673d42bf43dd3028d5a3 Mon Sep 17 00:00:00 2001 From: Ran Date: Tue, 8 Aug 2023 18:01:40 +0800 Subject: [PATCH 52/72] fix build failure --- releases/release-7.3.0.md | 34 ++++++++++++---------------------- 1 file changed, 12 insertions(+), 22 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index d058328fc589d..17b27617ad5ae 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -201,7 +201,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | TiKV | [`storage.block-cache.capacity`](/tikv-configuration-file.md#capacity) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to compensate for the memory overhead of memtables, TiKV changes the default value from 45% to 30% of the size of total system memory. | | TiKV | [`rocksdb.write-buffer-limit`](/tikv-configuration-file.md#write-buffer-limit-new-in-v660) | Modified | To reduce the memory overhead of memtables, when `storage.engine="raft-kv"`, TiKV changes the default value from 25% of the memory of the machine to `0`, which means no limit. When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), TiKV changes the default value from 25% to 20% of the memory of the machine. | | TiKV | [`rocksdb.lockcf.write-buffer-size`](/tikv-configuration-file.md#write-buffer-size) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to speed up compaction on lockcf, TiKV changes the default value from `"32MB"` to `"4MB"`. | -| TiFlash | [`storage.format_version`](/tiflash/tiflash-configuration.md) | Modified | Introduces a new DTFile format `format_version = 5` to reduce the number of physical files by merging smaller files. Note that this format is experimental and not enabled by default. +| TiFlash | [`storage.format_version`](/tiflash/tiflash-configuration.md) | Modified | Introduces a new DTFile format `format_version = 5` to reduce the number of physical files by merging smaller files. Note that this format is experimental and not enabled by default. |TiDB Lightning | `conflict.max-record-rows` | Newly added | The new version of strategy to handle conflicting data. It controls the maximum number of rows in the `conflict_records` table. The default value is 100. | |TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | |TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace”` or `conflict.strategy=“ignore"`, you can set it as a maxint. | @@ -223,29 +223,25 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ## Improvements -+ TiDB ++ TiDB - - Introduce a new system variable [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) to control whether the `EXPLAIN` statement executes subqueries in advance during the optimization phase [#22076](https://github.com/pingcap/tidb/issues/22076) @[winoros](https://github.com/winoros) **tw@Oreoxmt** - When [Global Kill](/tidb-configuration-file#enable-global-kill-new-in-v610) is enabled, you can terminate the current session by pressing Control+C [#8854](https://github.com/pingcap/tidb/issues/8854) @[pingyu](https://github.com/pingyu) **tw@Oreoxmt** - Support the `IS_FREE_LOCK()` and `IS_USED_LOCK()` locking functions [#44493](https://github.com/pingcap/tidb/issues/44493) @[dveeden](https://github.com/dveeden) - Optimize the performance of reading the dumped chunks from disk [#45125](https://github.com/pingcap/tidb/issues/45125) @[YangKeao](https://github.com/YangKeao) -+ TiKV ++ TiKV - - Add the `Max gap of safe-ts` and `Min safe ts region` metrics and introduce the `tikv-ctl get_region_read_progress` command to better observe and diagnose the status of resolved-ts and safe-ts [#15082](https://github.com/tikv/tikv/issues/15082) [@ekexium](https://github.com/ekexium) -+ PD ++ PD - - Support blocking the Swagger API by default when the Swagger server is not enabled [#6786](https://github.com/tikv/pd/issues/6786) @[bufferflies](https://github.com/bufferflies) - Improve the high availability of etcd [#6554](https://github.com/tikv/pd/issues/6554) [#6442](https://github.com/tikv/pd/issues/6442) @[lhy1024](https://github.com/lhy1024) - Reduce the memory consumption of `GetRegions` requests [#6835](https://github.com/tikv/pd/issues/6835) @[lhy1024](https://github.com/lhy1024) -+ TiFlash ++ TiFlash - - Support a new DTFile format version [`storage.format_version = 5`](/tiflash/tiflash-configuration.md) to reduce the number of physical files (experimental) [#7595](https://github.com/pingcap/tiflash/issues/7595) @[hongyunyan](https://github.com/hongyunyan) + Tools @@ -255,9 +251,8 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - When backing up data to Azure Blob Storage using BR, you can specify either an encryption scope or an encryption key for server-side encryption [#45025](https://github.com/pingcap/tidb/issues/45025) @[Leavrth](https://github.com/Leavrth) **tw@Oreoxmt** - note [#issue](链接) @[贡献者 GitHub ID](链接) - + TiCDC + + TiCDC - - Optimize the message size output using the Open Protocol to make it include only the updated column values when sending `UPDATE` type events [#9336](https://github.com/pingcap/tiflow/issues/9336) @[3AceShowHand](https://github.com/3AceShowHand) - Storage Sink now supports hexadecimal encoding for HEX formatted data, making it compatible with AWS DMS format specifications [#9373](https://github.com/pingcap/tiflow/issues/9373) @[CharlesCheung96](https://github.com/CharlesCheung96) - Kafka Sink supports [sending only handle key data](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) when the message is too large, reducing the size of the message [#9382](https://github.com/pingcap/tiflow/issues/9382) @[3AceShowHand](https://github.com/3AceShowHand) **tw@ran-huang** @@ -279,9 +274,8 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ## Bug fixes -+ TiDB ++ TiDB - - Fix the issue that when the MySQL Cursor Fetch protocol is used, the memory consumption of result sets might exceed the `tidb_mem_quota_query` limit and causes TiDB OOM. After the fix, TiDB will automatically write result sets to the disk to release memory [#43233](https://github.com/pingcap/tidb/issues/43233) @[YangKeao](https://github.com/YangKeao) - Fix the TiDB panic issue caused by data race [#45561](https://github.com/pingcap/tidb/issues/45561) @[genliqi](https://github.com/gengliqi) - Fix the hang-up issue that occurs when queries with `indexMerge` are killed [#45279](https://github.com/pingcap/tidb/issues/45279) @[xzhangxian1008](https://github.com/xzhangxian1008) @@ -302,15 +296,13 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Fix the `duplicate entry` error that occurs when restoring a table with `AUTO_ID_CACHE=1` using BR [#44716](https://github.com/pingcap/tidb/issues/44716) @[tiancaiamao](https://github.com/tiancaiamao) - Fix the issue that the time consumed for executing `TRUNCATE TABLE` is inconsistent with the task execution time shown in `ADMIN SHOW DDL JOBS` [#44785](https://github.com/pingcap/tidb/issues/44785) @[tangenta](https://github.com/tangenta) - Fix the issue that upgrading TiDB gets stuck when reading metadata takes longer than one DDL lease [#45176](https://github.com/pingcap/tidb/issues/45176) @[zimulala](https://github.com/zimulala) - -+ TiKV - ++ TiKV + - Fix the issue that reading data during GC might cause TiKV panic in some rare cases [#15109](https://github.com/tikv/tikv/issues/15109) [@MyonKeminta](https://github.com/MyonKeminta) -+ PD ++ PD - - Fix the issue that restarting PD might cause the `default` resource group to be reinitialized [#6787](https://github.com/tikv/pd/issues/6787) @[glorv](https://github.com/glorv) - Fix the issue that when etcd is already started but the client has not yet connected to it, calling the client might cause PD to panic [#6860](https://github.com/tikv/pd/issues/6860) @[HuSharp](https://github.com/HuSharp) - Fix the issue that the `health-check` output of a Region is inconsistent with the Region information returned by querying the Region ID [#6560](https://github.com/tikv/pd/issues/6560) @[JmPotato](https://github.com/JmPotato) @@ -318,9 +310,8 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Fix the issue that Placement Rules select TiFlash learners that do not meet the rules [#6662](https://github.com/tikv/pd/issues/6662) @[rleungx](https://github.com/rleungx) - Fix the issue that unhealthy peers cannot be removed when rule checker selects peers [#6559](https://github.com/tikv/pd/issues/6559) [nolouch](https://github.com/nolouch) -+ TiFlash ++ TiFlash - - Fix the issue that TiFlash cannot replicate partitioned tables successfully due to deadlocks [#7758](https://github.com/pingcap/tiflash/issues/7758) @[hongyunyan](https://github.com/hongyunyan) - Fix the issue that the `INFORMATION_SCHEMA.TIFLASH_REPLICA` system table contains tables that users do not have privileges to access [#7795](https://github.com/pingcap/tiflash/issues/7795) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) - Fix the issue that when there are multiple HashAgg operators within the same MPP task, the compilation of the MPP task might take an excessively long time, severely affecting query performance [#7810](https://github.com/pingcap/tiflash/issues/7810) @[SeaRise](https://github.com/SeaRise) @@ -332,9 +323,8 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - note [#issue](链接) @[贡献者 GitHub ID](链接) - note [#issue](链接) @[贡献者 GitHub ID](链接) - + TiCDC + + TiCDC - - Fix the issue that changefeeds would fail due to the temporary unavailability of PD [#9294](https://github.com/pingcap/tiflow/issues/9294) @[asddongmen](https://github.com/asddongmen) - Fix the data inconsistency issue that might occur when some TiCDC nodes were isolated from the network [#9344](https://github.com/pingcap/tiflow/issues/9344) @[CharlesCheung96](https://github.com/CharlesCheung96) - Fix the issue that when Kafka Sink encounters errors it might indefinitely block changefeed progress [#9309](https://github.com/pingcap/tiflow/issues/9309) @[hicqu](https://github.com/hicqu) From 2b3558f290b8269b3c2fb48fc7c396431a31a2ff Mon Sep 17 00:00:00 2001 From: Ran Date: Tue, 8 Aug 2023 18:39:17 +0800 Subject: [PATCH 53/72] Apply suggestions from code review --- releases/release-7.3.0.md | 53 +++++++-------------------------------- 1 file changed, 9 insertions(+), 44 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 17b27617ad5ae..b3bc4b15566ca 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -177,18 +177,17 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ### System variables -| 变量名 | 修改类型（包括新增/修改/删除） | 描述 | +| Variable name | Change type | Description | |--------|------------------------------|------| | [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) | Newly added | Controls whether the `EXPLAIN` statement disables the execution of constant subqueries that can be expanded at the optimization stage. | | [`tidb_skip_missing_partition_stats`](/system-variables.md#tidb_skip_missing_partition_stats-new-in-v730) | Newly added | This variable controls the generation of GlobalStats when partition statistics are missing. | -| [`tidb_remove_orderby_in_subquery`](/system-variables.md#tidb_remove_orderby_in_subquery-从-v610-版本开始引入) | 修改 | 经进一步的测试后，该变量默认值从 `OFF` 修改为 `ON`，即优化器改写会移除子查询中的 `ORDER BY` 子句。 | | [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-new-in-v730) | Newly added | Controls the strategy for selecting TiFlash replicas when a query requires the TiFlash engine. | -| | | | -| | | | +| [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-new-in-v720) | Modified | This system variable takes effect starting from v7.3.0. It controls whether non-recursive Common Table Expressions (CTEs) can be executed in TiFlash MPP. | +| [`tidb_lock_unchanged_keys`](/system-variables.md#tiflash_replica_read#tidb_lock_unchanged_keys-new-in-v711-and-v730) | Newly added | This variable is used to control in certain scenarios whether to lock the keys that are involved but not modified in a transaction. | ### Configuration file parameters -| 配置文件 | 配置项 | 修改类型 | 描述 | +| Configuration file | Configuration parameter | Change type | Description | | -------- | -------- | -------- | -------- | | TiDB | [`enable-32bits-connection-id`](/tidb-configuration-file.md#enable-32bits-connection-id-new-in-v730) | Newly added | Controls whether to enable the 32-bit connection ID feature. | | TiKV | [`coprocessor.region-bucket-size`](/tikv-configuration-file.md#region-bucket-size-new-in-v610) | Modified | Changes the default value from `96MiB` to `50MiB`. | @@ -201,7 +200,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | TiKV | [`storage.block-cache.capacity`](/tikv-configuration-file.md#capacity) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to compensate for the memory overhead of memtables, TiKV changes the default value from 45% to 30% of the size of total system memory. | | TiKV | [`rocksdb.write-buffer-limit`](/tikv-configuration-file.md#write-buffer-limit-new-in-v660) | Modified | To reduce the memory overhead of memtables, when `storage.engine="raft-kv"`, TiKV changes the default value from 25% of the memory of the machine to `0`, which means no limit. When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), TiKV changes the default value from 25% to 20% of the memory of the machine. | | TiKV | [`rocksdb.lockcf.write-buffer-size`](/tikv-configuration-file.md#write-buffer-size) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to speed up compaction on lockcf, TiKV changes the default value from `"32MB"` to `"4MB"`. | -| TiFlash | [`storage.format_version`](/tiflash/tiflash-configuration.md) | Modified | Introduces a new DTFile format `format_version = 5` to reduce the number of physical files by merging smaller files. Note that this format is experimental and not enabled by default. +| TiFlash | [`storage.format_version`](/tiflash/tiflash-configuration.md) | Modified | Introduces a new DTFile format `format_version = 5` to reduce the number of physical files by merging smaller files. Note that this format is experimental and not enabled by default. | |TiDB Lightning | `conflict.max-record-rows` | Newly added | The new version of strategy to handle conflicting data. It controls the maximum number of rows in the `conflict_records` table. The default value is 100. | |TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | |TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace”` or `conflict.strategy=“ignore"`, you can set it as a maxint. | @@ -209,17 +208,14 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- |TiDB Lightning | `tikv-importer.on-duplicate` | Deprecated | Controls action to do when trying to insert a conflicting record in the logical import mode. Startign from v7.3.0, this parameter is replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). | |TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | |TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. **tw:qiancai** | -| Data Migration | `strict-optimistic-shard-mode` | 新增 | 用于兼容历史版本 2.0 分库分表同步 DDL 的行为。当用户选择乐观模式时，可以启用该参数，开启后，乐观模式下，同步任务遇到二类 DDL 时，整个任务会中断，在多个表的 DDL变更有依赖关系的场景，可以及时中断，用户手动处理完各表的 DDL 后，再继续同步数据，保障上下游数据的一致性。 **tw@ran-huang** | +|BR | azblob.encryption-scope | Newly added | BR provides encryption scope support for Azure Blob Storage. | +|BR | azblob.encryption-key | Newly added | BR provides encryption key support for Azure Blob Storage. | | TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | | TiCDC | [`sink.csv.binary-encoding-method`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | The encoding method of binary data, which can be `'base64'` or `'hex'`. The default value is `'base64'`. | -### 系统表 +### System tables -- 新增系统表 `mysql.tidb_timers` 用来存储系统内部定时器的元信息。 - -## Deprecated features - -- note [#issue](链接) @[贡献者 GitHub ID](链接) +- Add a new system table `mysql.tidb_timers` to store the metadata of internal timers. ## Improvements @@ -249,28 +245,12 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + Backup & Restore (BR) - When backing up data to Azure Blob Storage using BR, you can specify either an encryption scope or an encryption key for server-side encryption [#45025](https://github.com/pingcap/tidb/issues/45025) @[Leavrth](https://github.com/Leavrth) **tw@Oreoxmt** - - note [#issue](链接) @[贡献者 GitHub ID](链接) + TiCDC - Optimize the message size output using the Open Protocol to make it include only the updated column values when sending `UPDATE` type events [#9336](https://github.com/pingcap/tiflow/issues/9336) @[3AceShowHand](https://github.com/3AceShowHand) - Storage Sink now supports hexadecimal encoding for HEX formatted data, making it compatible with AWS DMS format specifications [#9373](https://github.com/pingcap/tiflow/issues/9373) @[CharlesCheung96](https://github.com/CharlesCheung96) - Kafka Sink supports [sending only handle key data](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) when the message is too large, reducing the size of the message [#9382](https://github.com/pingcap/tiflow/issues/9382) @[3AceShowHand](https://github.com/3AceShowHand) **tw@ran-huang** - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - + TiDB Data Migration (DM) - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - + TiDB Lightning - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - + TiUP - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) ## Bug fixes @@ -318,11 +298,6 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + Tools - + Backup & Restore (BR) - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - + TiCDC - Fix the issue that changefeeds would fail due to the temporary unavailability of PD [#9294](https://github.com/pingcap/tiflow/issues/9294) @[asddongmen](https://github.com/asddongmen) @@ -331,20 +306,10 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Fix the panic issue that might occur when the TiCDC node status changes [#9354](https://github.com/pingcap/tiflow/issues/9354) @[sdojjy](https://github.com/sdojjy) - Fix the encoding error for the default `ENUM` values [#9259](https://github.com/pingcap/tiflow/issues/9259) @[3AceShowHand](https://github.com/3AceShowHand) - + TiDB Data Migration (DM) - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - + TiDB Lightning - Fix the issue that executing checksum after TiDB Lightning completes import might encounter SSL errors [#45462](https://github.com/pingcap/tidb/issues/45462) @[D3Hunter](https://github.com/D3Hunter) - + TiUP - - - note [#issue](链接) @[贡献者 GitHub ID](链接) - - note [#issue](链接) @[贡献者 GitHub ID](链接) - ## Contributors We would like to thank the following contributors from the TiDB community: From f99563a0f876a498a1aeed7fc698cdd71696519f Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 8 Aug 2023 18:58:01 +0800 Subject: [PATCH 54/72] Apply suggestions from code review --- releases/release-7.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index b3bc4b15566ca..df2742b63d913 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -228,7 +228,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiKV - - Add the `Max gap of safe-ts` and `Min safe ts region` metrics and introduce the `tikv-ctl get_region_read_progress` command to better observe and diagnose the status of resolved-ts and safe-ts [#15082](https://github.com/tikv/tikv/issues/15082) [@ekexium](https://github.com/ekexium) + - Add the `Max gap of safe-ts` and `Min safe ts region` metrics and introduce the `tikv-ctl get_region_read_progress` command to better observe and diagnose the status of resolved-ts and safe-ts [#15082](https://github.com/tikv/tikv/issues/15082) @[ekexium](https://github.com/ekexium) + PD @@ -279,7 +279,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiKV - - Fix the issue that reading data during GC might cause TiKV panic in some rare cases [#15109](https://github.com/tikv/tikv/issues/15109) [@MyonKeminta](https://github.com/MyonKeminta) + - Fix the issue that reading data during GC might cause TiKV panic in some rare cases [#15109](https://github.com/tikv/tikv/issues/15109) @[MyonKeminta](https://github.com/MyonKeminta) + PD From 4937bbf2d9358d8fc497c7c8d29394c6c7e9a45d Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 9 Aug 2023 10:53:06 +0800 Subject: [PATCH 55/72] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index df2742b63d913..83830c194041f 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -24,14 +24,14 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- Scalability and Performance - TiDB Lightning supports Partitioned Raft KV - TiDB Lightning now supports the new Partitioned Raft KV architecture, as part of the near-term GA of the architecture. + TiDB Lightning supports Partitioned Raft KV (experimental) + TiDB Lightning now supports the new Partitioned Raft KV architecture, as part of the near-term GA of the architecture. Reliability and Availability - Add automatic conflict detection and resolution on imports - The TiDB Lightning Physical Import Mode supports a new version of conflict detection, which implements the semantics of replacing (`replace`) or ignoring (`ignore`) conflict data when encountering conflicts. It automatically handles conflict data for you while improving the performance of conflict resolution. + Add automatic conflict detection and resolution on data imports + The TiDB Lightning Physical Import Mode supports a new version of conflict detection, which implements the semantics of replacing (replace) or ignoring (ignore) conflict data when encountering conflicts. It automatically handles conflict data for you while improving the performance of conflict resolution. Manual management of runaway queries (experimental) From 4dd82596e241ab123e7046e0bd3e5679feda22e4 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 10:54:33 +0800 Subject: [PATCH 56/72] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 83830c194041f..78b3a06bfe559 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -248,7 +248,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiCDC - - Optimize the message size output using the Open Protocol to make it include only the updated column values when sending `UPDATE` type events [#9336](https://github.com/pingcap/tiflow/issues/9336) @[3AceShowHand](https://github.com/3AceShowHand) + - Optimize the message size of the Open Protocol output to make it include only the updated column values when sending `UPDATE` events [#9336](https://github.com/pingcap/tiflow/issues/9336) @[3AceShowHand](https://github.com/3AceShowHand) - Storage Sink now supports hexadecimal encoding for HEX formatted data, making it compatible with AWS DMS format specifications [#9373](https://github.com/pingcap/tiflow/issues/9373) @[CharlesCheung96](https://github.com/CharlesCheung96) - Kafka Sink supports [sending only handle key data](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) when the message is too large, reducing the size of the message [#9382](https://github.com/pingcap/tiflow/issues/9382) @[3AceShowHand](https://github.com/3AceShowHand) **tw@ran-huang** @@ -301,14 +301,14 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiCDC - Fix the issue that changefeeds would fail due to the temporary unavailability of PD [#9294](https://github.com/pingcap/tiflow/issues/9294) @[asddongmen](https://github.com/asddongmen) - - Fix the data inconsistency issue that might occur when some TiCDC nodes were isolated from the network [#9344](https://github.com/pingcap/tiflow/issues/9344) @[CharlesCheung96](https://github.com/CharlesCheung96) + - Fix the data inconsistency issue that might occur when some TiCDC nodes are isolated from the network [#9344](https://github.com/pingcap/tiflow/issues/9344) @[CharlesCheung96](https://github.com/CharlesCheung96) - Fix the issue that when Kafka Sink encounters errors it might indefinitely block changefeed progress [#9309](https://github.com/pingcap/tiflow/issues/9309) @[hicqu](https://github.com/hicqu) - Fix the panic issue that might occur when the TiCDC node status changes [#9354](https://github.com/pingcap/tiflow/issues/9354) @[sdojjy](https://github.com/sdojjy) - Fix the encoding error for the default `ENUM` values [#9259](https://github.com/pingcap/tiflow/issues/9259) @[3AceShowHand](https://github.com/3AceShowHand) + TiDB Lightning - - Fix the issue that executing checksum after TiDB Lightning completes import might encounter SSL errors [#45462](https://github.com/pingcap/tidb/issues/45462) @[D3Hunter](https://github.com/D3Hunter) + - Fix the issue that executing checksum after TiDB Lightning completes import might get SSL errors [#45462](https://github.com/pingcap/tidb/issues/45462) @[D3Hunter](https://github.com/D3Hunter) ## Contributors From 6d4494833f363296ae4808fe4dcab6798fa051b4 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 9 Aug 2023 11:22:17 +0800 Subject: [PATCH 57/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 78b3a06bfe559..d1bb1646be6e9 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -62,7 +62,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- For a cluster that is deployed in multiple data centers and each data center has complete TiFlash data replicas, you can configure this strategy to only select TiFlash replicas from the current data center. This means data scanning and MPP calculations are performed only on TiFlash nodes in the current data center, which avoids excessive network data transmission across data centers. - For more information，see [documentation](/system-variables.md/system-variables.md#tiflash_replica_read-new-in-v730). + For more information, see [documentation](/system-variables.md/system-variables.md#tiflash_replica_read-new-in-v730). * TiFlash supports Runtime Filter within nodes [#40220](https://github.com/pingcap/tidb/issues/40220) @[elsa0520](https://github.com/elsa0520) **tw@ran-huang** From d4d04d1e085fb8e1a4f472b3f07c022076e5380a Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 9 Aug 2023 11:22:39 +0800 Subject: [PATCH 58/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index d1bb1646be6e9..dde821804e4a9 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -106,7 +106,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- * List and List COLUMNS partitioned tables support default partitions [#20679](https://github.com/pingcap/tidb/issues/20679) @[mjonss](https://github.com/mjonss) @[bb7133](https://github.com/bb7133) **tw@qiancai** - Before v7.3.0, when you use the `INSERT` statement to insert data into List or List COLUMNS partitioned tables, the data needs to meet the specified partitioning conditions of the table. If the data to be inserted does not meet any of these conditions, either the execution of the statement will fail or the non-compliant data will be ignored. + Before v7.3.0, when you use the `INSERT` statement to insert data into a List or List COLUMNS partitioned table, the data needs to meet the specified partitioning conditions of the table. If the data to be inserted does not meet any of these conditions, either the execution of the statement will fail or the non-compliant data will be ignored. Starting from v7.3.0, List and List COLUMNS partitioned tables support default partitions. After a default partition is created, if the data to be inserted does not meet any partitioning condition, it will be written to the default partition. This feature improves the usability of List and List COLUMNS partitioning, avoiding the execution failure of the `INSERT` statement or data being ignored due to data that does not meet partitioning conditions. From 13a2e3c4c9568ba64c0b9b525a827000ecf9d1a1 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 14:20:29 +0800 Subject: [PATCH 59/72] Apply suggestions from code review Co-authored-by: Roger Song Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index dde821804e4a9..45f3483d7c809 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -35,7 +35,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- Manual management of runaway queries (experimental) - Query timeouts might occur on any TiKV node but now resource groups can manage queries by global parallel execution time and either deprioritize or kill them. Allowing operators to mark target queries by exact SQL text, SQL digest, or plan digest and deal with the queries at a resource group level, this feature gives you much more control over the potential impact of unexpected large queries on a cluster. + Queries might take longer than you expect. With the new watch list of resource groups, you can now manage queries more effectively and either deprioritize or kill them. Allowing operators to mark target queries by exact SQL text, SQL digest, or plan digest and deal with the queries at a resource group level, this feature gives you much more control over the potential impact of unexpected large queries on a cluster. SQL @@ -70,7 +70,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- This feature is disabled by default in v7.3.0. To enable this feature, set the system variable [`tidb_runtime_filter_mode`](/system-variables.md#tidb_runtime_filter_mode-new-in-v720) to `LOCAL`. - For more information, refer to [user documentation](/runtime-filter.md). + For more information, see [documentation](/runtime-filter.md). * TiFlash supports executing common table expressions (CTEs) (experimental) [#43333](https://github.com/pingcap/tidb/issues/43333) @[winoros](https://github.com/winoros) **tw@ran-huang** @@ -92,7 +92,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - [`NO_HASH_JOIN()`](/optimizer-hints.md#no_hash_joint1_name--tl_name-) selects join methods other than hash join. - [`NO_INDEX_HASH_JOIN()`](/optimizer-hints.md#no_index_hash_joint1_name--tl_name-) selects join methods other than [index nested loop hash join](/optimizer-hints.md#inl_hash_join). - For more information, refer to [user documentation](/optimizer-hints). + For more information, see [documentation](/optimizer-hints.md). * Manually mark queries that use resources more than expected (experimental) [#43691](https://github.com/pingcap/tidb/issues/43691) @[Connor1996](https://github.com/Connor1996) @[CabinfeverB](https://github.com/CabinfeverB) **tw@hfxsd** @@ -130,9 +130,9 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- * Plan Replayer supports exporting historical statistics [#45038](https://github.com/pingcap/tidb/issues/45038) @[time-and-fate](https://github.com/time-and-fate) **tw@ran-huang** - Starting from v7.3.0, with the newly added [`dump with stats as of timestamp`](/sql-plan-replayer.md) clause, Plan Replayer can export the statistics of specified SQL-related objects at a specific point in time. During the diagnosis of execution plan issues, accurately capturing historical statistics can help analyze more precisely how the execution plan was generated at the time when the issue occurred. This helps identify the root cause of the issue and greatly improves efficiency in diagnosing execution plan issues. + Starting from v7.3.0, with the newly added [`dump with stats as of timestamp`](/sql-plan-replayer.md) clause, you can use Plan Replayer to export the statistics of specified SQL-related objects at a specific point in time. During the diagnosis of execution plan issues, accurately capturing historical statistics can help analyze more precisely how the execution plan was generated at the time when the issue occurred. This helps identify the root cause of the issue and greatly improves efficiency in diagnosing execution plan issues. - For more information, refer to [user documentation](/sql-plan-replayer.md). + For more information, see [documentation](/sql-plan-replayer.md). ### Data migration @@ -156,9 +156,9 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ## Compatibility changes -> **注意：** +> **Note:** > -> 以下为从 v7.1.0 升级至当前版本 (v7.2.0) 所需兼容性变更信息。如果从 v7.0.0 或之前版本升级到当前版本，可能也需要考虑和查看中间版本 release notes 中提到的兼容性变更信息。 +> This section provides compatibility changes you need to know when you upgrade from v7.2.0 to the current version (v7.3.0). If you are upgrading from v7.1.0 or earlier versions to the current version, you might also need to check the compatibility changes introduced in intermediate versions. ### Behavior changes @@ -205,7 +205,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- |TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | |TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace”` or `conflict.strategy=“ignore"`, you can set it as a maxint. | |TiDB Lightning | `enable-diagnose-logs` | Newly added | Controls whether to enable the diagnostic logs. The default value is `false`, that is, only the logs related to the import are output, and the logs of other dependent components are not output. When you set it to `true`, logs from both the import process and other dependent components are output, and GRPC debugging is enabled, which can be used for diagnosis. | -|TiDB Lightning | `tikv-importer.on-duplicate` | Deprecated | Controls action to do when trying to insert a conflicting record in the logical import mode. Startign from v7.3.0, this parameter is replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). | +|TiDB Lightning | `tikv-importer.on-duplicate` | Deprecated | Controls action to do when trying to insert a conflicting record in the logical import mode. Starting from v7.3.0, this parameter is replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). | |TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | |TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. **tw:qiancai** | |BR | azblob.encryption-scope | Newly added | BR provides encryption scope support for Azure Blob Storage. | From 4f7781763444d093b0a4bc0db755e12ad3157eb7 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 16:17:46 +0800 Subject: [PATCH 60/72] add dup notes --- releases/release-7.3.0.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 45f3483d7c809..0825484c1c074 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -276,6 +276,8 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Fix the `duplicate entry` error that occurs when restoring a table with `AUTO_ID_CACHE=1` using BR [#44716](https://github.com/pingcap/tidb/issues/44716) @[tiancaiamao](https://github.com/tiancaiamao) - Fix the issue that the time consumed for executing `TRUNCATE TABLE` is inconsistent with the task execution time shown in `ADMIN SHOW DDL JOBS` [#44785](https://github.com/pingcap/tidb/issues/44785) @[tangenta](https://github.com/tangenta) - Fix the issue that upgrading TiDB gets stuck when reading metadata takes longer than one DDL lease [#45176](https://github.com/pingcap/tidb/issues/45176) @[zimulala](https://github.com/zimulala) + - Fix the issue that the query result of the `SELECT CAST(n AS CHAR)` statement is incorrect when `n` in the statement is a negative number [#44786](https://github.com/pingcap/tidb/issues/44786) @[xhebox](https://github.com/xhebox) + - Fix the issue that queries might return incorrect results when `tidb_opt_agg_push_down` is enabled [#44795](https://github.com/pingcap/tidb/issues/44795) @[AilinKid](https://github.com/AilinKid) + TiKV @@ -309,6 +311,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + TiDB Lightning - Fix the issue that executing checksum after TiDB Lightning completes import might get SSL errors [#45462](https://github.com/pingcap/tidb/issues/45462) @[D3Hunter](https://github.com/D3Hunter) + - Fix the issue that in Logical Import Mode, deleting tables downstream during import might cause TiDB Lightning metadata not to be updated in time [#44614](https://github.com/pingcap/tidb/issues/44614) @[dsdashun](https://github.com/dsdashun) ## Contributors From e8ebc6bba9eddca9a81b696e12edaa42fb03821f Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 16:25:14 +0800 Subject: [PATCH 61/72] fix links Signed-off-by: Ran --- releases/release-7.3.0.md | 63 +++++++++++++++++++-------------------- 1 file changed, 30 insertions(+), 33 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 0825484c1c074..fd2d9025d3920 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -51,20 +51,19 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - ## Feature details ### Performance -* TiFlash supports the replica selection strategy [#44106](https://github.com/pingcap/tidb/issues/44106) @[XuHuaiyu](https://github.com/XuHuaiyu) **tw@qiancai** +* TiFlash supports the replica selection strategy [#44106](https://github.com/pingcap/tidb/issues/44106) @[XuHuaiyu](https://github.com/XuHuaiyu) Before v7.3.0, TiFlash uses replicas from all its nodes for data scanning and MPP calculations to maximize performance. Starting from v7.3.0, TiFlash introduces the replica selection strategy and lets you configure it using the [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-new-in-v730) system variable. This strategy supports selecting specific replicas based on the [zone attributes](/schedule-replicas-by-topology-labels.md#optional-configure-labels-for-tidb) of nodes and scheduling specific nodes for data scanning and MPP calculations. For a cluster that is deployed in multiple data centers and each data center has complete TiFlash data replicas, you can configure this strategy to only select TiFlash replicas from the current data center. This means data scanning and MPP calculations are performed only on TiFlash nodes in the current data center, which avoids excessive network data transmission across data centers. - For more information, see [documentation](/system-variables.md/system-variables.md#tiflash_replica_read-new-in-v730). + For more information, see [documentation](/system-variables.md#tiflash_replica_read-new-in-v730). -* TiFlash supports Runtime Filter within nodes [#40220](https://github.com/pingcap/tidb/issues/40220) @[elsa0520](https://github.com/elsa0520) **tw@ran-huang** +* TiFlash supports Runtime Filter within nodes [#40220](https://github.com/pingcap/tidb/issues/40220) @[elsa0520](https://github.com/elsa0520) Runtime Filter is a **dynamic predicate** generated during the query planning phase. In the process of table joining, these dynamic predicates can effectively filter out rows that do not meet the join conditions, reducing scan time and network overhead, and improving the efficiency of table joining. Starting from v7.3.0, TiFlash supports Runtime Filter within nodes, improving the overall performance of analytical queries. In some TPC-DS workloads, the performance can be improved by 10% to 50%. @@ -72,7 +71,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- For more information, see [documentation](/runtime-filter.md). -* TiFlash supports executing common table expressions (CTEs) (experimental) [#43333](https://github.com/pingcap/tidb/issues/43333) @[winoros](https://github.com/winoros) **tw@ran-huang** +* TiFlash supports executing common table expressions (CTEs) (experimental) [#43333](https://github.com/pingcap/tidb/issues/43333) @[winoros](https://github.com/winoros) Before v7.3.0, the MPP engine of TiFlash cannot execute queries that contain CTEs by default. To achieve the best execution performance within the MPP framework, you need to use the system variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-introduced-since-v630) to enforce inlining CTE. @@ -82,7 +81,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ### Reliability -* Add new optimizer hints [#45520](https://github.com/pingcap/tidb/issues/45520) @[qw4990](https://github.com/qw4990) **tw@ran-huang** +* Add new optimizer hints [#45520](https://github.com/pingcap/tidb/issues/45520) @[qw4990](https://github.com/qw4990) In v7.3.0, TiDB introduces several new optimizer hints to control the join methods between tables, including: @@ -94,7 +93,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- For more information, see [documentation](/optimizer-hints.md). -* Manually mark queries that use resources more than expected (experimental) [#43691](https://github.com/pingcap/tidb/issues/43691) @[Connor1996](https://github.com/Connor1996) @[CabinfeverB](https://github.com/CabinfeverB) **tw@hfxsd** +* Manually mark queries that use resources more than expected (experimental) [#43691](https://github.com/pingcap/tidb/issues/43691) @[Connor1996](https://github.com/Connor1996) @[CabinfeverB](https://github.com/CabinfeverB) In v7.2.0, TiDB automatically manages queries that use resources more than expected (Runaway Query) by automatically downgrading or canceling runaway queries. In actual practice, rules alone cannot cover all cases. Therefore, TiDB v7.3.0 introduces the ability to manually mark runaway queries. With the new command [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md), you can mark runaway queries based on SQL text, SQL Digest, or execution plan, and the marked runaway queries can be downgraded or cancelled. @@ -104,7 +103,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ### SQL -* List and List COLUMNS partitioned tables support default partitions [#20679](https://github.com/pingcap/tidb/issues/20679) @[mjonss](https://github.com/mjonss) @[bb7133](https://github.com/bb7133) **tw@qiancai** +* List and List COLUMNS partitioned tables support default partitions [#20679](https://github.com/pingcap/tidb/issues/20679) @[mjonss](https://github.com/mjonss) @[bb7133](https://github.com/bb7133) Before v7.3.0, when you use the `INSERT` statement to insert data into a List or List COLUMNS partitioned table, the data needs to meet the specified partitioning conditions of the table. If the data to be inserted does not meet any of these conditions, either the execution of the statement will fail or the non-compliant data will be ignored. @@ -122,13 +121,13 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ### Observability -* Show the progress of collecting statistics [#44033](https://github.com/pingcap/tidb/issues/44033) @[hawkingrei](https://github.com/hawkingrei) **tw@Oreoxmt** +* Show the progress of collecting statistics [#44033](https://github.com/pingcap/tidb/issues/44033) @[hawkingrei](https://github.com/hawkingrei) Collecting statistics for large tables often takes a long time. In previous versions, you cannot see the progress of collecting statistics, and therefore cannot predict the completion time. TiDB v7.3.0 introduces a feature to show the progress of collecting statistics. You can view the overall workload, current progress, and estimated completion time for each subtask using the system table `mysql.analyze_jobs` or `SHOW ANALYZE STATUS`. In scenarios such as large-scale data import and SQL performance optimization, this feature helps you understand the overall task progress and improves the user experience. For more information, see [documentation](/sql-statements/sql-statement-show-analyze-status.md). -* Plan Replayer supports exporting historical statistics [#45038](https://github.com/pingcap/tidb/issues/45038) @[time-and-fate](https://github.com/time-and-fate) **tw@ran-huang** +* Plan Replayer supports exporting historical statistics [#45038](https://github.com/pingcap/tidb/issues/45038) @[time-and-fate](https://github.com/time-and-fate) Starting from v7.3.0, with the newly added [`dump with stats as of timestamp`](/sql-plan-replayer.md) clause, you can use Plan Replayer to export the statistics of specified SQL-related objects at a specific point in time. During the diagnosis of execution plan issues, accurately capturing historical statistics can help analyze more precisely how the execution plan was generated at the time when the issue occurred. This helps identify the root cause of the issue and greatly improves efficiency in diagnosing execution plan issues. @@ -136,7 +135,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ### Data migration -* TiDB Lightning introduces a new version of conflict data detection and handling strategy [#41629](https://github.com/pingcap/tidb/issues/41629) @[lance6716](https://github.com/lance6716) **tw@hfxsd** +* TiDB Lightning introduces a new version of conflict data detection and handling strategy [#41629](https://github.com/pingcap/tidb/issues/41629) @[lance6716](https://github.com/lance6716) In previous versions, TiDB Lightning uses different conflict detection and handling methods for Logical Import Mode and Physical Import Mode, which are complex to configure and not easy for users to understand. In addition, Physical Import Mode cannot handle conflicts using the `replace` or `ignore` strategy. Starting from v7.3.0, TiDB Lightning introduces a unified conflict detection and handling strategy for both Logical Import Mode and Physical Import Mode. You can choose to report an error (`error`), replace (`replace`) or ignore (`ignore`) conflicting data when encountering conflicts. You can limit the number of conflict records, such as the task is interrupted and terminated after processing a specified number of conflict records. Furthermore, the system can record conflicting data for troubleshooting. @@ -144,11 +143,11 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- For more information, see [documentation](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#conflict-detection). -* TiDB Lightning supports Partitioned Raft KV (experimental) [#14916](https://github.com/tikv/tikv/issues/14916) @[GMHDBJD](https://github.com/GMHDBJD) **tw@hfxsd** +* TiDB Lightning supports Partitioned Raft KV (experimental) [#14916](https://github.com/tikv/tikv/issues/14916) @[GMHDBJD](https://github.com/GMHDBJD) TiDB Lightning now supports Partitioned Raft KV. This feature helps improve the data import performance of TiDB Lightning. -* TiDB Lightning introduces a new parameter `enable-diagnose-log` to enhance troubleshooting by printing more diagnostic logs [#45497](https://github.com/pingcap/tidb/issues/45497) @[D3Hunter](https://github.com/D3Hunter) **tw@hfxsd** +* TiDB Lightning introduces a new parameter `enable-diagnose-log` to enhance troubleshooting by printing more diagnostic logs [#45497](https://github.com/pingcap/tidb/issues/45497) @[D3Hunter](https://github.com/D3Hunter) By default, this feature is disabled and TiDB Lightning only prints logs containing `lightning/main`. When enabled, TiDB Lightning prints logs for all packages (including `client-go` and `tidb`) to help diagnose issues related to `client-go` and `tidb`. @@ -162,14 +161,12 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ### Behavior changes - - -* TiDB Lightning **tw@hfxsd** +* TiDB Lightning - `tikv-importer.on-duplicate` is deprecated and replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). - The `max-error` parameter, which controls the maximum number of non-fatal errors that TiDB Lightning can tolerate before stopping the migration task, no longer limits import data conflicts. The [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) parameter now controls the maximum number of conflicting records that can be tolerated. -* TiCDC **tw@ran-huang** +* TiCDC - When Kafka sink uses Avro protocol, if the `force-replicate` parameter is set to `true`, TiCDC reports an error when creating a changefeed. - Due to incompatibility between `delete-only-output-handle-key-columns` and `force-replicate` parameters, when both parameters are enabled, TiCDC reports an error when creating a changefeed. @@ -207,7 +204,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- |TiDB Lightning | `enable-diagnose-logs` | Newly added | Controls whether to enable the diagnostic logs. The default value is `false`, that is, only the logs related to the import are output, and the logs of other dependent components are not output. When you set it to `true`, logs from both the import process and other dependent components are output, and GRPC debugging is enabled, which can be used for diagnosis. | |TiDB Lightning | `tikv-importer.on-duplicate` | Deprecated | Controls action to do when trying to insert a conflicting record in the logical import mode. Starting from v7.3.0, this parameter is replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). | |TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | -|TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. **tw:qiancai** | +|TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. **tw:qiancai** | |BR | azblob.encryption-scope | Newly added | BR provides encryption scope support for Azure Blob Storage. | |BR | azblob.encryption-key | Newly added | BR provides encryption key support for Azure Blob Storage. | | TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | @@ -219,24 +216,24 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ## Improvements -+ TiDB ++ TiDB - - Introduce a new system variable [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) to control whether the `EXPLAIN` statement executes subqueries in advance during the optimization phase [#22076](https://github.com/pingcap/tidb/issues/22076) @[winoros](https://github.com/winoros) **tw@Oreoxmt** - - When [Global Kill](/tidb-configuration-file#enable-global-kill-new-in-v610) is enabled, you can terminate the current session by pressing Control+C [#8854](https://github.com/pingcap/tidb/issues/8854) @[pingyu](https://github.com/pingyu) **tw@Oreoxmt** + - Introduce a new system variable [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) to control whether the `EXPLAIN` statement executes subqueries in advance during the optimization phase [#22076](https://github.com/pingcap/tidb/issues/22076) @[winoros](https://github.com/winoros) + - When [Global Kill](/tidb-configuration-file.md#enable-global-kill-new-in-v610) is enabled, you can terminate the current session by pressing Control+C [#8854](https://github.com/pingcap/tidb/issues/8854) @[pingyu](https://github.com/pingyu) - Support the `IS_FREE_LOCK()` and `IS_USED_LOCK()` locking functions [#44493](https://github.com/pingcap/tidb/issues/44493) @[dveeden](https://github.com/dveeden) - Optimize the performance of reading the dumped chunks from disk [#45125](https://github.com/pingcap/tidb/issues/45125) @[YangKeao](https://github.com/YangKeao) -+ TiKV ++ TiKV - Add the `Max gap of safe-ts` and `Min safe ts region` metrics and introduce the `tikv-ctl get_region_read_progress` command to better observe and diagnose the status of resolved-ts and safe-ts [#15082](https://github.com/tikv/tikv/issues/15082) @[ekexium](https://github.com/ekexium) -+ PD ++ PD - Support blocking the Swagger API by default when the Swagger server is not enabled [#6786](https://github.com/tikv/pd/issues/6786) @[bufferflies](https://github.com/bufferflies) - Improve the high availability of etcd [#6554](https://github.com/tikv/pd/issues/6554) [#6442](https://github.com/tikv/pd/issues/6442) @[lhy1024](https://github.com/lhy1024) - Reduce the memory consumption of `GetRegions` requests [#6835](https://github.com/tikv/pd/issues/6835) @[lhy1024](https://github.com/lhy1024) -+ TiFlash ++ TiFlash - Support a new DTFile format version [`storage.format_version = 5`](/tiflash/tiflash-configuration.md) to reduce the number of physical files (experimental) [#7595](https://github.com/pingcap/tiflash/issues/7595) @[hongyunyan](https://github.com/hongyunyan) @@ -244,19 +241,19 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + Backup & Restore (BR) - - When backing up data to Azure Blob Storage using BR, you can specify either an encryption scope or an encryption key for server-side encryption [#45025](https://github.com/pingcap/tidb/issues/45025) @[Leavrth](https://github.com/Leavrth) **tw@Oreoxmt** + - When backing up data to Azure Blob Storage using BR, you can specify either an encryption scope or an encryption key for server-side encryption [#45025](https://github.com/pingcap/tidb/issues/45025) @[Leavrth](https://github.com/Leavrth) - + TiCDC + + TiCDC - Optimize the message size of the Open Protocol output to make it include only the updated column values when sending `UPDATE` events [#9336](https://github.com/pingcap/tiflow/issues/9336) @[3AceShowHand](https://github.com/3AceShowHand) - Storage Sink now supports hexadecimal encoding for HEX formatted data, making it compatible with AWS DMS format specifications [#9373](https://github.com/pingcap/tiflow/issues/9373) @[CharlesCheung96](https://github.com/CharlesCheung96) - - Kafka Sink supports [sending only handle key data](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) when the message is too large, reducing the size of the message [#9382](https://github.com/pingcap/tiflow/issues/9382) @[3AceShowHand](https://github.com/3AceShowHand) **tw@ran-huang** + - Kafka Sink supports [sending only handle key data](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) when the message is too large, reducing the size of the message [#9382](https://github.com/pingcap/tiflow/issues/9382) @[3AceShowHand](https://github.com/3AceShowHand) ## Bug fixes -+ TiDB ++ TiDB - - Fix the issue that when the MySQL Cursor Fetch protocol is used, the memory consumption of result sets might exceed the `tidb_mem_quota_query` limit and causes TiDB OOM. After the fix, TiDB will automatically write result sets to the disk to release memory [#43233](https://github.com/pingcap/tidb/issues/43233) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that when the MySQL Cursor Fetch protocol is used, the memory consumption of result sets might exceed the `tidb_mem_quota_query` limit and causes TiDB OOM. After the fix, TiDB will automatically write result sets to the disk to release memory [#43233](https://github.com/pingcap/tidb/issues/43233) @[YangKeao](https://github.com/YangKeao) - Fix the TiDB panic issue caused by data race [#45561](https://github.com/pingcap/tidb/issues/45561) @[genliqi](https://github.com/gengliqi) - Fix the hang-up issue that occurs when queries with `indexMerge` are killed [#45279](https://github.com/pingcap/tidb/issues/45279) @[xzhangxian1008](https://github.com/xzhangxian1008) - Fix the issue that query results in MPP mode are incorrect when `tidb_enable_parallel_apply` is enabled [#45299](https://github.com/pingcap/tidb/issues/45299) @[windtalker](https://github.com/windtalker) @@ -279,11 +276,11 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Fix the issue that the query result of the `SELECT CAST(n AS CHAR)` statement is incorrect when `n` in the statement is a negative number [#44786](https://github.com/pingcap/tidb/issues/44786) @[xhebox](https://github.com/xhebox) - Fix the issue that queries might return incorrect results when `tidb_opt_agg_push_down` is enabled [#44795](https://github.com/pingcap/tidb/issues/44795) @[AilinKid](https://github.com/AilinKid) -+ TiKV ++ TiKV - Fix the issue that reading data during GC might cause TiKV panic in some rare cases [#15109](https://github.com/tikv/tikv/issues/15109) @[MyonKeminta](https://github.com/MyonKeminta) -+ PD ++ PD - Fix the issue that restarting PD might cause the `default` resource group to be reinitialized [#6787](https://github.com/tikv/pd/issues/6787) @[glorv](https://github.com/glorv) - Fix the issue that when etcd is already started but the client has not yet connected to it, calling the client might cause PD to panic [#6860](https://github.com/tikv/pd/issues/6860) @[HuSharp](https://github.com/HuSharp) @@ -292,7 +289,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Fix the issue that Placement Rules select TiFlash learners that do not meet the rules [#6662](https://github.com/tikv/pd/issues/6662) @[rleungx](https://github.com/rleungx) - Fix the issue that unhealthy peers cannot be removed when rule checker selects peers [#6559](https://github.com/tikv/pd/issues/6559) [nolouch](https://github.com/nolouch) -+ TiFlash ++ TiFlash - Fix the issue that TiFlash cannot replicate partitioned tables successfully due to deadlocks [#7758](https://github.com/pingcap/tiflash/issues/7758) @[hongyunyan](https://github.com/hongyunyan) - Fix the issue that the `INFORMATION_SCHEMA.TIFLASH_REPLICA` system table contains tables that users do not have privileges to access [#7795](https://github.com/pingcap/tiflash/issues/7795) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) @@ -300,7 +297,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- + Tools - + TiCDC + + TiCDC - Fix the issue that changefeeds would fail due to the temporary unavailability of PD [#9294](https://github.com/pingcap/tiflow/issues/9294) @[asddongmen](https://github.com/asddongmen) - Fix the data inconsistency issue that might occur when some TiCDC nodes are isolated from the network [#9344](https://github.com/pingcap/tiflow/issues/9344) @[CharlesCheung96](https://github.com/CharlesCheung96) From b5430222ee4726fff524930d1355f1eaff426c15 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 16:30:48 +0800 Subject: [PATCH 62/72] fix link anchors Signed-off-by: Ran --- releases/release-7.3.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index fd2d9025d3920..1418f3054c134 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -73,7 +73,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- * TiFlash supports executing common table expressions (CTEs) (experimental) [#43333](https://github.com/pingcap/tidb/issues/43333) @[winoros](https://github.com/winoros) - Before v7.3.0, the MPP engine of TiFlash cannot execute queries that contain CTEs by default. To achieve the best execution performance within the MPP framework, you need to use the system variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-introduced-since-v630) to enforce inlining CTE. + Before v7.3.0, the MPP engine of TiFlash cannot execute queries that contain CTEs by default. To achieve the best execution performance within the MPP framework, you need to use the system variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) to enforce inlining CTE. Starting from v7.3.0, TiFlash's MPP engine supports executing queries with CTEs without inlining them, allowing for optimal query execution within the MPP framework. In TPC-DS benchmark tests, compared with inlining CTEs, this feature has shown a 20% improvement in overall query execution speed for queries containing CTE. @@ -111,7 +111,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- Note that this feature is a TiDB extension to MySQL syntax. For a partitioned table with a default partition, the data in the table cannot be directly replicated to MySQL. - For more information, see [documentation](/partitioned-table.md#list-partition). + For more information, see [documentation](/partitioned-table.md#list-partitioning). ### DB operations @@ -180,7 +180,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | [`tidb_skip_missing_partition_stats`](/system-variables.md#tidb_skip_missing_partition_stats-new-in-v730) | Newly added | This variable controls the generation of GlobalStats when partition statistics are missing. | | [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-new-in-v730) | Newly added | Controls the strategy for selecting TiFlash replicas when a query requires the TiFlash engine. | | [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-new-in-v720) | Modified | This system variable takes effect starting from v7.3.0. It controls whether non-recursive Common Table Expressions (CTEs) can be executed in TiFlash MPP. | -| [`tidb_lock_unchanged_keys`](/system-variables.md#tiflash_replica_read#tidb_lock_unchanged_keys-new-in-v711-and-v730) | Newly added | This variable is used to control in certain scenarios whether to lock the keys that are involved but not modified in a transaction. | +| [`tidb_lock_unchanged_keys`](/system-variables.md#tidb_lock_unchanged_keys-new-in-v711-and-v730) | Newly added | This variable is used to control in certain scenarios whether to lock the keys that are involved but not modified in a transaction. | ### Configuration file parameters From a437dbffa80b8fa211a30d64a0d367e428353bce Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 16:40:35 +0800 Subject: [PATCH 63/72] add toc and other files --- TOC.md | 4 +++- releases/release-7.3.0.md | 2 +- releases/release-notes.md | 4 ++++ releases/release-timeline.md | 1 + 4 files changed, 9 insertions(+), 2 deletions(-) diff --git a/TOC.md b/TOC.md index cfb39595e32e0..1c8f2b8483e6a 100644 --- a/TOC.md +++ b/TOC.md @@ -4,7 +4,7 @@ - [Docs Home](https://docs.pingcap.com/) - About TiDB - [TiDB Introduction](/overview.md) - - [TiDB 7.2 Release Notes](/releases/release-7.2.0.md) + - [TiDB 7.3 Release Notes](/releases/release-7.3.0.md) - [Features](/basic-features.md) - [MySQL Compatibility](/mysql-compatibility.md) - [TiDB Limitations](/tidb-limitations.md) @@ -990,6 +990,8 @@ - [Release Timeline](/releases/release-timeline.md) - [TiDB Versioning](/releases/versioning.md) - [TiDB Installation Packages](/binary-package.md) + - v7.3 + - [7.3.0-DMR](/releases/release-7.3.0.md) - v7.2 - [7.2.0-DMR](/releases/release-7.2.0.md) - v7.1 diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 1418f3054c134..0462a181be471 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -5,7 +5,7 @@ summary: Learn about the new features, compatibility changes, improvements, and # TiDB 7.3.0 Release Notes -Release date: xx xx, 2023 +Release date: August 10, 2023 TiDB version: 7.3.0 diff --git a/releases/release-notes.md b/releases/release-notes.md index 2dce4ee2afe1d..284ac0899d247 100644 --- a/releases/release-notes.md +++ b/releases/release-notes.md @@ -5,6 +5,10 @@ aliases: ['/docs/dev/releases/release-notes/','/docs/dev/releases/rn/'] # TiDB Release Notes +## 7.3 + +- [7.3.0-DMR](/releases/release-7.3.0.md): 2023-08-10 + ## 7.2 - [7.2.0-DMR](/releases/release-7.2.0.md): 2023-06-29 diff --git a/releases/release-timeline.md b/releases/release-timeline.md index 66537118fb5d1..ceb33a69e73dc 100644 --- a/releases/release-timeline.md +++ b/releases/release-timeline.md @@ -9,6 +9,7 @@ This document shows all the released TiDB versions in reverse chronological orde | Version | Release Date | | :--- | :--- | +| [7.3.0-DMR](/releases/release-7.3.0.md) | 2023-08-10 | | [7.1.1](/releases/release-7.1.1.md) | 2023-07-24 | | [6.1.7](/releases/release-6.1.7.md) | 2023-07-12 | | [7.2.0-DMR](/releases/release-7.2.0.md) | 2023-06-29 | From bdaf51f58bd8a13d056cad7de6f1ca7739ec1b61 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 18:05:10 +0800 Subject: [PATCH 64/72] Apply suggestions from code review Co-authored-by: Aolin --- releases/release-7.3.0.md | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 0462a181be471..3d45cc3583565 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -40,7 +40,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- SQL Enhance operator control over query stability by adding more optimizer hints to the query planner - Added hints: NO_INDEX_JOIN(), NO_MERGE_JOIN(), NO_INDEX_MERGE_JOIN(), NO_HASH_JOIN(), NO_INDEX_HASH_JOIN() + Added hints: NO_INDEX_JOIN(), NO_MERGE_JOIN(), NO_INDEX_MERGE_JOIN(), NO_HASH_JOIN(), NO_INDEX_HASH_JOIN() @@ -91,7 +91,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - [`NO_HASH_JOIN()`](/optimizer-hints.md#no_hash_joint1_name--tl_name-) selects join methods other than hash join. - [`NO_INDEX_HASH_JOIN()`](/optimizer-hints.md#no_index_hash_joint1_name--tl_name-) selects join methods other than [index nested loop hash join](/optimizer-hints.md#inl_hash_join). - For more information, see [documentation](/optimizer-hints.md). + For more information, see [documentation](/optimizer-hints.md). * Manually mark queries that use resources more than expected (experimental) [#43691](https://github.com/pingcap/tidb/issues/43691) @[Connor1996](https://github.com/Connor1996) @[CabinfeverB](https://github.com/CabinfeverB) @@ -113,12 +113,6 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- For more information, see [documentation](/partitioned-table.md#list-partitioning). -### DB operations - -- note 1 - -- note 2 - ### Observability * Show the progress of collecting statistics [#44033](https://github.com/pingcap/tidb/issues/44033) @[hawkingrei](https://github.com/hawkingrei) @@ -179,8 +173,8 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) | Newly added | Controls whether the `EXPLAIN` statement disables the execution of constant subqueries that can be expanded at the optimization stage. | | [`tidb_skip_missing_partition_stats`](/system-variables.md#tidb_skip_missing_partition_stats-new-in-v730) | Newly added | This variable controls the generation of GlobalStats when partition statistics are missing. | | [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-new-in-v730) | Newly added | Controls the strategy for selecting TiFlash replicas when a query requires the TiFlash engine. | -| [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-new-in-v720) | Modified | This system variable takes effect starting from v7.3.0. It controls whether non-recursive Common Table Expressions (CTEs) can be executed in TiFlash MPP. | | [`tidb_lock_unchanged_keys`](/system-variables.md#tidb_lock_unchanged_keys-new-in-v711-and-v730) | Newly added | This variable is used to control in certain scenarios whether to lock the keys that are involved but not modified in a transaction. | +| [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-new-in-v720) | Modified | This system variable takes effect starting from v7.3.0. It controls whether non-recursive Common Table Expressions (CTEs) can be executed in TiFlash MPP. | ### Configuration file parameters @@ -199,12 +193,12 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | TiKV | [`rocksdb.lockcf.write-buffer-size`](/tikv-configuration-file.md#write-buffer-size) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to speed up compaction on lockcf, TiKV changes the default value from `"32MB"` to `"4MB"`. | | TiFlash | [`storage.format_version`](/tiflash/tiflash-configuration.md) | Modified | Introduces a new DTFile format `format_version = 5` to reduce the number of physical files by merging smaller files. Note that this format is experimental and not enabled by default. | |TiDB Lightning | `conflict.max-record-rows` | Newly added | The new version of strategy to handle conflicting data. It controls the maximum number of rows in the `conflict_records` table. The default value is 100. | -|TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | -|TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace”` or `conflict.strategy=“ignore"`, you can set it as a maxint. | +|TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | +|TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace"` or `conflict.strategy="ignore"`, you can set it as a maxint. | |TiDB Lightning | `enable-diagnose-logs` | Newly added | Controls whether to enable the diagnostic logs. The default value is `false`, that is, only the logs related to the import are output, and the logs of other dependent components are not output. When you set it to `true`, logs from both the import process and other dependent components are output, and GRPC debugging is enabled, which can be used for diagnosis. | |TiDB Lightning | `tikv-importer.on-duplicate` | Deprecated | Controls action to do when trying to insert a conflicting record in the logical import mode. Starting from v7.3.0, this parameter is replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). | |TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | -|TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. **tw:qiancai** | +|TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. | |BR | azblob.encryption-scope | Newly added | BR provides encryption scope support for Azure Blob Storage. | |BR | azblob.encryption-key | Newly added | BR provides encryption key support for Azure Blob Storage. | | TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | @@ -287,7 +281,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Fix the issue that the `health-check` output of a Region is inconsistent with the Region information returned by querying the Region ID [#6560](https://github.com/tikv/pd/issues/6560) @[JmPotato](https://github.com/JmPotato) - Fix the issue that failed learner peers in `unsafe recovery` are ignored in `auto-detect` mode [#6690](https://github.com/tikv/pd/issues/6690) @[v01dstar](https://github.com/v01dstar) - Fix the issue that Placement Rules select TiFlash learners that do not meet the rules [#6662](https://github.com/tikv/pd/issues/6662) @[rleungx](https://github.com/rleungx) - - Fix the issue that unhealthy peers cannot be removed when rule checker selects peers [#6559](https://github.com/tikv/pd/issues/6559) [nolouch](https://github.com/nolouch) + - Fix the issue that unhealthy peers cannot be removed when rule checker selects peers [#6559](https://github.com/tikv/pd/issues/6559) @[nolouch](https://github.com/nolouch) + TiFlash From 24670781ec515182aef4b62febe67dd939ad60f6 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 18:06:42 +0800 Subject: [PATCH 65/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 3d45cc3583565..64b8c34d57a88 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -155,6 +155,10 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ### Behavior changes +* Backup & Restore (BR) + + - BR adds an empty cluster check before performing a full data restoration. By default, restoring data to a non-empty cluster is not allowed. If you want to force the restoration, you can use the `--filter` option to specify the corresponding table name to restore data to. + * TiDB Lightning - `tikv-importer.on-duplicate` is deprecated and replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). From 7817c631369fb15e29f95f20da96685c7e7b0011 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 18:17:31 +0800 Subject: [PATCH 66/72] adjust order of sysvar and configs Signed-off-by: Ran --- releases/release-7.3.0.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 64b8c34d57a88..9e4b520afc6e5 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -155,7 +155,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- ### Behavior changes -* Backup & Restore (BR) +* Backup & Restore (BR) - BR adds an empty cluster check before performing a full data restoration. By default, restoring data to a non-empty cluster is not allowed. If you want to force the restoration, you can use the `--filter` option to specify the corresponding table name to restore data to. @@ -174,11 +174,11 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | Variable name | Change type | Description | |--------|------------------------------|------| +| [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-new-in-v720) | Modified | This system variable takes effect starting from v7.3.0. It controls whether non-recursive Common Table Expressions (CTEs) can be executed in TiFlash MPP. | +| [`tidb_lock_unchanged_keys`](/system-variables.md#tidb_lock_unchanged_keys-new-in-v711-and-v730) | Newly added | This variable is used to control in certain scenarios whether to lock the keys that are involved but not modified in a transaction. | | [`tidb_opt_enable_non_eval_scalar_subquery`](/system-variables.md#tidb_opt_enable_non_eval_scalar_subquery-new-in-v730) | Newly added | Controls whether the `EXPLAIN` statement disables the execution of constant subqueries that can be expanded at the optimization stage. | | [`tidb_skip_missing_partition_stats`](/system-variables.md#tidb_skip_missing_partition_stats-new-in-v730) | Newly added | This variable controls the generation of GlobalStats when partition statistics are missing. | | [`tiflash_replica_read`](/system-variables.md#tiflash_replica_read-new-in-v730) | Newly added | Controls the strategy for selecting TiFlash replicas when a query requires the TiFlash engine. | -| [`tidb_lock_unchanged_keys`](/system-variables.md#tidb_lock_unchanged_keys-new-in-v711-and-v730) | Newly added | This variable is used to control in certain scenarios whether to lock the keys that are involved but not modified in a transaction. | -| [`tidb_opt_enable_mpp_shared_cte_execution`](/system-variables.md#tidb_opt_enable_mpp_shared_cte_execution-new-in-v720) | Modified | This system variable takes effect starting from v7.3.0. It controls whether non-recursive Common Table Expressions (CTEs) can be executed in TiFlash MPP. | ### Configuration file parameters @@ -186,25 +186,25 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | -------- | -------- | -------- | -------- | | TiDB | [`enable-32bits-connection-id`](/tidb-configuration-file.md#enable-32bits-connection-id-new-in-v730) | Newly added | Controls whether to enable the 32-bit connection ID feature. | | TiKV | [`coprocessor.region-bucket-size`](/tikv-configuration-file.md#region-bucket-size-new-in-v610) | Modified | Changes the default value from `96MiB` to `50MiB`. | -| TiKV | [rocksdb.\[defaultcf\|writecf\|lockcf\].compaction-guard-min-output-file-size](/tikv-configuration-file.md#compaction-guard-min-output-file-size) | Modified | Changes the default value from `"1MB"` to `"8MB"` to resolve the issue that compaction speed cannot keep up with the write speed during large data writes. | -| TiKV | [rocksdb.\[defaultcf\|writecf\|lockcf\].format-version](/tikv-configuration-file.md#format-version-new-in-v620) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), Ribbon filter is used. Therefore, TiKV changes the default value from `2` to `5`. | | TiKV | [`raft-engine.format-version`](/tikv-configuration-file.md#format-version-new-in-v630) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), Ribbon filter is used. Therefore, TiKV changes the default value from `2` to `5`. | | TiKV | [`raftdb.max-total-wal-size`](/tikv-configuration-file.md#max-total-wal-size-1) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), TiKV skips writing WAL. Therefore, TiKV changes the default value from `"4GB"` to `1`, meaning that WAL is disabled. | +| TiKV | [rocksdb.\[defaultcf\|writecf\|lockcf\].compaction-guard-min-output-file-size](/tikv-configuration-file.md#compaction-guard-min-output-file-size) | Modified | Changes the default value from `"1MB"` to `"8MB"` to resolve the issue that compaction speed cannot keep up with the write speed during large data writes. | +| TiKV | [rocksdb.\[defaultcf\|writecf\|lockcf\].format-version](/tikv-configuration-file.md#format-version-new-in-v620) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), Ribbon filter is used. Therefore, TiKV changes the default value from `2` to `5`. | +| TiKV | [`rocksdb.lockcf.write-buffer-size`](/tikv-configuration-file.md#write-buffer-size) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to speed up compaction on lockcf, TiKV changes the default value from `"32MB"` to `"4MB"`. | | TiKV | [`rocksdb.max-total-wal-size`](/tikv-configuration-file.md#max-total-wal-size) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), TiKV skips writing WAL. Therefore, TiKV changes the default value from `"4GB"` to `1`, meaning that WAL is disabled. | | TiKV | [`rocksdb.stats-dump-period`](/tikv-configuration-file.md#stats-dump-period) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to disable redundant log printing, changes the default value from `"10m"` to `"0"`. | -| TiKV | [`storage.block-cache.capacity`](/tikv-configuration-file.md#capacity) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to compensate for the memory overhead of memtables, TiKV changes the default value from 45% to 30% of the size of total system memory. | | TiKV | [`rocksdb.write-buffer-limit`](/tikv-configuration-file.md#write-buffer-limit-new-in-v660) | Modified | To reduce the memory overhead of memtables, when `storage.engine="raft-kv"`, TiKV changes the default value from 25% of the memory of the machine to `0`, which means no limit. When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), TiKV changes the default value from 25% to 20% of the memory of the machine. | -| TiKV | [`rocksdb.lockcf.write-buffer-size`](/tikv-configuration-file.md#write-buffer-size) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to speed up compaction on lockcf, TiKV changes the default value from `"32MB"` to `"4MB"`. | +| TiKV | [`storage.block-cache.capacity`](/tikv-configuration-file.md#capacity) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), to compensate for the memory overhead of memtables, TiKV changes the default value from 45% to 30% of the size of total system memory. | | TiFlash | [`storage.format_version`](/tiflash/tiflash-configuration.md) | Modified | Introduces a new DTFile format `format_version = 5` to reduce the number of physical files by merging smaller files. Note that this format is experimental and not enabled by default. | -|TiDB Lightning | `conflict.max-record-rows` | Newly added | The new version of strategy to handle conflicting data. It controls the maximum number of rows in the `conflict_records` table. The default value is 100. | -|TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | -|TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace"` or `conflict.strategy="ignore"`, you can set it as a maxint. | -|TiDB Lightning | `enable-diagnose-logs` | Newly added | Controls whether to enable the diagnostic logs. The default value is `false`, that is, only the logs related to the import are output, and the logs of other dependent components are not output. When you set it to `true`, logs from both the import process and other dependent components are output, and GRPC debugging is enabled, which can be used for diagnosis. | -|TiDB Lightning | `tikv-importer.on-duplicate` | Deprecated | Controls action to do when trying to insert a conflicting record in the logical import mode. Starting from v7.3.0, this parameter is replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). | -|TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | +| TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | +| TiDB Lightning | `tikv-importer.on-duplicate` | Deprecated | Controls action to do when trying to insert a conflicting record in the logical import mode. Starting from v7.3.0, this parameter is replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). | +| TiDB Lightning | `conflict.max-record-rows` | Newly added | The new version of strategy to handle conflicting data. It controls the maximum number of rows in the `conflict_records` table. The default value is 100. | +| TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | +| TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace"` or `conflict.strategy="ignore"`, you can set it as a maxint. | +| TiDB Lightning | `enable-diagnose-logs` | Newly added | Controls whether to enable the diagnostic logs. The default value is `false`, that is, only the logs related to the import are output, and the logs of other dependent components are not output. When you set it to `true`, logs from both the import process and other dependent components are output, and GRPC debugging is enabled, which can be used for diagnosis. | |TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. | -|BR | azblob.encryption-scope | Newly added | BR provides encryption scope support for Azure Blob Storage. | -|BR | azblob.encryption-key | Newly added | BR provides encryption key support for Azure Blob Storage. | +|BR | `azblob.encryption-scope` | Newly added | BR provides encryption scope support for Azure Blob Storage. | +|BR | `azblob.encryption-key` | Newly added | BR provides encryption key support for Azure Blob Storage. | | TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | | TiCDC | [`sink.csv.binary-encoding-method`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | The encoding method of binary data, which can be `'base64'` or `'hex'`. The default value is `'base64'`. | From 3f5ea826d4767e994636a0085610cdc6681083bf Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Wed, 9 Aug 2023 18:49:10 +0800 Subject: [PATCH 67/72] add a missing column for `sink.csv.binary-encoding-method` --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 9e4b520afc6e5..29ab4dc2693b1 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -206,7 +206,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- |BR | `azblob.encryption-scope` | Newly added | BR provides encryption scope support for Azure Blob Storage. | |BR | `azblob.encryption-key` | Newly added | BR provides encryption key support for Azure Blob Storage. | | TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | -| TiCDC | [`sink.csv.binary-encoding-method`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | The encoding method of binary data, which can be `'base64'` or `'hex'`. The default value is `'base64'`. | +| TiCDC | [`sink.csv.binary-encoding-method`](/ticdc/ticdc-changefeed-config.md#changefeed-configuration-parameters) | Newly added | The encoding method of binary data, which can be `'base64'` or `'hex'`. The default value is `'base64'`. | ### System tables From 57b89f279119be6e91fb2326e0dfcebd0626c72a Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 20:21:27 +0800 Subject: [PATCH 68/72] Update releases/release-7.3.0.md Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 29ab4dc2693b1..f8711eeb4703b 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -202,7 +202,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | | TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace"` or `conflict.strategy="ignore"`, you can set it as a maxint. | | TiDB Lightning | `enable-diagnose-logs` | Newly added | Controls whether to enable the diagnostic logs. The default value is `false`, that is, only the logs related to the import are output, and the logs of other dependent components are not output. When you set it to `true`, logs from both the import process and other dependent components are output, and GRPC debugging is enabled, which can be used for diagnosis. | -|TiDB Lightning | `tikv-importer.parallel-import` | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. | +|TiDB Lightning | [`tikv-importer.parallel-import`](/tidb-lightning/tidb-lightning-configuration.md) | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. | |BR | `azblob.encryption-scope` | Newly added | BR provides encryption scope support for Azure Blob Storage. | |BR | `azblob.encryption-key` | Newly added | BR provides encryption key support for Azure Blob Storage. | | TiCDC | [`large-message-handle-option`](/ticdc/ticdc-sink-to-kafka.md#handle-messages-that-exceed-the-kafka-topic-limit) | Newly added | Empty by default, which means that when the message size exceeds the limit of Kafka topic, the changefeed fails. When this configuration is set to `"handle-key-only"`, if the message exceeds the size limit, only the handle key will be sent to reduce the message size; if the reduced message still exceeds the limit, then the changefeed fails. | From 0694b3b945af62780559a07e887f86d094603957 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 9 Aug 2023 20:21:35 +0800 Subject: [PATCH 69/72] Update releases/release-7.3.0.md Co-authored-by: Grace Cai --- releases/release-7.3.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index f8711eeb4703b..b8c7c0b6241fe 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -198,10 +198,10 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | TiFlash | [`storage.format_version`](/tiflash/tiflash-configuration.md) | Modified | Introduces a new DTFile format `format_version = 5` to reduce the number of physical files by merging smaller files. Note that this format is experimental and not enabled by default. | | TiDB Lightning | `tikv-importer.incremental-import` | Deleted | TiDB Lightning parallel import parameter. Because it could easily be mistaken as an incremental import parameter, this parameter is now renamed to `tikv-importer.parallel-import`. If a user passes in the old parameter name, it will be automatically converted to the new one. | | TiDB Lightning | `tikv-importer.on-duplicate` | Deprecated | Controls action to do when trying to insert a conflicting record in the logical import mode. Starting from v7.3.0, this parameter is replaced by [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task). | -| TiDB Lightning | `conflict.max-record-rows` | Newly added | The new version of strategy to handle conflicting data. It controls the maximum number of rows in the `conflict_records` table. The default value is 100. | -| TiDB Lightning | `conflict.strategy` | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | -| TiDB Lightning | `conflict.threshold` | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace"` or `conflict.strategy="ignore"`, you can set it as a maxint. | -| TiDB Lightning | `enable-diagnose-logs` | Newly added | Controls whether to enable the diagnostic logs. The default value is `false`, that is, only the logs related to the import are output, and the logs of other dependent components are not output. When you set it to `true`, logs from both the import process and other dependent components are output, and GRPC debugging is enabled, which can be used for diagnosis. | +| TiDB Lightning | [`conflict.max-record-rows`](/tidb-lightning/tidb-lightning-configuration.md) | Newly added | The new version of strategy to handle conflicting data. It controls the maximum number of rows in the `conflict_records` table. The default value is 100. | +| TiDB Lightning | [`conflict.strategy`](/tidb-lightning/tidb-lightning-configuration.md) | Newly added | The new version of strategy to handle conflicting data. It includes the following options: "" (TiDB Lightning does not detect and process conflicting data), `error` (terminate the import and report an error if a primary or unique key conflict is detected in the imported data), `replace` (when encountering data with conflicting primary or unique keys, the new data is retained and the old data is overwritten.), `ignore` (when encountering data with conflicting primary or unique keys, the old data is retained and the new data is ignored.). The default value is "", that is, TiDB Lightning does not detect and process conflicting data. | +| TiDB Lightning | [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md) | Newly added | Controls the upper limit of the conflicting data. When `conflict.strategy="error"`, the default value is `0`. When `conflict.strategy="replace"` or `conflict.strategy="ignore"`, you can set it as a maxint. | +| TiDB Lightning | [`enable-diagnose-logs`](/tidb-lightning/tidb-lightning-configuration.md) | Newly added | Controls whether to enable the diagnostic logs. The default value is `false`, that is, only the logs related to the import are output, and the logs of other dependent components are not output. When you set it to `true`, logs from both the import process and other dependent components are output, and GRPC debugging is enabled, which can be used for diagnosis. | |TiDB Lightning | [`tikv-importer.parallel-import`](/tidb-lightning/tidb-lightning-configuration.md) | Newly added | TiDB Lightning parallel import parameter. It replaces the existing `tikv-importer.incremental-import` parameter, which could be mistaken as an incremental import parameter and misused. | |BR | `azblob.encryption-scope` | Newly added | BR provides encryption scope support for Azure Blob Storage. | |BR | `azblob.encryption-key` | Newly added | BR provides encryption key support for Azure Blob Storage. | From e02062b470df1c114b4cee4ea4de3a78bc13d5c4 Mon Sep 17 00:00:00 2001 From: Ran Date: Fri, 11 Aug 2023 10:06:03 +0800 Subject: [PATCH 70/72] Apply suggestions from code review --- releases/release-7.3.0.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index b8c7c0b6241fe..d0dca61dc0476 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -212,6 +212,13 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Add a new system table `mysql.tidb_timers` to store the metadata of internal timers. +## Deprecated features + +* TiDB + + - The [`Fast Analyze`](/system-variables.md#tidb_enable_fast_analyze) feature (experimental) for statistics will be deprecated in v7.5.0. + - The [incremental collection](/statistics.md#incremental-collection) feature for statistics will be deprecated in v7.5.0. + ## Improvements + TiDB @@ -220,6 +227,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - When [Global Kill](/tidb-configuration-file.md#enable-global-kill-new-in-v610) is enabled, you can terminate the current session by pressing Control+C [#8854](https://github.com/pingcap/tidb/issues/8854) @[pingyu](https://github.com/pingyu) - Support the `IS_FREE_LOCK()` and `IS_USED_LOCK()` locking functions [#44493](https://github.com/pingcap/tidb/issues/44493) @[dveeden](https://github.com/dveeden) - Optimize the performance of reading the dumped chunks from disk [#45125](https://github.com/pingcap/tidb/issues/45125) @[YangKeao](https://github.com/YangKeao) + - Optimize the overestimation issue of the inner table of Index Join by using Optimizer Fix Controls [#44855](https://github.com/pingcap/tidb/issues/44855) @[time-and-fate](https://github.com/time-and-fate) + TiKV @@ -273,6 +281,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- - Fix the issue that upgrading TiDB gets stuck when reading metadata takes longer than one DDL lease [#45176](https://github.com/pingcap/tidb/issues/45176) @[zimulala](https://github.com/zimulala) - Fix the issue that the query result of the `SELECT CAST(n AS CHAR)` statement is incorrect when `n` in the statement is a negative number [#44786](https://github.com/pingcap/tidb/issues/44786) @[xhebox](https://github.com/xhebox) - Fix the issue that queries might return incorrect results when `tidb_opt_agg_push_down` is enabled [#44795](https://github.com/pingcap/tidb/issues/44795) @[AilinKid](https://github.com/AilinKid) + - Fix the issue of wrong results that occurs when a query with `current_date()` uses plan cache [#45086](https://github.com/pingcap/tidb/issues/45086) @[qw4990](https://github.com/qw4990) + TiKV From 241a924522ea9ed2510ed7316d4d7bb03276d0f0 Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 14 Aug 2023 11:26:37 +0800 Subject: [PATCH 71/72] Update releases/release-7.3.0.md --- releases/release-7.3.0.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index d0dca61dc0476..4fdad43a34bf5 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -185,6 +185,8 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- | Configuration file | Configuration parameter | Change type | Description | | -------- | -------- | -------- | -------- | | TiDB | [`enable-32bits-connection-id`](/tidb-configuration-file.md#enable-32bits-connection-id-new-in-v730) | Newly added | Controls whether to enable the 32-bit connection ID feature. | +| TiDB | [`in-mem-slow-query-recent-num`](/tidb-configuration-file.md#in-mem-slow-query-recent-num-new-in-v730) | Newly added | Controls the number of recently used slow queries that are cached in memory. | +| TiDB | [`in-mem-slow-query-topn-num`](/tidb-configuration-file.md#in-mem-slow-query-topn-num-new-in-v730) | Newly added | Controls the number of slowest queries that are cached in memory. | | TiKV | [`coprocessor.region-bucket-size`](/tikv-configuration-file.md#region-bucket-size-new-in-v610) | Modified | Changes the default value from `96MiB` to `50MiB`. | | TiKV | [`raft-engine.format-version`](/tikv-configuration-file.md#format-version-new-in-v630) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), Ribbon filter is used. Therefore, TiKV changes the default value from `2` to `5`. | | TiKV | [`raftdb.max-total-wal-size`](/tikv-configuration-file.md#max-total-wal-size-1) | Modified | When using Partitioned Raft KV (`storage.engine="partitioned-raft-kv"`), TiKV skips writing WAL. Therefore, TiKV changes the default value from `"4GB"` to `1`, meaning that WAL is disabled. | From 2aed1432a0e21d0b665906e14d0c18216e3c8d57 Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 14 Aug 2023 13:56:21 +0800 Subject: [PATCH 72/72] Apply suggestions from code review --- releases/release-7.3.0.md | 2 +- releases/release-notes.md | 2 +- releases/release-timeline.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 4fdad43a34bf5..828736d7e5ad5 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -5,7 +5,7 @@ summary: Learn about the new features, compatibility changes, improvements, and # TiDB 7.3.0 Release Notes -Release date: August 10, 2023 +Release date: August 14, 2023 TiDB version: 7.3.0 diff --git a/releases/release-notes.md b/releases/release-notes.md index 284ac0899d247..1be217dadcb2b 100644 --- a/releases/release-notes.md +++ b/releases/release-notes.md @@ -7,7 +7,7 @@ aliases: ['/docs/dev/releases/release-notes/','/docs/dev/releases/rn/'] ## 7.3 -- [7.3.0-DMR](/releases/release-7.3.0.md): 2023-08-10 +- [7.3.0-DMR](/releases/release-7.3.0.md): 2023-08-14 ## 7.2 diff --git a/releases/release-timeline.md b/releases/release-timeline.md index ceb33a69e73dc..d00e28f916296 100644 --- a/releases/release-timeline.md +++ b/releases/release-timeline.md @@ -9,7 +9,7 @@ This document shows all the released TiDB versions in reverse chronological orde | Version | Release Date | | :--- | :--- | -| [7.3.0-DMR](/releases/release-7.3.0.md) | 2023-08-10 | +| [7.3.0-DMR](/releases/release-7.3.0.md) | 2023-08-14 | | [7.1.1](/releases/release-7.1.1.md) | 2023-07-24 | | [6.1.7](/releases/release-6.1.7.md) | 2023-07-12 | | [7.2.0-DMR](/releases/release-7.2.0.md) | 2023-06-29 |