From df006e4045ad63b9593a5f7eef8934db5efb7ec2 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Sat, 27 Aug 2022 16:27:05 +0800 Subject: [PATCH 001/104] add 6.3.0 release notes --- TOC.md | 2 + releases/release-6.3.0.md | 216 +++++++++++++++++++++++++++++++++++ releases/release-notes.md | 4 + releases/release-timeline.md | 1 + 4 files changed, 223 insertions(+) create mode 100644 releases/release-6.3.0.md diff --git a/TOC.md b/TOC.md index 4eb72098bf1e6..e88afd1d565a1 100644 --- a/TOC.md +++ b/TOC.md @@ -883,6 +883,8 @@ - [Release Timeline](/releases/release-timeline.md) - [TiDB Versioning](/releases/versioning.md) - [TiDB Installation Packages](/binary-package.md) + - v6.3 + - [6.3.0-DMR](/releases/release-6.3.0.md) - v6.2 - [6.2.0-DMR](/releases/release-6.2.0.md) - v6.1 diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md new file mode 100644 index 0000000000000..1a4b5cd773358 --- /dev/null +++ b/releases/release-6.3.0.md @@ -0,0 +1,216 @@ +--- +title: TiDB 6.3.0 Release Notes +--- + +# TiDB v6.3.0 Release Notes + +Release date: xx xx, 2022 + +TiDB version: 6.3.0-DMR + +In v6.3.0-DMR, the key new features and improvements are as follows: + +* Key feature 1 +* Key feature 2 +* Key feature 3 +* ... + +## New features + +### SQL + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +### Security + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +### Observability + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +### Performance + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +### Transactions + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +### Stability + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +### Ease of use + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +### MySQL compatibility + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +### Data migration + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +### TiDB data share subscription + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +### Deployment and maintenance + +* Feature title + + Short description of the feature + + [User document]() [#issue]() @[Contributor GitHub ID]() + +## Compatibility changes + +### System variables + +| Variable name | Change type (newly added, modified, or deleted) | Description | +| --- | --- | --- | +| | | | +| | | | +| | | | +| | | | + +### Configuration file parameters + +| Configuration file | Configuration | Change type | Description | +| --- | --- | --- | --- | +| | | | | +| | | | | +| | | | | +| | | | | + +### Others + +## Removed feature + +## Improvements + ++ TiDB + + - note [#issue]() @[Contributor GitHub ID]() + ++ TiKV + + - note [#issue]() @[Contributor GitHub ID]() + ++ PD + + - note [#issue]() @[Contributor GitHub ID]() + ++ TiFlash + + - note [#issue]() @[Contributor GitHub ID]() + ++ Tools + + - Backup & Restore (BR) + + - note [#issue]() @[Contributor GitHub ID]() + + - TiCDC + + - note [#issue]() @[Contributor GitHub ID]() + + - TiDB Data Migration (DM) + + - note [#issue]() @[Contributor GitHub ID]() + + - TiDB Lightning + + - note [#issue]() @[Contributor GitHub ID]() + + - TiUP + + - note [#issue]() @[Contributor GitHub ID]() + +## Bug fixes + ++ TiDB + + - note [#issue]() @[Contributor GitHub ID]() + ++ TiKV + + - note [#issue]() @[Contributor GitHub ID]() + ++ PD + + - note [#issue]() @[Contributor GitHub ID]() + ++ TiFlash + + - note [#issue]() @[Contributor GitHub ID]() + ++ Tools + + + Backup & Restore (BR) + + - note [#issue]() @[Contributor GitHub ID]() + + - TiCDC + + - note [#issue]() @[Contributor GitHub ID]() + + - TiDB Data Migration (DM) + + - note [#issue]() @[Contributor GitHub ID]() + + - TiDB Lightning + + - note [#issue]() @[Contributor GitHub ID]() + + - TiUP + + - note [#issue]() @[Contributor GitHub ID]() + +## Contributors + +We would like to thank the following contributors from the TiDB community: + +- [Contributor GitHub ID]() diff --git a/releases/release-notes.md b/releases/release-notes.md index b48fca27cb628..26967494cadea 100644 --- a/releases/release-notes.md +++ b/releases/release-notes.md @@ -5,6 +5,10 @@ aliases: ['/docs/dev/releases/release-notes/','/docs/dev/releases/rn/'] # TiDB Release Notes +## 6.3 + +- [6.3.0-DMR](/releases/release-6.3.0.md) + ## 6.2 - [6.2.0-DMR](/releases/release-6.2.0.md) diff --git a/releases/release-timeline.md b/releases/release-timeline.md index 52d3babc512cc..1e09fdd5e4b0d 100644 --- a/releases/release-timeline.md +++ b/releases/release-timeline.md @@ -9,6 +9,7 @@ This document shows all the released TiDB versions in reverse chronological orde | Version | Release Date | | :--- | :--- | +| [6.3.0-DMR](/releases/release-6.3.0.md) | 2022-xx-xx | | [6.2.0-DMR](/releases/release-6.2.0.md) | 2022-08-23 | | [5.4.2](/releases/release-5.4.2.md) | 2022-07-08 | | [5.3.2](/releases/release-5.3.2.md) | 2022-06-29 | From be2aacd66ed782109ba5268c7ba82e5709c381b6 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Tue, 13 Sep 2022 16:24:50 +0800 Subject: [PATCH 002/104] copy new feature description in zh to en for translation --- releases/release-6.3.0.md | 177 +++++++++++++++++++++++++++++--------- 1 file changed, 137 insertions(+), 40 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 1a4b5cd773358..e270c629bc888 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -10,100 +10,197 @@ TiDB version: 6.3.0-DMR In v6.3.0-DMR, the key new features and improvements are as follows: -* Key feature 1 -* Key feature 2 -* Key feature 3 -* ... +- TiKV/TiFlash 静态加密支持国密算法 SM4 +- TiDB 支持基于国密算法 SM3 插件的身份验证 +- SQL语句CREATE USER / ALTER USER支持ACCOUNT LOCK/UNLOCK 选项 +- JSON 数据类型和 JSON 函数 GA +- TiDB 支持 Null Aware Anti Join +- 提供“执行时间”的细粒度指标 +- 分区表新增简化 Range 分区的语法糖衣，避免在 DDL 中枚举所有分区 +- Range Columns 分区方式在 PARTITION BY RANGE COLUMNS (column_list) 处支持定义多列 ## New features ### SQL -* Feature title +* 新增简化 Range 分区定义的语法糖衣 [#issue]() @[mjonss](https://github.com/mjonss) - Short description of the feature + [新的定义 Range 分区的方式](用户文档链接)，不需要枚举所有分区，可大幅度缩短现有 Range 分区表定义语句冗长的书写方式，语义与原有 Range 分区等价。 - [User document]() [#issue]() @[Contributor GitHub ID]() +* Range Columns 分区方式支持定义多列 [#issue]() @[mjonss](https://github.com/mjonss) + + 支持 [PARTITION BY RANGE COLUMNS (column_list)](用户文档链接)，`column_list` 不再限定为单一列，基本功能与 MySQL 等同。 + +* 分区表 EXCHANGE PARTITION 功能 GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) + + [EXCHANGE PARTITION 功能](用户文档链接)通过性能和稳定性提升，由实验功能转为正式功能。 + +* 增加支持以下[窗口分析函数](/tiflash/tiflash-supported-pushdown-calculations.md)：[#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) + + * `LEAD` + * `LAG` + +* `CREATE USER` 支持 `ACCOUNT LOCK/UNLOCK` 选项 [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) + + 在执行 [`CREATE USER`](/sql-statements/sql-statement-create-user.md) 创建用户时，允许使用 `ACCOUNT LOCK/UNLOCK` 选项，限定被创建的用户是否被锁定。锁定后的用户不能正常登录数据库。 + +* `ALTER USER` 支持 `ACCOUNT LOCK/UNLOCK` 选项 [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) + + 对于已存在的用户，可以通过 [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) 使用 `ACCOUNT LOCK/UNLOCK` 选项，修改用户的锁定状态。 + +* JSON 数据类型和 JSON 函数 GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) + + JSON 是一种流行的数据格式，被大量的程序设计所采用。TiDB 在早期版本就引入了 [JSON 支持](/data-type-json.md)， 兼容 MySQL 的 JSON 数据类型 和一部分 JSON 函数。在 v6.3.0 版本中，我们正式将这些功能 GA ，用户可以安全地在生产环境中使用 JSON 相关的功能。 JSON 功能的 GA，为 TiDB 提供了更丰富的数据类型支持，同时也进一步提升的 TiDB 对 MySQL 的兼容能力。 + +* 提供轻量级元数据锁提升 DDL 变更过程 DML 的成功率 [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) + + TiDB 采用 F1 论文中的在线异步变更算法实现 DDL 变更：事务在执行时会获取开始时对应的元数据快照，如果事务执行过程中相关表上发生了元数据的更改，为了保证数据的一致性，TiDB 会返回 `Information schema is changed` 的异常，导致用户事务提交失败。为了解决这个问题，在 v6.3.0 版本中，TiDB 在 Online DDL 算法中引入了[元数据锁](/metadata-lock.md) 特性，通过协调表元数据变更过程中 DML 语句和 DDL 语句的优先级，让执行中的 DDL 语句等待持有旧版本元数据的 DML 语句提交，尽可能避免 DML 语句报错。 + +* 提升添加索引的性能并减少对 DML 事务的影响 [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) + + 开启[添加索引加速](/system-variables.md#tidb_ddl_enable_fast_reorg-从-v630-版本开始引入)后，TiDB 预期提升添加索引性能为原来的 3 倍。 + +* TiDB 支持更多正则表达式 [#](https://github.com/pingcap/tidb/issues/) @[windtalker](https://github.com/windtalker) + + TiDB 新增支持 [`REGEXP_SUBSTR`、`REGEXP_REPLACE`、`REGEXP_LIKE` 和 `REGEXP_INSTR` 4 个正则表达式函数](/functions-and-operators/string-functions.md#支持的函数)，支持基于模式的模糊查询和正则替换等操作。 ### Security -* Feature title +* 静态加密 TiKV 支持国密算法 SM4 [#13041](https://github.com/tikv/tikv/issues/13041) @[jiayang-zheng](https://github.com/jiayang-zheng) + + TiKV 静态加密中新增[加密算法 SM4](/encryption-at-rest.md)，用户在配置静态加密时，支持配置 data-encryption-method 参数为 "sm4-ctr"，以启用基于国密算法SM4的静态加密能力。 + +* 静态加密 TiFlash 支持国密算法 SM4 [#5714](https://github.com/pingcap/tiflash/issues/5714) @[lidezhu](https://github.com/lidezhu) - Short description of the feature + TiFlash 静态加密中新增[加密算法 SM4](/encryption-at-rest.md)，用户在配置静态加密时，支持配置 data-encryption-method 参数为 "sm4-ctr"，以启用基于国密算法SM4的静态加密能力。 - [User document]() [#issue]() @[Contributor GitHub ID]() +* TiDB 支持国密算法 SM3 的身份验证 [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) + + TiDB 身份验证新增基于国密算法 SM3 的插件 [tidb_sm3_password](/system-variables.md#default_authentication_plugin)，启用此插件后，用户密码将通过SM3进行加密存储和验证。 + +* JDBC 支持国密算法 SM3 的身份验证 [issue]() @[lastincisor](https://github.com/lastincisor) + + 用户密码的身份验证需要客户端的支持，现在 [JDBC 支持国密算法 SM3](/develop/dev-guide-choose-driver-or-orm.md#java-drivers) 的能力，用户可以通过 JDBC 连接到 TiDB 使用国密算法 SM3 的身份验证能力。 ### Observability -* Feature title +* 提供“执行时间”的细粒度指标 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) + + 性能问题的诊断通常从时间入手。 对执行时间的细粒度观测能力，是衡量数据库可观测性的重要标准。 TiDB 在新版中正式提供了细化数据指标，用于[对执行时间进行细化观测](/latency-breakdown.md)。 通过完整而又细分的指标数据，用户可以清晰的了解数据库的主要的时间消耗，进而快速发现关键问题，节省故障诊断的时间。 “执行时间”的细粒度观测能力，使得 TiDB 的观测性迈上了一个新的台阶。 + +* 增强的 slow log 和 trace 语句 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) + + 在新版本中 TiDB [增强了 slow log 的内容和 trace 命令的输出](/latency-breakdown.md)。用户可以观测到 SQL 语句执行过程中，从 tidb parse 到 kv rocksdb 落盘全链路的延迟数据，进一步增强 TiDB 的诊断能力。 - Short description of the feature +* Dashboard 中显示死锁的历史记录 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) - [User document]() [#issue]() @[Contributor GitHub ID]() + 新版本将死锁的历史记录加入到了 Dashboard 中。 当用户通过 Dashboard 的慢日志等手段发现某些 SQL 等待锁的时间较长的时候，Dashboard 上的死锁的历史记录有助于对问题的分析，提供了诊断的易用性。 ### Performance -* Feature title +* TiFlash 调整 FastScan 功能使用方式（实验特性） [#5252](https://github.com/pingcap/tiflash/issues/5252) @[hongyunyan](https://github.com/hongyunyan) - Short description of the feature + TiFlash 从 v6.2.0 版本开始引入的快速扫描功能 (FastScan)，性能上符合预期，但是使用方式上缺乏灵活性。因此，TiFlash 在 v6.3.0 版本[调整 FastScan 功能的使用方式](/develop/dev-guide-use-fastscan.md)，停止使用对表设定是否开启 FastScan 功能的方式，改为使用系统变量 [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-从-v630-版本开始引入) 控制是否开启 FastScan 功能。 - [User document]() [#issue]() @[Contributor GitHub ID]() + 从 v6.2.0 版本升级到 v6.3.0 版本时，在 v6.2.0 版本的所有 FastScan 设定将失效，需要重新使用变量方式进行 FastScan 设定，但不影响数据的正常读取。从更早版本升级到 v6.3.0 时，所有会话默认不开启 FastScan 功能，而是保持一致性的数据扫描功能。 + +* TiFlash 优化提升多并发场景下的数据扫描性能 [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) + + TiFlash 通过合并相同数据的读取操作，减少对于相同数据的重复读取，优化了多并发任务情况下的资源开销，[提升多并发下的数据扫描性能](/tiflash/tiflash-configuration.md#配置文件-tiflashtoml)。避免了以往在多并发任务下，如果涉及相同数据，同一份数据需要在每个任务中分别进行读取的情况，以及可能出现在同一时间内对相同数据进行多次读取的情况。 + + 该功能在 v6.2.0 版本以实验特性发布，并在 v6.3.0 版本作为正式功能发布。 + +* TiFlash 副本同步性能优化 [#5237](https://github.com/pingcap/tiflash/issues/5237) @[breezewish](https://github.com/breezewish) + + TiFlash 使用 Raft 协议与 TiKV 进行副本数据同步。在 v6.3.0 版本之前，同步大量副本数据时往往需要比较长的时间。v6.3.0 版本优化了 TiFlash 副本同步机制，大幅度提升了副本同步速度。因此，使用 BR 恢复数据、使用 Lightning 导入数据，或全新增加 TiFlash 副本时，副本将更迅速地完成同步，用户可以更及时地使用 TiFlash 进行查询。此外，在 TiFlash 扩缩容或修改 TiFlash 副本数时，TiFlash 副本也将更快地达到安全、均衡的状态。 + +* TiKV 日志循环使用 [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) + + TiKV Raft Engine 默认开启[日志循环](/tikv-configuration-file.md#enable-log-recycle-new-in-v630)功能。该特性显著降低 Raft 日志追加过程中的长尾延迟，提升了 TiKV 写入负载下的性能。 + +* TiDB 支持 Null Aware Anti Join [#issue]() @[Arenatlx](https://github.com/Arenatlx) + + TiDB 在新版本中引入了新的连接类型 [Null Aware Anti Join (NAAJ)](/explain-subqueries.md#null-aware-semi-joinin-any-子查询)。 NAAJ 在集合操作时能够感知集合是否为空，或是否有空值，优化了一部分操作比如`IN`、`= ANY` 的执行效率，提升 SQL 性能。 + +* 增加优化器 hint 控制哈希连接的驱动端 [#issue]() @[Reminiscent](https://github.com/Reminiscent) + + 在新版本中，优化器引入了两个新的 [hint `HASH_JOIN_BUILD()` 和 `HASH_JOIN_PROBE()`](/optimizer-hints.md) 用来指定哈希连接，并指定其驱动端和被驱动端。 在没有选到最优执行计划的情况下，提供了更丰富的干预手段。 + +* 会话级允许 CTE 内联展开 [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) + + 在 v6.2.0 中， 我们引入在优化器提示 `MERGE` ， 允许对 CTE 内联进行展开，使得 CTE 查询结果的消费者能够在 TiFlash 内并行执行。 在版本 v6.3.0 中，我们添加了会话级变量 [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-从-v630-版本开始引入)，允许在会话级修改这个行为，提升了易用性。 ### Transactions -* Feature title +* 悲观事务可以延迟唯一性检查 [#36579](https://github.com/pingcap/tidb/issues/36579) @[ekexium](https://github.com/ekexium) - Short description of the feature + 提供系统变量 `tidb_constraint_check_in_place_pessimistic` 来控制悲观事务中[唯一约束](/constraints.md#唯一约束)检查的时间点。当变量设为 `ON` 时，TiDB 会把加锁操作和唯一约束检测推迟到必要的时候进行，以此提升批量 DML 操作的性能。 - [User document]() [#issue]() @[Contributor GitHub ID]() +* 优化 Read-Committed 隔离级别中对 TSO 的获取 [#36812](https://github.com/pingcap/tidb/issues/36812) @[TonsnakeLin](https://github.com/TonsnakeLin) + + 在 Read-Committed 隔离级别中， 引入新的系统变量 [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-从-v630-版本开始引入) 控制语句对 TSO 的获取方式。在 Plan Cache 命中的情况下，通过降低对 TSO 的获取频率提升批量 DML 的执行效率，降低跑批类任务的执行时间。 ### Stability -* Feature title +* TiKV 后台 IO 任务限制增强 (TiKV I/O Rate Limiter) [#10867](https://github.com/tikv/tikv/issues/10867) @[tabokie](https://github.com/tabokie) + + 改进算法，提供对[磁盘读 I/O 的动态限流能力](/tikv-configuration-file.md#storageio-rate-limit)。 + +* 优化 `IN` 条件元素过多引发的大量内存消耗 [#30755](https://github.com/pingcap/tidb/issues/30755) @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) + + 当 SQL 中的 `IN` 条件包含的元素过多时，TiDB 在优化器构造扫描范围时可能会消耗大量的内存。在新版中，TiDB 引入了系统变量 [`tidb_opt_range_max_size`](/system-variables.md#tidb_opt_range_max_size-从-v630-版本开始引入) 实现对扫描范围的内存控制机制，对这类操作进行了优化，减少内存消耗，提升 SQL 执行效率和系统稳定性。 + +* 修改优化统计信息过期时的默认加载策略 [#issue]() @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) + + 在 v5.3.0 版本时，TiDB 引入系统变量 [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-从-v530-版本开始引入) 控制优化器在统计信息过期时的行为，默认为 `ON`，即保持旧版本行为不变：当 SQL 涉及的对象的统计信息过期时，优化器认为该表上除总行数以外的统计信息不再可靠，转而使用 pseudo 统计信息。 经过一系列测试和用户实际场景分析，TiDB 在新版本中将 `tidb_enable_pseudo_for_outdated_stats` 的默认值改为 `OFF`，即使统计信息过期，优化器也仍会使用该表上的统计信息，这有利于执行计划的稳定性。 - Short description of the feature +* TiKV Titan 关闭功能正式发布 [#issue]() @[tabokie](https://github.com/tabokie) - [User document]() [#issue]() @[Contributor GitHub ID]() + 正式支持对在线 TiKV 节点[关闭 Titan 引擎](/titan-configuration#disable-titan)。 ### Ease of use -* Feature title +* 优化 TiFlash 数据同步进度的准确性 [#4902](https://github.com/pingcap/tiflash/issues/4902) @[hehechen](https://github.com/hehechen) - Short description of the feature + TiDB 的 `information_schema.tiflash_replica` 表中的 `PROGRESS` 字段表示对应表 TiFlash 副本与 TiKV 数据的同步进度。在之前的版本中，`PROCESS` 字段只显示创建 TiFlash 副本过程中的数据同步进度。当 TiFlash 副本创建完毕，进行数据导入后，该值不会更新数据同步进度。 - [User document]() [#issue]() @[Contributor GitHub ID]() + v6.3.0 版本改进了 TiFlash 副本数据同步进度更新机制，在创建 TiFlash 副本后，进行数据导入等操作，TiFlash 副本需要和 TiKV 数据进行同步时，[`information_schema.tiflash_replica` 表](/information-schema/information-schema-tiflash-replica.md)中的 `PROGRESS` 值将会更新，显示实际的数据同步进度。通过 TiFlash 数据同步进度的准确性优化，用户可以了解数据同步的实际进度，具有更好的使用体验。 ### MySQL compatibility -* Feature title +* 完善基于 SQL 的数据放置规则功能的兼容性 [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) - Short description of the feature + TiDB 在 v6.0.0 版本提供基于 SQL 的数据放置规则功能，但是由于实现机制冲突，该功能和构建 TiFlash 副本功能不兼容。v6.3.0 版本进行改进优化，[完善了这两个功能的兼容性](/placement-rules-in-sql.md#使用限制)。 - [User document]() [#issue]() @[Contributor GitHub ID]() +### Backup and restore -### Data migration +* PITR 支持 GCS 和 Azure Blob Storage 作为备份存储 [#issue]() @[joccau](https://github.com/joccau) -* Feature title + PITR 支持[以 GCS 或 Azure Blob Storage 作为备份存储目标]()。部署在 GCP 或者 Azure 上的用户，升级 TiDB 集群到 v6.3.0 后，就可以使用 PITR 功能了。 - Short description of the feature +* BR 支持 AWS S3 Object Lock [#issue]() @[3pointer](https://github.com/3pointer) - [User document]() [#issue]() @[Contributor GitHub ID]() + 用户在 S3 开启 [Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html) 后，可以防止备份数据写入后被修改或者删除。 -### TiDB data share subscription +### Data migration -* Feature title +* TiDB Lightning 支持将 Apache Hive 导出的 Parquet 文件导入到 TiDB [#issue]() @[buchuitoudegou](https://github.com/buchuitoudegou) - Short description of the feature + TiDB Lightning [支持将 Apache Hive 导出的 Parquet 文件导入到 TiDB](migrate-from-parquet-files-to-tidb.md)，从而实现 Hive 到 TiDB 之间的数据流转。 - [User document]() [#issue]() @[Contributor GitHub ID]() +* DM 支持对同步到 TiDB 的表增加字段并对该字段赋值 [#3262](https://github.com/pingcap/tiflow/pull/3262), [#3340](https://github.com/pingcap/tiflow/issues/3340) @[yufan022](https://github.com/yufan022) + + [DM 支持对同步到 TiDB 的表增加字段并对该字段赋值](用户文档链接)。在上游分库分表合并到下游TiDB的场景，可以用于区分目标表的记录是来自于上游那个分库分表。 + +### TiDB data share subscription -### Deployment and maintenance +* TiCDC 支持对多个异地目标数据源进行数据复制的复杂部署形态 [#issue]() @[sdojjy](https://github.com/sdojjy) -* Feature title + 为了提供一套 TiDB 集群的数据能复制到多个不同的异地数据系统的能力，自 v6.3.0 开始，TiCDC 节点可以[部署到多个不同的异地的机房](用户文档链接)中，来分别负责对应机房的数据复制任务，以支撑各种复杂的异地数据复制使用场景和部署形态。 - Short description of the feature +* TiCDC 支持维护上下游数据一致性快照 (Sync point) [#issue]() @[asddongmen](https://github.com/asddongmen) - [User document]() [#issue]() @[Contributor GitHub ID]() + 在灾备复制场景下，[TiCDC 支持周期性的维护一个下游数据快照](用户文档链接)，使得该下游快照能保持与上游数据的快照一致性。借助此能力，TiCDC 能更好的匹配读写分离应用场景，帮助用户降本增效。 ## Compatibility changes From 14fe4bac19e4305699f5ac7b97f1066c083e9f5c Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Tue, 13 Sep 2022 17:32:14 +0800 Subject: [PATCH 003/104] reminding --- releases/release-6.3.0.md | 76 +++++++++++++++++++-------------------- 1 file changed, 38 insertions(+), 38 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index e270c629bc888..ccb8c9d9329dc 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -23,144 +23,144 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### SQL -* 新增简化 Range 分区定义的语法糖衣 [#issue]() @[mjonss](https://github.com/mjonss) +* 新增简化 Range 分区定义的语法糖衣 [#issue]() @[mjonss](https://github.com/mjonss) **tw: hfxsd** [新的定义 Range 分区的方式](用户文档链接)，不需要枚举所有分区，可大幅度缩短现有 Range 分区表定义语句冗长的书写方式，语义与原有 Range 分区等价。 -* Range Columns 分区方式支持定义多列 [#issue]() @[mjonss](https://github.com/mjonss) +* Range Columns 分区方式支持定义多列 [#issue]() @[mjonss](https://github.com/mjonss) **tw: hfxsd** 支持 [PARTITION BY RANGE COLUMNS (column_list)](用户文档链接)，`column_list` 不再限定为单一列，基本功能与 MySQL 等同。 -* 分区表 EXCHANGE PARTITION 功能 GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) +* 分区表 EXCHANGE PARTITION 功能 GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) **tw: hfxsd** [EXCHANGE PARTITION 功能](用户文档链接)通过性能和稳定性提升，由实验功能转为正式功能。 -* 增加支持以下[窗口分析函数](/tiflash/tiflash-supported-pushdown-calculations.md)：[#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) +* 增加支持以下[窗口分析函数](/tiflash/tiflash-supported-pushdown-calculations.md)：[#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) **tw：shichun-0415** * `LEAD` * `LAG` -* `CREATE USER` 支持 `ACCOUNT LOCK/UNLOCK` 选项 [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) +* `CREATE USER` 支持 `ACCOUNT LOCK/UNLOCK` 选项 [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** 在执行 [`CREATE USER`](/sql-statements/sql-statement-create-user.md) 创建用户时，允许使用 `ACCOUNT LOCK/UNLOCK` 选项，限定被创建的用户是否被锁定。锁定后的用户不能正常登录数据库。 -* `ALTER USER` 支持 `ACCOUNT LOCK/UNLOCK` 选项 [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) +* `ALTER USER` 支持 `ACCOUNT LOCK/UNLOCK` 选项 [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** 对于已存在的用户，可以通过 [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) 使用 `ACCOUNT LOCK/UNLOCK` 选项，修改用户的锁定状态。 -* JSON 数据类型和 JSON 函数 GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) +* JSON 数据类型和 JSON 函数 GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) **tw: qiancai** JSON 是一种流行的数据格式，被大量的程序设计所采用。TiDB 在早期版本就引入了 [JSON 支持](/data-type-json.md)， 兼容 MySQL 的 JSON 数据类型 和一部分 JSON 函数。在 v6.3.0 版本中，我们正式将这些功能 GA ，用户可以安全地在生产环境中使用 JSON 相关的功能。 JSON 功能的 GA，为 TiDB 提供了更丰富的数据类型支持，同时也进一步提升的 TiDB 对 MySQL 的兼容能力。 -* 提供轻量级元数据锁提升 DDL 变更过程 DML 的成功率 [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) +* 提供轻量级元数据锁提升 DDL 变更过程 DML 的成功率 [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** TiDB 采用 F1 论文中的在线异步变更算法实现 DDL 变更：事务在执行时会获取开始时对应的元数据快照，如果事务执行过程中相关表上发生了元数据的更改，为了保证数据的一致性，TiDB 会返回 `Information schema is changed` 的异常，导致用户事务提交失败。为了解决这个问题，在 v6.3.0 版本中，TiDB 在 Online DDL 算法中引入了[元数据锁](/metadata-lock.md) 特性，通过协调表元数据变更过程中 DML 语句和 DDL 语句的优先级，让执行中的 DDL 语句等待持有旧版本元数据的 DML 语句提交，尽可能避免 DML 语句报错。 -* 提升添加索引的性能并减少对 DML 事务的影响 [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) +* 提升添加索引的性能并减少对 DML 事务的影响 [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** 开启[添加索引加速](/system-variables.md#tidb_ddl_enable_fast_reorg-从-v630-版本开始引入)后，TiDB 预期提升添加索引性能为原来的 3 倍。 -* TiDB 支持更多正则表达式 [#](https://github.com/pingcap/tidb/issues/) @[windtalker](https://github.com/windtalker) +* TiDB 支持更多正则表达式 [#](https://github.com/pingcap/tidb/issues/) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** TiDB 新增支持 [`REGEXP_SUBSTR`、`REGEXP_REPLACE`、`REGEXP_LIKE` 和 `REGEXP_INSTR` 4 个正则表达式函数](/functions-and-operators/string-functions.md#支持的函数)，支持基于模式的模糊查询和正则替换等操作。 ### Security -* 静态加密 TiKV 支持国密算法 SM4 [#13041](https://github.com/tikv/tikv/issues/13041) @[jiayang-zheng](https://github.com/jiayang-zheng) +* 静态加密 TiKV 支持国密算法 SM4 [#13041](https://github.com/tikv/tikv/issues/13041) @[jiayang-zheng](https://github.com/jiayang-zheng) **tw: TomShawn** TiKV 静态加密中新增[加密算法 SM4](/encryption-at-rest.md)，用户在配置静态加密时，支持配置 data-encryption-method 参数为 "sm4-ctr"，以启用基于国密算法SM4的静态加密能力。 -* 静态加密 TiFlash 支持国密算法 SM4 [#5714](https://github.com/pingcap/tiflash/issues/5714) @[lidezhu](https://github.com/lidezhu) +* 静态加密 TiFlash 支持国密算法 SM4 [#5714](https://github.com/pingcap/tiflash/issues/5714) @[lidezhu](https://github.com/lidezhu) **tw: TomShawn** TiFlash 静态加密中新增[加密算法 SM4](/encryption-at-rest.md)，用户在配置静态加密时，支持配置 data-encryption-method 参数为 "sm4-ctr"，以启用基于国密算法SM4的静态加密能力。 -* TiDB 支持国密算法 SM3 的身份验证 [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) +* TiDB 支持国密算法 SM3 的身份验证 [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** TiDB 身份验证新增基于国密算法 SM3 的插件 [tidb_sm3_password](/system-variables.md#default_authentication_plugin)，启用此插件后，用户密码将通过SM3进行加密存储和验证。 -* JDBC 支持国密算法 SM3 的身份验证 [issue]() @[lastincisor](https://github.com/lastincisor) +* JDBC 支持国密算法 SM3 的身份验证 [issue]() @[lastincisor](https://github.com/lastincisor) **tw：ran-huang** 用户密码的身份验证需要客户端的支持，现在 [JDBC 支持国密算法 SM3](/develop/dev-guide-choose-driver-or-orm.md#java-drivers) 的能力，用户可以通过 JDBC 连接到 TiDB 使用国密算法 SM3 的身份验证能力。 ### Observability -* 提供“执行时间”的细粒度指标 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) +* 提供“执行时间”的细粒度指标 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: Oreoxmt** 性能问题的诊断通常从时间入手。 对执行时间的细粒度观测能力，是衡量数据库可观测性的重要标准。 TiDB 在新版中正式提供了细化数据指标，用于[对执行时间进行细化观测](/latency-breakdown.md)。 通过完整而又细分的指标数据，用户可以清晰的了解数据库的主要的时间消耗，进而快速发现关键问题，节省故障诊断的时间。 “执行时间”的细粒度观测能力，使得 TiDB 的观测性迈上了一个新的台阶。 -* 增强的 slow log 和 trace 语句 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) +* 增强的 slow log 和 trace 语句 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: Oreoxmt** 在新版本中 TiDB [增强了 slow log 的内容和 trace 命令的输出](/latency-breakdown.md)。用户可以观测到 SQL 语句执行过程中，从 tidb parse 到 kv rocksdb 落盘全链路的延迟数据，进一步增强 TiDB 的诊断能力。 -* Dashboard 中显示死锁的历史记录 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) +* Dashboard 中显示死锁的历史记录 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **shichun-0415** 新版本将死锁的历史记录加入到了 Dashboard 中。 当用户通过 Dashboard 的慢日志等手段发现某些 SQL 等待锁的时间较长的时候，Dashboard 上的死锁的历史记录有助于对问题的分析，提供了诊断的易用性。 ### Performance -* TiFlash 调整 FastScan 功能使用方式（实验特性） [#5252](https://github.com/pingcap/tiflash/issues/5252) @[hongyunyan](https://github.com/hongyunyan) +* TiFlash 调整 FastScan 功能使用方式（实验特性） [#5252](https://github.com/pingcap/tiflash/issues/5252) @[hongyunyan](https://github.com/hongyunyan) **tw: hfxsd** TiFlash 从 v6.2.0 版本开始引入的快速扫描功能 (FastScan)，性能上符合预期，但是使用方式上缺乏灵活性。因此，TiFlash 在 v6.3.0 版本[调整 FastScan 功能的使用方式](/develop/dev-guide-use-fastscan.md)，停止使用对表设定是否开启 FastScan 功能的方式，改为使用系统变量 [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-从-v630-版本开始引入) 控制是否开启 FastScan 功能。 从 v6.2.0 版本升级到 v6.3.0 版本时，在 v6.2.0 版本的所有 FastScan 设定将失效，需要重新使用变量方式进行 FastScan 设定，但不影响数据的正常读取。从更早版本升级到 v6.3.0 时，所有会话默认不开启 FastScan 功能，而是保持一致性的数据扫描功能。 -* TiFlash 优化提升多并发场景下的数据扫描性能 [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) +* TiFlash 优化提升多并发场景下的数据扫描性能 [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) **tw: hfxsd** TiFlash 通过合并相同数据的读取操作，减少对于相同数据的重复读取，优化了多并发任务情况下的资源开销，[提升多并发下的数据扫描性能](/tiflash/tiflash-configuration.md#配置文件-tiflashtoml)。避免了以往在多并发任务下，如果涉及相同数据，同一份数据需要在每个任务中分别进行读取的情况，以及可能出现在同一时间内对相同数据进行多次读取的情况。 该功能在 v6.2.0 版本以实验特性发布，并在 v6.3.0 版本作为正式功能发布。 -* TiFlash 副本同步性能优化 [#5237](https://github.com/pingcap/tiflash/issues/5237) @[breezewish](https://github.com/breezewish) +* TiFlash 副本同步性能优化 [#5237](https://github.com/pingcap/tiflash/issues/5237) @[breezewish](https://github.com/breezewish) **tw: hfxsd** TiFlash 使用 Raft 协议与 TiKV 进行副本数据同步。在 v6.3.0 版本之前，同步大量副本数据时往往需要比较长的时间。v6.3.0 版本优化了 TiFlash 副本同步机制，大幅度提升了副本同步速度。因此，使用 BR 恢复数据、使用 Lightning 导入数据，或全新增加 TiFlash 副本时，副本将更迅速地完成同步，用户可以更及时地使用 TiFlash 进行查询。此外，在 TiFlash 扩缩容或修改 TiFlash 副本数时，TiFlash 副本也将更快地达到安全、均衡的状态。 -* TiKV 日志循环使用 [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) +* TiKV 日志循环使用 [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) **tw：ran-huang** TiKV Raft Engine 默认开启[日志循环](/tikv-configuration-file.md#enable-log-recycle-new-in-v630)功能。该特性显著降低 Raft 日志追加过程中的长尾延迟，提升了 TiKV 写入负载下的性能。 -* TiDB 支持 Null Aware Anti Join [#issue]() @[Arenatlx](https://github.com/Arenatlx) +* TiDB 支持 Null Aware Anti Join [#issue]() @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** TiDB 在新版本中引入了新的连接类型 [Null Aware Anti Join (NAAJ)](/explain-subqueries.md#null-aware-semi-joinin-any-子查询)。 NAAJ 在集合操作时能够感知集合是否为空，或是否有空值，优化了一部分操作比如`IN`、`= ANY` 的执行效率，提升 SQL 性能。 -* 增加优化器 hint 控制哈希连接的驱动端 [#issue]() @[Reminiscent](https://github.com/Reminiscent) +* 增加优化器 hint 控制哈希连接的驱动端 [#issue]() @[Reminiscent](https://github.com/Reminiscent) **tw: TomShawn** 在新版本中，优化器引入了两个新的 [hint `HASH_JOIN_BUILD()` 和 `HASH_JOIN_PROBE()`](/optimizer-hints.md) 用来指定哈希连接，并指定其驱动端和被驱动端。 在没有选到最优执行计划的情况下，提供了更丰富的干预手段。 -* 会话级允许 CTE 内联展开 [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) +* 会话级允许 CTE 内联展开 [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) **shichun-0415** 在 v6.2.0 中， 我们引入在优化器提示 `MERGE` ， 允许对 CTE 内联进行展开，使得 CTE 查询结果的消费者能够在 TiFlash 内并行执行。 在版本 v6.3.0 中，我们添加了会话级变量 [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-从-v630-版本开始引入)，允许在会话级修改这个行为，提升了易用性。 ### Transactions -* 悲观事务可以延迟唯一性检查 [#36579](https://github.com/pingcap/tidb/issues/36579) @[ekexium](https://github.com/ekexium) +* 悲观事务可以延迟唯一性检查 [#36579](https://github.com/pingcap/tidb/issues/36579) @[ekexium](https://github.com/ekexium) **tw: qiancai** 提供系统变量 `tidb_constraint_check_in_place_pessimistic` 来控制悲观事务中[唯一约束](/constraints.md#唯一约束)检查的时间点。当变量设为 `ON` 时，TiDB 会把加锁操作和唯一约束检测推迟到必要的时候进行，以此提升批量 DML 操作的性能。 -* 优化 Read-Committed 隔离级别中对 TSO 的获取 [#36812](https://github.com/pingcap/tidb/issues/36812) @[TonsnakeLin](https://github.com/TonsnakeLin) +* 优化 Read-Committed 隔离级别中对 TSO 的获取 [#36812](https://github.com/pingcap/tidb/issues/36812) @[TonsnakeLin](https://github.com/TonsnakeLin) **tw: TomShawn** 在 Read-Committed 隔离级别中， 引入新的系统变量 [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-从-v630-版本开始引入) 控制语句对 TSO 的获取方式。在 Plan Cache 命中的情况下，通过降低对 TSO 的获取频率提升批量 DML 的执行效率，降低跑批类任务的执行时间。 ### Stability -* TiKV 后台 IO 任务限制增强 (TiKV I/O Rate Limiter) [#10867](https://github.com/tikv/tikv/issues/10867) @[tabokie](https://github.com/tabokie) +* TiKV 后台 IO 任务限制增强 (TiKV I/O Rate Limiter) [#10867](https://github.com/tikv/tikv/issues/10867) @[tabokie](https://github.com/tabokie) **tw：ran-huang** 改进算法，提供对[磁盘读 I/O 的动态限流能力](/tikv-configuration-file.md#storageio-rate-limit)。 -* 优化 `IN` 条件元素过多引发的大量内存消耗 [#30755](https://github.com/pingcap/tidb/issues/30755) @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) +* 优化 `IN` 条件元素过多引发的大量内存消耗 [#30755](https://github.com/pingcap/tidb/issues/30755) @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) **tw: TomShawn** 当 SQL 中的 `IN` 条件包含的元素过多时，TiDB 在优化器构造扫描范围时可能会消耗大量的内存。在新版中，TiDB 引入了系统变量 [`tidb_opt_range_max_size`](/system-variables.md#tidb_opt_range_max_size-从-v630-版本开始引入) 实现对扫描范围的内存控制机制，对这类操作进行了优化，减少内存消耗，提升 SQL 执行效率和系统稳定性。 -* 修改优化统计信息过期时的默认加载策略 [#issue]() @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) +* 修改优化统计信息过期时的默认加载策略 [#issue]() @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) **tw: TomShawn** 在 v5.3.0 版本时，TiDB 引入系统变量 [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-从-v530-版本开始引入) 控制优化器在统计信息过期时的行为，默认为 `ON`，即保持旧版本行为不变：当 SQL 涉及的对象的统计信息过期时，优化器认为该表上除总行数以外的统计信息不再可靠，转而使用 pseudo 统计信息。 经过一系列测试和用户实际场景分析，TiDB 在新版本中将 `tidb_enable_pseudo_for_outdated_stats` 的默认值改为 `OFF`，即使统计信息过期，优化器也仍会使用该表上的统计信息，这有利于执行计划的稳定性。 -* TiKV Titan 关闭功能正式发布 [#issue]() @[tabokie](https://github.com/tabokie) +* TiKV Titan 关闭功能正式发布 [#issue]() @[tabokie](https://github.com/tabokie) **tw：ran-huang** 正式支持对在线 TiKV 节点[关闭 Titan 引擎](/titan-configuration#disable-titan)。 ### Ease of use -* 优化 TiFlash 数据同步进度的准确性 [#4902](https://github.com/pingcap/tiflash/issues/4902) @[hehechen](https://github.com/hehechen) +* 优化 TiFlash 数据同步进度的准确性 [#4902](https://github.com/pingcap/tiflash/issues/4902) @[hehechen](https://github.com/hehechen) **tw: qiancai** TiDB 的 `information_schema.tiflash_replica` 表中的 `PROGRESS` 字段表示对应表 TiFlash 副本与 TiKV 数据的同步进度。在之前的版本中，`PROCESS` 字段只显示创建 TiFlash 副本过程中的数据同步进度。当 TiFlash 副本创建完毕，进行数据导入后，该值不会更新数据同步进度。 @@ -168,37 +168,37 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### MySQL compatibility -* 完善基于 SQL 的数据放置规则功能的兼容性 [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) +* 完善基于 SQL 的数据放置规则功能的兼容性 [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) **tw: hfxsd** TiDB 在 v6.0.0 版本提供基于 SQL 的数据放置规则功能，但是由于实现机制冲突，该功能和构建 TiFlash 副本功能不兼容。v6.3.0 版本进行改进优化，[完善了这两个功能的兼容性](/placement-rules-in-sql.md#使用限制)。 ### Backup and restore -* PITR 支持 GCS 和 Azure Blob Storage 作为备份存储 [#issue]() @[joccau](https://github.com/joccau) +* PITR 支持 GCS 和 Azure Blob Storage 作为备份存储 [#issue]() @[joccau](https://github.com/joccau) **tw: shichun-0415** PITR 支持[以 GCS 或 Azure Blob Storage 作为备份存储目标]()。部署在 GCP 或者 Azure 上的用户，升级 TiDB 集群到 v6.3.0 后，就可以使用 PITR 功能了。 -* BR 支持 AWS S3 Object Lock [#issue]() @[3pointer](https://github.com/3pointer) +* BR 支持 AWS S3 Object Lock [#issue]() @[3pointer](https://github.com/3pointer) **tw: shichun-0415** 用户在 S3 开启 [Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html) 后，可以防止备份数据写入后被修改或者删除。 ### Data migration -* TiDB Lightning 支持将 Apache Hive 导出的 Parquet 文件导入到 TiDB [#issue]() @[buchuitoudegou](https://github.com/buchuitoudegou) +* TiDB Lightning 支持将 Apache Hive 导出的 Parquet 文件导入到 TiDB [#issue]() @[buchuitoudegou](https://github.com/buchuitoudegou) **tw：ran-huang** TiDB Lightning [支持将 Apache Hive 导出的 Parquet 文件导入到 TiDB](migrate-from-parquet-files-to-tidb.md)，从而实现 Hive 到 TiDB 之间的数据流转。 -* DM 支持对同步到 TiDB 的表增加字段并对该字段赋值 [#3262](https://github.com/pingcap/tiflow/pull/3262), [#3340](https://github.com/pingcap/tiflow/issues/3340) @[yufan022](https://github.com/yufan022) +* DM 支持对同步到 TiDB 的表增加字段并对该字段赋值 [#3262](https://github.com/pingcap/tiflow/pull/3262), [#3340](https://github.com/pingcap/tiflow/issues/3340) @[yufan022](https://github.com/yufan022) **tw：ran-huang** [DM 支持对同步到 TiDB 的表增加字段并对该字段赋值](用户文档链接)。在上游分库分表合并到下游TiDB的场景，可以用于区分目标表的记录是来自于上游那个分库分表。 ### TiDB data share subscription -* TiCDC 支持对多个异地目标数据源进行数据复制的复杂部署形态 [#issue]() @[sdojjy](https://github.com/sdojjy) +* TiCDC 支持对多个异地目标数据源进行数据复制的复杂部署形态 [#issue]() @[sdojjy](https://github.com/sdojjy) **tw：ran-huang** 为了提供一套 TiDB 集群的数据能复制到多个不同的异地数据系统的能力，自 v6.3.0 开始，TiCDC 节点可以[部署到多个不同的异地的机房](用户文档链接)中，来分别负责对应机房的数据复制任务，以支撑各种复杂的异地数据复制使用场景和部署形态。 -* TiCDC 支持维护上下游数据一致性快照 (Sync point) [#issue]() @[asddongmen](https://github.com/asddongmen) +* TiCDC 支持维护上下游数据一致性快照 (Sync point) [#issue]() @[asddongmen](https://github.com/asddongmen) **tw: TomShawn** 在灾备复制场景下，[TiCDC 支持周期性的维护一个下游数据快照](用户文档链接)，使得该下游快照能保持与上游数据的快照一致性。借助此能力，TiCDC 能更好的匹配读写分离应用场景，帮助用户降本增效。 From 79b5060870f2d7a7d83e2167ddc0d7339d30706c Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 14 Sep 2022 14:00:01 +0800 Subject: [PATCH 004/104] translate notes tw:@Oreoxmt --- releases/release-6.3.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index ccb8c9d9329dc..ce622d9044e62 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -56,13 +56,13 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiDB 采用 F1 论文中的在线异步变更算法实现 DDL 变更：事务在执行时会获取开始时对应的元数据快照，如果事务执行过程中相关表上发生了元数据的更改，为了保证数据的一致性，TiDB 会返回 `Information schema is changed` 的异常，导致用户事务提交失败。为了解决这个问题，在 v6.3.0 版本中，TiDB 在 Online DDL 算法中引入了[元数据锁](/metadata-lock.md) 特性，通过协调表元数据变更过程中 DML 语句和 DDL 语句的优先级，让执行中的 DDL 语句等待持有旧版本元数据的 DML 语句提交，尽可能避免 DML 语句报错。 -* 提升添加索引的性能并减少对 DML 事务的影响 [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** +* Improve the performance of adding indexes and reduce the impact on DML transactions [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** - 开启[添加索引加速](/system-variables.md#tidb_ddl_enable_fast_reorg-从-v630-版本开始引入)后，TiDB 预期提升添加索引性能为原来的 3 倍。 + To improve the speed of backfilling when creating an index, TiDB v6.3.0 introduces the acceleration of `ADD INDEX` and `CREATE INDEX` DDL operations using the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) system variable. When this feature is enabled, the TiDB performance of adding indexes is about three times faster than the original. -* TiDB 支持更多正则表达式 [#](https://github.com/pingcap/tidb/issues/) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** +* TiDB supports more regular expressions [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** - TiDB 新增支持 [`REGEXP_SUBSTR`、`REGEXP_REPLACE`、`REGEXP_LIKE` 和 `REGEXP_INSTR` 4 个正则表达式函数](/functions-and-operators/string-functions.md#支持的函数)，支持基于模式的模糊查询和正则替换等操作。 + TiDB v6.3.0 supports four regular expression functions, `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` introduced in MySQL 8.0, which support schema-based fuzzy query and regular replacement. For more details about the compatibility with MySQL, see [Regular expression compatibility with MySQL](/functions-and-operators/string-functions.md#regular-expression-compatibility-with-mysql). ### Security From db6b9f85e77489cc332489453007384d927772e9 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 14 Sep 2022 20:25:41 +0800 Subject: [PATCH 005/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index ce622d9044e62..6dd6855ba01b1 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -23,17 +23,17 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### SQL -* 新增简化 Range 分区定义的语法糖衣 [#issue]() @[mjonss](https://github.com/mjonss) **tw: hfxsd** +* 新增简化 Range 分区定义的语法糖（syntactic sugar）[#issue]() @[mjonss](https://github.com/mjonss) - [新的定义 Range 分区的方式](用户文档链接)，不需要枚举所有分区，可大幅度缩短现有 Range 分区表定义语句冗长的书写方式，语义与原有 Range 分区等价。 + [提供了新的定义 Range 分区的方式](/partitioned-table.md#range-interval-partitioning)，不需要枚举所有分区，可大幅度缩短现有 Range 分区表定义语句冗长的书写方式。语义与原有 Range 分区等价。 -* Range Columns 分区方式支持定义多列 [#issue]() @[mjonss](https://github.com/mjonss) **tw: hfxsd** +* Range Columns 分区方式支持定义多列 [#issue]() @[mjonss](https://github.com/mjonss) - 支持 [PARTITION BY RANGE COLUMNS (column_list)](用户文档链接)，`column_list` 不再限定为单一列，基本功能与 MySQL 等同。 + 支持 [PARTITION BY RANGE COLUMNS (column_list)](/partitioned-table.md#range-columns-partitioning)，`column_list` 不再限定为单一列，基本功能与 MySQL 等同。 -* 分区表 EXCHANGE PARTITION 功能 GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) **tw: hfxsd** +* 分区表 EXCHANGE PARTITION 功能 GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) - [EXCHANGE PARTITION 功能](用户文档链接)通过性能和稳定性提升，由实验功能转为正式功能。 + [EXCHANGE PARTITION 功能]((/partitioned-table.md#partition-management)通过性能和稳定性提升，由实验功能转为正式功能。 * 增加支持以下[窗口分析函数](/tiflash/tiflash-supported-pushdown-calculations.md)：[#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) **tw：shichun-0415** From ac03aa95735620e4ffc62d3fe28ed9c3f5cbe275 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 14 Sep 2022 20:39:28 +0800 Subject: [PATCH 006/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 6dd6855ba01b1..3e606ec130a08 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -23,17 +23,17 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### SQL -* 新增简化 Range 分区定义的语法糖（syntactic sugar）[#issue]() @[mjonss](https://github.com/mjonss) +* Add a new syntactic sugar to simplify Range partition definitions [#issue]() @[mjonss](https://github.com/mjonss) - [提供了新的定义 Range 分区的方式](/partitioned-table.md#range-interval-partitioning)，不需要枚举所有分区，可大幅度缩短现有 Range 分区表定义语句冗长的书写方式。语义与原有 Range 分区等价。 + [Provides INTERVAL partitioning as a new way of defining Range partitions](/partitioned-table.md#range-interval-partitioning). You do not need to enumerate all partitions, which drastically reduces the lengthy way of writing Range partition statements. The semantic is equivalent to the original Range partition. -* Range Columns 分区方式支持定义多列 [#issue]() @[mjonss](https://github.com/mjonss) +* Range Columns partitioning supports defining multiple columns [#issue]() @[mjonss](https://github.com/mjonss) - 支持 [PARTITION BY RANGE COLUMNS (column_list)](/partitioned-table.md#range-columns-partitioning)，`column_list` 不再限定为单一列，基本功能与 MySQL 等同。 + Support [PARTITION BY RANGE COLUMNS (column_list)](/partitioned-table.md#range-columns-partitioning). `column_list` is no longer limited to a single column. The basic feature is the same as MySQL. -* 分区表 EXCHANGE PARTITION 功能 GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) +* EXCHANGE PARTITION becomes GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) - [EXCHANGE PARTITION 功能]((/partitioned-table.md#partition-management)通过性能和稳定性提升，由实验功能转为正式功能。 + [EXCHANGE PARTITION]((/partitioned-table.md#partition-management) becomes GA after performance and stability improvements. * 增加支持以下[窗口分析函数](/tiflash/tiflash-supported-pushdown-calculations.md)：[#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) **tw：shichun-0415** From 83fbb18ccf4a089f622a0765c043ae01b30e4123 Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 15 Sep 2022 09:59:07 +0800 Subject: [PATCH 007/104] Apply suggestions from code review --- releases/release-6.3.0.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 3e606ec130a08..5ee37f9132113 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -40,13 +40,13 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * `LEAD` * `LAG` -* `CREATE USER` 支持 `ACCOUNT LOCK/UNLOCK` 选项 [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** +* The `CREATE USER` statement supports the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** - 在执行 [`CREATE USER`](/sql-statements/sql-statement-create-user.md) 创建用户时，允许使用 `ACCOUNT LOCK/UNLOCK` 选项，限定被创建的用户是否被锁定。锁定后的用户不能正常登录数据库。 + When you create a user using the [`CREATE USER`](/sql-statements/sql-statement-create-user.md) statement, you can specify whether the created user is locked using the `ACCOUNT LOCK/UNLOCK` option. A locked user cannot log in to the database. -* `ALTER USER` 支持 `ACCOUNT LOCK/UNLOCK` 选项 [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** +* The `ALTER USER` statement supports the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** - 对于已存在的用户，可以通过 [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) 使用 `ACCOUNT LOCK/UNLOCK` 选项，修改用户的锁定状态。 + You can modify the lock state of an existing user using the `ACCOUNT LOCK/UNLOCK` option in the [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) statement. * JSON 数据类型和 JSON 函数 GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) **tw: qiancai** @@ -74,13 +74,13 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiFlash 静态加密中新增[加密算法 SM4](/encryption-at-rest.md)，用户在配置静态加密时，支持配置 data-encryption-method 参数为 "sm4-ctr"，以启用基于国密算法SM4的静态加密能力。 -* TiDB 支持国密算法 SM3 的身份验证 [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** +* TiDB supports authentication with the SM3 algorithm [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** - TiDB 身份验证新增基于国密算法 SM3 的插件 [tidb_sm3_password](/system-variables.md#default_authentication_plugin)，启用此插件后，用户密码将通过SM3进行加密存储和验证。 + TiDB adds an authentication plugin [`tidb_sm3_password`](/system-variables.md#default_authentication_plugin) based on the SM3 algorithm. When this plugin is enabled, the user password is encrypted and validated using the SM3 algorithm. -* JDBC 支持国密算法 SM3 的身份验证 [issue]() @[lastincisor](https://github.com/lastincisor) **tw：ran-huang** +* JDBC supports authentication with the SM3 algorithm [issue]() @[lastincisor](https://github.com/lastincisor) **tw：ran-huang** - 用户密码的身份验证需要客户端的支持，现在 [JDBC 支持国密算法 SM3](/develop/dev-guide-choose-driver-or-orm.md#java-drivers) 的能力，用户可以通过 JDBC 连接到 TiDB 使用国密算法 SM3 的身份验证能力。 + Authenticating the user password needs client-side support. Now because [JDBC supports the SM3 algorithm](/develop/dev-guide-choose-driver-or-orm.md#java-drivers), you can connect to TiDB using SM3 authentication via JDBC. ### Observability @@ -114,9 +114,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiFlash 使用 Raft 协议与 TiKV 进行副本数据同步。在 v6.3.0 版本之前，同步大量副本数据时往往需要比较长的时间。v6.3.0 版本优化了 TiFlash 副本同步机制，大幅度提升了副本同步速度。因此，使用 BR 恢复数据、使用 Lightning 导入数据，或全新增加 TiFlash 副本时，副本将更迅速地完成同步，用户可以更及时地使用 TiFlash 进行查询。此外，在 TiFlash 扩缩容或修改 TiFlash 副本数时，TiFlash 副本也将更快地达到安全、均衡的状态。 -* TiKV 日志循环使用 [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) **tw：ran-huang** +* TiKV supports log recycling [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) **tw：ran-huang** - TiKV Raft Engine 默认开启[日志循环](/tikv-configuration-file.md#enable-log-recycle-new-in-v630)功能。该特性显著降低 Raft 日志追加过程中的长尾延迟，提升了 TiKV 写入负载下的性能。 + TiKV recycles log files in Raft Engine by default. This reduces the long tail latency during Raft log appending and improves performance under write workloads. * TiDB 支持 Null Aware Anti Join [#issue]() @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** @@ -154,9 +154,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: 在 v5.3.0 版本时，TiDB 引入系统变量 [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-从-v530-版本开始引入) 控制优化器在统计信息过期时的行为，默认为 `ON`，即保持旧版本行为不变：当 SQL 涉及的对象的统计信息过期时，优化器认为该表上除总行数以外的统计信息不再可靠，转而使用 pseudo 统计信息。 经过一系列测试和用户实际场景分析，TiDB 在新版本中将 `tidb_enable_pseudo_for_outdated_stats` 的默认值改为 `OFF`，即使统计信息过期，优化器也仍会使用该表上的统计信息，这有利于执行计划的稳定性。 -* TiKV Titan 关闭功能正式发布 [#issue]() @[tabokie](https://github.com/tabokie) **tw：ran-huang** +* The feature of disabling Titan becomes GA [#issue]() @[tabokie](https://github.com/tabokie) **tw：ran-huang** - 正式支持对在线 TiKV 节点[关闭 Titan 引擎](/titan-configuration#disable-titan)。 + You can [disable Titan](/titan-configuration#disable-titan) for online TiKV nodes. ### Ease of use @@ -184,9 +184,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Data migration -* TiDB Lightning 支持将 Apache Hive 导出的 Parquet 文件导入到 TiDB [#issue]() @[buchuitoudegou](https://github.com/buchuitoudegou) **tw：ran-huang** +* TiDB Lightning supports importing Parquet files exported by Apache Hive into TiDB [#issue]() @[buchuitoudegou](https://github.com/buchuitoudegou) **tw：ran-huang** - TiDB Lightning [支持将 Apache Hive 导出的 Parquet 文件导入到 TiDB](migrate-from-parquet-files-to-tidb.md)，从而实现 Hive 到 TiDB 之间的数据流转。 + TiDB Lightning supports importing Parquet files exported by Apache Hive into TiDB, thereby achieving data migration from Hive to TiDB. * DM 支持对同步到 TiDB 的表增加字段并对该字段赋值 [#3262](https://github.com/pingcap/tiflow/pull/3262), [#3340](https://github.com/pingcap/tiflow/issues/3340) @[yufan022](https://github.com/yufan022) **tw：ran-huang** From 68d6abac5cda12abd92d1652e491bf0157a4e553 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 15 Sep 2022 10:27:56 +0800 Subject: [PATCH 008/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 5ee37f9132113..eb2170d32cc3d 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -48,9 +48,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: You can modify the lock state of an existing user using the `ACCOUNT LOCK/UNLOCK` option in the [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) statement. -* JSON 数据类型和 JSON 函数 GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) **tw: qiancai** +* JSON data type and JSON functions become GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) **tw: qiancai** - JSON 是一种流行的数据格式，被大量的程序设计所采用。TiDB 在早期版本就引入了 [JSON 支持](/data-type-json.md)， 兼容 MySQL 的 JSON 数据类型 和一部分 JSON 函数。在 v6.3.0 版本中，我们正式将这些功能 GA ，用户可以安全地在生产环境中使用 JSON 相关的功能。 JSON 功能的 GA，为 TiDB 提供了更丰富的数据类型支持，同时也进一步提升的 TiDB 对 MySQL 的兼容能力。 + JSON is a popular data format adopted by a large number of programs. TiDB has introduced the [JSON support](/data-type-json.md) as experimental since an earlier version, compatible with MySQL's JSON data type and some JSON functions. In v6.3.0, the JSON support becomes GA, providing TiDB with richer data types, and further improving TiDB compatibility with MySQL. * 提供轻量级元数据锁提升 DDL 变更过程 DML 的成功率 [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** From 5cb001b75d6971582ce664b8cba47f6fb9498c7a Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 15 Sep 2022 10:30:23 +0800 Subject: [PATCH 009/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index eb2170d32cc3d..3ba30accf34ca 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -132,9 +132,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Transactions -* 悲观事务可以延迟唯一性检查 [#36579](https://github.com/pingcap/tidb/issues/36579) @[ekexium](https://github.com/ekexium) **tw: qiancai** +* Support deferring unique constraints in pessimistic transactions [#36579](https://github.com/pingcap/tidb/issues/36579) @[ekexium](https://github.com/ekexium) **tw: qiancai** - 提供系统变量 `tidb_constraint_check_in_place_pessimistic` 来控制悲观事务中[唯一约束](/constraints.md#唯一约束)检查的时间点。当变量设为 `ON` 时，TiDB 会把加锁操作和唯一约束检测推迟到必要的时候进行，以此提升批量 DML 操作的性能。 + You can use the [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) system variable to control when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions. This variable is disabled by default. When the variable is set to `ON`, TiDB will defer locking operations and unique constraint checks in pessimistic transactions until necessary, thus improving the performance of bulk DML operations. * 优化 Read-Committed 隔离级别中对 TSO 的获取 [#36812](https://github.com/pingcap/tidb/issues/36812) @[TonsnakeLin](https://github.com/TonsnakeLin) **tw: TomShawn** From 93443a035b61ac771d2be45f223d116dc17c7b27 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 15 Sep 2022 10:32:10 +0800 Subject: [PATCH 010/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 3ba30accf34ca..804b895648648 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -160,11 +160,11 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Ease of use -* 优化 TiFlash 数据同步进度的准确性 [#4902](https://github.com/pingcap/tiflash/issues/4902) @[hehechen](https://github.com/hehechen) **tw: qiancai** +* Improve the accuracy of TiFlash data replication progress [#4902](https://github.com/pingcap/tiflash/issues/4902) @[hehechen](https://github.com/hehechen) **tw: qiancai** - TiDB 的 `information_schema.tiflash_replica` 表中的 `PROGRESS` 字段表示对应表 TiFlash 副本与 TiKV 数据的同步进度。在之前的版本中，`PROCESS` 字段只显示创建 TiFlash 副本过程中的数据同步进度。当 TiFlash 副本创建完毕，进行数据导入后，该值不会更新数据同步进度。 + In TiDB, the `PROGRESS` field of the `information_schema.tiflash_replica` table is used to indicate the progress of data replication from the corresponding tables in TiKV to the TiFlash replicas. In earlier TiDB versions, the `PROCESS` field only provides the progress of data replication during the creation of the TiFlash replicas. After a TiFlash replica is created, if new data is imported to a corresponding table in TiKV, this field will not be updated to show the replication progress from TiKV to TiFlash for the new data. - v6.3.0 版本改进了 TiFlash 副本数据同步进度更新机制，在创建 TiFlash 副本后，进行数据导入等操作，TiFlash 副本需要和 TiKV 数据进行同步时，[`information_schema.tiflash_replica` 表](/information-schema/information-schema-tiflash-replica.md)中的 `PROGRESS` 值将会更新，显示实际的数据同步进度。通过 TiFlash 数据同步进度的准确性优化，用户可以了解数据同步的实际进度，具有更好的使用体验。 + In v6.3.0, TiDB improves the update mechanism of data replication progress for TiFlash replicas. After a TiFlash replica is created, if new data is imported to a corresponding table in TiKV, the `PROGRESS` value in the [`information_schema.tiflash_replica`](/information-schema/information-schema-tiflash-replica.md) table will be updated to show the actual replication progress from TiKV to TiFlash for the new data. With this improvement, you can easily view the actual progress of TiFlash data replication. ### MySQL compatibility From 5d0c31377ec4c3d7fadefc9aab7aba80349e1b58 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 15 Sep 2022 10:33:17 +0800 Subject: [PATCH 011/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 804b895648648..38395d68b2cd1 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -164,7 +164,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: In TiDB, the `PROGRESS` field of the `information_schema.tiflash_replica` table is used to indicate the progress of data replication from the corresponding tables in TiKV to the TiFlash replicas. In earlier TiDB versions, the `PROCESS` field only provides the progress of data replication during the creation of the TiFlash replicas. After a TiFlash replica is created, if new data is imported to a corresponding table in TiKV, this field will not be updated to show the replication progress from TiKV to TiFlash for the new data. - In v6.3.0, TiDB improves the update mechanism of data replication progress for TiFlash replicas. After a TiFlash replica is created, if new data is imported to a corresponding table in TiKV, the `PROGRESS` value in the [`information_schema.tiflash_replica`](/information-schema/information-schema-tiflash-replica.md) table will be updated to show the actual replication progress from TiKV to TiFlash for the new data. With this improvement, you can easily view the actual progress of TiFlash data replication. + Since v6.3.0, TiDB improves the update mechanism of data replication progress for TiFlash replicas. After a TiFlash replica is created, if new data is imported to a corresponding table in TiKV, the `PROGRESS` value in the [`information_schema.tiflash_replica`](/information-schema/information-schema-tiflash-replica.md) table will be updated to show the actual replication progress from TiKV to TiFlash for the new data. With this improvement, you can easily view the actual progress of TiFlash data replication. ### MySQL compatibility From 0b9f9ef52489202d436d686f555049e779ca7c82 Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 15 Sep 2022 10:59:04 +0800 Subject: [PATCH 012/104] Apply suggestions from code review --- releases/release-6.3.0.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 38395d68b2cd1..92acf7db21e85 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -188,20 +188,24 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiDB Lightning supports importing Parquet files exported by Apache Hive into TiDB, thereby achieving data migration from Hive to TiDB. -* DM 支持对同步到 TiDB 的表增加字段并对该字段赋值 [#3262](https://github.com/pingcap/tiflow/pull/3262), [#3340](https://github.com/pingcap/tiflow/issues/3340) @[yufan022](https://github.com/yufan022) **tw：ran-huang** +* DM supports adding a field to a table migrated to TiDB and assigning values to the field [#3262](https://github.com/pingcap/tiflow/pull/3262), [#3340](https://github.com/pingcap/tiflow/issues/3340) @[yufan022](https://github.com/yufan022) **tw：ran-huang** - [DM 支持对同步到 TiDB 的表增加字段并对该字段赋值](用户文档链接)。在上游分库分表合并到下游TiDB的场景，可以用于区分目标表的记录是来自于上游那个分库分表。 + DM supports [adding a field to a table migrated to TiDB and assigning values to the field](link). When you merge and migrate MySQL shards to TiDB, you can use the field to distinguish which shard the record is migrated from. ### TiDB data share subscription -* TiCDC 支持对多个异地目标数据源进行数据复制的复杂部署形态 [#issue]() @[sdojjy](https://github.com/sdojjy) **tw：ran-huang** +* TiCDC supports a deployment topology that can replicate data from multiple geo-distributed data sources [#issue]() @[sdojjy](https://github.com/sdojjy) **tw：ran-huang** - 为了提供一套 TiDB 集群的数据能复制到多个不同的异地数据系统的能力，自 v6.3.0 开始，TiCDC 节点可以[部署到多个不同的异地的机房](用户文档链接)中，来分别负责对应机房的数据复制任务，以支撑各种复杂的异地数据复制使用场景和部署形态。 + To support replicating data from a single TiDB cluster to multiple geo-distributed data systems, starting from v6.3.0, [TiCDC can be deployed in multiple IDCs](link) to replicate data for each IDC. This feature helps deliver the capability of geo-distributed data replication and deployment topology. * TiCDC 支持维护上下游数据一致性快照 (Sync point) [#issue]() @[asddongmen](https://github.com/asddongmen) **tw: TomShawn** 在灾备复制场景下，[TiCDC 支持周期性的维护一个下游数据快照](用户文档链接)，使得该下游快照能保持与上游数据的快照一致性。借助此能力，TiCDC 能更好的匹配读写分离应用场景，帮助用户降本增效。 +* TiCDC supports graceful upgrade [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) @[3AceShowHand](https://github.com/3AceShowHand) **tw:ran-huang** + + When TiCDC is deployed using TiUP (>=v1.11.0) or TiDB Operator (>=v1.3.8), you can gracefully upgrade the TiCDC cluster. During the upgrade, data replication latency is kept as low as 30 seconds. This improves stability, empowering TiCDC to better support latency-sensitive applications. + ## Compatibility changes ### System variables From 3c3c098057c9fd29f07ce0b43d27a90746c01235 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Thu, 15 Sep 2022 20:24:32 +0800 Subject: [PATCH 013/104] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> Co-authored-by: xixirangrang --- releases/release-6.3.0.md | 88 ++++++++++++++++++--------------------- 1 file changed, 40 insertions(+), 48 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 92acf7db21e85..5dc98d024e678 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -10,14 +10,14 @@ TiDB version: 6.3.0-DMR In v6.3.0-DMR, the key new features and improvements are as follows: -- TiKV/TiFlash 静态加密支持国密算法 SM4 -- TiDB 支持基于国密算法 SM3 插件的身份验证 -- SQL语句CREATE USER / ALTER USER支持ACCOUNT LOCK/UNLOCK 选项 -- JSON 数据类型和 JSON 函数 GA -- TiDB 支持 Null Aware Anti Join -- 提供“执行时间”的细粒度指标 -- 分区表新增简化 Range 分区的语法糖衣，避免在 DDL 中枚举所有分区 -- Range Columns 分区方式在 PARTITION BY RANGE COLUMNS (column_list) 处支持定义多列 +- TiKV and TiFlash support SM4 encryption at rest. +- TiDB supports authentication with the SM3 algorithm. +- The `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option. +- JSON data type and functions become generally available (GA). +- TiDB supports Null Aware Anti Join. +- TiDB provides execution time metrics at a finer granularity. +- A new syntactic sugar is added to simplify Range partition definitions. +- Range Columns partitioning supports defining multiple columns. ## New features @@ -35,7 +35,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: [EXCHANGE PARTITION]((/partitioned-table.md#partition-management) becomes GA after performance and stability improvements. -* 增加支持以下[窗口分析函数](/tiflash/tiflash-supported-pushdown-calculations.md)：[#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) **tw：shichun-0415** +* TiDB supports two more [window functions](/tiflash/tiflash-supported-pushdown-calculations.md) [#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) **tw：shichun-0415** * `LEAD` * `LAG` @@ -66,13 +66,13 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Security -* 静态加密 TiKV 支持国密算法 SM4 [#13041](https://github.com/tikv/tikv/issues/13041) @[jiayang-zheng](https://github.com/jiayang-zheng) **tw: TomShawn** +* TiKV supports the SM4 algorithm for encryption at rest. [#13041](https://github.com/tikv/tikv/issues/13041) @[jiayang-zheng](https://github.com/jiayang-zheng) - TiKV 静态加密中新增[加密算法 SM4](/encryption-at-rest.md)，用户在配置静态加密时，支持配置 data-encryption-method 参数为 "sm4-ctr"，以启用基于国密算法SM4的静态加密能力。 + Add the [SM4 algorithm](/encryption-at-rest.md) for TiKV encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the "data-encryption-method" configuration to "sm4-ctr". -* 静态加密 TiFlash 支持国密算法 SM4 [#5714](https://github.com/pingcap/tiflash/issues/5714) @[lidezhu](https://github.com/lidezhu) **tw: TomShawn** +* TiFlash supports the SM4 algorithm for encryption at rest. [#5714](https://github.com/pingcap/tiflash/issues/5714) @[lidezhu](https://github.com/lidezhu) - TiFlash 静态加密中新增[加密算法 SM4](/encryption-at-rest.md)，用户在配置静态加密时，支持配置 data-encryption-method 参数为 "sm4-ctr"，以启用基于国密算法SM4的静态加密能力。 + Add the [SM4 algorithm](/encryption-at-rest.md) for TiFlash encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the "data-encryption-method" configuration to "sm4-ctr". * TiDB supports authentication with the SM3 algorithm [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** @@ -92,27 +92,27 @@ In v6.3.0-DMR, the key new features and improvements are as follows: 在新版本中 TiDB [增强了 slow log 的内容和 trace 命令的输出](/latency-breakdown.md)。用户可以观测到 SQL 语句执行过程中，从 tidb parse 到 kv rocksdb 落盘全链路的延迟数据，进一步增强 TiDB 的诊断能力。 -* Dashboard 中显示死锁的历史记录 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **shichun-0415** +* TiDB Dashboard provides deadlock history information [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: shichun-0415** - 新版本将死锁的历史记录加入到了 Dashboard 中。 当用户通过 Dashboard 的慢日志等手段发现某些 SQL 等待锁的时间较长的时候，Dashboard 上的死锁的历史记录有助于对问题的分析，提供了诊断的易用性。 + From v6.3.0, TiDB Dashboard will provide deadlock history. If you find excessively long SQL lock waiting by analyzing slow logs or other information on TiDB Dashboard, you can turn to deadlock history for problem locating, which delivers better diagnosis experience. ### Performance -* TiFlash 调整 FastScan 功能使用方式（实验特性） [#5252](https://github.com/pingcap/tiflash/issues/5252) @[hongyunyan](https://github.com/hongyunyan) **tw: hfxsd** +* TiFlash changes the way FastScan is used (experimental) [#5252](https://github.com/pingcap/tiflash/issues/5252) @[hongyunyan](https://github.com/hongyunyan) - TiFlash 从 v6.2.0 版本开始引入的快速扫描功能 (FastScan)，性能上符合预期，但是使用方式上缺乏灵活性。因此，TiFlash 在 v6.3.0 版本[调整 FastScan 功能的使用方式](/develop/dev-guide-use-fastscan.md)，停止使用对表设定是否开启 FastScan 功能的方式，改为使用系统变量 [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-从-v630-版本开始引入) 控制是否开启 FastScan 功能。 + FastScan, introduced in TiFlash starting with v6.2.0, performed as expected, but lacked flexibility in how it was used. Therefore, in v6.3.0 [the way FastScan is used has changed](/develop/dev-guide-use-fastscan.md). TiFlash does not support using the switch to control the FastScan feature. Instead, it uses the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) to control whether to turn on the FastScan function. + + When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. You need to set the variable [`tiflash_fastscan`]. When you upgrade from an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistency. - 从 v6.2.0 版本升级到 v6.3.0 版本时，在 v6.2.0 版本的所有 FastScan 设定将失效，需要重新使用变量方式进行 FastScan 设定，但不影响数据的正常读取。从更早版本升级到 v6.3.0 时，所有会话默认不开启 FastScan 功能，而是保持一致性的数据扫描功能。 +* TiFlash optimizes to improve data scanning performance in multiple concurrency scenarios [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) -* TiFlash 优化提升多并发场景下的数据扫描性能 [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) **tw: hfxsd** + TiFlash reduces duplicate reads of the same data by combining read operations of the same data. It optimizes the resource overhead and improves the performance of data scanning in the case of concurrent tasks (/tiflash/tiflash-configuration.md#configure-the-tiflashtoml-file). It avoids the situation where the same data needs to be read separately in each task in multiple concurrent tasks, and avoids the possibility of multiple reads of the same data at the same time. - TiFlash 通过合并相同数据的读取操作，减少对于相同数据的重复读取，优化了多并发任务情况下的资源开销，[提升多并发下的数据扫描性能](/tiflash/tiflash-configuration.md#配置文件-tiflashtoml)。避免了以往在多并发任务下，如果涉及相同数据，同一份数据需要在每个任务中分别进行读取的情况，以及可能出现在同一时间内对相同数据进行多次读取的情况。 + This feature is experimental in v6.2.0, and becomes GA in v6.3.0. - 该功能在 v6.2.0 版本以实验特性发布，并在 v6.3.0 版本作为正式功能发布。 +* Improve performance of TiFlash data replication [#5237](https://github.com/pingcap/tiflash/issues/5237) @[breezewish](https://github.com/breezewish) -* TiFlash 副本同步性能优化 [#5237](https://github.com/pingcap/tiflash/issues/5237) @[breezewish](https://github.com/breezewish) **tw: hfxsd** - - TiFlash 使用 Raft 协议与 TiKV 进行副本数据同步。在 v6.3.0 版本之前，同步大量副本数据时往往需要比较长的时间。v6.3.0 版本优化了 TiFlash 副本同步机制，大幅度提升了副本同步速度。因此，使用 BR 恢复数据、使用 Lightning 导入数据，或全新增加 TiFlash 副本时，副本将更迅速地完成同步，用户可以更及时地使用 TiFlash 进行查询。此外，在 TiFlash 扩缩容或修改 TiFlash 副本数时，TiFlash 副本也将更快地达到安全、均衡的状态。 + TiFlash uses the Raft protocol for data replication with TiKV. Prior to v6.3.0, it often took long time to replicate large amounts of replica data. TiDB v6.3.0 optimizes the TiFlash data replication mechanism and significantly improves the replication speed. When you use BR to recover data, use TiDB Lightning to import data, or add new TiFlash replicas, the replicas can be replicated more quickly. You can query with TiFlash in a more timely manner. In addition, TiFlash replicas will also reach a secure and balanced state faster when you scale up, scale down, or modify the number of TiFlash replicas. * TiKV supports log recycling [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) **tw：ran-huang** @@ -122,13 +122,13 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiDB 在新版本中引入了新的连接类型 [Null Aware Anti Join (NAAJ)](/explain-subqueries.md#null-aware-semi-joinin-any-子查询)。 NAAJ 在集合操作时能够感知集合是否为空，或是否有空值，优化了一部分操作比如`IN`、`= ANY` 的执行效率，提升 SQL 性能。 -* 增加优化器 hint 控制哈希连接的驱动端 [#issue]() @[Reminiscent](https://github.com/Reminiscent) **tw: TomShawn** +* Add optimizer hints to control the build end of Hash Join [#issue]() @[Reminiscent](https://github.com/Reminiscent) **tw: TomShawn** - 在新版本中，优化器引入了两个新的 [hint `HASH_JOIN_BUILD()` 和 `HASH_JOIN_PROBE()`](/optimizer-hints.md) 用来指定哈希连接，并指定其驱动端和被驱动端。 在没有选到最优执行计划的情况下，提供了更丰富的干预手段。 + In v6.3.0, the TiDB optimizer introduces 2 hints, `HASH_JOIN_BUILD()` and `HASH_JOIN_PROBE()`, to specify the Hash Join, its probe end, and its build end. When the optimizer fails to select the optimal execution plan, you can use these hints to intervene with the plan. -* 会话级允许 CTE 内联展开 [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) **shichun-0415** +* Support session-level CTE inline [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) **tw: shichun-0415** - 在 v6.2.0 中， 我们引入在优化器提示 `MERGE` ， 允许对 CTE 内联进行展开，使得 CTE 查询结果的消费者能够在 TiFlash 内并行执行。 在版本 v6.3.0 中，我们添加了会话级变量 [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-从-v630-版本开始引入)，允许在会话级修改这个行为，提升了易用性。 + TiDB v6.2.0 introduced the `MERGE` hint in optimizers to allow CTE inline, so that the consumers of a CTE query result can be executed in parallel in TiFlash. In v6.3.0, a session variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) in introduced to allow CTE inline in sessions. This improves ease of use greatly. ### Transactions @@ -136,23 +136,15 @@ In v6.3.0-DMR, the key new features and improvements are as follows: You can use the [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) system variable to control when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions. This variable is disabled by default. When the variable is set to `ON`, TiDB will defer locking operations and unique constraint checks in pessimistic transactions until necessary, thus improving the performance of bulk DML operations. -* 优化 Read-Committed 隔离级别中对 TSO 的获取 [#36812](https://github.com/pingcap/tidb/issues/36812) @[TonsnakeLin](https://github.com/TonsnakeLin) **tw: TomShawn** +* Optimize the way of fetching TSO in the Read-Committed isolation level [#36812](https://github.com/pingcap/tidb/issues/36812) @[TonsnakeLin](https://github.com/TonsnakeLin) **tw: TomShawn** - 在 Read-Committed 隔离级别中， 引入新的系统变量 [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-从-v630-版本开始引入) 控制语句对 TSO 的获取方式。在 Plan Cache 命中的情况下，通过降低对 TSO 的获取频率提升批量 DML 的执行效率，降低跑批类任务的执行时间。 + In the Read-Committed isolation level, the system variable [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) is introduced to control how TSO is fetched. In the case of Plan Cache hit, TiDB improves the execution efficiency of batch DML statements by reducing the frequency of fetching TSO, and reduces the execution time of running tasks in batches. ### Stability -* TiKV 后台 IO 任务限制增强 (TiKV I/O Rate Limiter) [#10867](https://github.com/tikv/tikv/issues/10867) @[tabokie](https://github.com/tabokie) **tw：ran-huang** - - 改进算法，提供对[磁盘读 I/O 的动态限流能力](/tikv-configuration-file.md#storageio-rate-limit)。 - -* 优化 `IN` 条件元素过多引发的大量内存消耗 [#30755](https://github.com/pingcap/tidb/issues/30755) @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) **tw: TomShawn** - - 当 SQL 中的 `IN` 条件包含的元素过多时，TiDB 在优化器构造扫描范围时可能会消耗大量的内存。在新版中，TiDB 引入了系统变量 [`tidb_opt_range_max_size`](/system-variables.md#tidb_opt_range_max_size-从-v630-版本开始引入) 实现对扫描范围的内存控制机制，对这类操作进行了优化，减少内存消耗，提升 SQL 执行效率和系统稳定性。 - -* 修改优化统计信息过期时的默认加载策略 [#issue]() @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) **tw: TomShawn** +* Modify the default policy of loading statistics when statistics become outdated [#issue]() @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) **tw: TomShawn** - 在 v5.3.0 版本时，TiDB 引入系统变量 [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-从-v530-版本开始引入) 控制优化器在统计信息过期时的行为，默认为 `ON`，即保持旧版本行为不变：当 SQL 涉及的对象的统计信息过期时，优化器认为该表上除总行数以外的统计信息不再可靠，转而使用 pseudo 统计信息。 经过一系列测试和用户实际场景分析，TiDB 在新版本中将 `tidb_enable_pseudo_for_outdated_stats` 的默认值改为 `OFF`，即使统计信息过期，优化器也仍会使用该表上的统计信息，这有利于执行计划的稳定性。 + In v5.3.0, TiDB introduces the system variable [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) to control how the optimizer behaves when the statistics become outdated. The default value is `ON`, which means keeping the behavior of the old version: When statistics on objects involved in SQL are outdated, the optimizer considers that statistics (other than the total number of rows on the table) are no longer reliable and uses pseudo statistics instead. After tests and analyses of actual user scenarios, the default value of `tidb_enable_pseudo_for_outdated_stats` is changed to `OFF` in v6.3.0. Even if the statistics become outdated, the optimizer will still use the statistics on the table, which is good for the execution program stability. * The feature of disabling Titan becomes GA [#issue]() @[tabokie](https://github.com/tabokie) **tw：ran-huang** @@ -168,19 +160,19 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### MySQL compatibility -* 完善基于 SQL 的数据放置规则功能的兼容性 [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) **tw: hfxsd** +* Improve compatibility of SQL-based data Placement Rules [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) - TiDB 在 v6.0.0 版本提供基于 SQL 的数据放置规则功能，但是由于实现机制冲突，该功能和构建 TiFlash 副本功能不兼容。v6.3.0 版本进行改进优化，[完善了这两个功能的兼容性](/placement-rules-in-sql.md#使用限制)。 + TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature is not compatible with TiFlash due to conflicts in implementation mechanisms. TiDB v6.3.0 optimizes this feature, and [improves compatibility of SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). ### Backup and restore -* PITR 支持 GCS 和 Azure Blob Storage 作为备份存储 [#issue]() @[joccau](https://github.com/joccau) **tw: shichun-0415** +* PITR supports GCS and Azure Blob Storage as backup storage [#issue]() @[joccau](https://github.com/joccau) **tw: shichun-0415** - PITR 支持[以 GCS 或 Azure Blob Storage 作为备份存储目标]()。部署在 GCP 或者 Azure 上的用户，升级 TiDB 集群到 v6.3.0 后，就可以使用 PITR 功能了。 + PITR supports [GCS and Azure Blob Storage as backup storage](). If your TiDB is deployed on GCP or Azure, you can use the PITR feature after upgrading your cluster to v6.3.0. -* BR 支持 AWS S3 Object Lock [#issue]() @[3pointer](https://github.com/3pointer) **tw: shichun-0415** +* BR supports AWS S3 Object Lock [#issue]() @[3pointer](https://github.com/3pointer) **tw: shichun-0415** - 用户在 S3 开启 [Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html) 后，可以防止备份数据写入后被修改或者删除。 + After enabling [Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html) , you can protect backup data from being tampered with or deleted. ### Data migration @@ -198,9 +190,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: To support replicating data from a single TiDB cluster to multiple geo-distributed data systems, starting from v6.3.0, [TiCDC can be deployed in multiple IDCs](link) to replicate data for each IDC. This feature helps deliver the capability of geo-distributed data replication and deployment topology. -* TiCDC 支持维护上下游数据一致性快照 (Sync point) [#issue]() @[asddongmen](https://github.com/asddongmen) **tw: TomShawn** +* TiCDC supports keeping the snapshots consistent between the upstream and the downstream (sync point) [#issue]() @[asddongmen](https://github.com/asddongmen) **tw: TomShawn** - 在灾备复制场景下，[TiCDC 支持周期性的维护一个下游数据快照](用户文档链接)，使得该下游快照能保持与上游数据的快照一致性。借助此能力，TiCDC 能更好的匹配读写分离应用场景，帮助用户降本增效。 + In the scenarios of data replication for disaster recovery, TiCDC supports periodically maintaining a downstream data snapshot so that the downstream snapshot is consistent with the upstream snapshot. With this feature, TiCDC can better match the scenarios where reads and writes are separate, and help you lower the cost. * TiCDC supports graceful upgrade [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) @[3AceShowHand](https://github.com/3AceShowHand) **tw:ran-huang** From a678dcf2d44a794047d9687423cec83832739c55 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 15 Sep 2022 21:09:16 +0800 Subject: [PATCH 014/104] Apply suggestions from code review --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 5dc98d024e678..ca30859268c4c 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -23,11 +23,11 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### SQL -* Add a new syntactic sugar to simplify Range partition definitions [#issue]() @[mjonss](https://github.com/mjonss) +* Add a new syntactic sugar to simplify Range partition definitions [#35683](https://github.com/pingcap/tidb/issues/35683) @[mjonss](https://github.com/mjonss) [Provides INTERVAL partitioning as a new way of defining Range partitions](/partitioned-table.md#range-interval-partitioning). You do not need to enumerate all partitions, which drastically reduces the lengthy way of writing Range partition statements. The semantic is equivalent to the original Range partition. -* Range Columns partitioning supports defining multiple columns [#issue]() @[mjonss](https://github.com/mjonss) +* Range Columns partitioning supports defining multiple columns [#36636](https://github.com/pingcap/tidb/issues/36636) @[mjonss](https://github.com/mjonss) Support [PARTITION BY RANGE COLUMNS (column_list)](/partitioned-table.md#range-columns-partitioning). `column_list` is no longer limited to a single column. The basic feature is the same as MySQL. From 1c2ab7ca056cd741f5cce53ca46b08a48fab8e3f Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 15 Sep 2022 21:12:31 +0800 Subject: [PATCH 015/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index ca30859268c4c..40c9c0a5f1124 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -33,7 +33,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * EXCHANGE PARTITION becomes GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) - [EXCHANGE PARTITION]((/partitioned-table.md#partition-management) becomes GA after performance and stability improvements. + [EXCHANGE PARTITION](/partitioned-table.md#partition-management) becomes GA after performance and stability improvements. * TiDB supports two more [window functions](/tiflash/tiflash-supported-pushdown-calculations.md) [#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) **tw：shichun-0415** From 194f0f99c809477fb570b11a426b5a91faac72e1 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 15 Sep 2022 21:14:34 +0800 Subject: [PATCH 016/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 40c9c0a5f1124..ecc2e5fd583b2 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -100,7 +100,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiFlash changes the way FastScan is used (experimental) [#5252](https://github.com/pingcap/tiflash/issues/5252) @[hongyunyan](https://github.com/hongyunyan) - FastScan, introduced in TiFlash starting with v6.2.0, performed as expected, but lacked flexibility in how it was used. Therefore, in v6.3.0 [the way FastScan is used has changed](/develop/dev-guide-use-fastscan.md). TiFlash does not support using the switch to control the FastScan feature. Instead, it uses the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) to control whether to turn on the FastScan function. + FastScan, introduced in TiFlash starting with v6.2.0, performed as expected, but lacked flexibility in how it was used. Therefore, in v6.3.0 [the way of using FastScan has changed](/develop/dev-guide-use-fastscan.md). TiFlash does not support using the switch to control the FastScan feature. Instead, it uses the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) to control whether to turn on the FastScan function. When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. You need to set the variable [`tiflash_fastscan`]. When you upgrade from an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistency. From a96312752b48b33360b2271afcf6143cb956c359 Mon Sep 17 00:00:00 2001 From: aolin Date: Thu, 15 Sep 2022 22:07:03 +0800 Subject: [PATCH 017/104] update --- releases/release-6.3.0.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index ecc2e5fd583b2..e30a63a0f6039 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -52,17 +52,17 @@ In v6.3.0-DMR, the key new features and improvements are as follows: JSON is a popular data format adopted by a large number of programs. TiDB has introduced the [JSON support](/data-type-json.md) as experimental since an earlier version, compatible with MySQL's JSON data type and some JSON functions. In v6.3.0, the JSON support becomes GA, providing TiDB with richer data types, and further improving TiDB compatibility with MySQL. -* 提供轻量级元数据锁提升 DDL 变更过程 DML 的成功率 [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** +* Provide lightweight metadata lock to improve the DML success rate during DDL change [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** - TiDB 采用 F1 论文中的在线异步变更算法实现 DDL 变更：事务在执行时会获取开始时对应的元数据快照，如果事务执行过程中相关表上发生了元数据的更改，为了保证数据的一致性，TiDB 会返回 `Information schema is changed` 的异常，导致用户事务提交失败。为了解决这个问题，在 v6.3.0 版本中，TiDB 在 Online DDL 算法中引入了[元数据锁](/metadata-lock.md) 特性，通过协调表元数据变更过程中 DML 语句和 DDL 语句的优先级，让执行中的 DDL 语句等待持有旧版本元数据的 DML 语句提交，尽可能避免 DML 语句报错。 + TiDB uses the online asynchronous schema change algorithm to support changing metadata objects. When a transaction is executed, the transaction obtains the corresponding metadata snapshot at transaction start. If the metadata is changed during transaction, to ensure the data consistency, TiDB returns an `Information schema is changed` error and the transaction fails to commit. To solve the problem, TiDB v6.3.0 introduces [metadata lock](/metadata-lock.md) into the online DDL algorithm. To avoid most DML errors, TiDB coordinates the priority of DMLs and DDLs during table metadata change, and makes executing DDLs wait for the DMLs with old metadata to commit. -* Improve the performance of adding indexes and reduce the impact on DML transactions [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** +* Improve the performance of adding indexes and reduce its impact on DML transactions [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** - To improve the speed of backfilling when creating an index, TiDB v6.3.0 introduces the acceleration of `ADD INDEX` and `CREATE INDEX` DDL operations using the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) system variable. When this feature is enabled, the TiDB performance of adding indexes is about three times faster than the original. + To improve the speed of backfilling when creating an index, TiDB v6.3.0 accelerates the `ADD INDEX` and `CREATE INDEX` DDL operations when the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) system variable is enabled. The performance of adding indexes is about three times faster than the original when the feature is enabled. -* TiDB supports more regular expressions [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** +* Add support for `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` regular expression functions [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** - TiDB v6.3.0 supports four regular expression functions, `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` introduced in MySQL 8.0, which support schema-based fuzzy query and regular replacement. For more details about the compatibility with MySQL, see [Regular expression compatibility with MySQL](/functions-and-operators/string-functions.md#regular-expression-compatibility-with-mysql). + For more details about the compatibility with MySQL, see [Regular expression compatibility with MySQL](/functions-and-operators/string-functions.md#regular-expression-compatibility-with-mysql). ### Security @@ -84,13 +84,13 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Observability -* 提供“执行时间”的细粒度指标 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: Oreoxmt** +* TiDB provides fine-grained metrics of SQL query execution time [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: Oreoxmt** - 性能问题的诊断通常从时间入手。 对执行时间的细粒度观测能力，是衡量数据库可观测性的重要标准。 TiDB 在新版中正式提供了细化数据指标，用于[对执行时间进行细化观测](/latency-breakdown.md)。 通过完整而又细分的指标数据，用户可以清晰的了解数据库的主要的时间消耗，进而快速发现关键问题，节省故障诊断的时间。 “执行时间”的细粒度观测能力，使得 TiDB 的观测性迈上了一个新的台阶。 + TiDB v6.3.0 provides fine-grained data metrics for [detailed observation of execution time](/latency-breakdown.md). Through the complete and segmented metrics, you can clearly understand the main time consumption of SQL queries, and then quickly find key problems and save time in troubleshooting. -* 增强的 slow log 和 trace 语句 [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: Oreoxmt** +* Enhanced output for slow logs and `TRACE` statements [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: Oreoxmt** - 在新版本中 TiDB [增强了 slow log 的内容和 trace 命令的输出](/latency-breakdown.md)。用户可以观测到 SQL 语句执行过程中，从 tidb parse 到 kv rocksdb 落盘全链路的延迟数据，进一步增强 TiDB 的诊断能力。 + TiDB v6.3.0 enhances the output of slow logs and `TRACE`. You can observe the [full-link duration](/latency-breakdown.md) of SQL queries from TiDB parsing to KV RocksDB writing to disk, which further enhances the diagnostic capabilities. * TiDB Dashboard provides deadlock history information [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: shichun-0415** From 27ada4221265c6bc43e151fdb5f67598917a84ab Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 16 Sep 2022 10:42:21 +0800 Subject: [PATCH 018/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index e30a63a0f6039..959208d9d1e51 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -23,7 +23,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### SQL -* Add a new syntactic sugar to simplify Range partition definitions [#35683](https://github.com/pingcap/tidb/issues/35683) @[mjonss](https://github.com/mjonss) +* Add a new syntactic sugar Range INTERVAL partitioning to simplify Range partition definitions [#35683](https://github.com/pingcap/tidb/issues/35683) @[mjonss](https://github.com/mjonss) [Provides INTERVAL partitioning as a new way of defining Range partitions](/partitioned-table.md#range-interval-partitioning). You do not need to enumerate all partitions, which drastically reduces the lengthy way of writing Range partition statements. The semantic is equivalent to the original Range partition. From 1fbafcae00fb8983a71456aee0dbcb6a37779666 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 16 Sep 2022 16:28:58 +0800 Subject: [PATCH 019/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 959208d9d1e51..21d7c72667934 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -23,7 +23,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### SQL -* Add a new syntactic sugar Range INTERVAL partitioning to simplify Range partition definitions [#35683](https://github.com/pingcap/tidb/issues/35683) @[mjonss](https://github.com/mjonss) +* Add a new syntactic sugar (Range INTERVAL partitioning) to simplify Range partition definitions [#35683](https://github.com/pingcap/tidb/issues/35683) @[mjonss](https://github.com/mjonss) [Provides INTERVAL partitioning as a new way of defining Range partitions](/partitioned-table.md#range-interval-partitioning). You do not need to enumerate all partitions, which drastically reduces the lengthy way of writing Range partition statements. The semantic is equivalent to the original Range partition. From b32da95d2a3d9b1fcc8478a76ea1472cf203eb6f Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 19 Sep 2022 12:01:00 +0800 Subject: [PATCH 020/104] Apply suggestions from code review --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 21d7c72667934..36127acc52b29 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -118,9 +118,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiKV recycles log files in Raft Engine by default. This reduces the long tail latency during Raft log appending and improves performance under write workloads. -* TiDB 支持 Null Aware Anti Join [#issue]() @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** +* TiDB supports Null-Aware Anti Join] [#issue]() @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** - TiDB 在新版本中引入了新的连接类型 [Null Aware Anti Join (NAAJ)](/explain-subqueries.md#null-aware-semi-joinin-any-子查询)。 NAAJ 在集合操作时能够感知集合是否为空，或是否有空值，优化了一部分操作比如`IN`、`= ANY` 的执行效率，提升 SQL 性能。 + TiDB v6.3.0 introduces a new join type [Null-Aware Anti Join (NAAJ)](/explain-subqueries.md#null-aware-semi-joinin-any). NAAJ can be aware of whether the collection is empty or `NULL` when processing collection operations. This optimizes the execution efficiency of operations such as `IN` and `= ANY` and improves SQL performance. * Add optimizer hints to control the build end of Hash Join [#issue]() @[Reminiscent](https://github.com/Reminiscent) **tw: TomShawn** From 4bf013db123a6d7890cd6e3e35c4adf69a342ad7 Mon Sep 17 00:00:00 2001 From: joey-yez <104608045+joey-yez@users.noreply.github.com> Date: Mon, 19 Sep 2022 16:15:43 +0800 Subject: [PATCH 021/104] Update releases/release-6.3.0.md Co-authored-by: Grace Cai --- releases/release-6.3.0.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 36127acc52b29..ccbb45ab33e2a 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -152,12 +152,6 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Ease of use -* Improve the accuracy of TiFlash data replication progress [#4902](https://github.com/pingcap/tiflash/issues/4902) @[hehechen](https://github.com/hehechen) **tw: qiancai** - - In TiDB, the `PROGRESS` field of the `information_schema.tiflash_replica` table is used to indicate the progress of data replication from the corresponding tables in TiKV to the TiFlash replicas. In earlier TiDB versions, the `PROCESS` field only provides the progress of data replication during the creation of the TiFlash replicas. After a TiFlash replica is created, if new data is imported to a corresponding table in TiKV, this field will not be updated to show the replication progress from TiKV to TiFlash for the new data. - - Since v6.3.0, TiDB improves the update mechanism of data replication progress for TiFlash replicas. After a TiFlash replica is created, if new data is imported to a corresponding table in TiKV, the `PROGRESS` value in the [`information_schema.tiflash_replica`](/information-schema/information-schema-tiflash-replica.md) table will be updated to show the actual replication progress from TiKV to TiFlash for the new data. With this improvement, you can easily view the actual progress of TiFlash data replication. - ### MySQL compatibility * Improve compatibility of SQL-based data Placement Rules [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) From a9fece162dff2e7c29a7c37bb4316bb1ad4d5c89 Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 19 Sep 2022 17:38:59 +0800 Subject: [PATCH 022/104] Apply suggestions from code review --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index ccbb45ab33e2a..434089b30c6cc 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -52,7 +52,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: JSON is a popular data format adopted by a large number of programs. TiDB has introduced the [JSON support](/data-type-json.md) as experimental since an earlier version, compatible with MySQL's JSON data type and some JSON functions. In v6.3.0, the JSON support becomes GA, providing TiDB with richer data types, and further improving TiDB compatibility with MySQL. -* Provide lightweight metadata lock to improve the DML success rate during DDL change [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** +* Provide lightweight metadata lock to improve the DML success rate during DDL change (experimental) [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** TiDB uses the online asynchronous schema change algorithm to support changing metadata objects. When a transaction is executed, the transaction obtains the corresponding metadata snapshot at transaction start. If the metadata is changed during transaction, to ensure the data consistency, TiDB returns an `Information schema is changed` error and the transaction fails to commit. To solve the problem, TiDB v6.3.0 introduces [metadata lock](/metadata-lock.md) into the online DDL algorithm. To avoid most DML errors, TiDB coordinates the priority of DMLs and DDLs during table metadata change, and makes executing DDLs wait for the DMLs with old metadata to commit. From 9372f928b5a94a1b684b7f941dee50a7cc20b8c0 Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 20 Sep 2022 16:35:38 +0800 Subject: [PATCH 023/104] Apply suggestions from code review --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 434089b30c6cc..a577052f1da35 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -60,7 +60,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: To improve the speed of backfilling when creating an index, TiDB v6.3.0 accelerates the `ADD INDEX` and `CREATE INDEX` DDL operations when the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) system variable is enabled. The performance of adding indexes is about three times faster than the original when the feature is enabled. -* Add support for `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` regular expression functions [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** +* Improve MySQL 8.0 compatibility by adding support for `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` regular expression functions [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** For more details about the compatibility with MySQL, see [Regular expression compatibility with MySQL](/functions-and-operators/string-functions.md#regular-expression-compatibility-with-mysql). From 06f128e3d6dcfbf07e325584b38bdda03d6a8a8c Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Wed, 21 Sep 2022 08:51:05 +0800 Subject: [PATCH 024/104] add improvements and bug fixes --- releases/release-6.3.0.md | 172 ++++++++++++++++++++++++++++++++------ 1 file changed, 147 insertions(+), 25 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index a577052f1da35..cf3e4b8bc3996 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -220,37 +220,81 @@ In v6.3.0-DMR, the key new features and improvements are as follows: + TiDB - - note [#issue]() @[Contributor GitHub ID]() + - One `PLAN REPLAYER` command can export information about execution plans of multiple SQL statements, which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) + + - sql-infra + + - Extend partitioning syntax with INTERVAL for easier partitioning definition [#35827](https://github.com/pingcap/tidb/issues/35827) @[ymkzpx](https://github.com/ymkzpx) + - Grant privilege of a table to an user checks the target table exist first, in the past, the table name comparison works in a case sensitive manner, now it's changed to case insensitive. [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) + - Support AWS NLB proxy protocol [#36312](https://github.com/pingcap/tidb/issues/36312) @[hawkingrei](https://github.com/hawkingrei) + - Previously, TiDB users can set `init_connect` without any checking. From now on, the value of `init_connect` should be checked by the sql parser. [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) + + - execution + + - report error if json path has the wrong syntax. [#22525](https://github.com/pingcap/tidb/issues/22525) @[xiongjiwei](https://github.com/xiongjiwei) + - report error if json path has the wrong syntax. [#34959](https://github.com/pingcap/tidb/issues/34959) @[xiongjiwei](https://github.com/xiongjiwei) + + - planner + + - planner: just pop cte's handleHelper map out since it shouldn't be considered [#35758](https://github.com/pingcap/tidb/issues/35758) @[AilinKid](https://github.com/AilinKid) + TiKV - - note [#issue]() @[Contributor GitHub ID]() + - Add a new option to make unreachable_backoff of raftstore configurable. [#13054](https://github.com/tikv/tikv/issues/13054) + - Implement TSO batch list to improve tolerance to TSO service fault. [#12794](https://github.com/tikv/tikv/issues/12794) @[pingyu](https://github.com/pingyu) + - Make max_subcompactions dynamically changeable [#13145](https://github.com/tikv/tikv/issues/13145) @[ethercflow](https://github.com/ethercflow) + - Optimize the performance of merging empty regions [#12421](https://github.com/tikv/tikv/issues/12421) @[tabokie](https://github.com/tabokie) + PD - - note [#issue]() @[Contributor GitHub ID]() - + - Updates metrics query. Renames `metrics` to `monitoring` on TiDB Dashboard. [#5366](https://github.com/tikv/pd/issues/5366) @[YiniXu9506](https://github.com/YiniXu9506) + + TiFlash - - note [#issue]() @[Contributor GitHub ID]() + - compute + + - Support to pushdown elt to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) + - Support to pushdown leftShift to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) + - Support to pushdown castTimeAsDuration to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) + - Support to pushdown castTimeAsDuration to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) + - Support Planner Interpreter. [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) + - Support to pushdown hexInt and hexStr to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107), [#5462](https://github.com/pingcap/tiflash/issues/5462) + - Support to pushdown elt to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) + - Support to pushdown shiftLeft to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) + - Suppress the "tcp set inq" loggings [#4940](https://github.com/pingcap/tiflash/issues/4940) + - Support to pushdown CastTimeAsDuration to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) + + - storage + + - Calculate the io throughput in background in ReadLimiter [#5401](https://github.com/pingcap/tiflash/issues/5401), [#5091](https://github.com/pingcap/tiflash/issues/5091) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + Tools - - Backup & Restore (BR) + + TiDB Dashboard - - note [#issue]() @[Contributor GitHub ID]() + - Optimize the display of TiDB Dashboard [#issue]() @[Contributor GitHub ID]() + - Display the row count on SQL statement summary and slow query pages [#issue]() @[Contributor GitHub ID]() + - Optimize the display of some error messages [#issue]() @[Contributor GitHub ID]() - - TiCDC + + Backup & Restore (BR) - note [#issue]() @[Contributor GitHub ID]() - - TiDB Data Migration (DM) + + TiDB Binlog - - note [#issue]() @[Contributor GitHub ID]() + - Fix a bug that Drainer cannot send requests correctly to Pump when compressor is set to gzip [#1152](https://github.com/pingcap/tidb-binlog/issues/1152) @[lichunzhu](https://github.com/lichunzhu) - - TiDB Lightning + + TiCDC - - note [#issue]() @[Contributor GitHub ID]() + - Improve compatibility for MySQL 8.0 upstream [#6506](https://github.com/pingcap/tiflow/issues/6506) @[lance6716](https://github.com/lance6716) + + + TiDB Data Migration (DM) + + - Improve compatibility for MySQL 8.0 upstream [#6448](https://github.com/pingcap/tiflow/issues/6448) @[lance6716](https://github.com/lance6716) + + + TiDB Lightning + + - Add query parameters for S3 external storage URL, in order to support accessing the S3 data in another account by assuming a given role [#36891](https://github.com/pingcap/tidb/issues/36891) [dsdashun](https://github.com/dsdashun) - TiUP @@ -260,37 +304,115 @@ In v6.3.0-DMR, the key new features and improvements are as follows: + TiDB - - note [#issue]() @[Contributor GitHub ID]() + - Fix handling of prepared statement flags in the classic MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[hawkingrei](https://github.com/hawkingrei) + - update pd-client to ensure tidb-server get clusterID correctly [#36505](https://github.com/pingcap/tidb/issues/36505), [#36478](https://github.com/pingcap/tidb/issues/36478) @[Defined2014](https://github.com/Defined2014) + - Fix that incorrect TiDB states may appear on startup under very, very, very extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) + - Fix a bug that UnionScan's Next() function skips reading data when the passed chunk's capacity is 0 [#36903](https://github.com/pingcap/tidb/issues/36903) + - Fix a bug about variables information leak. [#37586](https://github.com/pingcap/tidb/issues/37586) + - Fix the issue that the action order of [#37058](https://github.com/pingcap/tidb/issues/37058) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that comparison between json opaque will cause panic [#37315](https://github.com/pingcap/tidb/issues/37315) + - Fix the issue that the single precision float cannot be used in json aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) + - fix that the result of expression castRealAsTime is inconsistent with mysql. [#37462](https://github.com/pingcap/tidb/issues/37462) + + - sql-infra + + - fix the bug prepare will not check privilege [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) + - When set `tidb_enable_noop_variable` to `WARN`, an error will be returned [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that when 'expression index' is defined, the value of `ORDINAL_POSITION` column of `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect. [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) + - Fix the issue that when setting a timestamp that is larger than `MAXINT32`, TiDB doesn't report an error like MySQL. [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) + - flashback cluster shouldn't support expr in timestamp [#37495](https://github.com/pingcap/tidb/issues/37495) @[Defined2014](https://github.com/Defined2014) + - Fix panic of enterprise plugin on 6.1 [#37319](https://github.com/pingcap/tidb/issues/37319) + - Fix incorrect output of `show create placement policy` with a policy of double quotes [#37526](https://github.com/pingcap/tidb/issues/37526) @[xhebox](https://github.com/xhebox) + - store flashback history in TiKV, avoid overlapped flashback TS range [#37585](https://github.com/pingcap/tidb/issues/37585) @[Defined2014](https://github.com/Defined2014) + - When exchange partition with temporary table, an error will be returned. [#37201](https://github.com/pingcap/tidb/issues/37201) + - planner: fix partition table getting error result when select `TIKV_REGION_STATUS` with `table_id`. [#37436](https://github.com/pingcap/tidb/issues/37436) @[zimulala](https://github.com/zimulala) + - In test_driver, parser didn't deal with RestoreStringWithoutCharset and RestoreStringWithoutDefaultCharset flags, add support for those two flags. [#37175](https://github.com/pingcap/tidb/issues/37175) @[Defined2014](https://github.com/Defined2014) + - planner: fix show View Privilege behave for view table [#34326](https://github.com/pingcap/tidb/issues/34326) @[hawkingrei](https://github.com/hawkingrei) + - Support send flashback RPC. [#37651](https://github.com/pingcap/tidb/issues/37651) @[Defined2014](https://github.com/Defined2014) + - Fix a wrong casting in building union plan [#31678](https://github.com/pingcap/tidb/issues/31678) @[bb7133](https://github.com/bb7133) + - support some adminStmt in read-only mode [#37631](https://github.com/pingcap/tidb/issues/37631) @[Defined2014](https://github.com/Defined2014) + - fix resume pd schedule and cancel for `flashback cluster` [#37584](https://github.com/pingcap/tidb/issues/37584) @[Defined2014](https://github.com/Defined2014) + - fix resume pd schedule and cancel for `flashback cluster` [#37580](https://github.com/pingcap/tidb/issues/37580) @[Defined2014](https://github.com/Defined2014) + + - execution + + - Fix wrong result when enabling dynamic mode in partition table for tiflash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) + - Fix the issue that the cast and comparison between binary string and json is incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the cast and comparison between binary string and json is incompatible with MySQL [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the json_objectagg and json_arrayagg is not compatible with MySQL on binary value. [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + + - transaction + + - bugfix: do not acquire pessimistic lock for non-unique index keys [#36235](https://github.com/pingcap/tidb/issues/36235) + - Fix the auto-commit mode change related transaction commit behaviours. [#36581](https://github.com/pingcap/tidb/issues/36581) @[cfzjywxk](https://github.com/cfzjywxk) + + - planner + + - fix update plan's projection elimination will cause column resolution error. [#37568](https://github.com/pingcap/tidb/issues/37568) @[AilinKid](https://github.com/AilinKid) + - planner: fix outer join reorder will push down its outer join condition [#37238](https://github.com/pingcap/tidb/issues/37238) @[AilinKid](https://github.com/AilinKid) + - make the both side operand of NAAJ & refuse partial column substitute in projection elimination [#37032](https://github.com/pingcap/tidb/issues/37032) @[AilinKid](https://github.com/AilinKid) + + - diagnosis + + - fix metric sql error [#35856](https://github.com/pingcap/tidb/issues/35856) @[Defined2014](https://github.com/Defined2014) + TiKV - - - note [#issue]() @[Contributor GitHub ID]() + + - fix the bug that the consume should be refresh if region heartbeat send failed. [#12934](https://github.com/tikv/tikv/issues/12934) @[bufferflies](https://github.com/bufferflies) + - Fix a bug that regions may be overlapped if raftstore is too busy [#13160](https://github.com/tikv/tikv/issues/13160) @[5kbpers](https://github.com/5kbpers) + - Fix potential deadlock in `RpcClient` when two read locks are interleaved by a write lock. [#12933](https://github.com/tikv/tikv/issues/12933) @[BurtonQin](https://github.com/BurtonQin) + - Fix a double-lock bug in components/engine_test [#13186](https://github.com/tikv/tikv/issues/13186) @[SpadeA-Tang](https://github.com/SpadeA-Tang) + - Fix plaintext iv debug assert while disable encryption. [#13081](https://github.com/tikv/tikv/issues/13081) @[jiayang-zheng](https://github.com/jiayang-zheng) + - Fix a expression error that causes unified read pool cpu cannot be shown correctly. [#13086](https://github.com/tikv/tikv/issues/13086) @[glorv](https://github.com/glorv) + - Fix the problem that QPS may drop to zero for several mintues when a tikv is partitioned. [#12966](https://github.com/tikv/tikv/issues/12966) @[cosven](https://github.com/cosven) + - remove call_option to avoid deadlock(RWR). [#13191](https://github.com/tikv/tikv/issues/13191) @[bufferflies](https://github.com/bufferflies) + - Reduce false-positive PessimisticLockNotFound errors in conflicting auto-commit workloads. [#13425](https://github.com/tikv/tikv/issues/13425) @[sticnarf](https://github.com/sticnarf) + - Fix a bug that may cause PiTR losing some data when there are too many adjacent short row putting. [#13281](https://github.com/tikv/tikv/issues/13281) @[YuJuncen](https://github.com/YuJuncen) + - Fix a bug that caused checkpoint not advanced when there are some long pessimistic transactions. [#13304](https://github.com/tikv/tikv/issues/13304) @[YuJuncen](https://github.com/YuJuncen) + PD - - note [#issue]() @[Contributor GitHub ID]() + - grpc: fix the wrong error handler [#5373](https://github.com/tikv/pd/issues/5373) @[bufferflies](https://github.com/bufferflies) + - Fix the issue that unhealthy region cause panic [#5491](https://github.com/tikv/pd/issues/5491) @[nolouch](https://github.com/nolouch) + - Fix the bug where the Learner Peer of TiFlash Replica might not be created. [#5401](https://github.com/tikv/pd/issues/5401) @[HunDunDM](https://github.com/HunDunDM) + TiFlash - - note [#issue]() @[Contributor GitHub ID]() + - compute + + - Fix the bug that window function may cause tiflash crash when canceled. [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) + - fix error data input for date(CAST(value AS DATETIME)) causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) + + - storage + + - fix the problem that there may be some obsolete data left in storage which cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) + - fix the problem that there may be some obsolete data left in storage which cannot be deleted [#5659](https://github.com/pingcap/tiflash/issues/5659) @[lidezhu](https://github.com/lidezhu) + - Fix the bug that page GC may block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) + Tools + Backup & Restore (BR) - - note [#issue]() @[Contributor GitHub ID]() + - Fix issues in "br/tests/up.sh". [#36743](https://github.com/pingcap/tidb/issues/36743) @[pingyu](https://github.com/pingyu) + - br: raw restore fail in integration test "br_rawkv [#36490](https://github.com/pingcap/tidb/issues/36490) @[pingyu](https://github.com/pingyu) + - Fix a bug that may cause the information of the checkpoint being stale. [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) + - Fix a bug caused when restoring with high `concurrency` the regions aren't balanced. [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) - - TiCDC + + TiCDC + + - handle error correctly with wrong pd address but with a grpc service [#6458](https://github.com/pingcap/tiflow/issues/6458) @[crelax](https://github.com/crelax) - - note [#issue]() @[Contributor GitHub ID]() + + TiDB Data Migration (DM) - - TiDB Data Migration (DM) + - Fix a problem that DM will report `Specified key was too long` error [#5315](https://github.com/pingcap/tiflow/issues/5315) @[lance6716](https://github.com/lance6716) + - Fix a bug that relay goroutine and upstream connections may leak when relay meet error [#6193](https://github.com/pingcap/tiflow/issues/6193) @[lance6716](https://github.com/lance6716) + - Fix when use "strict" collation_compatible, DM sometimes generate SQL with duplicated collation [#6832](https://github.com/pingcap/tiflow/issues/6832) @[lance6716](https://github.com/lance6716) + - Reduce the appearing time of the warning message "found error when getting timezone from binlog status_vars" in dm-worker log. [#6628](https://github.com/pingcap/tiflow/issues/6628) @[lyzx2001](https://github.com/lyzx2001) + - Fix a bug that latin1 data may be corrupt when replicating [#7028](https://github.com/pingcap/tiflow/issues/7028) @[lance6716](https://github.com/lance6716) - - note [#issue]() @[Contributor GitHub ID]() - - - TiDB Lightning + + TiDB Lightning - - note [#issue]() @[Contributor GitHub ID]() + - Fix the issue that TiDB Lightning does not support columns starting with slash, number, or non-ascii characters in Parquet files [#36980](https://github.com/pingcap/tidb/issues/36980) @[D3Hunter](https://github.com/D3Hunter) - TiUP From da64b5baae17eeff4b5c255c34a0739434ddaef3 Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 21 Sep 2022 13:03:16 +0800 Subject: [PATCH 025/104] apply suggestions from code review --- releases/release-6.3.0.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index cf3e4b8bc3996..5e387fb88d1f9 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -58,11 +58,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Improve the performance of adding indexes and reduce its impact on DML transactions [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** - To improve the speed of backfilling when creating an index, TiDB v6.3.0 accelerates the `ADD INDEX` and `CREATE INDEX` DDL operations when the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) system variable is enabled. The performance of adding indexes is about three times faster than the original when the feature is enabled. - -* Improve MySQL 8.0 compatibility by adding support for `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` regular expression functions [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** - - For more details about the compatibility with MySQL, see [Regular expression compatibility with MySQL](/functions-and-operators/string-functions.md#regular-expression-compatibility-with-mysql). + To improve the speed of backfilling when creating an index, TiDB v6.3.0 accelerates the `ADD INDEX` and `CREATE INDEX` DDL operations when the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) system variable is enabled. When the feature is enabled, the performance of adding indexes is about three times faster than previously. ### Security @@ -118,9 +114,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiKV recycles log files in Raft Engine by default. This reduces the long tail latency during Raft log appending and improves performance under write workloads. -* TiDB supports Null-Aware Anti Join] [#issue]() @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** +* TiDB supports Null-Aware Anti Join [#37525](https://github.com/pingcap/tidb/issues/37525) @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** - TiDB v6.3.0 introduces a new join type [Null-Aware Anti Join (NAAJ)](/explain-subqueries.md#null-aware-semi-joinin-any). NAAJ can be aware of whether the collection is empty or `NULL` when processing collection operations. This optimizes the execution efficiency of operations such as `IN` and `= ANY` and improves SQL performance. + TiDB v6.3.0 introduces a new join type [Null-Aware Anti Join (NAAJ)](/explain-subqueries.md#null-aware-anti-semi-join-not-in-and--all-subqueries). NAAJ can be aware of whether the collection is empty or `NULL` when processing collection operations. This optimizes the execution efficiency of operations such as `IN` and `= ANY` and improves SQL performance. * Add optimizer hints to control the build end of Hash Join [#issue]() @[Reminiscent](https://github.com/Reminiscent) **tw: TomShawn** @@ -154,6 +150,10 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### MySQL compatibility +* Improve MySQL 8.0 compatibility by adding support for `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` regular expression functions [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** + + For more details about the compatibility with MySQL, see [Regular expression compatibility with MySQL](/functions-and-operators/string-functions.md#regular-expression-compatibility-with-mysql). + * Improve compatibility of SQL-based data Placement Rules [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature is not compatible with TiFlash due to conflicts in implementation mechanisms. TiDB v6.3.0 optimizes this feature, and [improves compatibility of SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). From b20d896e79cde00b7409c2251dabd69568fdbbb8 Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 21 Sep 2022 17:52:59 +0800 Subject: [PATCH 026/104] apply suggestions from code review --- releases/release-6.3.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 5e387fb88d1f9..e445224d4c2d6 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -14,7 +14,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: - TiDB supports authentication with the SM3 algorithm. - The `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option. - JSON data type and functions become generally available (GA). -- TiDB supports Null Aware Anti Join. +- TiDB supports null-aware anti join. - TiDB provides execution time metrics at a finer granularity. - A new syntactic sugar is added to simplify Range partition definitions. - Range Columns partitioning supports defining multiple columns. @@ -54,7 +54,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Provide lightweight metadata lock to improve the DML success rate during DDL change (experimental) [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** - TiDB uses the online asynchronous schema change algorithm to support changing metadata objects. When a transaction is executed, the transaction obtains the corresponding metadata snapshot at transaction start. If the metadata is changed during transaction, to ensure the data consistency, TiDB returns an `Information schema is changed` error and the transaction fails to commit. To solve the problem, TiDB v6.3.0 introduces [metadata lock](/metadata-lock.md) into the online DDL algorithm. To avoid most DML errors, TiDB coordinates the priority of DMLs and DDLs during table metadata change, and makes executing DDLs wait for the DMLs with old metadata to commit. + TiDB uses the online asynchronous schema change algorithm to support changing metadata objects. When a transaction is executed, it obtains the corresponding metadata snapshot at the transaction start. If the metadata is changed during a transaction, to ensure data consistency, TiDB returns an `Information schema is changed` error and the transaction fails to commit. To solve the problem, TiDB v6.3.0 introduces [metadata lock](/metadata-lock.md) into the online DDL algorithm. To avoid most DML errors, TiDB coordinates the priority of DMLs and DDLs during table metadata change and makes executing DDLs wait for the DMLs with old metadata to commit. * Improve the performance of adding indexes and reduce its impact on DML transactions [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** @@ -114,9 +114,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiKV recycles log files in Raft Engine by default. This reduces the long tail latency during Raft log appending and improves performance under write workloads. -* TiDB supports Null-Aware Anti Join [#37525](https://github.com/pingcap/tidb/issues/37525) @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** +* TiDB supports null-aware anti join [#37525](https://github.com/pingcap/tidb/issues/37525) @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** - TiDB v6.3.0 introduces a new join type [Null-Aware Anti Join (NAAJ)](/explain-subqueries.md#null-aware-anti-semi-join-not-in-and--all-subqueries). NAAJ can be aware of whether the collection is empty or `NULL` when processing collection operations. This optimizes the execution efficiency of operations such as `IN` and `= ANY` and improves SQL performance. + TiDB v6.3.0 introduces a new join type [Null-aware anti join (NAAJ)](/explain-subqueries.md#null-aware-anti-semi-join-not-in-and--all-subqueries). NAAJ can be aware of whether the collection is empty or `NULL` when processing collection operations. This optimizes the execution efficiency of operations such as `IN` and `= ANY` and improves SQL performance. * Add optimizer hints to control the build end of Hash Join [#issue]() @[Reminiscent](https://github.com/Reminiscent) **tw: TomShawn** From 519ac828728b5fa17807f7ccb662b5c71bf9f71f Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 21 Sep 2022 22:40:15 +0800 Subject: [PATCH 027/104] Apply suggestions from code review Co-authored-by: xixirangrang --- releases/release-6.3.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index e445224d4c2d6..a663dc2da43aa 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -96,13 +96,13 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiFlash changes the way FastScan is used (experimental) [#5252](https://github.com/pingcap/tiflash/issues/5252) @[hongyunyan](https://github.com/hongyunyan) - FastScan, introduced in TiFlash starting with v6.2.0, performed as expected, but lacked flexibility in how it was used. Therefore, in v6.3.0 [the way of using FastScan has changed](/develop/dev-guide-use-fastscan.md). TiFlash does not support using the switch to control the FastScan feature. Instead, it uses the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) to control whether to turn on the FastScan function. + FastScan, introduced in TiFlash starting with v6.2.0, performed as expected, but lacked flexibility. Therefore, in v6.3.0 [the way of using FastScan has changed](/develop/dev-guide-use-fastscan.md). TiFlash does not support using the switch to control the FastScan feature. Instead, it uses the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) to control whether to enable the FastScan function. When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. You need to set the variable [`tiflash_fastscan`]. When you upgrade from an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistency. * TiFlash optimizes to improve data scanning performance in multiple concurrency scenarios [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) - TiFlash reduces duplicate reads of the same data by combining read operations of the same data. It optimizes the resource overhead and improves the performance of data scanning in the case of concurrent tasks (/tiflash/tiflash-configuration.md#configure-the-tiflashtoml-file). It avoids the situation where the same data needs to be read separately in each task in multiple concurrent tasks, and avoids the possibility of multiple reads of the same data at the same time. + TiFlash reduces duplicate reads of the same data by combining read operations of the same data. It optimizes the resource overhead and [improves the performance of data scanning in the case of concurrent tasks](/tiflash/tiflash-configuration.md#configure-the-tiflashtoml-file). It avoids the situation where the same data needs to be read separately in each task in multiple concurrent tasks, and avoids the possibility of multiple reads of the same data at the same time. This feature is experimental in v6.2.0, and becomes GA in v6.3.0. @@ -112,7 +112,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiKV supports log recycling [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) **tw：ran-huang** - TiKV recycles log files in Raft Engine by default. This reduces the long tail latency during Raft log appending and improves performance under write workloads. + TiKV supports recycling log files in Raft Engine. This reduces the long tail latency in network disks during Raft log appending and improves performance under write workloads. * TiDB supports null-aware anti join [#37525](https://github.com/pingcap/tidb/issues/37525) @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** From 827bd81487448ad0010f915ca104f480d65bcc15 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Wed, 21 Sep 2022 22:44:32 +0800 Subject: [PATCH 028/104] add 9 notes generated between 9.14 and 9.21 --- releases/release-6.3.0.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index a663dc2da43aa..45a5a318c28ae 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -244,6 +244,8 @@ In v6.3.0-DMR, the key new features and improvements are as follows: - Implement TSO batch list to improve tolerance to TSO service fault. [#12794](https://github.com/tikv/tikv/issues/12794) @[pingyu](https://github.com/pingyu) - Make max_subcompactions dynamically changeable [#13145](https://github.com/tikv/tikv/issues/13145) @[ethercflow](https://github.com/ethercflow) - Optimize the performance of merging empty regions [#12421](https://github.com/tikv/tikv/issues/12421) @[tabokie](https://github.com/tabokie) + - Support more regular expression functions. [#13483](https://github.com/tikv/tikv/issues/13483) @[gengliqi](https://github.com/gengliqi) + - Support automatically scale read pool thread count based on the CPU usage. [#13313](https://github.com/tikv/tikv/issues/13313) @[glorv](https://github.com/glorv) + PD @@ -333,6 +335,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: - support some adminStmt in read-only mode [#37631](https://github.com/pingcap/tidb/issues/37631) @[Defined2014](https://github.com/Defined2014) - fix resume pd schedule and cancel for `flashback cluster` [#37584](https://github.com/pingcap/tidb/issues/37584) @[Defined2014](https://github.com/Defined2014) - fix resume pd schedule and cancel for `flashback cluster` [#37580](https://github.com/pingcap/tidb/issues/37580) @[Defined2014](https://github.com/Defined2014) + - Fix the issue that the user cannot update from json 'null' to NULL [#37852](https://github.com/pingcap/tidb/issues/37852) @[YangKeao](https://github.com/YangKeao) - execution @@ -369,6 +372,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: - Reduce false-positive PessimisticLockNotFound errors in conflicting auto-commit workloads. [#13425](https://github.com/tikv/tikv/issues/13425) @[sticnarf](https://github.com/sticnarf) - Fix a bug that may cause PiTR losing some data when there are too many adjacent short row putting. [#13281](https://github.com/tikv/tikv/issues/13281) @[YuJuncen](https://github.com/YuJuncen) - Fix a bug that caused checkpoint not advanced when there are some long pessimistic transactions. [#13304](https://github.com/tikv/tikv/issues/13304) @[YuJuncen](https://github.com/YuJuncen) + - Fix the issue that TiKV doesn't distinguish the `DATETIME/DATE/TIMESTAMP/TIME` and `STRING` in json type. [#13417](https://github.com/tikv/tikv/issues/13417) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that comparison between json bool and other json value is not compatible with TiDB and MySQL [#13386](https://github.com/tikv/tikv/issues/13386) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that comparison between json bool and other json value is not compatible with TiDB and MySQL [#37481](https://github.com/pingcap/tidb/issues/37481) @[YangKeao](https://github.com/YangKeao) + PD @@ -382,12 +388,14 @@ In v6.3.0-DMR, the key new features and improvements are as follows: - Fix the bug that window function may cause tiflash crash when canceled. [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - fix error data input for date(CAST(value AS DATETIME)) causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) + - fix that the result of expression casting real or decimal as time is inconsistent with mysql. [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) - storage - fix the problem that there may be some obsolete data left in storage which cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) - fix the problem that there may be some obsolete data left in storage which cannot be deleted [#5659](https://github.com/pingcap/tiflash/issues/5659) @[lidezhu](https://github.com/lidezhu) - Fix the bug that page GC may block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the panic issue after creating the primary index with a column containing `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) + Tools @@ -397,6 +405,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: - br: raw restore fail in integration test "br_rawkv [#36490](https://github.com/pingcap/tidb/issues/36490) @[pingyu](https://github.com/pingyu) - Fix a bug that may cause the information of the checkpoint being stale. [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) - Fix a bug caused when restoring with high `concurrency` the regions aren't balanced. [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) + - Fix a bug that may cause log backup checkpoint TS stuck when some weird ranged regions exist. [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) + TiCDC From 1e3d1e84d5d881e4e794feca9e5c62d140776850 Mon Sep 17 00:00:00 2001 From: joey-yez <104608045+joey-yez@users.noreply.github.com> Date: Thu, 22 Sep 2022 12:08:22 +0800 Subject: [PATCH 029/104] Update releases/release-6.3.0.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Daniël van Eeden --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 45a5a318c28ae..c450a95bedfd6 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -37,7 +37,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiDB supports two more [window functions](/tiflash/tiflash-supported-pushdown-calculations.md) [#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) **tw：shichun-0415** - * `LEAD` + * `LEAD()` * `LAG` * The `CREATE USER` statement supports the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** From 7375ff61b72143101838b3d12b51cfddd7c3c9ed Mon Sep 17 00:00:00 2001 From: joey-yez <104608045+joey-yez@users.noreply.github.com> Date: Thu, 22 Sep 2022 12:09:09 +0800 Subject: [PATCH 030/104] Update releases/release-6.3.0.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Daniël van Eeden --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index c450a95bedfd6..44e6e299978f9 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -38,7 +38,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiDB supports two more [window functions](/tiflash/tiflash-supported-pushdown-calculations.md) [#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) **tw：shichun-0415** * `LEAD()` - * `LAG` + * `LAG()` * The `CREATE USER` statement supports the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** From bfdf9f6124357241dfef24e6f794fbfd0bbb77a0 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Thu, 22 Sep 2022 14:46:38 +0800 Subject: [PATCH 031/104] add five more notes and non-PingCAPer contributors --- releases/release-6.3.0.md | 145 ++++++++++++++++++++++---------------- 1 file changed, 84 insertions(+), 61 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 44e6e299978f9..41664c151e64a 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -62,11 +62,11 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Security -* TiKV supports the SM4 algorithm for encryption at rest. [#13041](https://github.com/tikv/tikv/issues/13041) @[jiayang-zheng](https://github.com/jiayang-zheng) +* TiKV supports the SM4 algorithm for encryption at rest [#13041](https://github.com/tikv/tikv/issues/13041) @[jiayang-zheng](https://github.com/jiayang-zheng) Add the [SM4 algorithm](/encryption-at-rest.md) for TiKV encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the "data-encryption-method" configuration to "sm4-ctr". -* TiFlash supports the SM4 algorithm for encryption at rest. [#5714](https://github.com/pingcap/tiflash/issues/5714) @[lidezhu](https://github.com/lidezhu) +* TiFlash supports the SM4 algorithm for encryption at rest [#5714](https://github.com/pingcap/tiflash/issues/5714) @[lidezhu](https://github.com/lidezhu) Add the [SM4 algorithm](/encryption-at-rest.md) for TiFlash encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the "data-encryption-method" configuration to "sm4-ctr". @@ -97,7 +97,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiFlash changes the way FastScan is used (experimental) [#5252](https://github.com/pingcap/tiflash/issues/5252) @[hongyunyan](https://github.com/hongyunyan) FastScan, introduced in TiFlash starting with v6.2.0, performed as expected, but lacked flexibility. Therefore, in v6.3.0 [the way of using FastScan has changed](/develop/dev-guide-use-fastscan.md). TiFlash does not support using the switch to control the FastScan feature. Instead, it uses the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) to control whether to enable the FastScan function. - + When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. You need to set the variable [`tiflash_fastscan`]. When you upgrade from an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistency. * TiFlash optimizes to improve data scanning performance in multiple concurrency scenarios [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) @@ -106,7 +106,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: This feature is experimental in v6.2.0, and becomes GA in v6.3.0. -* Improve performance of TiFlash data replication [#5237](https://github.com/pingcap/tiflash/issues/5237) @[breezewish](https://github.com/breezewish) +* Improve performance of TiFlash data replication [#5237](https://github.com/pingcap/tiflash/issues/5237) @[breezewish](https://github.com/breezewish) TiFlash uses the Raft protocol for data replication with TiKV. Prior to v6.3.0, it often took long time to replicate large amounts of replica data. TiDB v6.3.0 optimizes the TiFlash data replication mechanism and significantly improves the replication speed. When you use BR to recover data, use TiDB Lightning to import data, or add new TiFlash replicas, the replicas can be replicated more quickly. You can query with TiFlash in a more timely manner. In addition, TiFlash replicas will also reach a secure and balanced state faster when you scale up, scale down, or modify the number of TiFlash replicas. @@ -120,7 +120,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Add optimizer hints to control the build end of Hash Join [#issue]() @[Reminiscent](https://github.com/Reminiscent) **tw: TomShawn** - In v6.3.0, the TiDB optimizer introduces 2 hints, `HASH_JOIN_BUILD()` and `HASH_JOIN_PROBE()`, to specify the Hash Join, its probe end, and its build end. When the optimizer fails to select the optimal execution plan, you can use these hints to intervene with the plan. + In v6.3.0, the TiDB optimizer introduces 2 hints, `HASH_JOIN_BUILD()` and `HASH_JOIN_PROBE()`, to specify the Hash Join, its probe end, and its build end. When the optimizer fails to select the optimal execution plan, you can use these hints to intervene with the plan. * Support session-level CTE inline [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) **tw: shichun-0415** @@ -156,7 +156,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Improve compatibility of SQL-based data Placement Rules [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) - TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature is not compatible with TiFlash due to conflicts in implementation mechanisms. TiDB v6.3.0 optimizes this feature, and [improves compatibility of SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). + TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature is not compatible with TiFlash due to conflicts in implementation mechanisms. TiDB v6.3.0 optimizes this feature, and [improves compatibility of SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). ### Backup and restore @@ -221,18 +221,20 @@ In v6.3.0-DMR, the key new features and improvements are as follows: + TiDB - One `PLAN REPLAYER` command can export information about execution plans of multiple SQL statements, which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) + - improve warning log when new connection arrives [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) - sql-infra - Extend partitioning syntax with INTERVAL for easier partitioning definition [#35827](https://github.com/pingcap/tidb/issues/35827) @[ymkzpx](https://github.com/ymkzpx) - - Grant privilege of a table to an user checks the target table exist first, in the past, the table name comparison works in a case sensitive manner, now it's changed to case insensitive. [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) + - Grant privilege of a table to an user checks the target table exist first, in the past, the table name comparison works in a case sensitive manner, now it's changed to case insensitive [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) - Support AWS NLB proxy protocol [#36312](https://github.com/pingcap/tidb/issues/36312) @[hawkingrei](https://github.com/hawkingrei) - - Previously, TiDB users can set `init_connect` without any checking. From now on, the value of `init_connect` should be checked by the sql parser. [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Previously, TiDB users can set `init_connect` without any checking. From now on, the value of `init_connect` should be checked by the sql parser [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Add support `flashback database` command [#37386](https://github.com/pingcap/tidb/issues/37386) @[tiancaiamao](https://github.com/tiancaiamao) - execution - - report error if json path has the wrong syntax. [#22525](https://github.com/pingcap/tidb/issues/22525) @[xiongjiwei](https://github.com/xiongjiwei) - - report error if json path has the wrong syntax. [#34959](https://github.com/pingcap/tidb/issues/34959) @[xiongjiwei](https://github.com/xiongjiwei) + - report error if json path has the wrong syntax [#22525](https://github.com/pingcap/tidb/issues/22525) @[xiongjiwei](https://github.com/xiongjiwei) + - report error if json path has the wrong syntax [#34959](https://github.com/pingcap/tidb/issues/34959) @[xiongjiwei](https://github.com/xiongjiwei) - planner @@ -240,27 +242,27 @@ In v6.3.0-DMR, the key new features and improvements are as follows: + TiKV - - Add a new option to make unreachable_backoff of raftstore configurable. [#13054](https://github.com/tikv/tikv/issues/13054) - - Implement TSO batch list to improve tolerance to TSO service fault. [#12794](https://github.com/tikv/tikv/issues/12794) @[pingyu](https://github.com/pingyu) + - Add a new option to make unreachable_backoff of raftstore configurable [#13054](https://github.com/tikv/tikv/issues/13054) + - Implement TSO batch list to improve tolerance to TSO service fault [#12794](https://github.com/tikv/tikv/issues/12794) @[pingyu](https://github.com/pingyu) - Make max_subcompactions dynamically changeable [#13145](https://github.com/tikv/tikv/issues/13145) @[ethercflow](https://github.com/ethercflow) - Optimize the performance of merging empty regions [#12421](https://github.com/tikv/tikv/issues/12421) @[tabokie](https://github.com/tabokie) - - Support more regular expression functions. [#13483](https://github.com/tikv/tikv/issues/13483) @[gengliqi](https://github.com/gengliqi) - - Support automatically scale read pool thread count based on the CPU usage. [#13313](https://github.com/tikv/tikv/issues/13313) @[glorv](https://github.com/glorv) + - Support more regular expression functions [#13483](https://github.com/tikv/tikv/issues/13483) @[gengliqi](https://github.com/gengliqi) + - Support automatically scale read pool thread count based on the CPU usage [#13313](https://github.com/tikv/tikv/issues/13313) @[glorv](https://github.com/glorv) + PD - - Updates metrics query. Renames `metrics` to `monitoring` on TiDB Dashboard. [#5366](https://github.com/tikv/pd/issues/5366) @[YiniXu9506](https://github.com/YiniXu9506) - + - Updates metrics query. Renames `metrics` to `monitoring` on TiDB Dashboard [#5366](https://github.com/tikv/pd/issues/5366) @[YiniXu9506](https://github.com/YiniXu9506) + + TiFlash - compute - + - Support to pushdown elt to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) - Support to pushdown leftShift to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) - Support to pushdown castTimeAsDuration to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - Support to pushdown castTimeAsDuration to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - - Support Planner Interpreter. [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - - Support to pushdown hexInt and hexStr to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107), [#5462](https://github.com/pingcap/tiflash/issues/5462) + - Support Planner Interpreter [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) + - Support to pushdown hexInt and hexStr to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107), [#5462](https://github.com/pingcap/tiflash/issues/5462) - Support to pushdown elt to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) - Support to pushdown shiftLeft to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) - Suppress the "tcp set inq" loggings [#4940](https://github.com/pingcap/tiflash/issues/4940) @@ -308,71 +310,74 @@ In v6.3.0-DMR, the key new features and improvements are as follows: - Fix handling of prepared statement flags in the classic MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[hawkingrei](https://github.com/hawkingrei) - update pd-client to ensure tidb-server get clusterID correctly [#36505](https://github.com/pingcap/tidb/issues/36505), [#36478](https://github.com/pingcap/tidb/issues/36478) @[Defined2014](https://github.com/Defined2014) - - Fix that incorrect TiDB states may appear on startup under very, very, very extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) - - Fix a bug that UnionScan's Next() function skips reading data when the passed chunk's capacity is 0 [#36903](https://github.com/pingcap/tidb/issues/36903) - - Fix a bug about variables information leak. [#37586](https://github.com/pingcap/tidb/issues/37586) + - Fix that incorrect TiDB states may appear on startup under very, very, very extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) + - Fix a bug that UnionScan's Next() function skips reading data when the passed chunk's capacity is 0 [#36903](https://github.com/pingcap/tidb/issues/36903) + - Fix a bug about variables information leak [#37586](https://github.com/pingcap/tidb/issues/37586) - Fix the issue that the action order of [#37058](https://github.com/pingcap/tidb/issues/37058) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that comparison between json opaque will cause panic [#37315](https://github.com/pingcap/tidb/issues/37315) + - Fix the issue that comparison between json opaque will cause panic [#37315](https://github.com/pingcap/tidb/issues/37315) - Fix the issue that the single precision float cannot be used in json aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) - - fix that the result of expression castRealAsTime is inconsistent with mysql. [#37462](https://github.com/pingcap/tidb/issues/37462) + - fix that the result of expression castRealAsTime is inconsistent with mysql [#37462](https://github.com/pingcap/tidb/issues/37462) - sql-infra - fix the bug prepare will not check privilege [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) - When set `tidb_enable_noop_variable` to `WARN`, an error will be returned [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that when 'expression index' is defined, the value of `ORDINAL_POSITION` column of `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect. [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) - - Fix the issue that when setting a timestamp that is larger than `MAXINT32`, TiDB doesn't report an error like MySQL. [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) + - Fix the issue that when 'expression index' is defined, the value of `ORDINAL_POSITION` column of `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) + - Fix the issue that when setting a timestamp that is larger than `MAXINT32`, TiDB doesn't report an error like MySQL [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) - flashback cluster shouldn't support expr in timestamp [#37495](https://github.com/pingcap/tidb/issues/37495) @[Defined2014](https://github.com/Defined2014) - Fix panic of enterprise plugin on 6.1 [#37319](https://github.com/pingcap/tidb/issues/37319) - Fix incorrect output of `show create placement policy` with a policy of double quotes [#37526](https://github.com/pingcap/tidb/issues/37526) @[xhebox](https://github.com/xhebox) - store flashback history in TiKV, avoid overlapped flashback TS range [#37585](https://github.com/pingcap/tidb/issues/37585) @[Defined2014](https://github.com/Defined2014) - - When exchange partition with temporary table, an error will be returned. [#37201](https://github.com/pingcap/tidb/issues/37201) - - planner: fix partition table getting error result when select `TIKV_REGION_STATUS` with `table_id`. [#37436](https://github.com/pingcap/tidb/issues/37436) @[zimulala](https://github.com/zimulala) - - In test_driver, parser didn't deal with RestoreStringWithoutCharset and RestoreStringWithoutDefaultCharset flags, add support for those two flags. [#37175](https://github.com/pingcap/tidb/issues/37175) @[Defined2014](https://github.com/Defined2014) + - When exchange partition with temporary table, an error will be returned [#37201](https://github.com/pingcap/tidb/issues/37201) + - planner: fix partition table getting error result when select `TIKV_REGION_STATUS` with `table_id` [#37436](https://github.com/pingcap/tidb/issues/37436) @[zimulala](https://github.com/zimulala) + - In test_driver, parser didn't deal with RestoreStringWithoutCharset and RestoreStringWithoutDefaultCharset flags, add support for those two flags [#37175](https://github.com/pingcap/tidb/issues/37175) @[Defined2014](https://github.com/Defined2014) - planner: fix show View Privilege behave for view table [#34326](https://github.com/pingcap/tidb/issues/34326) @[hawkingrei](https://github.com/hawkingrei) - - Support send flashback RPC. [#37651](https://github.com/pingcap/tidb/issues/37651) @[Defined2014](https://github.com/Defined2014) + - Support send flashback RPC [#37651](https://github.com/pingcap/tidb/issues/37651) @[Defined2014](https://github.com/Defined2014) - Fix a wrong casting in building union plan [#31678](https://github.com/pingcap/tidb/issues/31678) @[bb7133](https://github.com/bb7133) - support some adminStmt in read-only mode [#37631](https://github.com/pingcap/tidb/issues/37631) @[Defined2014](https://github.com/Defined2014) - fix resume pd schedule and cancel for `flashback cluster` [#37584](https://github.com/pingcap/tidb/issues/37584) @[Defined2014](https://github.com/Defined2014) - fix resume pd schedule and cancel for `flashback cluster` [#37580](https://github.com/pingcap/tidb/issues/37580) @[Defined2014](https://github.com/Defined2014) - Fix the issue that the user cannot update from json 'null' to NULL [#37852](https://github.com/pingcap/tidb/issues/37852) @[YangKeao](https://github.com/YangKeao) + - Optimize DDL history HTTP API, and add support for 'start_job_id' parameter [#35838](https://github.com/pingcap/tidb/issues/35838) @[tiancaiamao](https://github.com/tiancaiamao) + - fix inaccuate row_count num [#25968](https://github.com/pingcap/tidb/issues/25968) @[Defined2014](https://github.com/Defined2014) - execution - Fix wrong result when enabling dynamic mode in partition table for tiflash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) - Fix the issue that the cast and comparison between binary string and json is incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the cast and comparison between binary string and json is incompatible with MySQL [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that the json_objectagg and json_arrayagg is not compatible with MySQL on binary value. [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the json_objectagg and json_arrayagg is not compatible with MySQL on binary value [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - transaction - - bugfix: do not acquire pessimistic lock for non-unique index keys [#36235](https://github.com/pingcap/tidb/issues/36235) - - Fix the auto-commit mode change related transaction commit behaviours. [#36581](https://github.com/pingcap/tidb/issues/36581) @[cfzjywxk](https://github.com/cfzjywxk) + - bugfix: do not acquire pessimistic lock for non-unique index keys [#36235](https://github.com/pingcap/tidb/issues/36235) + - Fix the auto-commit mode change related transaction commit behaviours [#36581](https://github.com/pingcap/tidb/issues/36581) @[cfzjywxk](https://github.com/cfzjywxk) - planner - - fix update plan's projection elimination will cause column resolution error. [#37568](https://github.com/pingcap/tidb/issues/37568) @[AilinKid](https://github.com/AilinKid) - - planner: fix outer join reorder will push down its outer join condition [#37238](https://github.com/pingcap/tidb/issues/37238) @[AilinKid](https://github.com/AilinKid) - - make the both side operand of NAAJ & refuse partial column substitute in projection elimination [#37032](https://github.com/pingcap/tidb/issues/37032) @[AilinKid](https://github.com/AilinKid) + - fix update plan's projection elimination will cause column resolution error [#37568](https://github.com/pingcap/tidb/issues/37568) @[AilinKid](https://github.com/AilinKid) + - planner: fix outer join reorder will push down its outer join condition [#37238](https://github.com/pingcap/tidb/issues/37238) @[AilinKid](https://github.com/AilinKid) + - make the both side operand of NAAJ & refuse partial column substitute in projection elimination [#37032](https://github.com/pingcap/tidb/issues/37032) @[AilinKid](https://github.com/AilinKid) + - planner: correct the redundant field meaning in join full schema when join coalesce [#36420](https://github.com/pingcap/tidb/issues/36420) @[AilinKid](https://github.com/AilinKid) - diagnosis - - fix metric sql error [#35856](https://github.com/pingcap/tidb/issues/35856) @[Defined2014](https://github.com/Defined2014) + - fix metric sql error [#35856](https://github.com/pingcap/tidb/issues/35856) @[Defined2014](https://github.com/Defined2014) + TiKV - - - fix the bug that the consume should be refresh if region heartbeat send failed. [#12934](https://github.com/tikv/tikv/issues/12934) @[bufferflies](https://github.com/bufferflies) - - Fix a bug that regions may be overlapped if raftstore is too busy [#13160](https://github.com/tikv/tikv/issues/13160) @[5kbpers](https://github.com/5kbpers) - - Fix potential deadlock in `RpcClient` when two read locks are interleaved by a write lock. [#12933](https://github.com/tikv/tikv/issues/12933) @[BurtonQin](https://github.com/BurtonQin) - - Fix a double-lock bug in components/engine_test [#13186](https://github.com/tikv/tikv/issues/13186) @[SpadeA-Tang](https://github.com/SpadeA-Tang) - - Fix plaintext iv debug assert while disable encryption. [#13081](https://github.com/tikv/tikv/issues/13081) @[jiayang-zheng](https://github.com/jiayang-zheng) - - Fix a expression error that causes unified read pool cpu cannot be shown correctly. [#13086](https://github.com/tikv/tikv/issues/13086) @[glorv](https://github.com/glorv) - - Fix the problem that QPS may drop to zero for several mintues when a tikv is partitioned. [#12966](https://github.com/tikv/tikv/issues/12966) @[cosven](https://github.com/cosven) - - remove call_option to avoid deadlock(RWR). [#13191](https://github.com/tikv/tikv/issues/13191) @[bufferflies](https://github.com/bufferflies) - - Reduce false-positive PessimisticLockNotFound errors in conflicting auto-commit workloads. [#13425](https://github.com/tikv/tikv/issues/13425) @[sticnarf](https://github.com/sticnarf) - - Fix a bug that may cause PiTR losing some data when there are too many adjacent short row putting. [#13281](https://github.com/tikv/tikv/issues/13281) @[YuJuncen](https://github.com/YuJuncen) - - Fix a bug that caused checkpoint not advanced when there are some long pessimistic transactions. [#13304](https://github.com/tikv/tikv/issues/13304) @[YuJuncen](https://github.com/YuJuncen) - - Fix the issue that TiKV doesn't distinguish the `DATETIME/DATE/TIMESTAMP/TIME` and `STRING` in json type. [#13417](https://github.com/tikv/tikv/issues/13417) @[YangKeao](https://github.com/YangKeao) + + - fix the bug that the consume should be refresh if region heartbeat send failed [#12934](https://github.com/tikv/tikv/issues/12934) @[bufferflies](https://github.com/bufferflies) + - Fix a bug that regions may be overlapped if raftstore is too busy [#13160](https://github.com/tikv/tikv/issues/13160) @[5kbpers](https://github.com/5kbpers) + - Fix potential deadlock in `RpcClient` when two read locks are interleaved by a write lock [#12933](https://github.com/tikv/tikv/issues/12933) @[BurtonQin](https://github.com/BurtonQin) + - Fix a double-lock bug in components/engine_test [#13186](https://github.com/tikv/tikv/issues/13186) @[SpadeA-Tang](https://github.com/SpadeA-Tang) + - Fix plaintext iv debug assert while disable encryption [#13081](https://github.com/tikv/tikv/issues/13081) @[jiayang-zheng](https://github.com/jiayang-zheng) + - Fix a expression error that causes unified read pool cpu cannot be shown correctly [#13086](https://github.com/tikv/tikv/issues/13086) @[glorv](https://github.com/glorv) + - Fix the problem that QPS may drop to zero for several mintues when a tikv is partitioned [#12966](https://github.com/tikv/tikv/issues/12966) @[cosven](https://github.com/cosven) + - remove call_option to avoid deadlock(RWR) [#13191](https://github.com/tikv/tikv/issues/13191) @[bufferflies](https://github.com/bufferflies) + - Reduce false-positive PessimisticLockNotFound errors in conflicting auto-commit workloads [#13425](https://github.com/tikv/tikv/issues/13425) @[sticnarf](https://github.com/sticnarf) + - Fix a bug that may cause PiTR losing some data when there are too many adjacent short row putting [#13281](https://github.com/tikv/tikv/issues/13281) @[YuJuncen](https://github.com/YuJuncen) + - Fix a bug that caused checkpoint not advanced when there are some long pessimistic transactions [#13304](https://github.com/tikv/tikv/issues/13304) @[YuJuncen](https://github.com/YuJuncen) + - Fix the issue that TiKV doesn't distinguish the `DATETIME/DATE/TIMESTAMP/TIME` and `STRING` in json type [#13417](https://github.com/tikv/tikv/issues/13417) @[YangKeao](https://github.com/YangKeao) - Fix the issue that comparison between json bool and other json value is not compatible with TiDB and MySQL [#13386](https://github.com/tikv/tikv/issues/13386) @[YangKeao](https://github.com/YangKeao) - Fix the issue that comparison between json bool and other json value is not compatible with TiDB and MySQL [#37481](https://github.com/pingcap/tidb/issues/37481) @[YangKeao](https://github.com/YangKeao) @@ -380,15 +385,15 @@ In v6.3.0-DMR, the key new features and improvements are as follows: - grpc: fix the wrong error handler [#5373](https://github.com/tikv/pd/issues/5373) @[bufferflies](https://github.com/bufferflies) - Fix the issue that unhealthy region cause panic [#5491](https://github.com/tikv/pd/issues/5491) @[nolouch](https://github.com/nolouch) - - Fix the bug where the Learner Peer of TiFlash Replica might not be created. [#5401](https://github.com/tikv/pd/issues/5401) @[HunDunDM](https://github.com/HunDunDM) + - Fix the bug where the Learner Peer of TiFlash Replica might not be created [#5401](https://github.com/tikv/pd/issues/5401) @[HunDunDM](https://github.com/HunDunDM) + TiFlash - compute - - Fix the bug that window function may cause tiflash crash when canceled. [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) + - Fix the bug that window function may cause tiflash crash when canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - fix error data input for date(CAST(value AS DATETIME)) causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) - - fix that the result of expression casting real or decimal as time is inconsistent with mysql. [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) + - fix that the result of expression casting real or decimal as time is inconsistent with mysql [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) - storage @@ -401,14 +406,14 @@ In v6.3.0-DMR, the key new features and improvements are as follows: + Backup & Restore (BR) - - Fix issues in "br/tests/up.sh". [#36743](https://github.com/pingcap/tidb/issues/36743) @[pingyu](https://github.com/pingyu) + - Fix issues in "br/tests/up.sh" [#36743](https://github.com/pingcap/tidb/issues/36743) @[pingyu](https://github.com/pingyu) - br: raw restore fail in integration test "br_rawkv [#36490](https://github.com/pingcap/tidb/issues/36490) @[pingyu](https://github.com/pingyu) - - Fix a bug that may cause the information of the checkpoint being stale. [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) - - Fix a bug caused when restoring with high `concurrency` the regions aren't balanced. [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) - - Fix a bug that may cause log backup checkpoint TS stuck when some weird ranged regions exist. [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) + - Fix a bug that may cause the information of the checkpoint being stale [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) + - Fix a bug caused when restoring with high `concurrency` the regions aren't balanced [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) + - Fix a bug that may cause log backup checkpoint TS stuck when some weird ranged regions exist [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) + TiCDC - + - handle error correctly with wrong pd address but with a grpc service [#6458](https://github.com/pingcap/tiflow/issues/6458) @[crelax](https://github.com/crelax) + TiDB Data Migration (DM) @@ -416,12 +421,12 @@ In v6.3.0-DMR, the key new features and improvements are as follows: - Fix a problem that DM will report `Specified key was too long` error [#5315](https://github.com/pingcap/tiflow/issues/5315) @[lance6716](https://github.com/lance6716) - Fix a bug that relay goroutine and upstream connections may leak when relay meet error [#6193](https://github.com/pingcap/tiflow/issues/6193) @[lance6716](https://github.com/lance6716) - Fix when use "strict" collation_compatible, DM sometimes generate SQL with duplicated collation [#6832](https://github.com/pingcap/tiflow/issues/6832) @[lance6716](https://github.com/lance6716) - - Reduce the appearing time of the warning message "found error when getting timezone from binlog status_vars" in dm-worker log. [#6628](https://github.com/pingcap/tiflow/issues/6628) @[lyzx2001](https://github.com/lyzx2001) + - Reduce the appearing time of the warning message "found error when getting timezone from binlog status_vars" in dm-worker log [#6628](https://github.com/pingcap/tiflow/issues/6628) @[lyzx2001](https://github.com/lyzx2001) - Fix a bug that latin1 data may be corrupt when replicating [#7028](https://github.com/pingcap/tiflow/issues/7028) @[lance6716](https://github.com/lance6716) + TiDB Lightning - - Fix the issue that TiDB Lightning does not support columns starting with slash, number, or non-ascii characters in Parquet files [#36980](https://github.com/pingcap/tidb/issues/36980) @[D3Hunter](https://github.com/D3Hunter) + - Fix the issue that TiDB Lightning does not support columns starting with slash, number, or non-ascii characters in Parquet files [#36980](https://github.com/pingcap/tidb/issues/36980) @[D3Hunter](https://github.com/D3Hunter) - TiUP @@ -431,4 +436,22 @@ In v6.3.0-DMR, the key new features and improvements are as follows: We would like to thank the following contributors from the TiDB community: -- [Contributor GitHub ID]() +- @[AntiTopQuark](https://github.com/AntiTopQuark) +- @[eltociear](https://github.com/eltociear) +- @[morgo](https://github.com/morgo) +- @[fuzhe1989](https://github.com/fuzhe1989) +- @[crelax](https://github.com/crelax) +- @[Ziy1-Tan](https://github.com/Ziy1-Tan) +- @[AnnieoftheStars](https://github.com/AnnieoftheStars) +- @[An-DJ](https://github.com/An-DJ) +- @[erwadba](https://github.com/erwadba) +- @[whitekeepwork](https://github.com/whitekeepwork) +- @[blacktear23](https://github.com/blacktear23) +- @[rzrymiak](https://github.com/rzrymiak) +- @[AnnieoftheStars](https://github.com/AnnieoftheStars) +- @[jianzhiyao](https://github.com/jianzhiyao) +- @[peakji](https://github.com/peakji) +- @[joycse06](https://github.com/joycse06) +- @[onlyacat](https://github.com/onlyacat) +- @[tisonkun](https://github.com/tisonkun) +- @[BurtonQin](https://github.com/BurtonQin): First-time contributor From e0218a0d660aa5e665285ac72e6b26acfd1f5dcd Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 22 Sep 2022 15:22:01 +0800 Subject: [PATCH 032/104] Apply suggestions from code review --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 41664c151e64a..322fb2d14f110 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -17,7 +17,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: - TiDB supports null-aware anti join. - TiDB provides execution time metrics at a finer granularity. - A new syntactic sugar is added to simplify Range partition definitions. -- Range Columns partitioning supports defining multiple columns. +- Range COLUMNS partitioning supports defining multiple columns. ## New features @@ -27,7 +27,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: [Provides INTERVAL partitioning as a new way of defining Range partitions](/partitioned-table.md#range-interval-partitioning). You do not need to enumerate all partitions, which drastically reduces the lengthy way of writing Range partition statements. The semantic is equivalent to the original Range partition. -* Range Columns partitioning supports defining multiple columns [#36636](https://github.com/pingcap/tidb/issues/36636) @[mjonss](https://github.com/mjonss) +* Range COLUMNS partitioning supports defining multiple columns [#36636](https://github.com/pingcap/tidb/issues/36636) @[mjonss](https://github.com/mjonss) Support [PARTITION BY RANGE COLUMNS (column_list)](/partitioned-table.md#range-columns-partitioning). `column_list` is no longer limited to a single column. The basic feature is the same as MySQL. From 9aa64374de58edb34bdb1c681fe2e5a373f7a0db Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 22 Sep 2022 15:39:08 +0800 Subject: [PATCH 033/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 322fb2d14f110..73e06b4f4ac9c 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -40,12 +40,10 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * `LEAD()` * `LAG()` -* The `CREATE USER` statement supports the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** +* The `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** When you create a user using the [`CREATE USER`](/sql-statements/sql-statement-create-user.md) statement, you can specify whether the created user is locked using the `ACCOUNT LOCK/UNLOCK` option. A locked user cannot log in to the database. -* The `ALTER USER` statement supports the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** - You can modify the lock state of an existing user using the `ACCOUNT LOCK/UNLOCK` option in the [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) statement. * JSON data type and JSON functions become GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) **tw: qiancai** From a29309876dc463c0479ec6a6a547a3ebe56e919e Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 22 Sep 2022 17:02:23 +0800 Subject: [PATCH 034/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 73e06b4f4ac9c..1371d146990e8 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -25,7 +25,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Add a new syntactic sugar (Range INTERVAL partitioning) to simplify Range partition definitions [#35683](https://github.com/pingcap/tidb/issues/35683) @[mjonss](https://github.com/mjonss) - [Provides INTERVAL partitioning as a new way of defining Range partitions](/partitioned-table.md#range-interval-partitioning). You do not need to enumerate all partitions, which drastically reduces the lengthy way of writing Range partition statements. The semantic is equivalent to the original Range partition. + [Provide INTERVAL partitioning as a new way of defining Range partitions](/partitioned-table.md#range-interval-partitioning). You do not need to enumerate all partitions, which drastically reduces the lengthy way of writing Range partition statements. The syntax is equivalent to the original Range partition. * Range COLUMNS partitioning supports defining multiple columns [#36636](https://github.com/pingcap/tidb/issues/36636) @[mjonss](https://github.com/mjonss) From 2e38d7bcfd6e759569366360e3a376d3ac162291 Mon Sep 17 00:00:00 2001 From: Ran Date: Thu, 22 Sep 2022 17:35:42 +0800 Subject: [PATCH 035/104] Apply suggestions from code review Co-authored-by: Aolin --- releases/release-6.3.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 1371d146990e8..35c874f7dd8ca 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -110,7 +110,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiKV supports log recycling [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) **tw：ran-huang** - TiKV supports recycling log files in Raft Engine. This reduces the long tail latency in network disks during Raft log appending and improves performance under write workloads. + TiKV supports [recycling log files](/tikv-configuration-file.md#enable-log-recycle-new-in-v630) in Raft Engine. This reduces the long tail latency in network disks during Raft log appending and improves performance under write workloads. * TiDB supports null-aware anti join [#37525](https://github.com/pingcap/tidb/issues/37525) @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** @@ -142,7 +142,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * The feature of disabling Titan becomes GA [#issue]() @[tabokie](https://github.com/tabokie) **tw：ran-huang** - You can [disable Titan](/titan-configuration#disable-titan) for online TiKV nodes. + You can [disable Titan](/titan-configuration.md#disable-titan) for online TiKV nodes. ### Ease of use @@ -170,7 +170,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiDB Lightning supports importing Parquet files exported by Apache Hive into TiDB [#issue]() @[buchuitoudegou](https://github.com/buchuitoudegou) **tw：ran-huang** - TiDB Lightning supports importing Parquet files exported by Apache Hive into TiDB, thereby achieving data migration from Hive to TiDB. + TiDB Lightning supports [importing Parquet files exported by Apache Hive into TiDB](/tidb-lightning/tidb-lightning-data-source.md#parquet), thereby achieving data migration from Hive to TiDB. * DM supports adding a field to a table migrated to TiDB and assigning values to the field [#3262](https://github.com/pingcap/tiflow/pull/3262), [#3340](https://github.com/pingcap/tiflow/issues/3340) @[yufan022](https://github.com/yufan022) **tw：ran-huang** @@ -188,7 +188,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiCDC supports graceful upgrade [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) @[3AceShowHand](https://github.com/3AceShowHand) **tw:ran-huang** - When TiCDC is deployed using TiUP (>=v1.11.0) or TiDB Operator (>=v1.3.8), you can gracefully upgrade the TiCDC cluster. During the upgrade, data replication latency is kept as low as 30 seconds. This improves stability, empowering TiCDC to better support latency-sensitive applications. + When TiCDC is deployed using [TiUP](/ticdc/deploy-ticdc.md#rolling-upgrade-ticdc-using-tiup) (>=v1.11.0) or [TiDB Operator](https://docs.pingcap.com/tidb-in-kubernetes/v1.3/configure-a-tidb-cluster#configure-graceful-upgrade-for-ticdc-cluster) (>=v1.3.8), you can gracefully upgrade the TiCDC cluster. During the upgrade, data replication latency is kept as low as 30 seconds. This improves stability, empowering TiCDC to better support latency-sensitive applications. ## Compatibility changes From d37c64ab45b149aabcf9e02e72cde44c32170563 Mon Sep 17 00:00:00 2001 From: long0712 <87366891+ljun0712@users.noreply.github.com> Date: Thu, 22 Sep 2022 18:12:32 +0800 Subject: [PATCH 036/104] Update releases/release-6.3.0.md Co-authored-by: xixirangrang --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 35c874f7dd8ca..925ef49c80014 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -62,7 +62,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiKV supports the SM4 algorithm for encryption at rest [#13041](https://github.com/tikv/tikv/issues/13041) @[jiayang-zheng](https://github.com/jiayang-zheng) - Add the [SM4 algorithm](/encryption-at-rest.md) for TiKV encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the "data-encryption-method" configuration to "sm4-ctr". + Add the [SM4 algorithm](/encryption-at-rest.md) for TiKV encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the `data-encryption-method` configuration to `sm4-ctr`. * TiFlash supports the SM4 algorithm for encryption at rest [#5714](https://github.com/pingcap/tiflash/issues/5714) @[lidezhu](https://github.com/lidezhu) From b5704412dcfe23697c40f3f239a9ac687b385441 Mon Sep 17 00:00:00 2001 From: long0712 <87366891+ljun0712@users.noreply.github.com> Date: Thu, 22 Sep 2022 18:13:35 +0800 Subject: [PATCH 037/104] Update releases/release-6.3.0.md Co-authored-by: xixirangrang --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 925ef49c80014..061558f33de90 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -66,7 +66,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiFlash supports the SM4 algorithm for encryption at rest [#5714](https://github.com/pingcap/tiflash/issues/5714) @[lidezhu](https://github.com/lidezhu) - Add the [SM4 algorithm](/encryption-at-rest.md) for TiFlash encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the "data-encryption-method" configuration to "sm4-ctr". + Add the [SM4 algorithm](/encryption-at-rest.md) for TiFlash encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the `data-encryption-method` configuration to `sm4-ctr`. * TiDB supports authentication with the SM3 algorithm [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** From 4e35da00d4f0f736fc3c9e4c3e93e45502ef6478 Mon Sep 17 00:00:00 2001 From: long0712 <87366891+ljun0712@users.noreply.github.com> Date: Thu, 22 Sep 2022 18:15:57 +0800 Subject: [PATCH 038/104] Update releases/release-6.3.0.md Co-authored-by: Aolin --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 061558f33de90..5351c6288eac9 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -72,7 +72,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiDB adds an authentication plugin [`tidb_sm3_password`](/system-variables.md#default_authentication_plugin) based on the SM3 algorithm. When this plugin is enabled, the user password is encrypted and validated using the SM3 algorithm. -* JDBC supports authentication with the SM3 algorithm [issue]() @[lastincisor](https://github.com/lastincisor) **tw：ran-huang** +* TiDB JDBC supports authentication with the SM3 algorithm [#25](https://github.com/pingcap/mysql-connector-j/issues/25) @[lastincisor](https://github.com/lastincisor) **tw：ran-huang** Authenticating the user password needs client-side support. Now because [JDBC supports the SM3 algorithm](/develop/dev-guide-choose-driver-or-orm.md#java-drivers), you can connect to TiDB using SM3 authentication via JDBC. From c483772aeb443d305df6f55a580e1f0e01f9fba1 Mon Sep 17 00:00:00 2001 From: long0712 <87366891+ljun0712@users.noreply.github.com> Date: Thu, 22 Sep 2022 18:22:05 +0800 Subject: [PATCH 039/104] Update releases/release-6.3.0.md Co-authored-by: Aolin --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 5351c6288eac9..28090da64c9d6 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -74,7 +74,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiDB JDBC supports authentication with the SM3 algorithm [#25](https://github.com/pingcap/mysql-connector-j/issues/25) @[lastincisor](https://github.com/lastincisor) **tw：ran-huang** - Authenticating the user password needs client-side support. Now because [JDBC supports the SM3 algorithm](/develop/dev-guide-choose-driver-or-orm.md#java-drivers), you can connect to TiDB using SM3 authentication via JDBC. + Authenticating the user password needs client-side support. Now because [JDBC supports the SM3 algorithm](/develop/dev-guide-choose-driver-or-orm.md#java-drivers), you can connect to TiDB using SM3 authentication via TiDB-JDBC. ### Observability From de5932de094cd41ab16e9e9663ad6b320ee35fcc Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 22 Sep 2022 18:55:46 +0800 Subject: [PATCH 040/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 28090da64c9d6..166d2390eabd0 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -48,7 +48,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * JSON data type and JSON functions become GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) **tw: qiancai** - JSON is a popular data format adopted by a large number of programs. TiDB has introduced the [JSON support](/data-type-json.md) as experimental since an earlier version, compatible with MySQL's JSON data type and some JSON functions. In v6.3.0, the JSON support becomes GA, providing TiDB with richer data types, and further improving TiDB compatibility with MySQL. + JSON is a popular data format adopted by a large number of programs. TiDB has introduced the [JSON support](/data-type-json.md) as an experimental feature since an earlier version, compatible with MySQL's JSON data type and some JSON functions. + +In v6.3.0, the JSON data type and functions become GA in TiDB, which enriches TiDB’s data types, supports using JSON functions in [expression indexes](/sql-statements/sql-statement-create-index.md#expression-index) and [generated-columns](/generated-columns.md), and further improves TiDB’s compatibility with MySQL. * Provide lightweight metadata lock to improve the DML success rate during DDL change (experimental) [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** From 6c04286ed9528c0e698ff80fb59e854ce38e9ca5 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 22 Sep 2022 18:56:23 +0800 Subject: [PATCH 041/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 166d2390eabd0..ffd96591e967b 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -50,7 +50,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: JSON is a popular data format adopted by a large number of programs. TiDB has introduced the [JSON support](/data-type-json.md) as an experimental feature since an earlier version, compatible with MySQL's JSON data type and some JSON functions. -In v6.3.0, the JSON data type and functions become GA in TiDB, which enriches TiDB’s data types, supports using JSON functions in [expression indexes](/sql-statements/sql-statement-create-index.md#expression-index) and [generated-columns](/generated-columns.md), and further improves TiDB’s compatibility with MySQL. + In v6.3.0, the JSON data type and functions become GA in TiDB, which enriches TiDB’s data types, supports using JSON functions in [expression indexes](/sql-statements/sql-statement-create-index.md#expression-index) and [generated-columns](/generated-columns.md), and further improves TiDB’s compatibility with MySQL. * Provide lightweight metadata lock to improve the DML success rate during DDL change (experimental) [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** From 3e50122105c068dc91024d8693aeee9da716b9fb Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 22 Sep 2022 19:00:49 +0800 Subject: [PATCH 042/104] Apply suggestions from code review Co-authored-by: xixirangrang --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index ffd96591e967b..bb95fe7491235 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -128,9 +128,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Transactions -* Support deferring unique constraints in pessimistic transactions [#36579](https://github.com/pingcap/tidb/issues/36579) @[ekexium](https://github.com/ekexium) **tw: qiancai** +* Support deferring checks of unique constraints in pessimistic transactions [#36579](https://github.com/pingcap/tidb/issues/36579) @[ekexium](https://github.com/ekexium) **tw: qiancai** - You can use the [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) system variable to control when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions. This variable is disabled by default. When the variable is set to `ON`, TiDB will defer locking operations and unique constraint checks in pessimistic transactions until necessary, thus improving the performance of bulk DML operations. + You can use the [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) system variable to control when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions. This variable is disabled by default. When the variable is enabled (set to `ON`), TiDB will defer locking operations and unique constraint checks in pessimistic transactions until necessary, thus improving the performance of bulk DML operations. * Optimize the way of fetching TSO in the Read-Committed isolation level [#36812](https://github.com/pingcap/tidb/issues/36812) @[TonsnakeLin](https://github.com/TonsnakeLin) **tw: TomShawn** From 24cee79c403a3e54ed27991d3760c6d1df8ef841 Mon Sep 17 00:00:00 2001 From: long0712 <87366891+ljun0712@users.noreply.github.com> Date: Thu, 22 Sep 2022 19:11:28 +0800 Subject: [PATCH 043/104] Update release-6.3.0.md --- releases/release-6.3.0.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index bb95fe7491235..f0cba340912e7 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -10,7 +10,7 @@ TiDB version: 6.3.0-DMR In v6.3.0-DMR, the key new features and improvements are as follows: -- TiKV and TiFlash support SM4 encryption at rest. +- TiKV support SM4 encryption at rest. - TiDB supports authentication with the SM3 algorithm. - The `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option. - JSON data type and functions become generally available (GA). @@ -66,10 +66,6 @@ In v6.3.0-DMR, the key new features and improvements are as follows: Add the [SM4 algorithm](/encryption-at-rest.md) for TiKV encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the `data-encryption-method` configuration to `sm4-ctr`. -* TiFlash supports the SM4 algorithm for encryption at rest [#5714](https://github.com/pingcap/tiflash/issues/5714) @[lidezhu](https://github.com/lidezhu) - - Add the [SM4 algorithm](/encryption-at-rest.md) for TiFlash encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the `data-encryption-method` configuration to `sm4-ctr`. - * TiDB supports authentication with the SM3 algorithm [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** TiDB adds an authentication plugin [`tidb_sm3_password`](/system-variables.md#default_authentication_plugin) based on the SM3 algorithm. When this plugin is enabled, the user password is encrypted and validated using the SM3 algorithm. From 836942cfe3df126df35c86b13643f0d25b88858e Mon Sep 17 00:00:00 2001 From: long0712 <87366891+ljun0712@users.noreply.github.com> Date: Thu, 22 Sep 2022 19:15:08 +0800 Subject: [PATCH 044/104] Update release-6.3.0.md --- releases/release-6.3.0.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index f0cba340912e7..b2059d672fbe0 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -68,7 +68,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiDB supports authentication with the SM3 algorithm [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** - TiDB adds an authentication plugin [`tidb_sm3_password`](/system-variables.md#default_authentication_plugin) based on the SM3 algorithm. When this plugin is enabled, the user password is encrypted and validated using the SM3 algorithm. + TiDB adds an authentication plugin [`tidb_sm3_password`](/security-compatibility-with-mysql.md) based on the SM3 algorithm. When this plugin is enabled, the user password is encrypted and validated using the SM3 algorithm. * TiDB JDBC supports authentication with the SM3 algorithm [#25](https://github.com/pingcap/mysql-connector-j/issues/25) @[lastincisor](https://github.com/lastincisor) **tw：ran-huang** @@ -153,6 +153,12 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Improve compatibility of SQL-based data Placement Rules [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature is not compatible with TiFlash due to conflicts in implementation mechanisms. TiDB v6.3.0 optimizes this feature, and [improves compatibility of SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). + +* Improve compatibility: the `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) + + When you create a user using the [`CREATE USER`](/sql-statements/sql-statement-create-user.md) statement, you can specify whether the created user is locked using the `ACCOUNT LOCK/UNLOCK` option. A locked user cannot log in to the database. + + You can modify the lock state of an existing user using the `ACCOUNT LOCK/UNLOCK` option in the [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) statement. ### Backup and restore From 533c92de7d7d0f579525c5027dc0bbd5d03bbc83 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 23 Sep 2022 11:29:04 +0800 Subject: [PATCH 045/104] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-6.3.0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index b2059d672fbe0..7e85f2696f885 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -25,7 +25,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Add a new syntactic sugar (Range INTERVAL partitioning) to simplify Range partition definitions [#35683](https://github.com/pingcap/tidb/issues/35683) @[mjonss](https://github.com/mjonss) - [Provide INTERVAL partitioning as a new way of defining Range partitions](/partitioned-table.md#range-interval-partitioning). You do not need to enumerate all partitions, which drastically reduces the lengthy way of writing Range partition statements. The syntax is equivalent to the original Range partition. + TiDB provides [INTERVAL partitioning](/partitioned-table.md#range-interval-partitioning) as a new way of defining Range partitions. You do not need to enumerate all partitions, which drastically reduces the length of Range partitioning DDL statements. The syntax is equivalent to that of the original Range partitioning. * Range COLUMNS partitioning supports defining multiple columns [#36636](https://github.com/pingcap/tidb/issues/36636) @[mjonss](https://github.com/mjonss) @@ -90,21 +90,21 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Performance -* TiFlash changes the way FastScan is used (experimental) [#5252](https://github.com/pingcap/tiflash/issues/5252) @[hongyunyan](https://github.com/hongyunyan) +* TiFlash changes the way of using FastScan (experimental) [#5252](https://github.com/pingcap/tiflash/issues/5252) @[hongyunyan](https://github.com/hongyunyan) - FastScan, introduced in TiFlash starting with v6.2.0, performed as expected, but lacked flexibility. Therefore, in v6.3.0 [the way of using FastScan has changed](/develop/dev-guide-use-fastscan.md). TiFlash does not support using the switch to control the FastScan feature. Instead, it uses the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) to control whether to enable the FastScan function. + In v6.2.0, TiFlash introduces the FastScan feature, which brings expected performance improvements but lacks flexibility in use. Therefore, in v6.3.0, TiFlash changes [the way of using FastScan](/develop/dev-guide-use-fastscan.md): the `ALTER TABLE ... SET TIFLASH MODE ...` syntax to enable or disable FastScan is deprecated. Instead, you can use the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) to easily control whether to enable FastScan. When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. You need to set the variable [`tiflash_fastscan`]. When you upgrade from an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistency. * TiFlash optimizes to improve data scanning performance in multiple concurrency scenarios [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) - TiFlash reduces duplicate reads of the same data by combining read operations of the same data. It optimizes the resource overhead and [improves the performance of data scanning in the case of concurrent tasks](/tiflash/tiflash-configuration.md#configure-the-tiflashtoml-file). It avoids the situation where the same data needs to be read separately in each task in multiple concurrent tasks, and avoids the possibility of multiple reads of the same data at the same time. + TiFlash reduces duplicate reads of the same data by combining read operations of the same data. It optimizes the resource overhead and [improves the performance of data scanning in the case of concurrent tasks](/tiflash/tiflash-configuration.md#configure-the-tiflashtoml-file). For multiple concurrent tasks, it avoids the situation where each task needs to read the same data separately, and avoids the possibility of multiple reads of the same data at the same time. This feature is experimental in v6.2.0, and becomes GA in v6.3.0. * Improve performance of TiFlash data replication [#5237](https://github.com/pingcap/tiflash/issues/5237) @[breezewish](https://github.com/breezewish) - TiFlash uses the Raft protocol for data replication with TiKV. Prior to v6.3.0, it often took long time to replicate large amounts of replica data. TiDB v6.3.0 optimizes the TiFlash data replication mechanism and significantly improves the replication speed. When you use BR to recover data, use TiDB Lightning to import data, or add new TiFlash replicas, the replicas can be replicated more quickly. You can query with TiFlash in a more timely manner. In addition, TiFlash replicas will also reach a secure and balanced state faster when you scale up, scale down, or modify the number of TiFlash replicas. + TiFlash uses the Raft protocol for data replication from TiKV. Prior to v6.3.0, it often took a long time to replicate large amounts of replica data. TiDB v6.3.0 optimizes the TiFlash data replication mechanism and significantly improves the replication speed. When you use BR to recover data, use TiDB Lightning to import data, or add new TiFlash replicas, the TiFlash replicas can be replicated more quickly. You can query with TiFlash in a more timely manner. In addition, TiFlash replicas will also reach a secure and balanced state faster when you scale up, scale down, or modify the number of TiFlash replicas. * TiKV supports log recycling [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) **tw：ran-huang** From 2356062c5847d03f250a533b6eb4abfe2ff27389 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 23 Sep 2022 11:33:04 +0800 Subject: [PATCH 046/104] Update releases/release-6.3.0.md Co-authored-by: Grace Cai --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 7e85f2696f885..f6ff9485164d1 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -96,7 +96,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. You need to set the variable [`tiflash_fastscan`]. When you upgrade from an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistency. -* TiFlash optimizes to improve data scanning performance in multiple concurrency scenarios [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) +* TiFlash optimizes data scanning performance in scenarios of multiple concurrency tasks [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) TiFlash reduces duplicate reads of the same data by combining read operations of the same data. It optimizes the resource overhead and [improves the performance of data scanning in the case of concurrent tasks](/tiflash/tiflash-configuration.md#configure-the-tiflashtoml-file). For multiple concurrent tasks, it avoids the situation where each task needs to read the same data separately, and avoids the possibility of multiple reads of the same data at the same time. From e23ce93066e463728da21c640a1e532cfc0f1497 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 23 Sep 2022 11:39:43 +0800 Subject: [PATCH 047/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index f6ff9485164d1..3d40f03957134 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -50,7 +50,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: JSON is a popular data format adopted by a large number of programs. TiDB has introduced the [JSON support](/data-type-json.md) as an experimental feature since an earlier version, compatible with MySQL's JSON data type and some JSON functions. - In v6.3.0, the JSON data type and functions become GA in TiDB, which enriches TiDB’s data types, supports using JSON functions in [expression indexes](/sql-statements/sql-statement-create-index.md#expression-index) and [generated-columns](/generated-columns.md), and further improves TiDB’s compatibility with MySQL. + In TiDB v6.3.0, the JSON data type and functions become GA, which enriches TiDB’s data types, supports using JSON functions in [expression indexes](/sql-statements/sql-statement-create-index.md#expression-index) and [generated-columns](/generated-columns.md), and further improves TiDB’s compatibility with MySQL. * Provide lightweight metadata lock to improve the DML success rate during DDL change (experimental) [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** From 8b08722664157e815531de74949e56eb990fd1f4 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Fri, 23 Sep 2022 15:22:14 +0800 Subject: [PATCH 048/104] refine --- releases/release-6.3.0.md | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 3d40f03957134..4de5db15d8195 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -40,18 +40,6 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * `LEAD()` * `LAG()` -* The `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** - - When you create a user using the [`CREATE USER`](/sql-statements/sql-statement-create-user.md) statement, you can specify whether the created user is locked using the `ACCOUNT LOCK/UNLOCK` option. A locked user cannot log in to the database. - - You can modify the lock state of an existing user using the `ACCOUNT LOCK/UNLOCK` option in the [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) statement. - -* JSON data type and JSON functions become GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) **tw: qiancai** - - JSON is a popular data format adopted by a large number of programs. TiDB has introduced the [JSON support](/data-type-json.md) as an experimental feature since an earlier version, compatible with MySQL's JSON data type and some JSON functions. - - In TiDB v6.3.0, the JSON data type and functions become GA, which enriches TiDB’s data types, supports using JSON functions in [expression indexes](/sql-statements/sql-statement-create-index.md#expression-index) and [generated-columns](/generated-columns.md), and further improves TiDB’s compatibility with MySQL. - * Provide lightweight metadata lock to improve the DML success rate during DDL change (experimental) [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** TiDB uses the online asynchronous schema change algorithm to support changing metadata objects. When a transaction is executed, it obtains the corresponding metadata snapshot at the transaction start. If the metadata is changed during a transaction, to ensure data consistency, TiDB returns an `Information schema is changed` error and the transaction fails to commit. To solve the problem, TiDB v6.3.0 introduces [metadata lock](/metadata-lock.md) into the online DDL algorithm. To avoid most DML errors, TiDB coordinates the priority of DMLs and DDLs during table metadata change and makes executing DDLs wait for the DMLs with old metadata to commit. @@ -142,6 +130,10 @@ In v6.3.0-DMR, the key new features and improvements are as follows: You can [disable Titan](/titan-configuration.md#disable-titan) for online TiKV nodes. +* Use `static` partition pruning when GlobalStats are not ready [#37535](https://github.com/pingcap/tidb/issues/37535) @[Yisaer](https://github.com/Yisaer) + + When [`dynamic pruning`](/partitioned-table.md#dynamic-pruning-mode) is enabled, the optimizer selects execution plans based on [GlobalStats](/statistics.md#collect-statistics-of-partitioned-tables-in-dynamic-pruning-mode). Before GlobalStats are fully collected, using pseudo statistics might cause performance regression. In v6.3.0, this issue is addressed by maintaining the `static` mode if you enable dynamic pruning before GlobalStats are collected. TiDB remains in the `static` mode until GlobalStats are collected. This ensures performance stability when you change the partition pruning settings. + ### Ease of use ### MySQL compatibility @@ -153,13 +145,19 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Improve compatibility of SQL-based data Placement Rules [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature is not compatible with TiFlash due to conflicts in implementation mechanisms. TiDB v6.3.0 optimizes this feature, and [improves compatibility of SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). - -* Improve compatibility: the `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) - + +* The `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** + When you create a user using the [`CREATE USER`](/sql-statements/sql-statement-create-user.md) statement, you can specify whether the created user is locked using the `ACCOUNT LOCK/UNLOCK` option. A locked user cannot log in to the database. You can modify the lock state of an existing user using the `ACCOUNT LOCK/UNLOCK` option in the [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) statement. +* JSON data type and JSON functions become GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) **tw: qiancai** + + JSON is a popular data format adopted by a large number of programs. TiDB has introduced the [JSON support](/data-type-json.md) as an experimental feature since an earlier version, compatible with MySQL's JSON data type and some JSON functions. + + In TiDB v6.3.0, the JSON data type and functions become GA, which enriches TiDB’s data types, supports using JSON functions in [expression indexes](/sql-statements/sql-statement-create-index.md#expression-index) and [generated-columns](/generated-columns.md), and further improves TiDB’s compatibility with MySQL. + ### Backup and restore * PITR supports GCS and Azure Blob Storage as backup storage [#issue]() @[joccau](https://github.com/joccau) **tw: shichun-0415** @@ -216,6 +214,11 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Others +* Improve MySQL compatibility by supporting the `ACCOUNT LOCK` and `ACCOUNT UNLOCK` options. +* Log backup supports GCS and Azure Blob Storage as backup storage. +* Log backup is now compatible with the `exchange partition` DDL. +* The SQL statement `ALTER TABLE ...SET TiFLASH MODE ...` previously used for enabling [fastscan](/develop/dev-guide-use-fastscan.md) is deprecated, and replaced by the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630). When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. You need to set the variable [`tiflash_fastscan`]. When you upgrade from an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistency. + ## Removed feature ## Improvements From 32b2098f7ddabc0dd7ee1646578dd3a0addd4485 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Fri, 23 Sep 2022 15:31:34 +0800 Subject: [PATCH 049/104] Apply suggestions from code review Co-authored-by: Aolin Co-authored-by: xixirangrang Co-authored-by: Ran --- releases/release-6.3.0.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 4de5db15d8195..6b508d19cf0f1 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -2,7 +2,7 @@ title: TiDB 6.3.0 Release Notes --- -# TiDB v6.3.0 Release Notes +# TiDB 6.3.0 Release Notes Release date: xx xx, 2022 @@ -74,7 +74,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiDB Dashboard provides deadlock history information [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: shichun-0415** - From v6.3.0, TiDB Dashboard will provide deadlock history. If you find excessively long SQL lock waiting by analyzing slow logs or other information on TiDB Dashboard, you can turn to deadlock history for problem locating, which delivers better diagnosis experience. + From v6.3.0, TiDB Dashboard provides deadlock history. If you find excessively long SQL lock waiting by analyzing slow logs or other information on TiDB Dashboard, you can check the deadlock history to locate the root cause, which delivers better diagnosis experience. ### Performance @@ -82,7 +82,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: In v6.2.0, TiFlash introduces the FastScan feature, which brings expected performance improvements but lacks flexibility in use. Therefore, in v6.3.0, TiFlash changes [the way of using FastScan](/develop/dev-guide-use-fastscan.md): the `ALTER TABLE ... SET TIFLASH MODE ...` syntax to enable or disable FastScan is deprecated. Instead, you can use the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) to easily control whether to enable FastScan. - When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. You need to set the variable [`tiflash_fastscan`]. When you upgrade from an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistency. + When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. You need to set the variable [`tiflash_fastscan`]. When you upgrade from v6.2.0 or an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistency. * TiFlash optimizes data scanning performance in scenarios of multiple concurrency tasks [#5376](https://github.com/pingcap/tiflash/issues/5376) @[JinheLin](https://github.com/JinheLin) @@ -118,7 +118,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Optimize the way of fetching TSO in the Read-Committed isolation level [#36812](https://github.com/pingcap/tidb/issues/36812) @[TonsnakeLin](https://github.com/TonsnakeLin) **tw: TomShawn** - In the Read-Committed isolation level, the system variable [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) is introduced to control how TSO is fetched. In the case of Plan Cache hit, TiDB improves the execution efficiency of batch DML statements by reducing the frequency of fetching TSO, and reduces the execution time of running tasks in batches. + In the Read-Committed isolation level, the system variable [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) is introduced to control how TSO is fetched. In the case of Plan Cache hit, TiDB improves the execution efficiency of batch DML statements by reducing the frequency of fetching TSO, and reduces the execution time of running tasks in batch. ### Stability @@ -138,7 +138,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### MySQL compatibility -* Improve MySQL 8.0 compatibility by adding support for `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` regular expression functions [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** +* Improve MySQL 8.0 compatibility by adding support for four regular expression functions: `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** For more details about the compatibility with MySQL, see [Regular expression compatibility with MySQL](/functions-and-operators/string-functions.md#regular-expression-compatibility-with-mysql). @@ -166,7 +166,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * BR supports AWS S3 Object Lock [#issue]() @[3pointer](https://github.com/3pointer) **tw: shichun-0415** - After enabling [Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html) , you can protect backup data from being tampered with or deleted. + After enabling [S3 Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html), you can protect backup data from being tampered with or deleted. ### Data migration @@ -182,7 +182,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiCDC supports a deployment topology that can replicate data from multiple geo-distributed data sources [#issue]() @[sdojjy](https://github.com/sdojjy) **tw：ran-huang** - To support replicating data from a single TiDB cluster to multiple geo-distributed data systems, starting from v6.3.0, [TiCDC can be deployed in multiple IDCs](link) to replicate data for each IDC. This feature helps deliver the capability of geo-distributed data replication and deployment topology. + To support replicating data from a single TiDB cluster to multiple geo-distributed data systems, starting from v6.3.0, [you can deploy TiCDC in multiple IDCs](link) to replicate data for each IDC. This feature helps deliver the capability of geo-distributed data replication and deployment topology. * TiCDC supports keeping the snapshots consistent between the upstream and the downstream (sync point) [#issue]() @[asddongmen](https://github.com/asddongmen) **tw: TomShawn** @@ -221,6 +221,8 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ## Removed feature +Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://github.com/streamnative/kop) provided by StreamNative can be used as an alternative. + ## Improvements + TiDB From 6f3dc9994e4c435084abff0d80a0e94f15d51e09 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Sun, 25 Sep 2022 18:51:41 +0800 Subject: [PATCH 050/104] add compatibility changes --- releases/release-6.3.0.md | 44 ++++++++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 10 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 6b508d19cf0f1..10ab5989b4b1d 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -174,9 +174,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiDB Lightning supports [importing Parquet files exported by Apache Hive into TiDB](/tidb-lightning/tidb-lightning-data-source.md#parquet), thereby achieving data migration from Hive to TiDB. -* DM supports adding a field to a table migrated to TiDB and assigning values to the field [#3262](https://github.com/pingcap/tiflow/pull/3262), [#3340](https://github.com/pingcap/tiflow/issues/3340) @[yufan022](https://github.com/yufan022) **tw：ran-huang** +* DM adds a new configuration item `safe-mode-duration` in the task configuration file [#6224] (https://github.com/pingcap/tiflow/issues/6224) @[[okJiang](https://github.com/okJiang)] **tw：ran-huang** - DM supports [adding a field to a table migrated to TiDB and assigning values to the field](link). When you merge and migrate MySQL shards to TiDB, you can use the field to distinguish which shard the record is migrated from. + DM adds a new configuration item `safe-mode-duration` in the [task configuration file](/task-configuration-file-full.md). You can adjust the automatic safe mode duration after DM exits abnormally. The default value is 60 seconds. When `safe-mode-duration` is set to `"0s"`, DM does not automatically enter safe mode after an abnormal restart. ### TiDB data share subscription @@ -198,19 +198,43 @@ In v6.3.0-DMR, the key new features and improvements are as follows: | Variable name | Change type (newly added, modified, or deleted) | Description | | --- | --- | --- | -| | | | -| | | | -| | | | -| | | | +| [`default_authentication_plugin`](/system-variables.md#default_authentication_plugin) | Modified | Added a new option `tidb_sm3_password`. When this variable is set to `tidb_sm3_password`, the `tidb_sm3_password` method is used as the encryption algorithm. | +| [`tidb_adaptive_closest_read_threshold`](/system-variables.md#tidb_adaptive_closest_read_threshold-new-in-v630) | Newly added | This variable is used to control the threshold at which the TiDB server prefers to send read requests to the replica in the same region as the TiDB server when [`tidb_replica_read`](#tidb_replica_read-new-in-v40) is set to `closest-adaptive`. | +| [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) | Newly added | This variable is used to control when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions | +| [`tidb_ddl_disk_quota`](/system-variables.md#tidb_ddl_disk_quota-new-in-v630) | Newly added | This variable only takes effect when [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) is enabled. It sets the usage limit of local storage during backfilling when creating an index. | +| [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) | Newly added | This variable controls whether to enable the acceleration of `ADD INDEX` and `CREATE INDEX` DDl operations to improve the speed of backfilling when creating an index. | +| [`tidb_enable_clustered_index`](/system-variables.md#tidb_enable_clustered_index-new-in-v50) | Modified | The default value of this variable changes from `INT_ONLY` to `ON`. | +| [`tidb_enable_exchange_partition`](/system-variables.md#tidb_enable_exchange_partition) | Deprecated | This variable controls whether to enable the [`exchange partitions with tables`](/partitioned-table.md#partition-management) feature. The default value is `ON`, that is, exchange partitions with tables is enabled by default. | +| [`tidb_enable_metadata_lock`](/system-variables.md#tidb_enable_metadata_lock-new-in-v630)| Newly added | This variable is used to set whether to enable the [Metadata lock](/metadata-lock.md) feature. | +| [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Modified | This variable controls the behavior of the optimizer on using statistics of a table when the statistics are outdated. The default value changes from `ON` to `OFF`, which means the optimizer still keeps using the statistics of the table even if the statistics of this table is outdated. | +| [`tidb_enable_rate_limit_action`](/system-variables.md#tidb_enable_rate_limit_action) | Modified | This variable controls whether to enable the dynamic memory control feature for the operator that reads data. When this variable is set to `ON`, the memory usage might not be under the control of [tidb_mem_quota_query](/system-variables.md#tidb_mem_quota_query). Therefore, the default value is changed from `ON` to `OFF`. | +| [`tidb_enable_unsafe_substitute`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | This variable controls whether to replace expressions with generated columns in an unsafe way. | +| [`tidb_last_plan_replayer_token`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | This variable is read-only and is used to obtain the result of the last `PLAN REPLAYER DUMP` execution in the current session. | +| [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) | Newly added | This variable is used to control whether common table expressions (CTEs) in the entire session are inlined or not. The default value is `OFF`, which means that inlining CTE is not enforced by default. | +| [`tidb_opt_three_stage_distinct_agg`](/system-variables.md#tidb_opt_three_stage_distinct_agg-new-in-v630) | Newly added | This variable specifies whether to rewrite a `COUNT(DISTINCT)` aggregation into a three-stage aggregation in MPP mode. The default value is `ON`. | +| [`tidb_partition_prune_mode`](/system-variables.md#tidb_partition_prune_mode-从-v51-版本开始引入) | Modified | Specifies whether to enable dynamic pruning. Since v6.3.0, the default value changes to `dynamic`. | +| [`tidb_rc_read_check_ts`](/system-variables.md#tidb_rc_read_check_ts-new-in-v600) | Modified | This variable is used to optimize the timestamp acquisition, which is suitable for scenarios with read-committed isolation level where read-write conflicts are rare. This feature is oriented to specific service workloads and might cause performance regression in other scenarios. For this reason, since v6.3.0, the scope of this variable changes from `GLOBAL | SESSION` to `INSTANCE`. That means you can enable this feature for specific TiDB instances. | +| [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) | Newly added | This variable is used to optimize the acquisition of timestamps and is suitable for scenarios with few point-write conflicts in `READ-COMMITTED` isolation level of pessimistic transactions. Enabling this variable can avoid the latency and overhead brought by obtaining the global timestamps during the execution of point-write statements | +| [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) | Newly added | This variable controls whether to enable FastScan. If [FastScan](/develop/dev-guide-use-fastscan.md) is enabled (set to `ON`), TiFlash provides more efficient query performance, but does not guarantee the accuracy of the query results or data consistency. | ### Configuration file parameters | Configuration file | Configuration | Change type | Description | | --- | --- | --- | --- | -| | | | | -| | | | | -| | | | | -| | | | | +| TiDB | [`temp-dir`](/tidb-configuration-file.md#temp-dir-new-in-v630) | Newly added | Specifies the file system location used by TiDB to store temporary data. If a feature requires local storage in TiDB nodes, TiDB stores the corresponding temporary data in this location. The default value is `/tmp/tidb`. | +| TiKV | [`auto-adjust-pool-size`](/tikv-configuration-file.md#auto-adjust-pool-size-new-in-v630) | Newly added | Controls whether to automatically adjust the thread pool size. When it is enabled, the read performance of TiKV is optimized by automatically adjusting the UnifyReadPool thread pool size based on the current CPU usage.| +| TiKV | [`data-encryption-method`](/tikv-configuration-file.md#data-encryption-method) | Modified | Introduces a new value option `sm4-ctr`. When this configuration item is set to `sm4-ctr`, data is encrypted using SM4 before being stored. | +| TiKV | [`enable-log-recycle`](/tikv-configuration-file.md#enable-log-recycle-new-in-v630) | Newly added | Determines whether to recycle stale log files in Raft Engine. When it is enabled, logically purged log files will be reserved for recycling. This reduces the long tail latency on write workloads. This configuration item is only available when [format-version](/tikv-configuration-file.md#format-version-new-in-v630) is >= 2. | +| TiKV | [`format-version`](/tikv-configuration-file.md#format-version-new-in-v630) | Newly added | Specifies the version of log files in Raft Engine. The default log file version is `1` for TiKV earlier than v6.3.0. The log files can be read by TiKV >= v6.1.0. The default log file version is `2` for TiKV v6.3.0 and later. TiKV v6.3.0 and later can read the log files. | +| TiKV | [`log-backup.enable`](/tikv-configuration-file.md#enable-new-in-v620) | Modified | Since v6.3.0, the default value changes from `false` to `true`. | +| TiKV | [`log-backup.max-flush-interval`](/tikv-configuration-file.md#max-flush-interval-new-in-v620) | Modified | Since v6.3.0, the default value changes from `5min` to `3min`. | +| PD | [enable-diagnostic](/pd-configuration-file.md#enable-diagnostic-new-in-v630) | Newly added | Controls whether to enable the diagnostic feature. The default value is `false`. | +| TiFlash | [`dt_enable_read_thread`](/tiflash/tiflash-configuration.md#configure-the-tiflash-learnertoml-file) | Deprecated | Since v6.3.0, this configuration item is deprecated. The thread pool is used to handle read requests from the storage engine by default and cannot be disabled. | +| DM | [`safe-mode-duration`](/dm/task-configuration-file-full.md#task-configuration-file-template-advanced) | Newly added | Specifies the duration of the automatic safe mode. | +| TiCDC | [`enable-sync-point`](/ticdc/manage-ticdc.md#task-configuration-file) | Newly added | Specifies whether to enable the Syncpoint feature. | +| TiCDC | [`sync-point-interval`](/ticdc/manage-ticdc.md#task-configuration-file) | Newly added | Specifies the interval at which Syncpoint aligns the upstream and downstream snapshots. | +| TiCDC | [`sync-point-retention`](/ticdc/manage-ticdc.md#task-configuration-file) | Newly added | Specifies how long the data is retained by Syncpoint in the downstream table. When this duration is exceeded, the data is cleaned up. | +| TiCDC | [`sink-uri.memory`](/ticdc/manage-ticdc.md#create-a-replication-task) | Deprecated | This configuration item is deprecated. It is not recommended to use it in any situation. | ### Others From b0e3aa99bb01082053a7a1470e8febc7862e89e4 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Mon, 26 Sep 2022 12:59:23 +0800 Subject: [PATCH 051/104] Apply suggestions from code review Co-authored-by: tangenta Co-authored-by: ekexium --- releases/release-6.3.0.md | 36 ++++++++++++++---------------------- 1 file changed, 14 insertions(+), 22 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 10ab5989b4b1d..ceffafaaf2290 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -251,16 +251,13 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiDB - - One `PLAN REPLAYER` command can export information about execution plans of multiple SQL statements, which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) - - improve warning log when new connection arrives [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) + - [`PLAN REPLAYER`](/sql-plan-replayer) can be used on multiple SQL statements, which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) + - Improve warning log when new connection arrives [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) - sql-infra - - Extend partitioning syntax with INTERVAL for easier partitioning definition [#35827](https://github.com/pingcap/tidb/issues/35827) @[ymkzpx](https://github.com/ymkzpx) - Grant privilege of a table to an user checks the target table exist first, in the past, the table name comparison works in a case sensitive manner, now it's changed to case insensitive [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) - - Support AWS NLB proxy protocol [#36312](https://github.com/pingcap/tidb/issues/36312) @[hawkingrei](https://github.com/hawkingrei) - Previously, TiDB users can set `init_connect` without any checking. From now on, the value of `init_connect` should be checked by the sql parser [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) - - Add support `flashback database` command [#37386](https://github.com/pingcap/tidb/issues/37386) @[tiancaiamao](https://github.com/tiancaiamao) - execution @@ -351,26 +348,19 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - sql-infra - - fix the bug prepare will not check privilege [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) - - When set `tidb_enable_noop_variable` to `WARN`, an error will be returned [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that `PREAPRE` statements do not check privileges [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) + - System variable `tidb_enable_noop_variable` cannot be set to `WARN` [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) - Fix the issue that when 'expression index' is defined, the value of `ORDINAL_POSITION` column of `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) - Fix the issue that when setting a timestamp that is larger than `MAXINT32`, TiDB doesn't report an error like MySQL [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) - - flashback cluster shouldn't support expr in timestamp [#37495](https://github.com/pingcap/tidb/issues/37495) @[Defined2014](https://github.com/Defined2014) - - Fix panic of enterprise plugin on 6.1 [#37319](https://github.com/pingcap/tidb/issues/37319) - - Fix incorrect output of `show create placement policy` with a policy of double quotes [#37526](https://github.com/pingcap/tidb/issues/37526) @[xhebox](https://github.com/xhebox) - - store flashback history in TiKV, avoid overlapped flashback TS range [#37585](https://github.com/pingcap/tidb/issues/37585) @[Defined2014](https://github.com/Defined2014) - - When exchange partition with temporary table, an error will be returned [#37201](https://github.com/pingcap/tidb/issues/37201) - - planner: fix partition table getting error result when select `TIKV_REGION_STATUS` with `table_id` [#37436](https://github.com/pingcap/tidb/issues/37436) @[zimulala](https://github.com/zimulala) - - In test_driver, parser didn't deal with RestoreStringWithoutCharset and RestoreStringWithoutDefaultCharset flags, add support for those two flags [#37175](https://github.com/pingcap/tidb/issues/37175) @[Defined2014](https://github.com/Defined2014) - - planner: fix show View Privilege behave for view table [#34326](https://github.com/pingcap/tidb/issues/34326) @[hawkingrei](https://github.com/hawkingrei) - - Support send flashback RPC [#37651](https://github.com/pingcap/tidb/issues/37651) @[Defined2014](https://github.com/Defined2014) - - Fix a wrong casting in building union plan [#31678](https://github.com/pingcap/tidb/issues/31678) @[bb7133](https://github.com/bb7133) - - support some adminStmt in read-only mode [#37631](https://github.com/pingcap/tidb/issues/37631) @[Defined2014](https://github.com/Defined2014) - - fix resume pd schedule and cancel for `flashback cluster` [#37584](https://github.com/pingcap/tidb/issues/37584) @[Defined2014](https://github.com/Defined2014) - - fix resume pd schedule and cancel for `flashback cluster` [#37580](https://github.com/pingcap/tidb/issues/37580) @[Defined2014](https://github.com/Defined2014) - - Fix the issue that the user cannot update from json 'null' to NULL [#37852](https://github.com/pingcap/tidb/issues/37852) @[YangKeao](https://github.com/YangKeao) + - Fix the panic issue of enterprise plugin on 6.1 [#37319](https://github.com/pingcap/tidb/issues/37319) @[xhebox](https://github.com/xhebox) + - Fix the incorrect output of `SHOW CREATE PLACEMENT POLICY` [#37526](https://github.com/pingcap/tidb/issues/37526) @[xhebox](https://github.com/xhebox) + - Disallow exchange partition with temporary table [#37201](https://github.com/pingcap/tidb/issues/37201) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that query on `INFORMATION_SCHEMA.TIKV_REGION_STATUS` returns an incorrect result @[zimulala](https://github.com/zimulala) + - Fix the issue that `EXPLAIN` query on views does not check privileges [#34326](https://github.com/pingcap/tidb/issues/34326) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that the user cannot update from JSON 'null' to NULL [#37852](https://github.com/pingcap/tidb/issues/37852) @[YangKeao](https://github.com/YangKeao) - Optimize DDL history HTTP API, and add support for 'start_job_id' parameter [#35838](https://github.com/pingcap/tidb/issues/35838) @[tiancaiamao](https://github.com/tiancaiamao) - - fix inaccuate row_count num [#25968](https://github.com/pingcap/tidb/issues/25968) @[Defined2014](https://github.com/Defined2014) + - Fix the issue that `row_count` of DDL jobs is inaccurate [#25968](https://github.com/pingcap/tidb/issues/25968) @[Defined2014](https://github.com/Defined2014) + - Fix the issue that `FLASHBACK TABLE` does not work properly [#37386](https://github.com/pingcap/tidb/issues/37386) @[tiancaiamao](https://github.com/tiancaiamao) - execution @@ -383,6 +373,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - bugfix: do not acquire pessimistic lock for non-unique index keys [#36235](https://github.com/pingcap/tidb/issues/36235) - Fix the auto-commit mode change related transaction commit behaviours [#36581](https://github.com/pingcap/tidb/issues/36581) @[cfzjywxk](https://github.com/cfzjywxk) + - Fix the issue explain analyze with DML executors may respond to the client before the transaction commit has finished [#37273](https://github.com/pingcap/tidb/issues/37373) @[cfzjywxk](https://github.com/cfzjywxk) - planner @@ -390,6 +381,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - planner: fix outer join reorder will push down its outer join condition [#37238](https://github.com/pingcap/tidb/issues/37238) @[AilinKid](https://github.com/AilinKid) - make the both side operand of NAAJ & refuse partial column substitute in projection elimination [#37032](https://github.com/pingcap/tidb/issues/37032) @[AilinKid](https://github.com/AilinKid) - planner: correct the redundant field meaning in join full schema when join coalesce [#36420](https://github.com/pingcap/tidb/issues/36420) @[AilinKid](https://github.com/AilinKid) + - Fix a wrong casting in building union plan [#31678](https://github.com/pingcap/tidb/issues/31678) @[bb7133](https://github.com/bb7133) - diagnosis From c47cba367a148c50424fb6e71c2edd3ac2ee16df Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Mon, 26 Sep 2022 13:26:08 +0800 Subject: [PATCH 052/104] fix notes for tiflash (compute) --- releases/release-6.3.0.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index ceffafaaf2290..809dc83cbf8cd 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -288,13 +288,9 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Support to pushdown elt to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) - Support to pushdown leftShift to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) - Support to pushdown castTimeAsDuration to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - - Support to pushdown castTimeAsDuration to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - Support Planner Interpreter [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - - Support to pushdown hexInt and hexStr to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107), [#5462](https://github.com/pingcap/tiflash/issues/5462) - - Support to pushdown elt to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) - - Support to pushdown shiftLeft to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) + - Support to pushdown hex to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) - Suppress the "tcp set inq" loggings [#4940](https://github.com/pingcap/tiflash/issues/4940) - - Support to pushdown CastTimeAsDuration to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - storage From e8951e9d70258210f4bf9c95c8abe40aca211532 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Mon, 26 Sep 2022 13:35:29 +0800 Subject: [PATCH 053/104] remove notes for tidb dashboard --- releases/release-6.3.0.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 809dc83cbf8cd..5b6e187edc1af 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -298,12 +298,6 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + Tools - + TiDB Dashboard - - - Optimize the display of TiDB Dashboard [#issue]() @[Contributor GitHub ID]() - - Display the row count on SQL statement summary and slow query pages [#issue]() @[Contributor GitHub ID]() - - Optimize the display of some error messages [#issue]() @[Contributor GitHub ID]() - + Backup & Restore (BR) - note [#issue]() @[Contributor GitHub ID]() From c603c33c9ee54af3fa1d4b9ece35d40a53bdd2ae Mon Sep 17 00:00:00 2001 From: Aolin Date: Mon, 26 Sep 2022 13:56:08 +0800 Subject: [PATCH 054/104] Apply suggestions from code review Co-authored-by: xixirangrang --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 5b6e187edc1af..2d1525267d99f 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -42,11 +42,11 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Provide lightweight metadata lock to improve the DML success rate during DDL change (experimental) [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** - TiDB uses the online asynchronous schema change algorithm to support changing metadata objects. When a transaction is executed, it obtains the corresponding metadata snapshot at the transaction start. If the metadata is changed during a transaction, to ensure data consistency, TiDB returns an `Information schema is changed` error and the transaction fails to commit. To solve the problem, TiDB v6.3.0 introduces [metadata lock](/metadata-lock.md) into the online DDL algorithm. To avoid most DML errors, TiDB coordinates the priority of DMLs and DDLs during table metadata change and makes executing DDLs wait for the DMLs with old metadata to commit. + TiDB uses the online asynchronous schema change algorithm to support changing metadata objects. When a transaction is executed, it obtains the corresponding metadata snapshot at the transaction start. If the metadata is changed during a transaction, to ensure data consistency, TiDB returns an `Information schema is changed` error and the transaction fails to commit. To solve the problem, TiDB v6.3.0 introduces [metadata lock](/metadata-lock.md) into the online DDL algorithm. To avoid DML errors whenever possible, TiDB coordinates the priority of DMLs and DDLs during table metadata change, and makes executing DDLs wait for the DMLs with old metadata to commit. * Improve the performance of adding indexes and reduce its impact on DML transactions [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** - To improve the speed of backfilling when creating an index, TiDB v6.3.0 accelerates the `ADD INDEX` and `CREATE INDEX` DDL operations when the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) system variable is enabled. When the feature is enabled, the performance of adding indexes is about three times faster than previously. + To improve the speed of backfilling when creating an index, TiDB v6.3.0 accelerates the `ADD INDEX` and `CREATE INDEX` DDL operations when the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) system variable is enabled. When the feature is enabled, the performance of adding indexes is about trippled. ### Security From a55af7f6a13a07dbcb1cd88b6119292b5b38a5f9 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 26 Sep 2022 13:59:36 +0800 Subject: [PATCH 055/104] Update releases/release-6.3.0.md Co-authored-by: xixirangrang --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 2d1525267d99f..845ce7868e984 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -186,7 +186,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiCDC supports keeping the snapshots consistent between the upstream and the downstream (sync point) [#issue]() @[asddongmen](https://github.com/asddongmen) **tw: TomShawn** - In the scenarios of data replication for disaster recovery, TiCDC supports periodically maintaining a downstream data snapshot so that the downstream snapshot is consistent with the upstream snapshot. With this feature, TiCDC can better match the scenarios where reads and writes are separate, and help you lower the cost. + In the scenarios of data replication for disaster recovery, TiCDC supports periodically maintaining a downstream data snapshot so that the downstream snapshot is consistent with the upstream snapshot. With this feature, TiCDC can better support the scenarios where reads and writes are separate, and help you lower the cost. * TiCDC supports graceful upgrade [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) @[3AceShowHand](https://github.com/3AceShowHand) **tw:ran-huang** From 3b42e355dfc876efd936d84aa625ad73f6a3c359 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Mon, 26 Sep 2022 14:04:20 +0800 Subject: [PATCH 056/104] Apply suggestions from code review Co-authored-by: xixirangrang --- releases/release-6.3.0.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 845ce7868e984..848506d5ced1d 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -108,7 +108,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Support session-level CTE inline [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) **tw: shichun-0415** - TiDB v6.2.0 introduced the `MERGE` hint in optimizers to allow CTE inline, so that the consumers of a CTE query result can be executed in parallel in TiFlash. In v6.3.0, a session variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) in introduced to allow CTE inline in sessions. This improves ease of use greatly. + TiDB v6.2.0 introduced the `MERGE` hint in optimizers to allow CTE inline, so that the consumers of a CTE query result can execute it in parallel in TiFlash. In v6.3.0, a session variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) is introduced to allow CTE inline in sessions. This improves ease of use greatly. ### Transactions @@ -410,7 +410,6 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - storage - - fix the problem that there may be some obsolete data left in storage which cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) - fix the problem that there may be some obsolete data left in storage which cannot be deleted [#5659](https://github.com/pingcap/tiflash/issues/5659) @[lidezhu](https://github.com/lidezhu) - Fix the bug that page GC may block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) - Fix the panic issue after creating the primary index with a column containing `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) From 2bd20842edd319a37d5ef3ec1dc2e4f0593ecea1 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Mon, 26 Sep 2022 14:48:49 +0800 Subject: [PATCH 057/104] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- releases/release-6.3.0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 848506d5ced1d..18fe491a27cd6 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -74,7 +74,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * TiDB Dashboard provides deadlock history information [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: shichun-0415** - From v6.3.0, TiDB Dashboard provides deadlock history. If you find excessively long SQL lock waiting by analyzing slow logs or other information on TiDB Dashboard, you can check the deadlock history to locate the root cause, which delivers better diagnosis experience. + From v6.3.0, TiDB Dashboard provides deadlock history. If you check the slow log in TiDB Dashboard and find the lock waiting time of some SQL statements to be excessively long, you can check the deadlock history to locate the root cause, which makes your diagnosis easier. ### Performance @@ -106,9 +106,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: In v6.3.0, the TiDB optimizer introduces 2 hints, `HASH_JOIN_BUILD()` and `HASH_JOIN_PROBE()`, to specify the Hash Join, its probe end, and its build end. When the optimizer fails to select the optimal execution plan, you can use these hints to intervene with the plan. -* Support session-level CTE inline [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) **tw: shichun-0415** +* Support session-level common table expressions (CTE) inline [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) **tw: shichun-0415** - TiDB v6.2.0 introduced the `MERGE` hint in optimizers to allow CTE inline, so that the consumers of a CTE query result can execute it in parallel in TiFlash. In v6.3.0, a session variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) is introduced to allow CTE inline in sessions. This improves ease of use greatly. + TiDB v6.2.0 introduced the `MERGE` hint in optimizers to allow CTE inline, so that the consumers of a CTE query result can execute it in parallel in TiFlash. In v6.3.0, a session variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) is introduced to allow CTE inline in sessions. This can greatly improve the ease of use. ### Transactions @@ -124,7 +124,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Modify the default policy of loading statistics when statistics become outdated [#issue]() @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) **tw: TomShawn** - In v5.3.0, TiDB introduces the system variable [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) to control how the optimizer behaves when the statistics become outdated. The default value is `ON`, which means keeping the behavior of the old version: When statistics on objects involved in SQL are outdated, the optimizer considers that statistics (other than the total number of rows on the table) are no longer reliable and uses pseudo statistics instead. After tests and analyses of actual user scenarios, the default value of `tidb_enable_pseudo_for_outdated_stats` is changed to `OFF` in v6.3.0. Even if the statistics become outdated, the optimizer will still use the statistics on the table, which is good for the execution program stability. + In v5.3.0, TiDB introduced the system variable [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) to control how the optimizer behaves when the statistics become outdated. The default value is `ON`, which means keeping the behavior of the old version: When the statistics on objects that are involved in a SQL statement are outdated, the optimizer considers that statistics (other than the total number of rows on the table) are no longer reliable and uses pseudo statistics instead. After tests and analyses of actual user scenarios, the default value of `tidb_enable_pseudo_for_outdated_stats` is changed to `OFF` since v6.3.0. Even if the statistics become outdated, the optimizer will still use the statistics on the table, which makes the execution plan more stable. * The feature of disabling Titan becomes GA [#issue]() @[tabokie](https://github.com/tabokie) **tw：ran-huang** @@ -160,7 +160,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Backup and restore -* PITR supports GCS and Azure Blob Storage as backup storage [#issue]() @[joccau](https://github.com/joccau) **tw: shichun-0415** +* PITR supports GCS and Azure Blob Storage as backup storages [#issue]() @[joccau](https://github.com/joccau) **tw: shichun-0415** PITR supports [GCS and Azure Blob Storage as backup storage](). If your TiDB is deployed on GCP or Azure, you can use the PITR feature after upgrading your cluster to v6.3.0. From 981d9365d7a27a0c2674e63a568ee69b719c5fe2 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Mon, 26 Sep 2022 15:12:55 +0800 Subject: [PATCH 058/104] add three tiflash notes and five br notes --- releases/release-6.3.0.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 18fe491a27cd6..b31cb50320fda 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -291,6 +291,8 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Support Planner Interpreter [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - Support to pushdown hex to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) - Suppress the "tcp set inq" loggings [#4940](https://github.com/pingcap/tiflash/issues/4940) + - Improve the accuracy of memory tracker in TiFlash [#5610](https://github.com/pingcap/tiflash/pull/5610) + - Improve the performance of string column with `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collation [#5294](https://github.com/pingcap/tiflash/issues/5294) - storage @@ -300,7 +302,8 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + Backup & Restore (BR) - - note [#issue]() @[Contributor GitHub ID]() + - PITR now aggregates a batch of files, which would greatly reduce the number of backup file. [#13232](https://github.com/tikv/tikv/issues/13232) @[Leavrth](https://github.com/Leavrth) + - PITR now supports automatically config TiFlash replica number after the restoration. [#37208](https://github.com/pingcap/tidb/issues/37208) @[YuJuncen](https://github.com/YuJuncen) + TiDB Binlog @@ -405,7 +408,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - compute - Fix the bug that window function may cause tiflash crash when canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - - fix error data input for date(CAST(value AS DATETIME)) causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) + - Fix the bug that wrong data input for `cast(value as datetime)` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) - fix that the result of expression casting real or decimal as time is inconsistent with mysql [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) - storage @@ -422,7 +425,9 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - br: raw restore fail in integration test "br_rawkv [#36490](https://github.com/pingcap/tidb/issues/36490) @[pingyu](https://github.com/pingyu) - Fix a bug that may cause the information of the checkpoint being stale [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) - Fix a bug caused when restoring with high `concurrency` the regions aren't balanced [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) - - Fix a bug that may cause log backup checkpoint TS stuck when some weird ranged regions exist [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) + - Fix a bug that may cause log backup checkpoint TS stuck when TiCDC exists in cluster [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) + Fix a bug that caused: when the backup meta v2 enabled, there may be too many meta files. [#37244](https://github.com/pingcap/tidb/issues/37244) [@MoCuishle28](https://github.com/MoCuishle28) + - Fix a bug that may lead to the backup / restore failure if some special character in the authorize key of external storages. [#37469](https://github.com/pingcap/tidb/issues/37469) [@MoCuishle28](https://github.com/MoCuishle28) + TiCDC From ae9bf8659a281d1bf1fbc61188ccdd7452151007 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Mon, 26 Sep 2022 19:36:39 +0800 Subject: [PATCH 059/104] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index b31cb50320fda..73b6cd0362db5 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -162,11 +162,11 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * PITR supports GCS and Azure Blob Storage as backup storages [#issue]() @[joccau](https://github.com/joccau) **tw: shichun-0415** - PITR supports [GCS and Azure Blob Storage as backup storage](). If your TiDB is deployed on GCP or Azure, you can use the PITR feature after upgrading your cluster to v6.3.0. + PITR supports [GCS](/br/backup-storage-gcs.md) and [Azure Blob Storage](/br/backup-storage-azblob.md) as backup storages. If your TiDB cluster is deployed on GCP or Azure, you can use the PITR feature after upgrading your cluster to v6.3.0. * BR supports AWS S3 Object Lock [#issue]() @[3pointer](https://github.com/3pointer) **tw: shichun-0415** - After enabling [S3 Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html), you can protect backup data from being tampered with or deleted. + You can protect backup data on AWS from being tampered with or deleted by enabling [S3 Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html). ### Data migration From a5dbb1027a0cd46890f47c0f0b6dbd11f8e462e8 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Mon, 26 Sep 2022 20:40:59 +0800 Subject: [PATCH 060/104] categorize unsorted tidb notes, add a dm note, and fix br notes --- releases/release-6.3.0.md | 38 +++++++++++++++----------------------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 73b6cd0362db5..13fbfb8c741dd 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -251,22 +251,23 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiDB - - [`PLAN REPLAYER`](/sql-plan-replayer) can be used on multiple SQL statements, which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) - - Improve warning log when new connection arrives [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) - - sql-infra - Grant privilege of a table to an user checks the target table exist first, in the past, the table name comparison works in a case sensitive manner, now it's changed to case insensitive [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) - Previously, TiDB users can set `init_connect` without any checking. From now on, the value of `init_connect` should be checked by the sql parser [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Improve warning log when new connection arrives [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) - execution - report error if json path has the wrong syntax [#22525](https://github.com/pingcap/tidb/issues/22525) @[xiongjiwei](https://github.com/xiongjiwei) - report error if json path has the wrong syntax [#34959](https://github.com/pingcap/tidb/issues/34959) @[xiongjiwei](https://github.com/xiongjiwei) + - Improve the performance of Join by fix a false sharing problem. [#37641](https://github.com/pingcap/tidb/issues/37641) + - Improve the MySQL compatibility of `CAST` function in terms of float-to-time conversion [#37462](https://github.com/pingcap/tidb/issues/37462) @[mengxin9014](https://github.com/mengxin9014) - planner - planner: just pop cte's handleHelper map out since it shouldn't be considered [#35758](https://github.com/pingcap/tidb/issues/35758) @[AilinKid](https://github.com/AilinKid) + - [`PLAN REPLAYER`](/sql-plan-replayer) can be used on multiple SQL statements, which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) + TiKV @@ -316,29 +317,16 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiDB Data Migration (DM) - Improve compatibility for MySQL 8.0 upstream [#6448](https://github.com/pingcap/tiflow/issues/6448) @[lance6716](https://github.com/lance6716) + - Optimize DDL by executing DDL asynchronously when encounter "invalid connection" [#4689](https://github.com/pingcap/tiflow/issues/4689) @[lyzx2001](https://github.com/lyzx2001) + TiDB Lightning - Add query parameters for S3 external storage URL, in order to support accessing the S3 data in another account by assuming a given role [#36891](https://github.com/pingcap/tidb/issues/36891) [dsdashun](https://github.com/dsdashun) - - TiUP - - - note [#issue]() @[Contributor GitHub ID]() - ## Bug fixes + TiDB - - Fix handling of prepared statement flags in the classic MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[hawkingrei](https://github.com/hawkingrei) - - update pd-client to ensure tidb-server get clusterID correctly [#36505](https://github.com/pingcap/tidb/issues/36505), [#36478](https://github.com/pingcap/tidb/issues/36478) @[Defined2014](https://github.com/Defined2014) - - Fix that incorrect TiDB states may appear on startup under very, very, very extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) - - Fix a bug that UnionScan's Next() function skips reading data when the passed chunk's capacity is 0 [#36903](https://github.com/pingcap/tidb/issues/36903) - - Fix a bug about variables information leak [#37586](https://github.com/pingcap/tidb/issues/37586) - - Fix the issue that the action order of [#37058](https://github.com/pingcap/tidb/issues/37058) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that comparison between json opaque will cause panic [#37315](https://github.com/pingcap/tidb/issues/37315) - - Fix the issue that the single precision float cannot be used in json aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) - - fix that the result of expression castRealAsTime is inconsistent with mysql [#37462](https://github.com/pingcap/tidb/issues/37462) - - sql-infra - Fix the issue that `PREAPRE` statements do not check privileges [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) @@ -354,6 +342,9 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Optimize DDL history HTTP API, and add support for 'start_job_id' parameter [#35838](https://github.com/pingcap/tidb/issues/35838) @[tiancaiamao](https://github.com/tiancaiamao) - Fix the issue that `row_count` of DDL jobs is inaccurate [#25968](https://github.com/pingcap/tidb/issues/25968) @[Defined2014](https://github.com/Defined2014) - Fix the issue that `FLASHBACK TABLE` does not work properly [#37386](https://github.com/pingcap/tidb/issues/37386) @[tiancaiamao](https://github.com/tiancaiamao) + - Fix handling of prepared statement flags in the classic MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[dveeden](https://github.com/dveeden) + - Fix data-race issue on start up in rare cases [#36791](https://github.com/pingcap/tidb/issues/36791) @[xhebox](https://github.com/xhebox) + - Fix the issue that `information_schema.variables_info` does not respect SEM [#37586](https://github.com/pingcap/tidb/issues/37586) @[CbcWestwolf](https://github.com/CbcWestwolf) - execution @@ -361,6 +352,9 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix the issue that the cast and comparison between binary string and json is incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the cast and comparison between binary string and json is incompatible with MySQL [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the json_objectagg and json_arrayagg is not compatible with MySQL on binary value [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that comparison between json opaque causes panic [#37315](https://github.com/pingcap/tidb/issues/37315) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the single precision float cannot be used in json aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that `UNION` may returns unexpected empty result [#36903](https://github.com/pingcap/tidb/issues/36903) @[tiancaiamao](https://github.com/tiancaiamao) - transaction @@ -378,7 +372,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - diagnosis - - fix metric sql error [#35856](https://github.com/pingcap/tidb/issues/35856) @[Defined2014](https://github.com/Defined2014) + - Fix incorrect `PromQL` [#35856](https://github.com/pingcap/tidb/issues/35856) @[Defined2014](https://github.com/Defined2014) + TiKV @@ -399,7 +393,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + PD - - grpc: fix the wrong error handler [#5373](https://github.com/tikv/pd/issues/5373) @[bufferflies](https://github.com/bufferflies) + - Fix PD panics caused by the issue that gRPC handles errors inappropriately when `enable-forwarding` is enabled [#5373](https://github.com/tikv/pd/issues/5373) @[bufferflies](https://github.com/bufferflies) - Fix the issue that unhealthy region cause panic [#5491](https://github.com/tikv/pd/issues/5491) @[nolouch](https://github.com/nolouch) - Fix the bug where the Learner Peer of TiFlash Replica might not be created [#5401](https://github.com/tikv/pd/issues/5401) @[HunDunDM](https://github.com/HunDunDM) @@ -413,7 +407,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - storage - - fix the problem that there may be some obsolete data left in storage which cannot be deleted [#5659](https://github.com/pingcap/tiflash/issues/5659) @[lidezhu](https://github.com/lidezhu) + - fix the problem that there may be some obsolete data left in storage which cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) - Fix the bug that page GC may block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) - Fix the panic issue after creating the primary index with a column containing `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) @@ -421,12 +415,10 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + Backup & Restore (BR) - - Fix issues in "br/tests/up.sh" [#36743](https://github.com/pingcap/tidb/issues/36743) @[pingyu](https://github.com/pingyu) - - br: raw restore fail in integration test "br_rawkv [#36490](https://github.com/pingcap/tidb/issues/36490) @[pingyu](https://github.com/pingyu) - Fix a bug that may cause the information of the checkpoint being stale [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) - Fix a bug caused when restoring with high `concurrency` the regions aren't balanced [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) - Fix a bug that may cause log backup checkpoint TS stuck when TiCDC exists in cluster [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) - Fix a bug that caused: when the backup meta v2 enabled, there may be too many meta files. [#37244](https://github.com/pingcap/tidb/issues/37244) [@MoCuishle28](https://github.com/MoCuishle28) + - Fix a bug that caused: when the backup meta v2 enabled, there may be too many meta files. [#37244](https://github.com/pingcap/tidb/issues/37244) [@MoCuishle28](https://github.com/MoCuishle28) - Fix a bug that may lead to the backup / restore failure if some special character in the authorize key of external storages. [#37469](https://github.com/pingcap/tidb/issues/37469) [@MoCuishle28](https://github.com/MoCuishle28) + TiCDC From fdaeb718f77c9338c4314196ff6d5976d2aabff3 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 27 Sep 2022 10:33:42 +0800 Subject: [PATCH 061/104] Apply suggestions from code review --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 13fbfb8c741dd..1ba157e1fcc55 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -23,11 +23,11 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### SQL -* Add a new syntactic sugar (Range INTERVAL partitioning) to simplify Range partition definitions [#35683](https://github.com/pingcap/tidb/issues/35683) @[mjonss](https://github.com/mjonss) +* Add a new syntactic sugar (Range INTERVAL partitioning) to simplify Range partition definitions (experimental) [#35683](https://github.com/pingcap/tidb/issues/35683) @[mjonss](https://github.com/mjonss) TiDB provides [INTERVAL partitioning](/partitioned-table.md#range-interval-partitioning) as a new way of defining Range partitions. You do not need to enumerate all partitions, which drastically reduces the length of Range partitioning DDL statements. The syntax is equivalent to that of the original Range partitioning. -* Range COLUMNS partitioning supports defining multiple columns [#36636](https://github.com/pingcap/tidb/issues/36636) @[mjonss](https://github.com/mjonss) +* Range COLUMNS partitioning supports defining multiple columns (experimental) [#36636](https://github.com/pingcap/tidb/issues/36636) @[mjonss](https://github.com/mjonss) Support [PARTITION BY RANGE COLUMNS (column_list)](/partitioned-table.md#range-columns-partitioning). `column_list` is no longer limited to a single column. The basic feature is the same as MySQL. From 74d1dae1c535ef1c5e8b943efd750f7f6ef959b8 Mon Sep 17 00:00:00 2001 From: Aolin Date: Tue, 27 Sep 2022 11:37:33 +0800 Subject: [PATCH 062/104] Apply suggestions from code review Co-authored-by: Ran --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 1ba157e1fcc55..230e0c2f88cef 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -44,7 +44,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiDB uses the online asynchronous schema change algorithm to support changing metadata objects. When a transaction is executed, it obtains the corresponding metadata snapshot at the transaction start. If the metadata is changed during a transaction, to ensure data consistency, TiDB returns an `Information schema is changed` error and the transaction fails to commit. To solve the problem, TiDB v6.3.0 introduces [metadata lock](/metadata-lock.md) into the online DDL algorithm. To avoid DML errors whenever possible, TiDB coordinates the priority of DMLs and DDLs during table metadata change, and makes executing DDLs wait for the DMLs with old metadata to commit. -* Improve the performance of adding indexes and reduce its impact on DML transactions [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** +* Improve the performance of adding indexes and reduce its impact on DML transactions (experimental) [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** To improve the speed of backfilling when creating an index, TiDB v6.3.0 accelerates the `ADD INDEX` and `CREATE INDEX` DDL operations when the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) system variable is enabled. When the feature is enabled, the performance of adding indexes is about trippled. @@ -176,7 +176,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * DM adds a new configuration item `safe-mode-duration` in the task configuration file [#6224] (https://github.com/pingcap/tiflow/issues/6224) @[[okJiang](https://github.com/okJiang)] **tw：ran-huang** - DM adds a new configuration item `safe-mode-duration` in the [task configuration file](/task-configuration-file-full.md). You can adjust the automatic safe mode duration after DM exits abnormally. The default value is 60 seconds. When `safe-mode-duration` is set to `"0s"`, DM does not automatically enter safe mode after an abnormal restart. + DM adds a new configuration item `safe-mode-duration` in the [task configuration file](/task-configuration-file-full.md). You can adjust the automatic safe mode duration after DM exits abnormally. The default value is 60 seconds. If `safe-mode-duration` is set to `"0s"`, an error is reported when DM tries to enter safe mode after an abnormal restart. ### TiDB data share subscription From 0cdb3a04e5af1139038e446215ef65bd9e604964 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 27 Sep 2022 13:25:04 +0800 Subject: [PATCH 063/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 230e0c2f88cef..5d0b4622ccb2a 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -27,7 +27,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiDB provides [INTERVAL partitioning](/partitioned-table.md#range-interval-partitioning) as a new way of defining Range partitions. You do not need to enumerate all partitions, which drastically reduces the length of Range partitioning DDL statements. The syntax is equivalent to that of the original Range partitioning. -* Range COLUMNS partitioning supports defining multiple columns (experimental) [#36636](https://github.com/pingcap/tidb/issues/36636) @[mjonss](https://github.com/mjonss) +* Range COLUMNS partitioning supports defining multiple columns [#36636](https://github.com/pingcap/tidb/issues/36636) @[mjonss](https://github.com/mjonss) Support [PARTITION BY RANGE COLUMNS (column_list)](/partitioned-table.md#range-columns-partitioning). `column_list` is no longer limited to a single column. The basic feature is the same as MySQL. From a881d8a83202d9842817f580f74c16972a0c5029 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Tue, 27 Sep 2022 14:35:28 +0800 Subject: [PATCH 064/104] Apply suggestions from code review Co-authored-by: Jianyuan Jiang --- releases/release-6.3.0.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 5d0b4622ccb2a..fffcda81cb91c 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -313,6 +313,12 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiCDC - Improve compatibility for MySQL 8.0 upstream [#6506](https://github.com/pingcap/tiflow/issues/6506) @[lance6716](https://github.com/lance6716) + - Log DML start ts when MySQL sink meets error [#6460](https://github.com/pingcap/tiflow/issues/6460) @[overvenus](https://github.com/overvenus) + - Make the API `api/v1/health` reflects the health status of the TiCDC cluster [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) + - Introduce a new sink implement to increase throughput [#5928](https://github.com/pingcap/tiflow/issues/5928) @[hicqu]https://github.com/hicqu @[hi-rustin](https://github.com/hi-rustin) + - Deprecate the memory sorter and always use the unified sorter [#7087](https://github.com/pingcap/tiflow/issues/5928) @[hi-rustin](https://github.com/hi-rustin) + - Delete the deprecated pulsar sink [#7087](https://github.com/pingcap/tiflow/issues/7087) @[hi-rustin](https://github.com/hi-rustin) + - Improve replicating performance by discarding DDLs that are irrelevant with a changefeed. [#6447](https://github.com/pingcap/tiflow/issues/6447) @[asddongmen](https://github.com/asddongmen) + TiDB Data Migration (DM) @@ -423,7 +429,9 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiCDC - - handle error correctly with wrong pd address but with a grpc service [#6458](https://github.com/pingcap/tiflow/issues/6458) @[crelax](https://github.com/crelax) + - Handle error correctly with wrong PD address but with a grpc service [#6458](https://github.com/pingcap/tiflow/issues/6458) @[crelax](https://github.com/crelax) + - Fix a bug that `cdc cause cli changefeed list` does not contain failed changefeed [#6334](https://github.com/pingcap/tiflow/issues/6334) @[asddongmen](https://github.com/asddongmen) + - Fix a bug that may cause cdc unavailable if a changefeed initializing fails [#6859](https://github.com/pingcap/tiflow/issues/6859) @[asddongmen](https://github.com/asddongmen) + TiDB Data Migration (DM) From e2b69de46f6b9731e3cc99fc84f418a8153f263b Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Tue, 27 Sep 2022 17:52:13 +0800 Subject: [PATCH 065/104] Apply suggestions from code review --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index fffcda81cb91c..bfb3005fc5a99 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -102,7 +102,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiDB v6.3.0 introduces a new join type [Null-aware anti join (NAAJ)](/explain-subqueries.md#null-aware-anti-semi-join-not-in-and--all-subqueries). NAAJ can be aware of whether the collection is empty or `NULL` when processing collection operations. This optimizes the execution efficiency of operations such as `IN` and `= ANY` and improves SQL performance. -* Add optimizer hints to control the build end of Hash Join [#issue]() @[Reminiscent](https://github.com/Reminiscent) **tw: TomShawn** +* Add optimizer hints to control the build end of Hash Join [#35439](https://github.com/pingcap/tidb/issues/35439) @[Reminiscent](https://github.com/Reminiscent) **tw: TomShawn** In v6.3.0, the TiDB optimizer introduces 2 hints, `HASH_JOIN_BUILD()` and `HASH_JOIN_PROBE()`, to specify the Hash Join, its probe end, and its build end. When the optimizer fails to select the optimal execution plan, you can use these hints to intervene with the plan. @@ -122,7 +122,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Stability -* Modify the default policy of loading statistics when statistics become outdated [#issue]() @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) **tw: TomShawn** +* Modify the default policy of loading statistics when statistics become outdated [#27601](https://github.com/pingcap/tidb/issues/27601) @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) **tw: TomShawn** In v5.3.0, TiDB introduced the system variable [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) to control how the optimizer behaves when the statistics become outdated. The default value is `ON`, which means keeping the behavior of the old version: When the statistics on objects that are involved in a SQL statement are outdated, the optimizer considers that statistics (other than the total number of rows on the table) are no longer reliable and uses pseudo statistics instead. After tests and analyses of actual user scenarios, the default value of `tidb_enable_pseudo_for_outdated_stats` is changed to `OFF` since v6.3.0. Even if the statistics become outdated, the optimizer will still use the statistics on the table, which makes the execution plan more stable. From 33d30009602bef115466395167096c0ea2a1fd52 Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 28 Sep 2022 10:05:19 +0800 Subject: [PATCH 066/104] review TiKV and PD --- releases/release-6.3.0.md | 42 +++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 22 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index bfb3005fc5a99..2f17c61e44ff5 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -271,16 +271,17 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiKV - - Add a new option to make unreachable_backoff of raftstore configurable [#13054](https://github.com/tikv/tikv/issues/13054) - - Implement TSO batch list to improve tolerance to TSO service fault [#12794](https://github.com/tikv/tikv/issues/12794) @[pingyu](https://github.com/pingyu) - - Make max_subcompactions dynamically changeable [#13145](https://github.com/tikv/tikv/issues/13145) @[ethercflow](https://github.com/ethercflow) - - Optimize the performance of merging empty regions [#12421](https://github.com/tikv/tikv/issues/12421) @[tabokie](https://github.com/tabokie) + - Support configuring the `unreachable_backoff` item to avoid Raftstore broadcasting too many messages after one peer become unreachable [#13054](https://github.com/tikv/tikv/issues/13054) @[5kbpers](https://github.com/5kbpers) + - Improve the fault tolerance of TSO service [#12794](https://github.com/tikv/tikv/issues/12794) @[pingyu](https://github.com/pingyu) + - (dup) Support dynamically modifying the number of sub-compaction operations performed concurrently in RocksDB (`rocksdb.max-sub-compactions`) [#13145](https://github.com/tikv/tikv/issues/13145) @[ethercflow](https://github.com/ethercflow) + - Optimize the performance of merging empty Regions [#12421](https://github.com/tikv/tikv/issues/12421) @[tabokie](https://github.com/tabokie) - Support more regular expression functions [#13483](https://github.com/tikv/tikv/issues/13483) @[gengliqi](https://github.com/gengliqi) - - Support automatically scale read pool thread count based on the CPU usage [#13313](https://github.com/tikv/tikv/issues/13313) @[glorv](https://github.com/glorv) + - Support automatically adjust the thread pool size based on the CPU usage [#13313](https://github.com/tikv/tikv/issues/13313) @[glorv](https://github.com/glorv) + PD - - Updates metrics query. Renames `metrics` to `monitoring` on TiDB Dashboard [#5366](https://github.com/tikv/pd/issues/5366) @[YiniXu9506](https://github.com/YiniXu9506) + - Improve the query of the TiKV IO MBps metric in TiDB Dashboard [#5366](https://github.com/tikv/pd/issues/5366) @[YiniXu9506](https://github.com/YiniXu9506) + - Modify the URL in TiDB Dashboard from `metrics` to `monitoring` [#5366](https://github.com/tikv/pd/issues/5366) @[YiniXu9506](https://github.com/YiniXu9506) + TiFlash @@ -382,26 +383,23 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiKV - - fix the bug that the consume should be refresh if region heartbeat send failed [#12934](https://github.com/tikv/tikv/issues/12934) @[bufferflies](https://github.com/bufferflies) - - Fix a bug that regions may be overlapped if raftstore is too busy [#13160](https://github.com/tikv/tikv/issues/13160) @[5kbpers](https://github.com/5kbpers) - - Fix potential deadlock in `RpcClient` when two read locks are interleaved by a write lock [#12933](https://github.com/tikv/tikv/issues/12933) @[BurtonQin](https://github.com/BurtonQin) - - Fix a double-lock bug in components/engine_test [#13186](https://github.com/tikv/tikv/issues/13186) @[SpadeA-Tang](https://github.com/SpadeA-Tang) - - Fix plaintext iv debug assert while disable encryption [#13081](https://github.com/tikv/tikv/issues/13081) @[jiayang-zheng](https://github.com/jiayang-zheng) - - Fix a expression error that causes unified read pool cpu cannot be shown correctly [#13086](https://github.com/tikv/tikv/issues/13086) @[glorv](https://github.com/glorv) - - Fix the problem that QPS may drop to zero for several mintues when a tikv is partitioned [#12966](https://github.com/tikv/tikv/issues/12966) @[cosven](https://github.com/cosven) - - remove call_option to avoid deadlock(RWR) [#13191](https://github.com/tikv/tikv/issues/13191) @[bufferflies](https://github.com/bufferflies) - - Reduce false-positive PessimisticLockNotFound errors in conflicting auto-commit workloads [#13425](https://github.com/tikv/tikv/issues/13425) @[sticnarf](https://github.com/sticnarf) - - Fix a bug that may cause PiTR losing some data when there are too many adjacent short row putting [#13281](https://github.com/tikv/tikv/issues/13281) @[YuJuncen](https://github.com/YuJuncen) + - (dup) Fix the issue that PD does not reconnect to TiKV after the Region heartbeat is interrupted [#12934](https://github.com/tikv/tikv/issues/12934) @[bufferflies](https://github.com/bufferflies) + - (dup) Fix a bug that Regions might be overlapped if Raftstore is busy [#13160](https://github.com/tikv/tikv/issues/13160) @[5kbpers](https://github.com/5kbpers) + - (dup) Fix the issue that the PD client might cause deadlocks [#13191](https://github.com/tikv/tikv/issues/13191) @[bufferflies](https://github.com/bufferflies) [#12933](https://github.com/tikv/tikv/issues/12933) @[BurtonQin](https://github.com/BurtonQin) + - Fix the issue that TiKV might panic when encryption is disabled [#13081](https://github.com/tikv/tikv/issues/13081) @[jiayang-zheng](https://github.com/jiayang-zheng) + - (dup) Fix the wrong expression of `Unified Read Pool CPU` in Dashboard [#13086](https://github.com/tikv/tikv/issues/13086) @[glorv](https://github.com/glorv) + - Fix the issue that the TiKV service is unavailable for several minutes when a TiKV is in an isolated network environment [#12966](https://github.com/tikv/tikv/issues/12966) @[cosven](https://github.com/cosven) + - Fix the issue that TiKV mistakenly reports a `PessimisticLockNotFound` error [#13425](https://github.com/tikv/tikv/issues/13425) @[sticnarf](https://github.com/sticnarf) + - Fix a bug that PITR might cause data loss in some situations [#13281](https://github.com/tikv/tikv/issues/13281) @[YuJuncen](https://github.com/YuJuncen) - Fix a bug that caused checkpoint not advanced when there are some long pessimistic transactions [#13304](https://github.com/tikv/tikv/issues/13304) @[YuJuncen](https://github.com/YuJuncen) - - Fix the issue that TiKV doesn't distinguish the `DATETIME/DATE/TIMESTAMP/TIME` and `STRING` in json type [#13417](https://github.com/tikv/tikv/issues/13417) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that comparison between json bool and other json value is not compatible with TiDB and MySQL [#13386](https://github.com/tikv/tikv/issues/13386) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that comparison between json bool and other json value is not compatible with TiDB and MySQL [#37481](https://github.com/pingcap/tidb/issues/37481) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that TiKV does not distinguish the datetime type (`DATETIME`, `DATE`, `TIMESTAMP` and `TIME`) and `STRING` type in JSON [#13417](https://github.com/tikv/tikv/issues/13417) @[YangKeao](https://github.com/YangKeao) + - Fix incompatibility with MySQL of comparison between JSON bool and other JSON value [#13386](https://github.com/tikv/tikv/issues/13386) [#37481](https://github.com/pingcap/tidb/issues/37481) @[YangKeao](https://github.com/YangKeao) + PD - - Fix PD panics caused by the issue that gRPC handles errors inappropriately when `enable-forwarding` is enabled [#5373](https://github.com/tikv/pd/issues/5373) @[bufferflies](https://github.com/bufferflies) - - Fix the issue that unhealthy region cause panic [#5491](https://github.com/tikv/pd/issues/5491) @[nolouch](https://github.com/nolouch) - - Fix the bug where the Learner Peer of TiFlash Replica might not be created [#5401](https://github.com/tikv/pd/issues/5401) @[HunDunDM](https://github.com/HunDunDM) + - (dup) Fix PD panics caused by the issue that gRPC handles errors inappropriately when `enable-forwarding` is enabled [#5373](https://github.com/tikv/pd/issues/5373) @[bufferflies](https://github.com/bufferflies) + - Fix the issue that unhealthy Region might cause PD panic [#5491](https://github.com/tikv/pd/issues/5491) @[nolouch](https://github.com/nolouch) + - Fix the bug that the TiFlash learner replica might not be created [#5401](https://github.com/tikv/pd/issues/5401) @[HunDunDM](https://github.com/HunDunDM) + TiFlash From 363ad0a3e035f7242f6242af31cd2690647774e4 Mon Sep 17 00:00:00 2001 From: Aolin Date: Wed, 28 Sep 2022 13:53:37 +0800 Subject: [PATCH 067/104] Apply suggestions from code review Co-authored-by: Ran --- releases/release-6.3.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 2f17c61e44ff5..3e5f640901685 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -271,12 +271,12 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiKV - - Support configuring the `unreachable_backoff` item to avoid Raftstore broadcasting too many messages after one peer become unreachable [#13054](https://github.com/tikv/tikv/issues/13054) @[5kbpers](https://github.com/5kbpers) + - Support configuring the `unreachable_backoff` item to avoid Raftstore broadcasting too many messages after one peer becomes unreachable [#13054](https://github.com/tikv/tikv/issues/13054) @[5kbpers](https://github.com/5kbpers) - Improve the fault tolerance of TSO service [#12794](https://github.com/tikv/tikv/issues/12794) @[pingyu](https://github.com/pingyu) - (dup) Support dynamically modifying the number of sub-compaction operations performed concurrently in RocksDB (`rocksdb.max-sub-compactions`) [#13145](https://github.com/tikv/tikv/issues/13145) @[ethercflow](https://github.com/ethercflow) - Optimize the performance of merging empty Regions [#12421](https://github.com/tikv/tikv/issues/12421) @[tabokie](https://github.com/tabokie) - Support more regular expression functions [#13483](https://github.com/tikv/tikv/issues/13483) @[gengliqi](https://github.com/gengliqi) - - Support automatically adjust the thread pool size based on the CPU usage [#13313](https://github.com/tikv/tikv/issues/13313) @[glorv](https://github.com/glorv) + - Support automatically adjusting the thread pool size based on the CPU usage [#13313](https://github.com/tikv/tikv/issues/13313) @[glorv](https://github.com/glorv) + PD @@ -388,10 +388,10 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - (dup) Fix the issue that the PD client might cause deadlocks [#13191](https://github.com/tikv/tikv/issues/13191) @[bufferflies](https://github.com/bufferflies) [#12933](https://github.com/tikv/tikv/issues/12933) @[BurtonQin](https://github.com/BurtonQin) - Fix the issue that TiKV might panic when encryption is disabled [#13081](https://github.com/tikv/tikv/issues/13081) @[jiayang-zheng](https://github.com/jiayang-zheng) - (dup) Fix the wrong expression of `Unified Read Pool CPU` in Dashboard [#13086](https://github.com/tikv/tikv/issues/13086) @[glorv](https://github.com/glorv) - - Fix the issue that the TiKV service is unavailable for several minutes when a TiKV is in an isolated network environment [#12966](https://github.com/tikv/tikv/issues/12966) @[cosven](https://github.com/cosven) + - Fix the issue that the TiKV service is unavailable for several minutes when a TiKV instance is in an isolated network environment [#12966](https://github.com/tikv/tikv/issues/12966) @[cosven](https://github.com/cosven) - Fix the issue that TiKV mistakenly reports a `PessimisticLockNotFound` error [#13425](https://github.com/tikv/tikv/issues/13425) @[sticnarf](https://github.com/sticnarf) - Fix a bug that PITR might cause data loss in some situations [#13281](https://github.com/tikv/tikv/issues/13281) @[YuJuncen](https://github.com/YuJuncen) - - Fix a bug that caused checkpoint not advanced when there are some long pessimistic transactions [#13304](https://github.com/tikv/tikv/issues/13304) @[YuJuncen](https://github.com/YuJuncen) + - Fix a bug that causes checkpoint not advanced when there are some long pessimistic transactions [#13304](https://github.com/tikv/tikv/issues/13304) @[YuJuncen](https://github.com/YuJuncen) - Fix the issue that TiKV does not distinguish the datetime type (`DATETIME`, `DATE`, `TIMESTAMP` and `TIME`) and `STRING` type in JSON [#13417](https://github.com/tikv/tikv/issues/13417) @[YangKeao](https://github.com/YangKeao) - Fix incompatibility with MySQL of comparison between JSON bool and other JSON value [#13386](https://github.com/tikv/tikv/issues/13386) [#37481](https://github.com/pingcap/tidb/issues/37481) @[YangKeao](https://github.com/YangKeao) From daca2a11d543c616479fabb425573bdf11569de4 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 28 Sep 2022 14:12:32 +0800 Subject: [PATCH 068/104] Apply suggestions from code review Co-authored-by: Aolin --- releases/release-6.3.0.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 3e5f640901685..f2c203d90cbc2 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -304,8 +304,8 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + Backup & Restore (BR) - - PITR now aggregates a batch of files, which would greatly reduce the number of backup file. [#13232](https://github.com/tikv/tikv/issues/13232) @[Leavrth](https://github.com/Leavrth) - - PITR now supports automatically config TiFlash replica number after the restoration. [#37208](https://github.com/pingcap/tidb/issues/37208) @[YuJuncen](https://github.com/YuJuncen) + - PITR now aggregates a batch of files, which would greatly reduce the number of backup files required [#13232](https://github.com/tikv/tikv/issues/13232) @[Leavrth](https://github.com/Leavrth) + - PITR now supports automatically configuring the number of TiFlash replicas based on the upstream cluster configuration after the restoration [#37208](https://github.com/pingcap/tidb/issues/37208) @[YuJuncen](https://github.com/YuJuncen) + TiDB Binlog @@ -323,12 +323,12 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiDB Data Migration (DM) - - Improve compatibility for MySQL 8.0 upstream [#6448](https://github.com/pingcap/tiflow/issues/6448) @[lance6716](https://github.com/lance6716) + - Improve compatibility with MySQL 8.0 as data source [#6448](https://github.com/pingcap/tiflow/issues/6448) @[lance6716](https://github.com/lance6716) - Optimize DDL by executing DDL asynchronously when encounter "invalid connection" [#4689](https://github.com/pingcap/tiflow/issues/4689) @[lyzx2001](https://github.com/lyzx2001) + TiDB Lightning - - Add query parameters for S3 external storage URL, in order to support accessing the S3 data in another account by assuming a given role [#36891](https://github.com/pingcap/tidb/issues/36891) [dsdashun](https://github.com/dsdashun) + - Add query parameters for S3 external storage URL to support accessing the S3 data in another account by assuming a given role [#36891](https://github.com/pingcap/tidb/issues/36891) @[dsdashun](https://github.com/dsdashun) ## Bug fixes @@ -419,11 +419,11 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + Backup & Restore (BR) - - Fix a bug that may cause the information of the checkpoint being stale [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) - - Fix a bug caused when restoring with high `concurrency` the regions aren't balanced [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) - - Fix a bug that may cause log backup checkpoint TS stuck when TiCDC exists in cluster [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) + - Fix a bug that might cause the information of the checkpoint being stale [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) + - Fix a bug that the regions are not balanced because the concurrency is set too large during the restoration [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) + - Fix a bug that might cause log backup checkpoint TS stuck when TiCDC exists in the cluster [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) - Fix a bug that caused: when the backup meta v2 enabled, there may be too many meta files. [#37244](https://github.com/pingcap/tidb/issues/37244) [@MoCuishle28](https://github.com/MoCuishle28) - - Fix a bug that may lead to the backup / restore failure if some special character in the authorize key of external storages. [#37469](https://github.com/pingcap/tidb/issues/37469) [@MoCuishle28](https://github.com/MoCuishle28) + - Fix a bug that might lead to backup and restoration failure if some special character exists in the authorization key of external storage [#37469](https://github.com/pingcap/tidb/issues/37469) [@MoCuishle28](https://github.com/MoCuishle28) + TiCDC @@ -433,15 +433,15 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiDB Data Migration (DM) - - Fix a problem that DM will report `Specified key was too long` error [#5315](https://github.com/pingcap/tiflow/issues/5315) @[lance6716](https://github.com/lance6716) - - Fix a bug that relay goroutine and upstream connections may leak when relay meet error [#6193](https://github.com/pingcap/tiflow/issues/6193) @[lance6716](https://github.com/lance6716) - - Fix when use "strict" collation_compatible, DM sometimes generate SQL with duplicated collation [#6832](https://github.com/pingcap/tiflow/issues/6832) @[lance6716](https://github.com/lance6716) - - Reduce the appearing time of the warning message "found error when getting timezone from binlog status_vars" in dm-worker log [#6628](https://github.com/pingcap/tiflow/issues/6628) @[lyzx2001](https://github.com/lyzx2001) - - Fix a bug that latin1 data may be corrupt when replicating [#7028](https://github.com/pingcap/tiflow/issues/7028) @[lance6716](https://github.com/lance6716) + - Fix the issue that DM reports the `Specified key was too long` error [#5315](https://github.com/pingcap/tiflow/issues/5315) @[lance6716](https://github.com/lance6716) + - (dup) Fix goroutine leak when relay meets an error [#6193](https://github.com/pingcap/tiflow/issues/6193) @[lance6716](https://github.com/lance6716) + - Fix the bug that when `collation_compatible` is set to `"strict"`, DM might generate SQL with duplicated collation [#6832](https://github.com/pingcap/tiflow/issues/6832) @[lance6716](https://github.com/lance6716) + - Reduce the appearance of the warning message "found error when get timezone from binlog status_vars" in DM-worker log [#6628](https://github.com/pingcap/tiflow/issues/6628) @[lyzx2001](https://github.com/lyzx2001) + - Fix a bug that latin1 data might be corrupt during replication [#7028](https://github.com/pingcap/tiflow/issues/7028) @[lance6716](https://github.com/lance6716) + TiDB Lightning - - Fix the issue that TiDB Lightning does not support columns starting with slash, number, or non-ascii characters in Parquet files [#36980](https://github.com/pingcap/tidb/issues/36980) @[D3Hunter](https://github.com/D3Hunter) + - (dup) Fix the issue that TiDB Lightning does not support columns starting with slash, number, or non-ascii characters in Parquet files [#36980](https://github.com/pingcap/tidb/issues/36980) @[D3Hunter](https://github.com/D3Hunter) - TiUP From 6e9e9d35650aff23739c4ff494c279046c8fe57a Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 28 Sep 2022 14:17:03 +0800 Subject: [PATCH 069/104] Apply suggestions from code review --- releases/release-6.3.0.md | 41 +++++++++++++++++++++++++-------------- 1 file changed, 26 insertions(+), 15 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index f2c203d90cbc2..b69b5bb7bf97d 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -94,6 +94,10 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiFlash uses the Raft protocol for data replication from TiKV. Prior to v6.3.0, it often took a long time to replicate large amounts of replica data. TiDB v6.3.0 optimizes the TiFlash data replication mechanism and significantly improves the replication speed. When you use BR to recover data, use TiDB Lightning to import data, or add new TiFlash replicas, the TiFlash replicas can be replicated more quickly. You can query with TiFlash in a more timely manner. In addition, TiFlash replicas will also reach a secure and balanced state faster when you scale up, scale down, or modify the number of TiFlash replicas. +* TiFlash supports three-stage aggregation of individual `COUNT(DISTINCT)` [#37202](https://github.com/pingcap/tidb/issues/37202) @[fixdb](https://github.com/fixdb) + + TiFlash supports rewriting queries containing only one `COUNT(DISTINCT)` into a [three-stage aggregation](/system-variables.md#tidb_opt_three_stage_distinct_agg-new-in-v630). This improves concurrency and performance. + * TiKV supports log recycling [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) **tw：ran-huang** TiKV supports [recycling log files](/tikv-configuration-file.md#enable-log-recycle-new-in-v630) in Raft Engine. This reduces the long tail latency in network disks during Raft log appending and improves performance under write workloads. @@ -212,10 +216,17 @@ In v6.3.0-DMR, the key new features and improvements are as follows: | [`tidb_last_plan_replayer_token`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | This variable is read-only and is used to obtain the result of the last `PLAN REPLAYER DUMP` execution in the current session. | | [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) | Newly added | This variable is used to control whether common table expressions (CTEs) in the entire session are inlined or not. The default value is `OFF`, which means that inlining CTE is not enforced by default. | | [`tidb_opt_three_stage_distinct_agg`](/system-variables.md#tidb_opt_three_stage_distinct_agg-new-in-v630) | Newly added | This variable specifies whether to rewrite a `COUNT(DISTINCT)` aggregation into a three-stage aggregation in MPP mode. The default value is `ON`. | -| [`tidb_partition_prune_mode`](/system-variables.md#tidb_partition_prune_mode-从-v51-版本开始引入) | Modified | Specifies whether to enable dynamic pruning. Since v6.3.0, the default value changes to `dynamic`. | +| [`tidb_partition_prune_mode`](/system-variables.md#tidb_partition_prune_mode-new-in-v51) | Modified | Specifies whether to enable dynamic pruning. Since v6.3.0, the default value changes to `dynamic`. | | [`tidb_rc_read_check_ts`](/system-variables.md#tidb_rc_read_check_ts-new-in-v600) | Modified | This variable is used to optimize the timestamp acquisition, which is suitable for scenarios with read-committed isolation level where read-write conflicts are rare. This feature is oriented to specific service workloads and might cause performance regression in other scenarios. For this reason, since v6.3.0, the scope of this variable changes from `GLOBAL | SESSION` to `INSTANCE`. That means you can enable this feature for specific TiDB instances. | | [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) | Newly added | This variable is used to optimize the acquisition of timestamps and is suitable for scenarios with few point-write conflicts in `READ-COMMITTED` isolation level of pessimistic transactions. Enabling this variable can avoid the latency and overhead brought by obtaining the global timestamps during the execution of point-write statements | | [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) | Newly added | This variable controls whether to enable FastScan. If [FastScan](/develop/dev-guide-use-fastscan.md) is enabled (set to `ON`), TiFlash provides more efficient query performance, but does not guarantee the accuracy of the query results or data consistency. | +| [`sql_require_primary_key`](/system-variables.md#sql_require_primary_key-new-in-v630) | Newly added | This variable controls whether to enforce the requirement that a table has a primary key. After this variable is enabled, attempting to create or alter a table without a primary key will produce an error. | +| [`tidb_ddl_flashback_concurrency`](/system-variables.md#tidb_ddl_flashback_concurrency-new-in-v630) | Newly added | Controls the concurrency of `flashback cluster`. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | +| [`tidb_enable_foreign_key`](/system-variables.md#tidb_enable_foreign_key-new-in-v630) | Newly added | Controls whether to enable the `FOREIGN KEY` feature. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | +| [`tidb_enable_general_plan_cache`](/system-variables.md#tidb_enable_general_plan_cache-new-in-v630) | Newly added | Controls whether to enable the General Plan Cache feature. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | +| [`tidb_enable_null_aware_anti_join`](/system-variables.md#tidb_enable_null_aware_anti_join-new-in-v630) | Newly added | Controls whether TiDB applies Null Aware Hash Join when ANTI JOIN is generated by subqueries led by special set operators `NOT IN` and `!= ALL`. | +| [`tidb_enable_tiflash_read_for_write_stmt`](/system-variables.md#tidb_enable_tiflash_read_for_write_stmt-new-in-v630) | Newly added | Controls whether read requests in SQL write statements are pushed down to TiFlash. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | +| [`tidb_general_plan_cache_size`](/system-variables.md#tidb_general_plan_cache_size-new-in-v630) | Newly added | Controls the maximum number of execution plans that can be cached by General Plan Cache. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | ### Configuration file parameters @@ -253,9 +264,10 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - sql-infra - - Grant privilege of a table to an user checks the target table exist first, in the past, the table name comparison works in a case sensitive manner, now it's changed to case insensitive [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) - - Previously, TiDB users can set `init_connect` without any checking. From now on, the value of `init_connect` should be checked by the sql parser [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) - - Improve warning log when new connection arrives [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) + - TiDB is now case insensitive to the target table name when checking the table existence [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) + - Improved MySQL compatibility by adding the parsing process on the value of `init_connect` [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Improve log warning generated when new connections arrive [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) + - Optimize the HTTP API for querying DDL history jobs, and add support for the `start_job_id` parameter [#35838](https://github.com/pingcap/tidb/issues/35838) @[tiancaiamao](https://github.com/tiancaiamao) - execution @@ -336,21 +348,20 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - sql-infra - - Fix the issue that `PREAPRE` statements do not check privileges [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) - - System variable `tidb_enable_noop_variable` cannot be set to `WARN` [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that when 'expression index' is defined, the value of `ORDINAL_POSITION` column of `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) - - Fix the issue that when setting a timestamp that is larger than `MAXINT32`, TiDB doesn't report an error like MySQL [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) - - Fix the panic issue of enterprise plugin on 6.1 [#37319](https://github.com/pingcap/tidb/issues/37319) @[xhebox](https://github.com/xhebox) + - Fix the issue that privilege check is skipped for `PREAPRE` statements [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that the system variable `tidb_enable_noop_variable` cannot be set to `WARN` [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that when 'expression index' is defined, the `ORDINAL_POSITION` column of the `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) + - Fix the issue that TiDB does not report an error when the timestamp is larger than `MAXINT32` [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) + - Fix the issue that TiDB server cannot be started when the enterprise plugin is used [#37319](https://github.com/pingcap/tidb/issues/37319) @[xhebox](https://github.com/xhebox) - Fix the incorrect output of `SHOW CREATE PLACEMENT POLICY` [#37526](https://github.com/pingcap/tidb/issues/37526) @[xhebox](https://github.com/xhebox) - - Disallow exchange partition with temporary table [#37201](https://github.com/pingcap/tidb/issues/37201) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that query on `INFORMATION_SCHEMA.TIKV_REGION_STATUS` returns an incorrect result @[zimulala](https://github.com/zimulala) - - Fix the issue that `EXPLAIN` query on views does not check privileges [#34326](https://github.com/pingcap/tidb/issues/34326) @[hawkingrei](https://github.com/hawkingrei) - - Fix the issue that the user cannot update from JSON 'null' to NULL [#37852](https://github.com/pingcap/tidb/issues/37852) @[YangKeao](https://github.com/YangKeao) - - Optimize DDL history HTTP API, and add support for 'start_job_id' parameter [#35838](https://github.com/pingcap/tidb/issues/35838) @[tiancaiamao](https://github.com/tiancaiamao) + - Fix unexpected exchange partition with temporary tables [#37201](https://github.com/pingcap/tidb/issues/37201) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that querying `INFORMATION_SCHEMA.TIKV_REGION_STATUS` returns an incorrect result @[zimulala](https://github.com/zimulala) + - Fix the issue that the `EXPLAIN` query on views does not check privileges [#34326](https://github.com/pingcap/tidb/issues/34326) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that JSON `null` cannot be updated to `NULL` [#37852](https://github.com/pingcap/tidb/issues/37852) @[YangKeao](https://github.com/YangKeao) - Fix the issue that `row_count` of DDL jobs is inaccurate [#25968](https://github.com/pingcap/tidb/issues/25968) @[Defined2014](https://github.com/Defined2014) - Fix the issue that `FLASHBACK TABLE` does not work properly [#37386](https://github.com/pingcap/tidb/issues/37386) @[tiancaiamao](https://github.com/tiancaiamao) - Fix handling of prepared statement flags in the classic MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[dveeden](https://github.com/dveeden) - - Fix data-race issue on start up in rare cases [#36791](https://github.com/pingcap/tidb/issues/36791) @[xhebox](https://github.com/xhebox) + - (dup) Fix the issue of incorrect TiDB status that might appear on startup in some extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) @[xhebox](https://github.com/xhebox) - Fix the issue that `information_schema.variables_info` does not respect SEM [#37586](https://github.com/pingcap/tidb/issues/37586) @[CbcWestwolf](https://github.com/CbcWestwolf) - execution From bfb20f5c8474f1f78a361d5686353c343db1cc80 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 28 Sep 2022 14:31:33 +0800 Subject: [PATCH 070/104] Apply suggestions from code review --- releases/release-6.3.0.md | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index b69b5bb7bf97d..bc72aafd9de0b 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -271,10 +271,9 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - execution - - report error if json path has the wrong syntax [#22525](https://github.com/pingcap/tidb/issues/22525) @[xiongjiwei](https://github.com/xiongjiwei) - - report error if json path has the wrong syntax [#34959](https://github.com/pingcap/tidb/issues/34959) @[xiongjiwei](https://github.com/xiongjiwei) - - Improve the performance of Join by fix a false sharing problem. [#37641](https://github.com/pingcap/tidb/issues/37641) - - Improve the MySQL compatibility of `CAST` function in terms of float-to-time conversion [#37462](https://github.com/pingcap/tidb/issues/37462) @[mengxin9014](https://github.com/mengxin9014) + - Report errors when the JSON path has wrong syntax [#22525](https://github.com/pingcap/tidb/issues/22525) [#34959](https://github.com/pingcap/tidb/issues/34959) @[xiongjiwei](https://github.com/xiongjiwei) + - Improve the performance of Join operations by fixing a false sharing issue [#37641](https://github.com/pingcap/tidb/issues/37641) + - Improve the MySQL compatibility of the `CAST` function for the float-to-time conversion [#37462](https://github.com/pingcap/tidb/issues/37462) @[mengxin9014](https://github.com/mengxin9014) - planner @@ -366,19 +365,18 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - execution - - Fix wrong result when enabling dynamic mode in partition table for tiflash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) - - Fix the issue that the cast and comparison between binary string and json is incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that the cast and comparison between binary string and json is incompatible with MySQL [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that the json_objectagg and json_arrayagg is not compatible with MySQL on binary value [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that comparison between json opaque causes panic [#37315](https://github.com/pingcap/tidb/issues/37315) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that the single precision float cannot be used in json aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that `UNION` may returns unexpected empty result [#36903](https://github.com/pingcap/tidb/issues/36903) @[tiancaiamao](https://github.com/tiancaiamao) + - Fix the wrong result that occurs when enabling dynamic mode in partitioned tables for TiFlash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) + - Fix the issue that the cast and comparison between binary strings and JSON are incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that `json_objectagg` and `json_arrayagg` in TiDB are not compatible with MySQL on binary values [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the comparison between JSON opaque values causes panic [#37315](https://github.com/pingcap/tidb/issues/37315) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the single precision float cannot be used in JSON aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the `UNION` operator might return unexpected empty result [#36903](https://github.com/pingcap/tidb/issues/36903) @[tiancaiamao](https://github.com/tiancaiamao) - transaction - - bugfix: do not acquire pessimistic lock for non-unique index keys [#36235](https://github.com/pingcap/tidb/issues/36235) - - Fix the auto-commit mode change related transaction commit behaviours [#36581](https://github.com/pingcap/tidb/issues/36581) @[cfzjywxk](https://github.com/cfzjywxk) - - Fix the issue explain analyze with DML executors may respond to the client before the transaction commit has finished [#37273](https://github.com/pingcap/tidb/issues/37373) @[cfzjywxk](https://github.com/cfzjywxk) + - Fix the issue that pessimistic DML operations lock non-unique index keys [#36235](https://github.com/pingcap/tidb/issues/36235) + - Fix the transaction commit behaviours related to auto-commit mode change [#36581](https://github.com/pingcap/tidb/issues/36581) @[cfzjywxk](https://github.com/cfzjywxk) + - Fix the issue that the `EXPLAIN ANALYZE` statement with DML executors might return result before the transaction commit has finished [#37273](https://github.com/pingcap/tidb/issues/37373) @[cfzjywxk](https://github.com/cfzjywxk) - planner From 5934a6c16d5d5bb14dac75e9775cafa80684d354 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 28 Sep 2022 14:35:41 +0800 Subject: [PATCH 071/104] Apply suggestions from code review --- releases/release-6.3.0.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index bc72aafd9de0b..75213c41a505a 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -296,20 +296,20 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiFlash - - compute + - Compute - - Support to pushdown elt to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) - - Support to pushdown leftShift to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) - - Support to pushdown castTimeAsDuration to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - - Support Planner Interpreter [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - - Support to pushdown hex to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) - - Suppress the "tcp set inq" loggings [#4940](https://github.com/pingcap/tiflash/issues/4940) - - Improve the accuracy of memory tracker in TiFlash [#5610](https://github.com/pingcap/tiflash/pull/5610) - - Improve the performance of string column with `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collation [#5294](https://github.com/pingcap/tiflash/issues/5294) + - Support to push down the `elt` function to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) + - Support to push down the `leftShift` function to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) + - Support to push down the `castTimeAsDuration` function to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) + - Refactor TiFlash's interpreter, and support the new interpreter Planner [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) + - Support to push down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) + - Suppress the "cannot set inq" logs [#4940](https://github.com/pingcap/tiflash/issues/4940) + - Improve the accuracy of memory tracker in TiFlash [#5609](https://github.com/pingcap/tiflash/issues/5609) + - Improve the performance of string columns with the `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collations [#5294](https://github.com/pingcap/tiflash/issues/5294) - - storage + - Storage - - Calculate the io throughput in background in ReadLimiter [#5401](https://github.com/pingcap/tiflash/issues/5401), [#5091](https://github.com/pingcap/tiflash/issues/5091) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + - Calculate the I/O throughput in background in ReadLimiter [#5401](https://github.com/pingcap/tiflash/issues/5401), [#5091](https://github.com/pingcap/tiflash/issues/5091) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + Tools From 4d86031540588896c13c5516a1fa45429e86d311 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 28 Sep 2022 14:36:03 +0800 Subject: [PATCH 072/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 75213c41a505a..452a20236ad4a 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -412,17 +412,17 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiFlash - - compute + - Compute - - Fix the bug that window function may cause tiflash crash when canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - - Fix the bug that wrong data input for `cast(value as datetime)` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) - - fix that the result of expression casting real or decimal as time is inconsistent with mysql [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) + - Fix the bug that the WindowFunctionLeadLagBase may cause TifFlash to crash when canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) + - Fix the bug that wrong data input for `date(CAST(value AS DATETIME))` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) + - Fix that the result of expression casting real or decimal as time is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) - - storage + - Storage - - fix the problem that there may be some obsolete data left in storage which cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the issue that there may be some obsolete data left in storage which cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) - Fix the bug that page GC may block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) - - Fix the panic issue after creating the primary index with a column containing `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the panic issue that occurs after creating the primary index with a column containing the `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) + Tools From 5b8f4faafa3fb8d3d4b57ccc8834aad6d9bcf2d9 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 28 Sep 2022 14:53:45 +0800 Subject: [PATCH 073/104] refine system variable description --- releases/release-6.3.0.md | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 452a20236ad4a..e5646437371de 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -203,24 +203,24 @@ In v6.3.0-DMR, the key new features and improvements are as follows: | Variable name | Change type (newly added, modified, or deleted) | Description | | --- | --- | --- | | [`default_authentication_plugin`](/system-variables.md#default_authentication_plugin) | Modified | Added a new option `tidb_sm3_password`. When this variable is set to `tidb_sm3_password`, the `tidb_sm3_password` method is used as the encryption algorithm. | -| [`tidb_adaptive_closest_read_threshold`](/system-variables.md#tidb_adaptive_closest_read_threshold-new-in-v630) | Newly added | This variable is used to control the threshold at which the TiDB server prefers to send read requests to the replica in the same region as the TiDB server when [`tidb_replica_read`](#tidb_replica_read-new-in-v40) is set to `closest-adaptive`. | -| [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) | Newly added | This variable is used to control when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions | -| [`tidb_ddl_disk_quota`](/system-variables.md#tidb_ddl_disk_quota-new-in-v630) | Newly added | This variable only takes effect when [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) is enabled. It sets the usage limit of local storage during backfilling when creating an index. | -| [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) | Newly added | This variable controls whether to enable the acceleration of `ADD INDEX` and `CREATE INDEX` DDl operations to improve the speed of backfilling when creating an index. | +| [`tidb_adaptive_closest_read_threshold`](/system-variables.md#tidb_adaptive_closest_read_threshold-new-in-v630) | Newly added | Controls the threshold at which the TiDB server prefers to send read requests to the replica in the same region as the TiDB server when [`tidb_replica_read`](/system-variables.md#tidb_replica_read-new-in-v40) is set to `closest-adaptive`. | +| [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) | Newly added | Controls when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions. | +| [`tidb_ddl_disk_quota`](/system-variables.md#tidb_ddl_disk_quota-new-in-v630) | Newly added | Takes effect only when [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) is enabled. It sets the usage limit of local storage during backfilling when creating an index. | +| [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) | Newly added | Controls whether to enable the acceleration of `ADD INDEX` and `CREATE INDEX` DDl operations to improve the speed of backfilling when creating an index. | | [`tidb_enable_clustered_index`](/system-variables.md#tidb_enable_clustered_index-new-in-v50) | Modified | The default value of this variable changes from `INT_ONLY` to `ON`. | -| [`tidb_enable_exchange_partition`](/system-variables.md#tidb_enable_exchange_partition) | Deprecated | This variable controls whether to enable the [`exchange partitions with tables`](/partitioned-table.md#partition-management) feature. The default value is `ON`, that is, exchange partitions with tables is enabled by default. | -| [`tidb_enable_metadata_lock`](/system-variables.md#tidb_enable_metadata_lock-new-in-v630)| Newly added | This variable is used to set whether to enable the [Metadata lock](/metadata-lock.md) feature. | -| [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Modified | This variable controls the behavior of the optimizer on using statistics of a table when the statistics are outdated. The default value changes from `ON` to `OFF`, which means the optimizer still keeps using the statistics of the table even if the statistics of this table is outdated. | -| [`tidb_enable_rate_limit_action`](/system-variables.md#tidb_enable_rate_limit_action) | Modified | This variable controls whether to enable the dynamic memory control feature for the operator that reads data. When this variable is set to `ON`, the memory usage might not be under the control of [tidb_mem_quota_query](/system-variables.md#tidb_mem_quota_query). Therefore, the default value is changed from `ON` to `OFF`. | -| [`tidb_enable_unsafe_substitute`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | This variable controls whether to replace expressions with generated columns in an unsafe way. | -| [`tidb_last_plan_replayer_token`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | This variable is read-only and is used to obtain the result of the last `PLAN REPLAYER DUMP` execution in the current session. | -| [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) | Newly added | This variable is used to control whether common table expressions (CTEs) in the entire session are inlined or not. The default value is `OFF`, which means that inlining CTE is not enforced by default. | -| [`tidb_opt_three_stage_distinct_agg`](/system-variables.md#tidb_opt_three_stage_distinct_agg-new-in-v630) | Newly added | This variable specifies whether to rewrite a `COUNT(DISTINCT)` aggregation into a three-stage aggregation in MPP mode. The default value is `ON`. | +| [`tidb_enable_exchange_partition`](/system-variables.md#tidb_enable_exchange_partition) | Deprecated | Controls whether to enable the [`exchange partitions with tables`](/partitioned-table.md#partition-management) feature. The default value is `ON`, that is, exchange partitions with tables is enabled by default. | +| [`tidb_enable_metadata_lock`](/system-variables.md#tidb_enable_metadata_lock-new-in-v630)| Newly added | Specifies whether to enable the [Metadata lock](/metadata-lock.md) feature. | +| [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Modified | Controls the behavior of the optimizer on using statistics of a table when the statistics are outdated. The default value changes from `ON` to `OFF`, which means the optimizer still keeps using the statistics of the table even if the statistics of this table are outdated. | +| [`tidb_enable_rate_limit_action`](/system-variables.md#tidb_enable_rate_limit_action) | Modified | Controls whether to enable the dynamic memory control feature for the operator that reads data. When this variable is set to `ON`, the memory usage might not be under the control of [tidb_mem_quota_query](/system-variables.md#tidb_mem_quota_query). Therefore, the default value is changed from `ON` to `OFF`. | +| [`tidb_enable_unsafe_substitute`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | Controls whether to replace expressions with generated columns in an unsafe way. | +| [`tidb_last_plan_replayer_token`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | Read-only and used to obtain the result of the last `PLAN REPLAYER DUMP` execution in the current session. | +| [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) | Newly added | Controls whether common table expressions (CTEs) in the entire session are inlined or not. The default value is `OFF`, which means that inlining CTE is not enforced by default. | +| [`tidb_opt_three_stage_distinct_agg`](/system-variables.md#tidb_opt_three_stage_distinct_agg-new-in-v630) | Newly added | Specifies whether to rewrite a `COUNT(DISTINCT)` aggregation into a three-stage aggregation in MPP mode. The default value is `ON`. | | [`tidb_partition_prune_mode`](/system-variables.md#tidb_partition_prune_mode-new-in-v51) | Modified | Specifies whether to enable dynamic pruning. Since v6.3.0, the default value changes to `dynamic`. | -| [`tidb_rc_read_check_ts`](/system-variables.md#tidb_rc_read_check_ts-new-in-v600) | Modified | This variable is used to optimize the timestamp acquisition, which is suitable for scenarios with read-committed isolation level where read-write conflicts are rare. This feature is oriented to specific service workloads and might cause performance regression in other scenarios. For this reason, since v6.3.0, the scope of this variable changes from `GLOBAL | SESSION` to `INSTANCE`. That means you can enable this feature for specific TiDB instances. | -| [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) | Newly added | This variable is used to optimize the acquisition of timestamps and is suitable for scenarios with few point-write conflicts in `READ-COMMITTED` isolation level of pessimistic transactions. Enabling this variable can avoid the latency and overhead brought by obtaining the global timestamps during the execution of point-write statements | -| [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) | Newly added | This variable controls whether to enable FastScan. If [FastScan](/develop/dev-guide-use-fastscan.md) is enabled (set to `ON`), TiFlash provides more efficient query performance, but does not guarantee the accuracy of the query results or data consistency. | -| [`sql_require_primary_key`](/system-variables.md#sql_require_primary_key-new-in-v630) | Newly added | This variable controls whether to enforce the requirement that a table has a primary key. After this variable is enabled, attempting to create or alter a table without a primary key will produce an error. | +| [`tidb_rc_read_check_ts`](/system-variables.md#tidb_rc_read_check_ts-new-in-v600) | Modified | Used for optimizing the timestamp acquisition, which is suitable for scenarios with read-committed isolation level where read-write conflicts are rare. This feature is oriented to specific service workloads and might cause performance regression in other scenarios. For this reason, since v6.3.0, the scope of this variable changes from `GLOBAL | SESSION` to `INSTANCE`. That means you can enable this feature for specific TiDB instances. | +| [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) | Newly added | Used for optimizing the acquisition of timestamps and is suitable for scenarios with few point-write conflicts in `READ-COMMITTED` isolation level of pessimistic transactions. Enabling this variable can avoid the latency and overhead brought by obtaining the global timestamps during the execution of point-write statements | +| [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) | Newly added | Controls whether to enable FastScan. If [FastScan](/develop/dev-guide-use-fastscan.md) is enabled (set to `ON`), TiFlash provides more efficient query performance, but does not guarantee the accuracy of the query results or data consistency. | +| [`sql_require_primary_key`](/system-variables.md#sql_require_primary_key-new-in-v630) | Newly added | Controls whether to enforce the requirement that a table has a primary key. After this variable is enabled, attempting to create or alter a table without a primary key will produce an error. | | [`tidb_ddl_flashback_concurrency`](/system-variables.md#tidb_ddl_flashback_concurrency-new-in-v630) | Newly added | Controls the concurrency of `flashback cluster`. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | | [`tidb_enable_foreign_key`](/system-variables.md#tidb_enable_foreign_key-new-in-v630) | Newly added | Controls whether to enable the `FOREIGN KEY` feature. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | | [`tidb_enable_general_plan_cache`](/system-variables.md#tidb_enable_general_plan_cache-new-in-v630) | Newly added | Controls whether to enable the General Plan Cache feature. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | From 4a0bc47713db49c45efa3ab2a8fb2c2993626d74 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 28 Sep 2022 14:56:36 +0800 Subject: [PATCH 074/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index e5646437371de..2924de60b3f19 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -414,7 +414,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Compute - - Fix the bug that the WindowFunctionLeadLagBase may cause TifFlash to crash when canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) + - Fix the bug that the window function may cause TifFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - Fix the bug that wrong data input for `date(CAST(value AS DATETIME))` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) - Fix that the result of expression casting real or decimal as time is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) From 9095b8942e78a8f1bc187fe68fe9ad1231064e50 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 28 Sep 2022 15:06:10 +0800 Subject: [PATCH 075/104] refine wording for key features --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 2924de60b3f19..6165049f81925 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -10,8 +10,8 @@ TiDB version: 6.3.0-DMR In v6.3.0-DMR, the key new features and improvements are as follows: -- TiKV support SM4 encryption at rest. -- TiDB supports authentication with the SM3 algorithm. +- TiKV supports encryption at rest using the SM4 algorithm. +- TiDB supports authentication using the SM3 algorithm. - The `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option. - JSON data type and functions become generally available (GA). - TiDB supports null-aware anti join. From be2b0e3eed49c665e95878c3cfa20e878ac2ca03 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 28 Sep 2022 15:11:53 +0800 Subject: [PATCH 076/104] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- releases/release-6.3.0.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 6165049f81925..fb93f02196e49 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -264,9 +264,9 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - sql-infra - - TiDB is now case insensitive to the target table name when checking the table existence [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) - - Improved MySQL compatibility by adding the parsing process on the value of `init_connect` [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) - - Improve log warning generated when new connections arrive [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) + - TiDB is now case-insensitive to the target table name when checking the table existence [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) + - Improve MySQL compatibility by adding a parsing check when setting the value of `init_connect` [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Improve the log warning generated for new connections [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) - Optimize the HTTP API for querying DDL history jobs, and add support for the `start_job_id` parameter [#35838](https://github.com/pingcap/tidb/issues/35838) @[tiancaiamao](https://github.com/tiancaiamao) - execution @@ -347,21 +347,21 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - sql-infra - - Fix the issue that privilege check is skipped for `PREAPRE` statements [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that the system variable `tidb_enable_noop_variable` cannot be set to `WARN` [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that when 'expression index' is defined, the `ORDINAL_POSITION` column of the `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) + - Fix the issue that the privilege check is skipped for `PREAPRE` statements [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that the system variable `tidb_enable_noop_variable` can be set to `WARN` [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that when an expression index is defined, the `ORDINAL_POSITION` column of the `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) - Fix the issue that TiDB does not report an error when the timestamp is larger than `MAXINT32` [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) - Fix the issue that TiDB server cannot be started when the enterprise plugin is used [#37319](https://github.com/pingcap/tidb/issues/37319) @[xhebox](https://github.com/xhebox) - Fix the incorrect output of `SHOW CREATE PLACEMENT POLICY` [#37526](https://github.com/pingcap/tidb/issues/37526) @[xhebox](https://github.com/xhebox) - - Fix unexpected exchange partition with temporary tables [#37201](https://github.com/pingcap/tidb/issues/37201) @[lcwangchao](https://github.com/lcwangchao) + - Fix the unexpected `EXCHANGE PARTITION` behaviors with temporary tables [#37201](https://github.com/pingcap/tidb/issues/37201) @[lcwangchao](https://github.com/lcwangchao) - Fix the issue that querying `INFORMATION_SCHEMA.TIKV_REGION_STATUS` returns an incorrect result @[zimulala](https://github.com/zimulala) - Fix the issue that the `EXPLAIN` query on views does not check privileges [#34326](https://github.com/pingcap/tidb/issues/34326) @[hawkingrei](https://github.com/hawkingrei) - Fix the issue that JSON `null` cannot be updated to `NULL` [#37852](https://github.com/pingcap/tidb/issues/37852) @[YangKeao](https://github.com/YangKeao) - Fix the issue that `row_count` of DDL jobs is inaccurate [#25968](https://github.com/pingcap/tidb/issues/25968) @[Defined2014](https://github.com/Defined2014) - Fix the issue that `FLASHBACK TABLE` does not work properly [#37386](https://github.com/pingcap/tidb/issues/37386) @[tiancaiamao](https://github.com/tiancaiamao) - - Fix handling of prepared statement flags in the classic MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[dveeden](https://github.com/dveeden) + - Fix the issue of failing to handle `prepared` statement flags in the typical MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[dveeden](https://github.com/dveeden) - (dup) Fix the issue of incorrect TiDB status that might appear on startup in some extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) @[xhebox](https://github.com/xhebox) - - Fix the issue that `information_schema.variables_info` does not respect SEM [#37586](https://github.com/pingcap/tidb/issues/37586) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Fix the issue that `information_schema.variables_info` does not comply with security enhanced mode (SEM) [#37586](https://github.com/pingcap/tidb/issues/37586) @[CbcWestwolf](https://github.com/CbcWestwolf) - execution From 71a6750cebc93dcdb4eaf0a4b796e47730c211c6 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 28 Sep 2022 15:21:56 +0800 Subject: [PATCH 077/104] refine compatibility-others --- releases/release-6.3.0.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index fb93f02196e49..2a2d1d5d7e790 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -249,10 +249,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Others -* Improve MySQL compatibility by supporting the `ACCOUNT LOCK` and `ACCOUNT UNLOCK` options. * Log backup supports GCS and Azure Blob Storage as backup storage. * Log backup is now compatible with the `exchange partition` DDL. -* The SQL statement `ALTER TABLE ...SET TiFLASH MODE ...` previously used for enabling [fastscan](/develop/dev-guide-use-fastscan.md) is deprecated, and replaced by the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630). When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. You need to set the variable [`tiflash_fastscan`]. When you upgrade from an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistency. +* The SQL statement `ALTER TABLE ...SET TiFLASH MODE ...` previously used for enabling [fastscan](/develop/dev-guide-use-fastscan.md) is deprecated, and replaced by the system variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630). When you upgrade from v6.2.0 to v6.3.0, all FastScan settings in v6.2.0 will become invalid, but will not affect the normal reading of data. In this case, you need to configure the variable [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) to enable or disable FastScan. When you upgrade from an earlier version to v6.3.0, the FastScan feature is not enabled by default for all sessions to keep data consistent. ## Removed feature From 3d39a1f16d6a79c4701fe85d5be51bbfc65ce0e0 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 28 Sep 2022 16:00:45 +0800 Subject: [PATCH 078/104] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Aolin Co-authored-by: Daniël van Eeden --- releases/release-6.3.0.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 2a2d1d5d7e790..9ba305a4a3c39 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -202,13 +202,13 @@ In v6.3.0-DMR, the key new features and improvements are as follows: | Variable name | Change type (newly added, modified, or deleted) | Description | | --- | --- | --- | -| [`default_authentication_plugin`](/system-variables.md#default_authentication_plugin) | Modified | Added a new option `tidb_sm3_password`. When this variable is set to `tidb_sm3_password`, the `tidb_sm3_password` method is used as the encryption algorithm. | +| [`default_authentication_plugin`](/system-variables.md#default_authentication_plugin) | Modified | Adds a new option `tidb_sm3_password`. When this variable is set to `tidb_sm3_password`, SM3 is used as the encryption algorithm. | | [`tidb_adaptive_closest_read_threshold`](/system-variables.md#tidb_adaptive_closest_read_threshold-new-in-v630) | Newly added | Controls the threshold at which the TiDB server prefers to send read requests to the replica in the same region as the TiDB server when [`tidb_replica_read`](/system-variables.md#tidb_replica_read-new-in-v40) is set to `closest-adaptive`. | | [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) | Newly added | Controls when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions. | | [`tidb_ddl_disk_quota`](/system-variables.md#tidb_ddl_disk_quota-new-in-v630) | Newly added | Takes effect only when [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) is enabled. It sets the usage limit of local storage during backfilling when creating an index. | -| [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) | Newly added | Controls whether to enable the acceleration of `ADD INDEX` and `CREATE INDEX` DDl operations to improve the speed of backfilling when creating an index. | +| [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) | Newly added | Controls whether to enable the acceleration of `ADD INDEX` and `CREATE INDEX` DDL operations to improve the speed of backfilling when creating an index. | | [`tidb_enable_clustered_index`](/system-variables.md#tidb_enable_clustered_index-new-in-v50) | Modified | The default value of this variable changes from `INT_ONLY` to `ON`. | -| [`tidb_enable_exchange_partition`](/system-variables.md#tidb_enable_exchange_partition) | Deprecated | Controls whether to enable the [`exchange partitions with tables`](/partitioned-table.md#partition-management) feature. The default value is `ON`, that is, exchange partitions with tables is enabled by default. | +| [`tidb_enable_exchange_partition`](/system-variables.md#tidb_enable_exchange_partition) | Deprecated | Controls whether to enable the [`exchange partitions with tables`](/partitioned-table.md#partition-management) feature. The default value is `ON`, that is, `exchange partitions with tables` is enabled by default. | | [`tidb_enable_metadata_lock`](/system-variables.md#tidb_enable_metadata_lock-new-in-v630)| Newly added | Specifies whether to enable the [Metadata lock](/metadata-lock.md) feature. | | [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Modified | Controls the behavior of the optimizer on using statistics of a table when the statistics are outdated. The default value changes from `ON` to `OFF`, which means the optimizer still keeps using the statistics of the table even if the statistics of this table are outdated. | | [`tidb_enable_rate_limit_action`](/system-variables.md#tidb_enable_rate_limit_action) | Modified | Controls whether to enable the dynamic memory control feature for the operator that reads data. When this variable is set to `ON`, the memory usage might not be under the control of [tidb_mem_quota_query](/system-variables.md#tidb_mem_quota_query). Therefore, the default value is changed from `ON` to `OFF`. | @@ -217,8 +217,8 @@ In v6.3.0-DMR, the key new features and improvements are as follows: | [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) | Newly added | Controls whether common table expressions (CTEs) in the entire session are inlined or not. The default value is `OFF`, which means that inlining CTE is not enforced by default. | | [`tidb_opt_three_stage_distinct_agg`](/system-variables.md#tidb_opt_three_stage_distinct_agg-new-in-v630) | Newly added | Specifies whether to rewrite a `COUNT(DISTINCT)` aggregation into a three-stage aggregation in MPP mode. The default value is `ON`. | | [`tidb_partition_prune_mode`](/system-variables.md#tidb_partition_prune_mode-new-in-v51) | Modified | Specifies whether to enable dynamic pruning. Since v6.3.0, the default value changes to `dynamic`. | -| [`tidb_rc_read_check_ts`](/system-variables.md#tidb_rc_read_check_ts-new-in-v600) | Modified | Used for optimizing the timestamp acquisition, which is suitable for scenarios with read-committed isolation level where read-write conflicts are rare. This feature is oriented to specific service workloads and might cause performance regression in other scenarios. For this reason, since v6.3.0, the scope of this variable changes from `GLOBAL | SESSION` to `INSTANCE`. That means you can enable this feature for specific TiDB instances. | -| [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) | Newly added | Used for optimizing the acquisition of timestamps and is suitable for scenarios with few point-write conflicts in `READ-COMMITTED` isolation level of pessimistic transactions. Enabling this variable can avoid the latency and overhead brought by obtaining the global timestamps during the execution of point-write statements | +| [`tidb_rc_read_check_ts`](/system-variables.md#tidb_rc_read_check_ts-new-in-v600) | Modified | Used for optimizing the timestamp acquisition, which is suitable for scenarios with read-committed isolation level where read-write conflicts are rare. This feature is oriented to specific service workloads and might cause performance regression in other scenarios. For this reason, since v6.3.0, the scope of this variable changes from `GLOBAL \| SESSION` to `INSTANCE`. That means you can enable this feature for specific TiDB instances. | +| [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) | Newly added | Used for optimizing the acquisition of timestamps and is suitable for scenarios with few point-write conflicts in read-committed isolation level of pessimistic transactions. Enabling this variable can avoid the latency and overhead brought by obtaining the global timestamps during the execution of point-write statements | | [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) | Newly added | Controls whether to enable FastScan. If [FastScan](/develop/dev-guide-use-fastscan.md) is enabled (set to `ON`), TiFlash provides more efficient query performance, but does not guarantee the accuracy of the query results or data consistency. | | [`sql_require_primary_key`](/system-variables.md#sql_require_primary_key-new-in-v630) | Newly added | Controls whether to enforce the requirement that a table has a primary key. After this variable is enabled, attempting to create or alter a table without a primary key will produce an error. | | [`tidb_ddl_flashback_concurrency`](/system-variables.md#tidb_ddl_flashback_concurrency-new-in-v630) | Newly added | Controls the concurrency of `flashback cluster`. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | @@ -346,7 +346,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - sql-infra - - Fix the issue that the privilege check is skipped for `PREAPRE` statements [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that the privilege check is skipped for `PREPARE` statements [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) - Fix the issue that the system variable `tidb_enable_noop_variable` can be set to `WARN` [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) - Fix the issue that when an expression index is defined, the `ORDINAL_POSITION` column of the `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) - Fix the issue that TiDB does not report an error when the timestamp is larger than `MAXINT32` [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) From 4befc06db7fae5726fc156594cdddb20cc92f4f1 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 28 Sep 2022 16:27:40 +0800 Subject: [PATCH 079/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 9ba305a4a3c39..abb2fd0b3d278 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -413,7 +413,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Compute - - Fix the bug that the window function may cause TifFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) + - Fix the bug that the window function may cause TiFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - Fix the bug that wrong data input for `date(CAST(value AS DATETIME))` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) - Fix that the result of expression casting real or decimal as time is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) From 961b4a62f257c167c54f6c91a5e8597ede0a3ffd Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 28 Sep 2022 16:31:04 +0800 Subject: [PATCH 080/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index abb2fd0b3d278..8e85662535ae0 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -300,8 +300,8 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Support to push down the `elt` function to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) - Support to push down the `leftShift` function to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) - Support to push down the `castTimeAsDuration` function to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - - Refactor TiFlash's interpreter, and support the new interpreter Planner [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - - Support to push down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) + - Support to push down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) + - Refactor TiFlash's interpreter, and support the new interpreter Planner [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - Suppress the "cannot set inq" logs [#4940](https://github.com/pingcap/tiflash/issues/4940) - Improve the accuracy of memory tracker in TiFlash [#5609](https://github.com/pingcap/tiflash/issues/5609) - Improve the performance of string columns with the `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collations [#5294](https://github.com/pingcap/tiflash/issues/5294) From f5ca9ec273e71ffb0331d5e62dee9a8a3bcbb0d4 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 28 Sep 2022 16:35:31 +0800 Subject: [PATCH 081/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 8e85662535ae0..7c274959bc7aa 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -300,8 +300,8 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Support to push down the `elt` function to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) - Support to push down the `leftShift` function to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) - Support to push down the `castTimeAsDuration` function to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - - Support to push down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) - - Refactor TiFlash's interpreter, and support the new interpreter Planner [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) + - Support to push down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) + - Refactor TiFlash's interpreter, and support the new interpreter Planner [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - Suppress the "cannot set inq" logs [#4940](https://github.com/pingcap/tiflash/issues/4940) - Improve the accuracy of memory tracker in TiFlash [#5609](https://github.com/pingcap/tiflash/issues/5609) - Improve the performance of string columns with the `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collations [#5294](https://github.com/pingcap/tiflash/issues/5294) From ea1f9f5174e2bf188a796e8aad0abcf4562398c4 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 28 Sep 2022 16:44:04 +0800 Subject: [PATCH 082/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 7c274959bc7aa..c28348a0e33fe 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -302,9 +302,9 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Support to push down the `castTimeAsDuration` function to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - Support to push down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) - Refactor TiFlash's interpreter, and support the new interpreter Planner [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - - Suppress the "cannot set inq" logs [#4940](https://github.com/pingcap/tiflash/issues/4940) - - Improve the accuracy of memory tracker in TiFlash [#5609](https://github.com/pingcap/tiflash/issues/5609) - - Improve the performance of string columns with the `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collations [#5294](https://github.com/pingcap/tiflash/issues/5294) + - Suppress the "cannot set inq" logs [#4940](https://github.com/pingcap/tiflash/issues/4940) @[LittleFall](https://github.com/LittleFall) + - Improve the accuracy of memory tracker in TiFlash [#5609](https://github.com/pingcap/tiflash/issues/5609) @[bestwoody](https://github.com/bestwoody) + - Improve the performance of string columns with the `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collations [#5294](https://github.com/pingcap/tiflash/issues/5294) @[solotzg](https://github.com/solotzg) - Storage From bbc2dfd266e9963939802cf0aa938414f5822d27 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Wed, 28 Sep 2022 16:52:02 +0800 Subject: [PATCH 083/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index c28348a0e33fe..0a010874bf8d8 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -415,7 +415,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix the bug that the window function may cause TiFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - Fix the bug that wrong data input for `date(CAST(value AS DATETIME))` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) - - Fix that the result of expression casting real or decimal as time is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) + - Fix that issue that the result of `cast(Real/Decimal)AsTime` is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) - Storage From 9be75d82dc92cfae9e1acfb2488777b7a5d19f3f Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 28 Sep 2022 17:35:35 +0800 Subject: [PATCH 084/104] Apply suggestions from code review --- releases/release-6.3.0.md | 63 ++++++++++++++++++--------------------- 1 file changed, 29 insertions(+), 34 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 0a010874bf8d8..40b2a80a8c452 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -35,16 +35,16 @@ In v6.3.0-DMR, the key new features and improvements are as follows: [EXCHANGE PARTITION](/partitioned-table.md#partition-management) becomes GA after performance and stability improvements. -* TiDB supports two more [window functions](/tiflash/tiflash-supported-pushdown-calculations.md) [#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) **tw：shichun-0415** +* TiDB supports two more [window functions](/tiflash/tiflash-supported-pushdown-calculations.md) [#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) * `LEAD()` * `LAG()` -* Provide lightweight metadata lock to improve the DML success rate during DDL change (experimental) [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) **tw: Oreoxmt** +* Provide lightweight metadata lock to improve the DML success rate during DDL change (experimental) [#37275](https://github.com/pingcap/tidb/issues/37275) @[wjhuang2016](https://github.com/wjhuang2016) TiDB uses the online asynchronous schema change algorithm to support changing metadata objects. When a transaction is executed, it obtains the corresponding metadata snapshot at the transaction start. If the metadata is changed during a transaction, to ensure data consistency, TiDB returns an `Information schema is changed` error and the transaction fails to commit. To solve the problem, TiDB v6.3.0 introduces [metadata lock](/metadata-lock.md) into the online DDL algorithm. To avoid DML errors whenever possible, TiDB coordinates the priority of DMLs and DDLs during table metadata change, and makes executing DDLs wait for the DMLs with old metadata to commit. -* Improve the performance of adding indexes and reduce its impact on DML transactions (experimental) [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) **tw: Oreoxmt** +* Improve the performance of adding indexes and reduce its impact on DML transactions (experimental) [#35983](https://github.com/pingcap/tidb/issues/35983) @[benjamin2037](https://github.com/benjamin2037) To improve the speed of backfilling when creating an index, TiDB v6.3.0 accelerates the `ADD INDEX` and `CREATE INDEX` DDL operations when the [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) system variable is enabled. When the feature is enabled, the performance of adding indexes is about trippled. @@ -54,25 +54,25 @@ In v6.3.0-DMR, the key new features and improvements are as follows: Add the [SM4 algorithm](/encryption-at-rest.md) for TiKV encryption at rest. When you configure encryption at rest, you can enable the SM4 encryption capacity by setting the value of the `data-encryption-method` configuration to `sm4-ctr`. -* TiDB supports authentication with the SM3 algorithm [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** +* TiDB supports authentication with the SM3 algorithm [#36192](https://github.com/pingcap/tidb/issues/36192) @[CbcWestwolf](https://github.com/CbcWestwolf) TiDB adds an authentication plugin [`tidb_sm3_password`](/security-compatibility-with-mysql.md) based on the SM3 algorithm. When this plugin is enabled, the user password is encrypted and validated using the SM3 algorithm. -* TiDB JDBC supports authentication with the SM3 algorithm [#25](https://github.com/pingcap/mysql-connector-j/issues/25) @[lastincisor](https://github.com/lastincisor) **tw：ran-huang** +* TiDB JDBC supports authentication with the SM3 algorithm [#25](https://github.com/pingcap/mysql-connector-j/issues/25) @[lastincisor](https://github.com/lastincisor) Authenticating the user password needs client-side support. Now because [JDBC supports the SM3 algorithm](/develop/dev-guide-choose-driver-or-orm.md#java-drivers), you can connect to TiDB using SM3 authentication via TiDB-JDBC. ### Observability -* TiDB provides fine-grained metrics of SQL query execution time [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: Oreoxmt** +* TiDB provides fine-grained metrics of SQL query execution time [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) TiDB v6.3.0 provides fine-grained data metrics for [detailed observation of execution time](/latency-breakdown.md). Through the complete and segmented metrics, you can clearly understand the main time consumption of SQL queries, and then quickly find key problems and save time in troubleshooting. -* Enhanced output for slow logs and `TRACE` statements [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: Oreoxmt** +* Enhanced output for slow logs and `TRACE` statements [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) TiDB v6.3.0 enhances the output of slow logs and `TRACE`. You can observe the [full-link duration](/latency-breakdown.md) of SQL queries from TiDB parsing to KV RocksDB writing to disk, which further enhances the diagnostic capabilities. -* TiDB Dashboard provides deadlock history information [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) **tw: shichun-0415** +* TiDB Dashboard provides deadlock history information [#34106](https://github.com/pingcap/tidb/issues/34106) @[cfzjywxk](https://github.com/cfzjywxk) From v6.3.0, TiDB Dashboard provides deadlock history. If you check the slow log in TiDB Dashboard and find the lock waiting time of some SQL statements to be excessively long, you can check the deadlock history to locate the root cause, which makes your diagnosis easier. @@ -98,39 +98,39 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiFlash supports rewriting queries containing only one `COUNT(DISTINCT)` into a [three-stage aggregation](/system-variables.md#tidb_opt_three_stage_distinct_agg-new-in-v630). This improves concurrency and performance. -* TiKV supports log recycling [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) **tw：ran-huang** +* TiKV supports log recycling [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) TiKV supports [recycling log files](/tikv-configuration-file.md#enable-log-recycle-new-in-v630) in Raft Engine. This reduces the long tail latency in network disks during Raft log appending and improves performance under write workloads. -* TiDB supports null-aware anti join [#37525](https://github.com/pingcap/tidb/issues/37525) @[Arenatlx](https://github.com/Arenatlx) **tw: Oreoxmt** +* TiDB supports null-aware anti join [#37525](https://github.com/pingcap/tidb/issues/37525) @[Arenatlx](https://github.com/Arenatlx) TiDB v6.3.0 introduces a new join type [Null-aware anti join (NAAJ)](/explain-subqueries.md#null-aware-anti-semi-join-not-in-and--all-subqueries). NAAJ can be aware of whether the collection is empty or `NULL` when processing collection operations. This optimizes the execution efficiency of operations such as `IN` and `= ANY` and improves SQL performance. -* Add optimizer hints to control the build end of Hash Join [#35439](https://github.com/pingcap/tidb/issues/35439) @[Reminiscent](https://github.com/Reminiscent) **tw: TomShawn** +* Add optimizer hints to control the build end of Hash Join [#35439](https://github.com/pingcap/tidb/issues/35439) @[Reminiscent](https://github.com/Reminiscent) In v6.3.0, the TiDB optimizer introduces 2 hints, `HASH_JOIN_BUILD()` and `HASH_JOIN_PROBE()`, to specify the Hash Join, its probe end, and its build end. When the optimizer fails to select the optimal execution plan, you can use these hints to intervene with the plan. -* Support session-level common table expressions (CTE) inline [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) **tw: shichun-0415** +* Support session-level common table expressions (CTE) inline [#36514](https://github.com/pingcap/tidb/issues/36514) @[elsa0520](https://github.com/elsa0520) TiDB v6.2.0 introduced the `MERGE` hint in optimizers to allow CTE inline, so that the consumers of a CTE query result can execute it in parallel in TiFlash. In v6.3.0, a session variable [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) is introduced to allow CTE inline in sessions. This can greatly improve the ease of use. ### Transactions -* Support deferring checks of unique constraints in pessimistic transactions [#36579](https://github.com/pingcap/tidb/issues/36579) @[ekexium](https://github.com/ekexium) **tw: qiancai** +* Support deferring checks of unique constraints in pessimistic transactions [#36579](https://github.com/pingcap/tidb/issues/36579) @[ekexium](https://github.com/ekexium) You can use the [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) system variable to control when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions. This variable is disabled by default. When the variable is enabled (set to `ON`), TiDB will defer locking operations and unique constraint checks in pessimistic transactions until necessary, thus improving the performance of bulk DML operations. -* Optimize the way of fetching TSO in the Read-Committed isolation level [#36812](https://github.com/pingcap/tidb/issues/36812) @[TonsnakeLin](https://github.com/TonsnakeLin) **tw: TomShawn** +* Optimize the way of fetching TSO in the Read-Committed isolation level [#36812](https://github.com/pingcap/tidb/issues/36812) @[TonsnakeLin](https://github.com/TonsnakeLin) In the Read-Committed isolation level, the system variable [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) is introduced to control how TSO is fetched. In the case of Plan Cache hit, TiDB improves the execution efficiency of batch DML statements by reducing the frequency of fetching TSO, and reduces the execution time of running tasks in batch. ### Stability -* Modify the default policy of loading statistics when statistics become outdated [#27601](https://github.com/pingcap/tidb/issues/27601) @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) **tw: TomShawn** +* Modify the default policy of loading statistics when statistics become outdated [#27601](https://github.com/pingcap/tidb/issues/27601) @[xuyifangreeneyes](https://github.com/xuyifangreeneyes) In v5.3.0, TiDB introduced the system variable [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) to control how the optimizer behaves when the statistics become outdated. The default value is `ON`, which means keeping the behavior of the old version: When the statistics on objects that are involved in a SQL statement are outdated, the optimizer considers that statistics (other than the total number of rows on the table) are no longer reliable and uses pseudo statistics instead. After tests and analyses of actual user scenarios, the default value of `tidb_enable_pseudo_for_outdated_stats` is changed to `OFF` since v6.3.0. Even if the statistics become outdated, the optimizer will still use the statistics on the table, which makes the execution plan more stable. -* The feature of disabling Titan becomes GA [#issue]() @[tabokie](https://github.com/tabokie) **tw：ran-huang** +* The feature of disabling Titan becomes GA @[tabokie](https://github.com/tabokie) You can [disable Titan](/titan-configuration.md#disable-titan) for online TiKV nodes. @@ -142,7 +142,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### MySQL compatibility -* Improve MySQL 8.0 compatibility by adding support for four regular expression functions: `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) **tw: Oreoxmt** +* Improve MySQL 8.0 compatibility by adding support for four regular expression functions: `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) For more details about the compatibility with MySQL, see [Regular expression compatibility with MySQL](/functions-and-operators/string-functions.md#regular-expression-compatibility-with-mysql). @@ -150,13 +150,13 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature is not compatible with TiFlash due to conflicts in implementation mechanisms. TiDB v6.3.0 optimizes this feature, and [improves compatibility of SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). -* The `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) **tw：ran-huang** +* The `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) When you create a user using the [`CREATE USER`](/sql-statements/sql-statement-create-user.md) statement, you can specify whether the created user is locked using the `ACCOUNT LOCK/UNLOCK` option. A locked user cannot log in to the database. You can modify the lock state of an existing user using the `ACCOUNT LOCK/UNLOCK` option in the [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) statement. -* JSON data type and JSON functions become GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) **tw: qiancai** +* JSON data type and JSON functions become GA [#36993](https://github.com/pingcap/tidb/issues/36993) @[xiongjiwei](https://github.com/xiongjiwei) JSON is a popular data format adopted by a large number of programs. TiDB has introduced the [JSON support](/data-type-json.md) as an experimental feature since an earlier version, compatible with MySQL's JSON data type and some JSON functions. @@ -164,35 +164,35 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Backup and restore -* PITR supports GCS and Azure Blob Storage as backup storages [#issue]() @[joccau](https://github.com/joccau) **tw: shichun-0415** +* PITR supports GCS and Azure Blob Storage as backup storages @[joccau](https://github.com/joccau) PITR supports [GCS](/br/backup-storage-gcs.md) and [Azure Blob Storage](/br/backup-storage-azblob.md) as backup storages. If your TiDB cluster is deployed on GCP or Azure, you can use the PITR feature after upgrading your cluster to v6.3.0. -* BR supports AWS S3 Object Lock [#issue]() @[3pointer](https://github.com/3pointer) **tw: shichun-0415** +* BR supports AWS S3 Object Lock [#13442](https://github.com/tikv/tikv/issues/13442) @[3pointer](https://github.com/3pointer) You can protect backup data on AWS from being tampered with or deleted by enabling [S3 Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html). ### Data migration -* TiDB Lightning supports importing Parquet files exported by Apache Hive into TiDB [#issue]() @[buchuitoudegou](https://github.com/buchuitoudegou) **tw：ran-huang** +* TiDB Lightning supports importing Parquet files exported by Apache Hive into TiDB [#37536](https://github.com/pingcap/tidb/issues/37536) @[buchuitoudegou](https://github.com/buchuitoudegou) TiDB Lightning supports [importing Parquet files exported by Apache Hive into TiDB](/tidb-lightning/tidb-lightning-data-source.md#parquet), thereby achieving data migration from Hive to TiDB. -* DM adds a new configuration item `safe-mode-duration` in the task configuration file [#6224] (https://github.com/pingcap/tiflow/issues/6224) @[[okJiang](https://github.com/okJiang)] **tw：ran-huang** +* DM adds a new configuration item `safe-mode-duration` in the task configuration file [#6224](https://github.com/pingcap/tiflow/issues/6224) @[okJiang](https://github.com/okJiang) DM adds a new configuration item `safe-mode-duration` in the [task configuration file](/task-configuration-file-full.md). You can adjust the automatic safe mode duration after DM exits abnormally. The default value is 60 seconds. If `safe-mode-duration` is set to `"0s"`, an error is reported when DM tries to enter safe mode after an abnormal restart. ### TiDB data share subscription -* TiCDC supports a deployment topology that can replicate data from multiple geo-distributed data sources [#issue]() @[sdojjy](https://github.com/sdojjy) **tw：ran-huang** +* TiCDC supports a deployment topology that can replicate data from multiple geo-distributed data sources [#5301](https://github.com/pingcap/tiflow/issues/5301) @[sdojjy](https://github.com/sdojjy) - To support replicating data from a single TiDB cluster to multiple geo-distributed data systems, starting from v6.3.0, [you can deploy TiCDC in multiple IDCs](link) to replicate data for each IDC. This feature helps deliver the capability of geo-distributed data replication and deployment topology. + To support replicating data from a single TiDB cluster to multiple geo-distributed data systems, starting from v6.3.0, [you can deploy TiCDC in multiple IDCs](/ticdc/deploy-ticdc.md) to replicate data for each IDC. This feature helps deliver the capability of geo-distributed data replication and deployment topology. -* TiCDC supports keeping the snapshots consistent between the upstream and the downstream (sync point) [#issue]() @[asddongmen](https://github.com/asddongmen) **tw: TomShawn** +* TiCDC supports keeping the snapshots consistent between the upstream and the downstream (sync point) [#6977](https://github.com/pingcap/tiflow/issues/6977) @[asddongmen](https://github.com/asddongmen) - In the scenarios of data replication for disaster recovery, TiCDC supports periodically maintaining a downstream data snapshot so that the downstream snapshot is consistent with the upstream snapshot. With this feature, TiCDC can better support the scenarios where reads and writes are separate, and help you lower the cost. + In the scenarios of data replication for disaster recovery, TiCDC supports [periodically maintaining a downstream data snapshot](/sync-diff-inspector/upstream-downstream-diff.md#data-check-for-tidb-upstream-and-downstream-clusters) so that the downstream snapshot is consistent with the upstream snapshot. With this feature, TiCDC can better support the scenarios where reads and writes are separate, and help you lower the cost. -* TiCDC supports graceful upgrade [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) @[3AceShowHand](https://github.com/3AceShowHand) **tw:ran-huang** +* TiCDC supports graceful upgrade [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) @[3AceShowHand](https://github.com/3AceShowHand) When TiCDC is deployed using [TiUP](/ticdc/deploy-ticdc.md#rolling-upgrade-ticdc-using-tiup) (>=v1.11.0) or [TiDB Operator](https://docs.pingcap.com/tidb-in-kubernetes/v1.3/configure-a-tidb-cluster#configure-graceful-upgrade-for-ticdc-cluster) (>=v1.3.8), you can gracefully upgrade the TiCDC cluster. During the upgrade, data replication latency is kept as low as 30 seconds. This improves stability, empowering TiCDC to better support latency-sensitive applications. @@ -276,7 +276,6 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - planner - - planner: just pop cte's handleHelper map out since it shouldn't be considered [#35758](https://github.com/pingcap/tidb/issues/35758) @[AilinKid](https://github.com/AilinKid) - [`PLAN REPLAYER`](/sql-plan-replayer) can be used on multiple SQL statements, which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) + TiKV @@ -361,7 +360,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix the issue of failing to handle `prepared` statement flags in the typical MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[dveeden](https://github.com/dveeden) - (dup) Fix the issue of incorrect TiDB status that might appear on startup in some extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) @[xhebox](https://github.com/xhebox) - Fix the issue that `information_schema.variables_info` does not comply with security enhanced mode (SEM) [#37586](https://github.com/pingcap/tidb/issues/37586) @[CbcWestwolf](https://github.com/CbcWestwolf) - + - Fix wrong casting in building the plan of union [#31678](https://github.com/pingcap/tidb/issues/31678) @[bb7133](https://github.com/bb7133) - execution - Fix the wrong result that occurs when enabling dynamic mode in partitioned tables for TiFlash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) @@ -451,10 +450,6 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - (dup) Fix the issue that TiDB Lightning does not support columns starting with slash, number, or non-ascii characters in Parquet files [#36980](https://github.com/pingcap/tidb/issues/36980) @[D3Hunter](https://github.com/D3Hunter) - - TiUP - - - note [#issue]() @[Contributor GitHub ID]() - ## Contributors We would like to thank the following contributors from the TiDB community: From 92756a33f82391e7d1326ee7b5217b8633e31082 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 28 Sep 2022 17:39:16 +0800 Subject: [PATCH 085/104] Apply suggestions from code review Co-authored-by: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> --- releases/release-6.3.0.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 40b2a80a8c452..0677637209857 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -271,8 +271,8 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - execution - Report errors when the JSON path has wrong syntax [#22525](https://github.com/pingcap/tidb/issues/22525) [#34959](https://github.com/pingcap/tidb/issues/34959) @[xiongjiwei](https://github.com/xiongjiwei) - - Improve the performance of Join operations by fixing a false sharing issue [#37641](https://github.com/pingcap/tidb/issues/37641) - - Improve the MySQL compatibility of the `CAST` function for the float-to-time conversion [#37462](https://github.com/pingcap/tidb/issues/37462) @[mengxin9014](https://github.com/mengxin9014) + - Improve the performance of Join operations by fixing a false sharing issue [#37641](https://github.com/pingcap/tidb/issues/37641) @[gengliqi](https://github.com/gengliqi) + - Fix the issue that the result of the `castRealAsTime` expression is inconsistent with MySQL [#37462](https://github.com/pingcap/tidb/issues/37462) @[mengxin9014](https://github.com/mengxin9014) - planner @@ -364,7 +364,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - execution - Fix the wrong result that occurs when enabling dynamic mode in partitioned tables for TiFlash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) - - Fix the issue that the cast and comparison between binary strings and JSON are incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the cast and comparison between binary strings and JSON in TiDB are incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - Fix the issue that `json_objectagg` and `json_arrayagg` in TiDB are not compatible with MySQL on binary values [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the comparison between JSON opaque values causes panic [#37315](https://github.com/pingcap/tidb/issues/37315) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the single precision float cannot be used in JSON aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) @@ -372,9 +372,9 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - transaction - - Fix the issue that pessimistic DML operations lock non-unique index keys [#36235](https://github.com/pingcap/tidb/issues/36235) - - Fix the transaction commit behaviours related to auto-commit mode change [#36581](https://github.com/pingcap/tidb/issues/36581) @[cfzjywxk](https://github.com/cfzjywxk) - - Fix the issue that the `EXPLAIN ANALYZE` statement with DML executors might return result before the transaction commit has finished [#37273](https://github.com/pingcap/tidb/issues/37373) @[cfzjywxk](https://github.com/cfzjywxk) + - Fix the issue that pessimistic DML operations lock non-unique index keys [#36235](https://github.com/pingcap/tidb/issues/36235) @[ekexium](https://github.com/ekexium) + - Fix the issue that `auto-commit` change affects transaction commit behaviours [#36581](https://github.com/pingcap/tidb/issues/36581) @[cfzjywxk](https://github.com/cfzjywxk) + - Fix the issue that the `EXPLAIN ANALYZE` statement with DML executors might return result before the transaction commit finishes [#37373](https://github.com/pingcap/tidb/issues/37373) @[cfzjywxk](https://github.com/cfzjywxk) - planner From 24287bc56be1c3dabef1cbe5133aebf7f451e0f5 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 28 Sep 2022 18:39:34 +0800 Subject: [PATCH 086/104] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-6.3.0.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 0677637209857..f98b7a663dbdf 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -132,7 +132,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * The feature of disabling Titan becomes GA @[tabokie](https://github.com/tabokie) - You can [disable Titan](/titan-configuration.md#disable-titan) for online TiKV nodes. + You can [disable Titan](/storage-engine/titan-configuration.md#disable-titan) for online TiKV nodes. * Use `static` partition pruning when GlobalStats are not ready [#37535](https://github.com/pingcap/tidb/issues/37535) @[Yisaer](https://github.com/Yisaer) @@ -180,7 +180,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * DM adds a new configuration item `safe-mode-duration` in the task configuration file [#6224](https://github.com/pingcap/tiflow/issues/6224) @[okJiang](https://github.com/okJiang) - DM adds a new configuration item `safe-mode-duration` in the [task configuration file](/task-configuration-file-full.md). You can adjust the automatic safe mode duration after DM exits abnormally. The default value is 60 seconds. If `safe-mode-duration` is set to `"0s"`, an error is reported when DM tries to enter safe mode after an abnormal restart. + DM adds a new configuration item `safe-mode-duration` in the [task configuration file](/dm/task-configuration-file-full.md). You can adjust the automatic safe mode duration after DM exits abnormally. The default value is 60 seconds. If `safe-mode-duration` is set to `"0s"`, an error is reported when DM tries to enter safe mode after an abnormal restart. ### TiDB data share subscription @@ -272,7 +272,6 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Report errors when the JSON path has wrong syntax [#22525](https://github.com/pingcap/tidb/issues/22525) [#34959](https://github.com/pingcap/tidb/issues/34959) @[xiongjiwei](https://github.com/xiongjiwei) - Improve the performance of Join operations by fixing a false sharing issue [#37641](https://github.com/pingcap/tidb/issues/37641) @[gengliqi](https://github.com/gengliqi) - - Fix the issue that the result of the `castRealAsTime` expression is inconsistent with MySQL [#37462](https://github.com/pingcap/tidb/issues/37462) @[mengxin9014](https://github.com/mengxin9014) - planner @@ -325,7 +324,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Improve compatibility for MySQL 8.0 upstream [#6506](https://github.com/pingcap/tiflow/issues/6506) @[lance6716](https://github.com/lance6716) - Log DML start ts when MySQL sink meets error [#6460](https://github.com/pingcap/tiflow/issues/6460) @[overvenus](https://github.com/overvenus) - Make the API `api/v1/health` reflects the health status of the TiCDC cluster [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) - - Introduce a new sink implement to increase throughput [#5928](https://github.com/pingcap/tiflow/issues/5928) @[hicqu]https://github.com/hicqu @[hi-rustin](https://github.com/hi-rustin) + - Introduce a new sink implement to increase throughput [#5928](https://github.com/pingcap/tiflow/issues/5928) @[hicqu](https://github.com/hicqu) @[hi-rustin](https://github.com/hi-rustin) - Deprecate the memory sorter and always use the unified sorter [#7087](https://github.com/pingcap/tiflow/issues/5928) @[hi-rustin](https://github.com/hi-rustin) - Delete the deprecated pulsar sink [#7087](https://github.com/pingcap/tiflow/issues/7087) @[hi-rustin](https://github.com/hi-rustin) - Improve replicating performance by discarding DDLs that are irrelevant with a changefeed. [#6447](https://github.com/pingcap/tiflow/issues/6447) @[asddongmen](https://github.com/asddongmen) @@ -360,7 +359,8 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix the issue of failing to handle `prepared` statement flags in the typical MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[dveeden](https://github.com/dveeden) - (dup) Fix the issue of incorrect TiDB status that might appear on startup in some extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) @[xhebox](https://github.com/xhebox) - Fix the issue that `information_schema.variables_info` does not comply with security enhanced mode (SEM) [#37586](https://github.com/pingcap/tidb/issues/37586) @[CbcWestwolf](https://github.com/CbcWestwolf) - - Fix wrong casting in building the plan of union [#31678](https://github.com/pingcap/tidb/issues/31678) @[bb7133](https://github.com/bb7133) + - Fix the issue that casting string to string goes wrong in queries with `UNION` [#31678](https://github.com/pingcap/tidb/issues/31678) @[cbcwestwolf](https://github.com/cbcwestwolf) + - execution - Fix the wrong result that occurs when enabling dynamic mode in partitioned tables for TiFlash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) @@ -369,6 +369,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix the issue that the comparison between JSON opaque values causes panic [#37315](https://github.com/pingcap/tidb/issues/37315) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the single precision float cannot be used in JSON aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the `UNION` operator might return unexpected empty result [#36903](https://github.com/pingcap/tidb/issues/36903) @[tiancaiamao](https://github.com/tiancaiamao) + - Fix the issue that the result of the `castRealAsTime` expression is inconsistent with MySQL [#37462](https://github.com/pingcap/tidb/issues/37462) @[mengxin9014](https://github.com/mengxin9014) - transaction @@ -381,7 +382,6 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - fix update plan's projection elimination will cause column resolution error [#37568](https://github.com/pingcap/tidb/issues/37568) @[AilinKid](https://github.com/AilinKid) - planner: fix outer join reorder will push down its outer join condition [#37238](https://github.com/pingcap/tidb/issues/37238) @[AilinKid](https://github.com/AilinKid) - make the both side operand of NAAJ & refuse partial column substitute in projection elimination [#37032](https://github.com/pingcap/tidb/issues/37032) @[AilinKid](https://github.com/AilinKid) - - planner: correct the redundant field meaning in join full schema when join coalesce [#36420](https://github.com/pingcap/tidb/issues/36420) @[AilinKid](https://github.com/AilinKid) - Fix a wrong casting in building union plan [#31678](https://github.com/pingcap/tidb/issues/31678) @[bb7133](https://github.com/bb7133) - diagnosis From b0db48f6b153d35b7e7372048be982ffac5ca7f9 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 28 Sep 2022 20:20:07 +0800 Subject: [PATCH 087/104] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-6.3.0.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index f98b7a663dbdf..1f5bd1bfd4082 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -275,7 +275,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - planner - - [`PLAN REPLAYER`](/sql-plan-replayer) can be used on multiple SQL statements, which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) + - Support exporting the execution plan information of multiple SQL statements at a time using [`PLAN REPLAYER`](/sql-plan-replayer.md), which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) + TiKV @@ -379,10 +379,10 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - planner - - fix update plan's projection elimination will cause column resolution error [#37568](https://github.com/pingcap/tidb/issues/37568) @[AilinKid](https://github.com/AilinKid) - - planner: fix outer join reorder will push down its outer join condition [#37238](https://github.com/pingcap/tidb/issues/37238) @[AilinKid](https://github.com/AilinKid) - - make the both side operand of NAAJ & refuse partial column substitute in projection elimination [#37032](https://github.com/pingcap/tidb/issues/37032) @[AilinKid](https://github.com/AilinKid) - - Fix a wrong casting in building union plan [#31678](https://github.com/pingcap/tidb/issues/31678) @[bb7133](https://github.com/bb7133) + - Fix the issue that the UPDATE statements incorrectly eliminate the projection in some cases, which causes the `Can't find column` error [#37568](https://github.com/pingcap/tidb/issues/37568) @[AilinKid](https://github.com/AilinKid) + - (dup) Fix the issue that the Join Reorder operation will mistakenly push down its Outer Join condition [#37238](https://github.com/pingcap/tidb/issues/37238) @[AilinKid](https://github.com/AilinKid) + - Fix the issue that the `IN` and `NOT IN` subqueries in some patterns report the `Can't find column` error [#37032](https://github.com/pingcap/tidb/issues/37032) @[AilinKid](https://github.com/AilinKid) + - Fix the issue that `Can't find column` is reported if an `UPDATE` statement contains common table expressions (CTE) [#35758](https://github.com/pingcap/tidb/issues/35758) @[AilinKid](https://github.com/AilinKid) - diagnosis From 9a8a4ee7c608e8b1580957c35a278b850b9969db Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 28 Sep 2022 20:27:16 +0800 Subject: [PATCH 088/104] Update releases/release-6.3.0.md --- releases/release-6.3.0.md | 1 - 1 file changed, 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 1f5bd1bfd4082..a993f48e04985 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -429,7 +429,6 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix a bug that might cause the information of the checkpoint being stale [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) - Fix a bug that the regions are not balanced because the concurrency is set too large during the restoration [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) - Fix a bug that might cause log backup checkpoint TS stuck when TiCDC exists in the cluster [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) - - Fix a bug that caused: when the backup meta v2 enabled, there may be too many meta files. [#37244](https://github.com/pingcap/tidb/issues/37244) [@MoCuishle28](https://github.com/MoCuishle28) - Fix a bug that might lead to backup and restoration failure if some special character exists in the authorization key of external storage [#37469](https://github.com/pingcap/tidb/issues/37469) [@MoCuishle28](https://github.com/MoCuishle28) + TiCDC From 7053f7eed4e39a7080bddc61118feb4eb6ec9af9 Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Wed, 28 Sep 2022 20:32:17 +0800 Subject: [PATCH 089/104] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index a993f48e04985..6f7eedd2cfab1 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -433,8 +433,8 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiCDC - - Handle error correctly with wrong PD address but with a grpc service [#6458](https://github.com/pingcap/tiflow/issues/6458) @[crelax](https://github.com/crelax) - - Fix a bug that `cdc cause cli changefeed list` does not contain failed changefeed [#6334](https://github.com/pingcap/tiflow/issues/6334) @[asddongmen](https://github.com/asddongmen) + - Fix a bug that TiCDC returns an inaccurate error for a wrong PD address with a grpc service [#6458](https://github.com/pingcap/tiflow/issues/6458) @[crelax](https://github.com/crelax) + - Fix a bug that the `cdc cause cli changefeed list` command does not return failed changefeeds [#6334](https://github.com/pingcap/tiflow/issues/6334) @[asddongmen](https://github.com/asddongmen) - Fix a bug that may cause cdc unavailable if a changefeed initializing fails [#6859](https://github.com/pingcap/tiflow/issues/6859) @[asddongmen](https://github.com/asddongmen) + TiDB Data Migration (DM) From 200196c817ae01bbfb03541f3525df25211d491d Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 29 Sep 2022 11:28:07 +0800 Subject: [PATCH 090/104] Apply suggestions from code review --- releases/release-6.3.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 6f7eedd2cfab1..1a863bd9b414e 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -322,12 +322,12 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiCDC - Improve compatibility for MySQL 8.0 upstream [#6506](https://github.com/pingcap/tiflow/issues/6506) @[lance6716](https://github.com/lance6716) - - Log DML start ts when MySQL sink meets error [#6460](https://github.com/pingcap/tiflow/issues/6460) @[overvenus](https://github.com/overvenus) + - Support logging `start ts` of DML statements when MySQL sink gets an error [#6460](https://github.com/pingcap/tiflow/issues/6460) @[overvenus](https://github.com/overvenus) - Make the API `api/v1/health` reflects the health status of the TiCDC cluster [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) - - Introduce a new sink implement to increase throughput [#5928](https://github.com/pingcap/tiflow/issues/5928) @[hicqu](https://github.com/hicqu) @[hi-rustin](https://github.com/hi-rustin) + - Implement MQ sink and MySQL sink in the asynchronous mode to improve the sink throughput [#5928](https://github.com/pingcap/tiflow/issues/5928) @[hicqu](https://github.com/hicqu) @[hi-rustin](https://github.com/hi-rustin) - Deprecate the memory sorter and always use the unified sorter [#7087](https://github.com/pingcap/tiflow/issues/5928) @[hi-rustin](https://github.com/hi-rustin) - Delete the deprecated pulsar sink [#7087](https://github.com/pingcap/tiflow/issues/7087) @[hi-rustin](https://github.com/hi-rustin) - - Improve replicating performance by discarding DDLs that are irrelevant with a changefeed. [#6447](https://github.com/pingcap/tiflow/issues/6447) @[asddongmen](https://github.com/asddongmen) + - Improve replication performance by discarding DDL statements that are irrelevant with a changefeed [#6447](https://github.com/pingcap/tiflow/issues/6447) @[asddongmen](https://github.com/asddongmen) + TiDB Data Migration (DM) @@ -435,7 +435,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix a bug that TiCDC returns an inaccurate error for a wrong PD address with a grpc service [#6458](https://github.com/pingcap/tiflow/issues/6458) @[crelax](https://github.com/crelax) - Fix a bug that the `cdc cause cli changefeed list` command does not return failed changefeeds [#6334](https://github.com/pingcap/tiflow/issues/6334) @[asddongmen](https://github.com/asddongmen) - - Fix a bug that may cause cdc unavailable if a changefeed initializing fails [#6859](https://github.com/pingcap/tiflow/issues/6859) @[asddongmen](https://github.com/asddongmen) + - Fix a bug that TiCDC is unavailable when changefeed initialization fails [#6859](https://github.com/pingcap/tiflow/issues/6859) @[asddongmen](https://github.com/asddongmen) + TiDB Data Migration (DM) From 9652bec0489c933fba8231c85b940c1ab2e172f2 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 29 Sep 2022 11:36:07 +0800 Subject: [PATCH 091/104] Apply suggestions from code review --- releases/release-6.3.0.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 1a863bd9b414e..0f318b9d0907a 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -245,7 +245,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: | TiCDC | [`enable-sync-point`](/ticdc/manage-ticdc.md#task-configuration-file) | Newly added | Specifies whether to enable the Syncpoint feature. | | TiCDC | [`sync-point-interval`](/ticdc/manage-ticdc.md#task-configuration-file) | Newly added | Specifies the interval at which Syncpoint aligns the upstream and downstream snapshots. | | TiCDC | [`sync-point-retention`](/ticdc/manage-ticdc.md#task-configuration-file) | Newly added | Specifies how long the data is retained by Syncpoint in the downstream table. When this duration is exceeded, the data is cleaned up. | -| TiCDC | [`sink-uri.memory`](/ticdc/manage-ticdc.md#create-a-replication-task) | Deprecated | This configuration item is deprecated. It is not recommended to use it in any situation. | +| TiCDC | [`sink-uri.memory`](/ticdc/manage-ticdc.md#create-a-replication-task) | Deprecated | The `memory` sorting is deprecated. It is not recommended to use it in any situation. You can use the `unified` sorting instead. | ### Others @@ -325,7 +325,6 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Support logging `start ts` of DML statements when MySQL sink gets an error [#6460](https://github.com/pingcap/tiflow/issues/6460) @[overvenus](https://github.com/overvenus) - Make the API `api/v1/health` reflects the health status of the TiCDC cluster [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) - Implement MQ sink and MySQL sink in the asynchronous mode to improve the sink throughput [#5928](https://github.com/pingcap/tiflow/issues/5928) @[hicqu](https://github.com/hicqu) @[hi-rustin](https://github.com/hi-rustin) - - Deprecate the memory sorter and always use the unified sorter [#7087](https://github.com/pingcap/tiflow/issues/5928) @[hi-rustin](https://github.com/hi-rustin) - Delete the deprecated pulsar sink [#7087](https://github.com/pingcap/tiflow/issues/7087) @[hi-rustin](https://github.com/hi-rustin) - Improve replication performance by discarding DDL statements that are irrelevant with a changefeed [#6447](https://github.com/pingcap/tiflow/issues/6447) @[asddongmen](https://github.com/asddongmen) From 9b348ecf837f18d25cf276e636aa01dc004d467b Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Thu, 29 Sep 2022 13:34:38 +0800 Subject: [PATCH 092/104] Apply suggestions from code review Co-authored-by: xixirangrang Co-authored-by: Grace Cai --- releases/release-6.3.0.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 0f318b9d0907a..2fe767ffce584 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -300,7 +300,6 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Support to push down the `castTimeAsDuration` function to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - Support to push down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) - Refactor TiFlash's interpreter, and support the new interpreter Planner [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - - Suppress the "cannot set inq" logs [#4940](https://github.com/pingcap/tiflash/issues/4940) @[LittleFall](https://github.com/LittleFall) - Improve the accuracy of memory tracker in TiFlash [#5609](https://github.com/pingcap/tiflash/issues/5609) @[bestwoody](https://github.com/bestwoody) - Improve the performance of string columns with the `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collations [#5294](https://github.com/pingcap/tiflash/issues/5294) @[solotzg](https://github.com/solotzg) @@ -411,14 +410,14 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Compute - - Fix the bug that the window function may cause TiFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - - Fix the bug that wrong data input for `date(CAST(value AS DATETIME))` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) - - Fix that issue that the result of `cast(Real/Decimal)AsTime` is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) + - Fix a bug that the window function might cause TiFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) + - Fix a bug that wrong data input for `date(CAST(value AS DATETIME))` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) + - Fix the issue that the result of `cast(Real/Decimal)AsTime` is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) - Storage - Fix the issue that there may be some obsolete data left in storage which cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) - - Fix the bug that page GC may block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix a bug that page GC might block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) - Fix the panic issue that occurs after creating the primary index with a column containing the `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) + Tools From 1c145d249e5b635da53422d23f452a7d5f713f0b Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 29 Sep 2022 13:38:46 +0800 Subject: [PATCH 093/104] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-6.3.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 2fe767ffce584..59357256901a4 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -295,10 +295,10 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Compute - - Support to push down the `elt` function to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) - - Support to push down the `leftShift` function to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) - - Support to push down the `castTimeAsDuration` function to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - - Support to push down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) + - Support pushing down the `elt` function to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) + - Support pushing down the `leftShift` function to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) + - Support pushing down the `castTimeAsDuration` function to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) + - Support pushing down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) - Refactor TiFlash's interpreter, and support the new interpreter Planner [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - Improve the accuracy of memory tracker in TiFlash [#5609](https://github.com/pingcap/tiflash/issues/5609) @[bestwoody](https://github.com/bestwoody) - Improve the performance of string columns with the `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collations [#5294](https://github.com/pingcap/tiflash/issues/5294) @[solotzg](https://github.com/solotzg) From f87d24a5de28aa949b2e6db6b127696755f3aaad Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 29 Sep 2022 13:51:51 +0800 Subject: [PATCH 094/104] Apply suggestions from code review Co-authored-by: xixirangrang --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 59357256901a4..466faa2eb2e22 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -322,10 +322,10 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Improve compatibility for MySQL 8.0 upstream [#6506](https://github.com/pingcap/tiflow/issues/6506) @[lance6716](https://github.com/lance6716) - Support logging `start ts` of DML statements when MySQL sink gets an error [#6460](https://github.com/pingcap/tiflow/issues/6460) @[overvenus](https://github.com/overvenus) - - Make the API `api/v1/health` reflects the health status of the TiCDC cluster [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) + - Enhance the `api/v1/health` API to return a more accurate health state of a TiCDC cluster (https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) - Implement MQ sink and MySQL sink in the asynchronous mode to improve the sink throughput [#5928](https://github.com/pingcap/tiflow/issues/5928) @[hicqu](https://github.com/hicqu) @[hi-rustin](https://github.com/hi-rustin) - Delete the deprecated pulsar sink [#7087](https://github.com/pingcap/tiflow/issues/7087) @[hi-rustin](https://github.com/hi-rustin) - - Improve replication performance by discarding DDL statements that are irrelevant with a changefeed [#6447](https://github.com/pingcap/tiflow/issues/6447) @[asddongmen](https://github.com/asddongmen) + - Improve replication performance by discarding DDL statements that are irrelevant to a changefeed [#6447](https://github.com/pingcap/tiflow/issues/6447) @[asddongmen](https://github.com/asddongmen) + TiDB Data Migration (DM) From 2499faa10ae43181f341e8d783b27292132f59f7 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 29 Sep 2022 14:04:01 +0800 Subject: [PATCH 095/104] Apply suggestions from code review Co-authored-by: Grace Cai --- releases/release-6.3.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 466faa2eb2e22..304f45e0d41c5 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -416,9 +416,9 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Storage - - Fix the issue that there may be some obsolete data left in storage which cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the issue that some obsolete data in storage cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) - Fix a bug that page GC might block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) - - Fix the panic issue that occurs after creating the primary index with a column containing the `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the panic that occurs after creating the primary index with a column containing the `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) + Tools From 62842bc91691db9064d3cd0d295c77db0c3d26eb Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Thu, 29 Sep 2022 14:33:49 +0800 Subject: [PATCH 096/104] refine format --- releases/release-6.3.0.md | 211 +++++++++++++++-------------------- releases/release-timeline.md | 2 +- 2 files changed, 89 insertions(+), 124 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 304f45e0d41c5..1c96f954ffe62 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -4,7 +4,7 @@ title: TiDB 6.3.0 Release Notes # TiDB 6.3.0 Release Notes -Release date: xx xx, 2022 +Release date: September 30, 2022 TiDB version: 6.3.0-DMR @@ -29,7 +29,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: * Range COLUMNS partitioning supports defining multiple columns [#36636](https://github.com/pingcap/tidb/issues/36636) @[mjonss](https://github.com/mjonss) - Support [PARTITION BY RANGE COLUMNS (column_list)](/partitioned-table.md#range-columns-partitioning). `column_list` is no longer limited to a single column. The basic feature is the same as MySQL. + TiDB supports [PARTITION BY RANGE COLUMNS (column_list)](/partitioned-table.md#range-columns-partitioning). `column_list` is no longer limited to a single column. The basic feature is the same as MySQL. * EXCHANGE PARTITION becomes GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) @@ -90,14 +90,14 @@ In v6.3.0-DMR, the key new features and improvements are as follows: This feature is experimental in v6.2.0, and becomes GA in v6.3.0. -* Improve performance of TiFlash data replication [#5237](https://github.com/pingcap/tiflash/issues/5237) @[breezewish](https://github.com/breezewish) +* TiFlash improves performance of data replication [#5237](https://github.com/pingcap/tiflash/issues/5237) @[breezewish](https://github.com/breezewish) TiFlash uses the Raft protocol for data replication from TiKV. Prior to v6.3.0, it often took a long time to replicate large amounts of replica data. TiDB v6.3.0 optimizes the TiFlash data replication mechanism and significantly improves the replication speed. When you use BR to recover data, use TiDB Lightning to import data, or add new TiFlash replicas, the TiFlash replicas can be replicated more quickly. You can query with TiFlash in a more timely manner. In addition, TiFlash replicas will also reach a secure and balanced state faster when you scale up, scale down, or modify the number of TiFlash replicas. * TiFlash supports three-stage aggregation of individual `COUNT(DISTINCT)` [#37202](https://github.com/pingcap/tidb/issues/37202) @[fixdb](https://github.com/fixdb) TiFlash supports rewriting queries containing only one `COUNT(DISTINCT)` into a [three-stage aggregation](/system-variables.md#tidb_opt_three_stage_distinct_agg-new-in-v630). This improves concurrency and performance. - + * TiKV supports log recycling [#214](https://github.com/tikv/raft-engine/issues/214) @[LykxSassinator](https://github.com/LykxSassinator) TiKV supports [recycling log files](/tikv-configuration-file.md#enable-log-recycle-new-in-v630) in Raft Engine. This reduces the long tail latency in network disks during Raft log appending and improves performance under write workloads. @@ -130,7 +130,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: In v5.3.0, TiDB introduced the system variable [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) to control how the optimizer behaves when the statistics become outdated. The default value is `ON`, which means keeping the behavior of the old version: When the statistics on objects that are involved in a SQL statement are outdated, the optimizer considers that statistics (other than the total number of rows on the table) are no longer reliable and uses pseudo statistics instead. After tests and analyses of actual user scenarios, the default value of `tidb_enable_pseudo_for_outdated_stats` is changed to `OFF` since v6.3.0. Even if the statistics become outdated, the optimizer will still use the statistics on the table, which makes the execution plan more stable. -* The feature of disabling Titan becomes GA @[tabokie](https://github.com/tabokie) +* Disabling Titan becomes GA @[tabokie](https://github.com/tabokie) You can [disable Titan](/storage-engine/titan-configuration.md#disable-titan) for online TiKV nodes. @@ -138,8 +138,6 @@ In v6.3.0-DMR, the key new features and improvements are as follows: When [`dynamic pruning`](/partitioned-table.md#dynamic-pruning-mode) is enabled, the optimizer selects execution plans based on [GlobalStats](/statistics.md#collect-statistics-of-partitioned-tables-in-dynamic-pruning-mode). Before GlobalStats are fully collected, using pseudo statistics might cause performance regression. In v6.3.0, this issue is addressed by maintaining the `static` mode if you enable dynamic pruning before GlobalStats are collected. TiDB remains in the `static` mode until GlobalStats are collected. This ensures performance stability when you change the partition pruning settings. -### Ease of use - ### MySQL compatibility * Improve MySQL 8.0 compatibility by adding support for four regular expression functions: `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) @@ -200,19 +198,26 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### System variables -| Variable name | Change type (newly added, modified, or deleted) | Description | +| Variable name | Change type | Description | | --- | --- | --- | | [`default_authentication_plugin`](/system-variables.md#default_authentication_plugin) | Modified | Adds a new option `tidb_sm3_password`. When this variable is set to `tidb_sm3_password`, SM3 is used as the encryption algorithm. | +| [`sql_require_primary_key`](/system-variables.md#sql_require_primary_key-new-in-v630) | Newly added | Controls whether to enforce the requirement that a table has a primary key. After this variable is enabled, attempting to create or alter a table without a primary key will produce an error. | | [`tidb_adaptive_closest_read_threshold`](/system-variables.md#tidb_adaptive_closest_read_threshold-new-in-v630) | Newly added | Controls the threshold at which the TiDB server prefers to send read requests to the replica in the same region as the TiDB server when [`tidb_replica_read`](/system-variables.md#tidb_replica_read-new-in-v40) is set to `closest-adaptive`. | | [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) | Newly added | Controls when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions. | | [`tidb_ddl_disk_quota`](/system-variables.md#tidb_ddl_disk_quota-new-in-v630) | Newly added | Takes effect only when [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) is enabled. It sets the usage limit of local storage during backfilling when creating an index. | | [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) | Newly added | Controls whether to enable the acceleration of `ADD INDEX` and `CREATE INDEX` DDL operations to improve the speed of backfilling when creating an index. | +| [`tidb_ddl_flashback_concurrency`](/system-variables.md#tidb_ddl_flashback_concurrency-new-in-v630) | Newly added | Controls the concurrency of `flashback cluster`. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | | [`tidb_enable_clustered_index`](/system-variables.md#tidb_enable_clustered_index-new-in-v50) | Modified | The default value of this variable changes from `INT_ONLY` to `ON`. | | [`tidb_enable_exchange_partition`](/system-variables.md#tidb_enable_exchange_partition) | Deprecated | Controls whether to enable the [`exchange partitions with tables`](/partitioned-table.md#partition-management) feature. The default value is `ON`, that is, `exchange partitions with tables` is enabled by default. | +| [`tidb_enable_foreign_key`](/system-variables.md#tidb_enable_foreign_key-new-in-v630) | Newly added | Controls whether to enable the `FOREIGN KEY` feature. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | +| [`tidb_enable_general_plan_cache`](/system-variables.md#tidb_enable_general_plan_cache-new-in-v630) | Newly added | Controls whether to enable the General Plan Cache feature. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | | [`tidb_enable_metadata_lock`](/system-variables.md#tidb_enable_metadata_lock-new-in-v630)| Newly added | Specifies whether to enable the [Metadata lock](/metadata-lock.md) feature. | +| [`tidb_enable_null_aware_anti_join`](/system-variables.md#tidb_enable_null_aware_anti_join-new-in-v630) | Newly added | Controls whether TiDB applies Null Aware Hash Join when ANTI JOIN is generated by subqueries led by special set operators `NOT IN` and `!= ALL`. | | [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Modified | Controls the behavior of the optimizer on using statistics of a table when the statistics are outdated. The default value changes from `ON` to `OFF`, which means the optimizer still keeps using the statistics of the table even if the statistics of this table are outdated. | | [`tidb_enable_rate_limit_action`](/system-variables.md#tidb_enable_rate_limit_action) | Modified | Controls whether to enable the dynamic memory control feature for the operator that reads data. When this variable is set to `ON`, the memory usage might not be under the control of [tidb_mem_quota_query](/system-variables.md#tidb_mem_quota_query). Therefore, the default value is changed from `ON` to `OFF`. | +| [`tidb_enable_tiflash_read_for_write_stmt`](/system-variables.md#tidb_enable_tiflash_read_for_write_stmt-new-in-v630) | Newly added | Controls whether read requests in SQL write statements are pushed down to TiFlash. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | | [`tidb_enable_unsafe_substitute`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | Controls whether to replace expressions with generated columns in an unsafe way. | +| [`tidb_general_plan_cache_size`](/system-variables.md#tidb_general_plan_cache_size-new-in-v630) | Newly added | Controls the maximum number of execution plans that can be cached by General Plan Cache. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | | [`tidb_last_plan_replayer_token`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | Read-only and used to obtain the result of the last `PLAN REPLAYER DUMP` execution in the current session. | | [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) | Newly added | Controls whether common table expressions (CTEs) in the entire session are inlined or not. The default value is `OFF`, which means that inlining CTE is not enforced by default. | | [`tidb_opt_three_stage_distinct_agg`](/system-variables.md#tidb_opt_three_stage_distinct_agg-new-in-v630) | Newly added | Specifies whether to rewrite a `COUNT(DISTINCT)` aggregation into a three-stage aggregation in MPP mode. The default value is `ON`. | @@ -220,13 +225,6 @@ In v6.3.0-DMR, the key new features and improvements are as follows: | [`tidb_rc_read_check_ts`](/system-variables.md#tidb_rc_read_check_ts-new-in-v600) | Modified | Used for optimizing the timestamp acquisition, which is suitable for scenarios with read-committed isolation level where read-write conflicts are rare. This feature is oriented to specific service workloads and might cause performance regression in other scenarios. For this reason, since v6.3.0, the scope of this variable changes from `GLOBAL \| SESSION` to `INSTANCE`. That means you can enable this feature for specific TiDB instances. | | [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) | Newly added | Used for optimizing the acquisition of timestamps and is suitable for scenarios with few point-write conflicts in read-committed isolation level of pessimistic transactions. Enabling this variable can avoid the latency and overhead brought by obtaining the global timestamps during the execution of point-write statements | | [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) | Newly added | Controls whether to enable FastScan. If [FastScan](/develop/dev-guide-use-fastscan.md) is enabled (set to `ON`), TiFlash provides more efficient query performance, but does not guarantee the accuracy of the query results or data consistency. | -| [`sql_require_primary_key`](/system-variables.md#sql_require_primary_key-new-in-v630) | Newly added | Controls whether to enforce the requirement that a table has a primary key. After this variable is enabled, attempting to create or alter a table without a primary key will produce an error. | -| [`tidb_ddl_flashback_concurrency`](/system-variables.md#tidb_ddl_flashback_concurrency-new-in-v630) | Newly added | Controls the concurrency of `flashback cluster`. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | -| [`tidb_enable_foreign_key`](/system-variables.md#tidb_enable_foreign_key-new-in-v630) | Newly added | Controls whether to enable the `FOREIGN KEY` feature. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | -| [`tidb_enable_general_plan_cache`](/system-variables.md#tidb_enable_general_plan_cache-new-in-v630) | Newly added | Controls whether to enable the General Plan Cache feature. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | -| [`tidb_enable_null_aware_anti_join`](/system-variables.md#tidb_enable_null_aware_anti_join-new-in-v630) | Newly added | Controls whether TiDB applies Null Aware Hash Join when ANTI JOIN is generated by subqueries led by special set operators `NOT IN` and `!= ALL`. | -| [`tidb_enable_tiflash_read_for_write_stmt`](/system-variables.md#tidb_enable_tiflash_read_for_write_stmt-new-in-v630) | Newly added | Controls whether read requests in SQL write statements are pushed down to TiFlash. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | -| [`tidb_general_plan_cache_size`](/system-variables.md#tidb_general_plan_cache_size-new-in-v630) | Newly added | Controls the maximum number of execution plans that can be cached by General Plan Cache. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | ### Configuration file parameters @@ -261,27 +259,19 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiDB - - sql-infra - - - TiDB is now case-insensitive to the target table name when checking the table existence [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) - - Improve MySQL compatibility by adding a parsing check when setting the value of `init_connect` [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) - - Improve the log warning generated for new connections [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) - - Optimize the HTTP API for querying DDL history jobs, and add support for the `start_job_id` parameter [#35838](https://github.com/pingcap/tidb/issues/35838) @[tiancaiamao](https://github.com/tiancaiamao) - - - execution - - - Report errors when the JSON path has wrong syntax [#22525](https://github.com/pingcap/tidb/issues/22525) [#34959](https://github.com/pingcap/tidb/issues/34959) @[xiongjiwei](https://github.com/xiongjiwei) - - Improve the performance of Join operations by fixing a false sharing issue [#37641](https://github.com/pingcap/tidb/issues/37641) @[gengliqi](https://github.com/gengliqi) - - - planner - - - Support exporting the execution plan information of multiple SQL statements at a time using [`PLAN REPLAYER`](/sql-plan-replayer.md), which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) + - TiDB is now case-insensitive to the target table name when checking the table existence [#34610](https://github.com/pingcap/tidb/issues/34610) @[tiancaiamao](https://github.com/tiancaiamao) + - Improve MySQL compatibility by adding a parsing check when setting the value of `init_connect` [#35324](https://github.com/pingcap/tidb/issues/35324) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Improve the log warning generated for new connections [#34964](https://github.com/pingcap/tidb/issues/34964) @[xiongjiwei](https://github.com/xiongjiwei) + - Optimize the HTTP API for querying DDL history jobs, and add support for the `start_job_id` parameter [#35838](https://github.com/pingcap/tidb/issues/35838) @[tiancaiamao](https://github.com/tiancaiamao) + - Report errors when the JSON path has wrong syntax [#22525](https://github.com/pingcap/tidb/issues/22525) [#34959](https://github.com/pingcap/tidb/issues/34959) @[xiongjiwei](https://github.com/xiongjiwei) + - Improve the performance of Join operations by fixing a false sharing issue [#37641](https://github.com/pingcap/tidb/issues/37641) @[gengliqi](https://github.com/gengliqi) + - Support exporting the execution plan information of multiple SQL statements at a time using [`PLAN REPLAYER`](/sql-plan-replayer.md), which makes troubleshooting more efficient [#37798](https://github.com/pingcap/tidb/issues/37798) @[Yisaer](https://github.com/Yisaer) + TiKV - Support configuring the `unreachable_backoff` item to avoid Raftstore broadcasting too many messages after one peer becomes unreachable [#13054](https://github.com/tikv/tikv/issues/13054) @[5kbpers](https://github.com/5kbpers) - Improve the fault tolerance of TSO service [#12794](https://github.com/tikv/tikv/issues/12794) @[pingyu](https://github.com/pingyu) - - (dup) Support dynamically modifying the number of sub-compaction operations performed concurrently in RocksDB (`rocksdb.max-sub-compactions`) [#13145](https://github.com/tikv/tikv/issues/13145) @[ethercflow](https://github.com/ethercflow) + - Support dynamically modifying the number of sub-compaction operations performed concurrently in RocksDB (`rocksdb.max-sub-compactions`) [#13145](https://github.com/tikv/tikv/issues/13145) @[ethercflow](https://github.com/ethercflow) - Optimize the performance of merging empty Regions [#12421](https://github.com/tikv/tikv/issues/12421) @[tabokie](https://github.com/tabokie) - Support more regular expression functions [#13483](https://github.com/tikv/tikv/issues/13483) @[gengliqi](https://github.com/gengliqi) - Support automatically adjusting the thread pool size based on the CPU usage [#13313](https://github.com/tikv/tikv/issues/13313) @[glorv](https://github.com/glorv) @@ -293,19 +283,14 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiFlash - - Compute - - - Support pushing down the `elt` function to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) - - Support pushing down the `leftShift` function to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) - - Support pushing down the `castTimeAsDuration` function to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) - - Support pushing down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) - - Refactor TiFlash's interpreter, and support the new interpreter Planner [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) - - Improve the accuracy of memory tracker in TiFlash [#5609](https://github.com/pingcap/tiflash/issues/5609) @[bestwoody](https://github.com/bestwoody) - - Improve the performance of string columns with the `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collations [#5294](https://github.com/pingcap/tiflash/issues/5294) @[solotzg](https://github.com/solotzg) - - - Storage - - - Calculate the I/O throughput in background in ReadLimiter [#5401](https://github.com/pingcap/tiflash/issues/5401), [#5091](https://github.com/pingcap/tiflash/issues/5091) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + - Support pushing down the `elt` function to TiFlash [#5104](https://github.com/pingcap/tiflash/issues/5104) @[Willendless](https://github.com/Willendless) + - Support pushing down the `leftShift` function to TiFlash [#5099](https://github.com/pingcap/tiflash/issues/5099) @[AnnieoftheStars](https://github.com/AnnieoftheStars) + - Support pushing down the `castTimeAsDuration` function to TiFlash [#5306](https://github.com/pingcap/tiflash/issues/5306) @[AntiTopQuark](https://github.com/AntiTopQuark) + - Support pushing down the `HexIntArg/HexStrArg` function to TiFlash [#5107](https://github.com/pingcap/tiflash/issues/5107) @[YangKeao](https://github.com/YangKeao) + - Refactor TiFlash's interpreter, and support the new interpreter Planner [#4739](https://github.com/pingcap/tiflash/issues/4739) @[SeaRise](https://github.com/SeaRise) + - Improve the accuracy of memory tracker in TiFlash [#5609](https://github.com/pingcap/tiflash/issues/5609) @[bestwoody](https://github.com/bestwoody) + - Improve the performance of string columns with the `UTF8_BIN/ASCII_BIN/LATIN1_BIN/UTF8MB4_BIN` collations [#5294](https://github.com/pingcap/tiflash/issues/5294) @[solotzg](https://github.com/solotzg) + - Calculate the I/O throughput in background in ReadLimiter [#5401](https://github.com/pingcap/tiflash/issues/5401), [#5091](https://github.com/pingcap/tiflash/issues/5091) @[Lloyd-Pottiger](https://github.com/Lloyd-Pottiger) + Tools @@ -322,10 +307,10 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Improve compatibility for MySQL 8.0 upstream [#6506](https://github.com/pingcap/tiflow/issues/6506) @[lance6716](https://github.com/lance6716) - Support logging `start ts` of DML statements when MySQL sink gets an error [#6460](https://github.com/pingcap/tiflow/issues/6460) @[overvenus](https://github.com/overvenus) - - Enhance the `api/v1/health` API to return a more accurate health state of a TiCDC cluster (https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) - - Implement MQ sink and MySQL sink in the asynchronous mode to improve the sink throughput [#5928](https://github.com/pingcap/tiflow/issues/5928) @[hicqu](https://github.com/hicqu) @[hi-rustin](https://github.com/hi-rustin) - - Delete the deprecated pulsar sink [#7087](https://github.com/pingcap/tiflow/issues/7087) @[hi-rustin](https://github.com/hi-rustin) - - Improve replication performance by discarding DDL statements that are irrelevant to a changefeed [#6447](https://github.com/pingcap/tiflow/issues/6447) @[asddongmen](https://github.com/asddongmen) + - Enhance the `api/v1/health` API to return a more accurate health state of a TiCDC cluster [#4757](https://github.com/pingcap/tiflow/issues/4757) @[overvenus](https://github.com/overvenus) + - Implement MQ sink and MySQL sink in the asynchronous mode to improve the sink throughput [#5928](https://github.com/pingcap/tiflow/issues/5928) @[hicqu](https://github.com/hicqu) @[hi-rustin](https://github.com/hi-rustin) + - Delete the deprecated pulsar sink [#7087](https://github.com/pingcap/tiflow/issues/7087) @[hi-rustin](https://github.com/hi-rustin) + - Improve replication performance by discarding DDL statements that are irrelevant to a changefeed [#6447](https://github.com/pingcap/tiflow/issues/6447) @[asddongmen](https://github.com/asddongmen) + TiDB Data Migration (DM) @@ -340,59 +325,45 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiDB - - sql-infra - - - Fix the issue that the privilege check is skipped for `PREPARE` statements [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that the system variable `tidb_enable_noop_variable` can be set to `WARN` [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that when an expression index is defined, the `ORDINAL_POSITION` column of the `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) - - Fix the issue that TiDB does not report an error when the timestamp is larger than `MAXINT32` [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) - - Fix the issue that TiDB server cannot be started when the enterprise plugin is used [#37319](https://github.com/pingcap/tidb/issues/37319) @[xhebox](https://github.com/xhebox) - - Fix the incorrect output of `SHOW CREATE PLACEMENT POLICY` [#37526](https://github.com/pingcap/tidb/issues/37526) @[xhebox](https://github.com/xhebox) - - Fix the unexpected `EXCHANGE PARTITION` behaviors with temporary tables [#37201](https://github.com/pingcap/tidb/issues/37201) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that querying `INFORMATION_SCHEMA.TIKV_REGION_STATUS` returns an incorrect result @[zimulala](https://github.com/zimulala) - - Fix the issue that the `EXPLAIN` query on views does not check privileges [#34326](https://github.com/pingcap/tidb/issues/34326) @[hawkingrei](https://github.com/hawkingrei) - - Fix the issue that JSON `null` cannot be updated to `NULL` [#37852](https://github.com/pingcap/tidb/issues/37852) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that `row_count` of DDL jobs is inaccurate [#25968](https://github.com/pingcap/tidb/issues/25968) @[Defined2014](https://github.com/Defined2014) - - Fix the issue that `FLASHBACK TABLE` does not work properly [#37386](https://github.com/pingcap/tidb/issues/37386) @[tiancaiamao](https://github.com/tiancaiamao) - - Fix the issue of failing to handle `prepared` statement flags in the typical MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[dveeden](https://github.com/dveeden) - - (dup) Fix the issue of incorrect TiDB status that might appear on startup in some extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) @[xhebox](https://github.com/xhebox) - - Fix the issue that `information_schema.variables_info` does not comply with security enhanced mode (SEM) [#37586](https://github.com/pingcap/tidb/issues/37586) @[CbcWestwolf](https://github.com/CbcWestwolf) - - Fix the issue that casting string to string goes wrong in queries with `UNION` [#31678](https://github.com/pingcap/tidb/issues/31678) @[cbcwestwolf](https://github.com/cbcwestwolf) - - - execution - - - Fix the wrong result that occurs when enabling dynamic mode in partitioned tables for TiFlash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) - - Fix the issue that the cast and comparison between binary strings and JSON in TiDB are incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that `json_objectagg` and `json_arrayagg` in TiDB are not compatible with MySQL on binary values [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that the comparison between JSON opaque values causes panic [#37315](https://github.com/pingcap/tidb/issues/37315) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that the single precision float cannot be used in JSON aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that the `UNION` operator might return unexpected empty result [#36903](https://github.com/pingcap/tidb/issues/36903) @[tiancaiamao](https://github.com/tiancaiamao) - - Fix the issue that the result of the `castRealAsTime` expression is inconsistent with MySQL [#37462](https://github.com/pingcap/tidb/issues/37462) @[mengxin9014](https://github.com/mengxin9014) - - - transaction - - - Fix the issue that pessimistic DML operations lock non-unique index keys [#36235](https://github.com/pingcap/tidb/issues/36235) @[ekexium](https://github.com/ekexium) - - Fix the issue that `auto-commit` change affects transaction commit behaviours [#36581](https://github.com/pingcap/tidb/issues/36581) @[cfzjywxk](https://github.com/cfzjywxk) - - Fix the issue that the `EXPLAIN ANALYZE` statement with DML executors might return result before the transaction commit finishes [#37373](https://github.com/pingcap/tidb/issues/37373) @[cfzjywxk](https://github.com/cfzjywxk) - - - planner - - - Fix the issue that the UPDATE statements incorrectly eliminate the projection in some cases, which causes the `Can't find column` error [#37568](https://github.com/pingcap/tidb/issues/37568) @[AilinKid](https://github.com/AilinKid) - - (dup) Fix the issue that the Join Reorder operation will mistakenly push down its Outer Join condition [#37238](https://github.com/pingcap/tidb/issues/37238) @[AilinKid](https://github.com/AilinKid) - - Fix the issue that the `IN` and `NOT IN` subqueries in some patterns report the `Can't find column` error [#37032](https://github.com/pingcap/tidb/issues/37032) @[AilinKid](https://github.com/AilinKid) - - Fix the issue that `Can't find column` is reported if an `UPDATE` statement contains common table expressions (CTE) [#35758](https://github.com/pingcap/tidb/issues/35758) @[AilinKid](https://github.com/AilinKid) - - - diagnosis - - - Fix incorrect `PromQL` [#35856](https://github.com/pingcap/tidb/issues/35856) @[Defined2014](https://github.com/Defined2014) + - Fix the issue that the privilege check is skipped for `PREPARE` statements [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that the system variable `tidb_enable_noop_variable` can be set to `WARN` [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that when an expression index is defined, the `ORDINAL_POSITION` column of the `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) + - Fix the issue that TiDB does not report an error when the timestamp is larger than `MAXINT32` [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) + - Fix the issue that TiDB server cannot be started when the enterprise plugin is used [#37319](https://github.com/pingcap/tidb/issues/37319) @[xhebox](https://github.com/xhebox) + - Fix the incorrect output of `SHOW CREATE PLACEMENT POLICY` [#37526](https://github.com/pingcap/tidb/issues/37526) @[xhebox](https://github.com/xhebox) + - Fix the unexpected `EXCHANGE PARTITION` behaviors with temporary tables [#37201](https://github.com/pingcap/tidb/issues/37201) @[lcwangchao](https://github.com/lcwangchao) + - Fix the issue that querying `INFORMATION_SCHEMA.TIKV_REGION_STATUS` returns an incorrect result @[zimulala](https://github.com/zimulala) + - Fix the issue that the `EXPLAIN` query on views does not check privileges [#34326](https://github.com/pingcap/tidb/issues/34326) @[hawkingrei](https://github.com/hawkingrei) + - Fix the issue that JSON `null` cannot be updated to `NULL` [#37852](https://github.com/pingcap/tidb/issues/37852) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that `row_count` of DDL jobs is inaccurate [#25968](https://github.com/pingcap/tidb/issues/25968) @[Defined2014](https://github.com/Defined2014) + - Fix the issue that `FLASHBACK TABLE` does not work properly [#37386](https://github.com/pingcap/tidb/issues/37386) @[tiancaiamao](https://github.com/tiancaiamao) + - Fix the issue of failing to handle `prepared` statement flags in the typical MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[dveeden](https://github.com/dveeden) + - Fix the issue of incorrect TiDB status that might appear on startup in some extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) @[xhebox](https://github.com/xhebox) + - Fix the issue that `information_schema.variables_info` does not comply with security enhanced mode (SEM) [#37586](https://github.com/pingcap/tidb/issues/37586) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Fix the issue that casting string to string goes wrong in queries with `UNION` [#31678](https://github.com/pingcap/tidb/issues/31678) @[cbcwestwolf](https://github.com/cbcwestwolf) + - Fix the wrong result that occurs when enabling dynamic mode in partitioned tables for TiFlash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) + - Fix the issue that the cast and comparison between binary strings and JSON in TiDB are incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that `json_objectagg` and `json_arrayagg` in TiDB are not compatible with MySQL on binary values [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the comparison between JSON opaque values causes panic [#37315](https://github.com/pingcap/tidb/issues/37315) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the single precision float cannot be used in JSON aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that the `UNION` operator might return unexpected empty result [#36903](https://github.com/pingcap/tidb/issues/36903) @[tiancaiamao](https://github.com/tiancaiamao) + - Fix the issue that the result of the `castRealAsTime` expression is inconsistent with MySQL [#37462](https://github.com/pingcap/tidb/issues/37462) @[mengxin9014](https://github.com/mengxin9014) + - Fix the issue that pessimistic DML operations lock non-unique index keys [#36235](https://github.com/pingcap/tidb/issues/36235) @[ekexium](https://github.com/ekexium) + - Fix the issue that `auto-commit` change affects transaction commit behaviours [#36581](https://github.com/pingcap/tidb/issues/36581) @[cfzjywxk](https://github.com/cfzjywxk) + - Fix the issue that the `EXPLAIN ANALYZE` statement with DML executors might return result before the transaction commit finishes [#37373](https://github.com/pingcap/tidb/issues/37373) @[cfzjywxk](https://github.com/cfzjywxk) + - Fix the issue that the UPDATE statements incorrectly eliminate the projection in some cases, which causes the `Can't find column` error [#37568](https://github.com/pingcap/tidb/issues/37568) @[AilinKid](https://github.com/AilinKid) + - Fix the issue that the Join Reorder operation will mistakenly push down its Outer Join condition [#37238](https://github.com/pingcap/tidb/issues/37238) @[AilinKid](https://github.com/AilinKid) + - Fix the issue that the `IN` and `NOT IN` subqueries in some patterns report the `Can't find column` error [#37032](https://github.com/pingcap/tidb/issues/37032) @[AilinKid](https://github.com/AilinKid) + - Fix the issue that `Can't find column` is reported if an `UPDATE` statement contains common table expressions (CTE) [#35758](https://github.com/pingcap/tidb/issues/35758) @[AilinKid](https://github.com/AilinKid) + - Fix incorrect `PromQL` [#35856](https://github.com/pingcap/tidb/issues/35856) @[Defined2014](https://github.com/Defined2014) + TiKV - - (dup) Fix the issue that PD does not reconnect to TiKV after the Region heartbeat is interrupted [#12934](https://github.com/tikv/tikv/issues/12934) @[bufferflies](https://github.com/bufferflies) - - (dup) Fix a bug that Regions might be overlapped if Raftstore is busy [#13160](https://github.com/tikv/tikv/issues/13160) @[5kbpers](https://github.com/5kbpers) - - (dup) Fix the issue that the PD client might cause deadlocks [#13191](https://github.com/tikv/tikv/issues/13191) @[bufferflies](https://github.com/bufferflies) [#12933](https://github.com/tikv/tikv/issues/12933) @[BurtonQin](https://github.com/BurtonQin) + - Fix the issue that PD does not reconnect to TiKV after the Region heartbeat is interrupted [#12934](https://github.com/tikv/tikv/issues/12934) @[bufferflies](https://github.com/bufferflies) + - Fix a bug that Regions might be overlapped if Raftstore is busy [#13160](https://github.com/tikv/tikv/issues/13160) @[5kbpers](https://github.com/5kbpers) + - Fix the issue that the PD client might cause deadlocks [#13191](https://github.com/tikv/tikv/issues/13191) @[bufferflies](https://github.com/bufferflies) [#12933](https://github.com/tikv/tikv/issues/12933) @[BurtonQin](https://github.com/BurtonQin) - Fix the issue that TiKV might panic when encryption is disabled [#13081](https://github.com/tikv/tikv/issues/13081) @[jiayang-zheng](https://github.com/jiayang-zheng) - - (dup) Fix the wrong expression of `Unified Read Pool CPU` in Dashboard [#13086](https://github.com/tikv/tikv/issues/13086) @[glorv](https://github.com/glorv) + - Fix the wrong expression of `Unified Read Pool CPU` in Dashboard [#13086](https://github.com/tikv/tikv/issues/13086) @[glorv](https://github.com/glorv) - Fix the issue that the TiKV service is unavailable for several minutes when a TiKV instance is in an isolated network environment [#12966](https://github.com/tikv/tikv/issues/12966) @[cosven](https://github.com/cosven) - Fix the issue that TiKV mistakenly reports a `PessimisticLockNotFound` error [#13425](https://github.com/tikv/tikv/issues/13425) @[sticnarf](https://github.com/sticnarf) - Fix a bug that PITR might cause data loss in some situations [#13281](https://github.com/tikv/tikv/issues/13281) @[YuJuncen](https://github.com/YuJuncen) @@ -402,23 +373,18 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + PD - - (dup) Fix PD panics caused by the issue that gRPC handles errors inappropriately when `enable-forwarding` is enabled [#5373](https://github.com/tikv/pd/issues/5373) @[bufferflies](https://github.com/bufferflies) + - Fix PD panics caused by the issue that gRPC handles errors inappropriately when `enable-forwarding` is enabled [#5373](https://github.com/tikv/pd/issues/5373) @[bufferflies](https://github.com/bufferflies) - Fix the issue that unhealthy Region might cause PD panic [#5491](https://github.com/tikv/pd/issues/5491) @[nolouch](https://github.com/nolouch) - Fix the bug that the TiFlash learner replica might not be created [#5401](https://github.com/tikv/pd/issues/5401) @[HunDunDM](https://github.com/HunDunDM) + TiFlash - - Compute - - - Fix a bug that the window function might cause TiFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - - Fix a bug that wrong data input for `date(CAST(value AS DATETIME))` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) - - Fix the issue that the result of `cast(Real/Decimal)AsTime` is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) - - - Storage - - - Fix the issue that some obsolete data in storage cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) - - Fix a bug that page GC might block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) - - Fix the panic that occurs after creating the primary index with a column containing the `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix a bug that the window function might cause TiFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) + - Fix a bug that wrong data input for `date(CAST(value AS DATETIME))` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) + - Fix the issue that the result of `cast(Real/Decimal)AsTime` is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) + - Fix the issue that some obsolete data in storage cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix a bug that page GC might block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the panic that occurs after creating the primary index with a column containing the `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) + Tools @@ -438,35 +404,34 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiDB Data Migration (DM) - Fix the issue that DM reports the `Specified key was too long` error [#5315](https://github.com/pingcap/tiflow/issues/5315) @[lance6716](https://github.com/lance6716) - - (dup) Fix goroutine leak when relay meets an error [#6193](https://github.com/pingcap/tiflow/issues/6193) @[lance6716](https://github.com/lance6716) + - Fix goroutine leak when relay meets an error [#6193](https://github.com/pingcap/tiflow/issues/6193) @[lance6716](https://github.com/lance6716) - Fix the bug that when `collation_compatible` is set to `"strict"`, DM might generate SQL with duplicated collation [#6832](https://github.com/pingcap/tiflow/issues/6832) @[lance6716](https://github.com/lance6716) - Reduce the appearance of the warning message "found error when get timezone from binlog status_vars" in DM-worker log [#6628](https://github.com/pingcap/tiflow/issues/6628) @[lyzx2001](https://github.com/lyzx2001) - - Fix a bug that latin1 data might be corrupt during replication [#7028](https://github.com/pingcap/tiflow/issues/7028) @[lance6716](https://github.com/lance6716) + - Fix a bug that latin1 data might be corrupted during replication [#7028](https://github.com/pingcap/tiflow/issues/7028) @[lance6716](https://github.com/lance6716) + TiDB Lightning - - (dup) Fix the issue that TiDB Lightning does not support columns starting with slash, number, or non-ascii characters in Parquet files [#36980](https://github.com/pingcap/tidb/issues/36980) @[D3Hunter](https://github.com/D3Hunter) + - Fix the issue that TiDB Lightning does not support columns starting with slash, number, or non-ascii characters in Parquet files [#36980](https://github.com/pingcap/tidb/issues/36980) @[D3Hunter](https://github.com/D3Hunter) ## Contributors We would like to thank the following contributors from the TiDB community: +- @[An-DJ](https://github.com/An-DJ) +- @[AnnieoftheStars](https://github.com/AnnieoftheStars) - @[AntiTopQuark](https://github.com/AntiTopQuark) +- @[blacktear23](https://github.com/blacktear23) +- @[BurtonQin](https://github.com/BurtonQin): First-time contributor +- @[crelax](https://github.com/crelax) - @[eltociear](https://github.com/eltociear) -- @[morgo](https://github.com/morgo) - @[fuzhe1989](https://github.com/fuzhe1989) -- @[crelax](https://github.com/crelax) -- @[Ziy1-Tan](https://github.com/Ziy1-Tan) -- @[AnnieoftheStars](https://github.com/AnnieoftheStars) -- @[An-DJ](https://github.com/An-DJ) - @[erwadba](https://github.com/erwadba) -- @[whitekeepwork](https://github.com/whitekeepwork) -- @[blacktear23](https://github.com/blacktear23) -- @[rzrymiak](https://github.com/rzrymiak) -- @[AnnieoftheStars](https://github.com/AnnieoftheStars) - @[jianzhiyao](https://github.com/jianzhiyao) -- @[peakji](https://github.com/peakji) - @[joycse06](https://github.com/joycse06) +- @[morgo](https://github.com/morgo) - @[onlyacat](https://github.com/onlyacat) +- @[peakji](https://github.com/peakji) +- @[rzrymiak](https://github.com/rzrymiak) - @[tisonkun](https://github.com/tisonkun) -- @[BurtonQin](https://github.com/BurtonQin): First-time contributor +- @[whitekeepwork](https://github.com/whitekeepwork) +- @[Ziy1-Tan](https://github.com/Ziy1-Tan) diff --git a/releases/release-timeline.md b/releases/release-timeline.md index cd357739fdc93..9dbc8573735bc 100644 --- a/releases/release-timeline.md +++ b/releases/release-timeline.md @@ -9,7 +9,7 @@ This document shows all the released TiDB versions in reverse chronological orde | Version | Release Date | | :--- | :--- | -| [6.3.0-DMR](/releases/release-6.3.0.md) | 2022-xx-xx | +| [6.3.0-DMR](/releases/release-6.3.0.md) | 2022-09-30 | | [5.3.3](/releases/release-5.3.3.md) | 2022-09-14 | | [6.1.1](/releases/release-6.1.1.md) | 2022-09-01 | | [6.2.0-DMR](/releases/release-6.2.0.md) | 2022-08-23 | From 57ba9dc8297c8ee37be711e193a848f34d6449a2 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 29 Sep 2022 15:14:25 +0800 Subject: [PATCH 097/104] Update releases/release-6.3.0.md Co-authored-by: Aolin --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 1c96f954ffe62..0cf9fb90facd5 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -381,7 +381,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix a bug that the window function might cause TiFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - Fix a bug that wrong data input for `date(CAST(value AS DATETIME))` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) - - Fix the issue that the result of `cast(Real/Decimal)AsTime` is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) + - Fix the issue that the result of `CAST(Real/Decimal AS time)` is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) - Fix the issue that some obsolete data in storage cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) - Fix a bug that page GC might block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) - Fix the panic that occurs after creating the primary index with a column containing the `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) From 91313f157b55ffc4f253953df097b5d831a854e4 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Thu, 29 Sep 2022 15:51:49 +0800 Subject: [PATCH 098/104] refine wording --- releases/release-6.3.0.md | 62 +++++++++++++++++++-------------------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 0cf9fb90facd5..af6eaeb9a7697 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -206,24 +206,24 @@ In v6.3.0-DMR, the key new features and improvements are as follows: | [`tidb_constraint_check_in_place_pessimistic`](/system-variables.md#tidb_constraint_check_in_place_pessimistic-new-in-v630) | Newly added | Controls when TiDB checks [unique constraints](/constraints.md#pessimistic-transactions) in pessimistic transactions. | | [`tidb_ddl_disk_quota`](/system-variables.md#tidb_ddl_disk_quota-new-in-v630) | Newly added | Takes effect only when [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) is enabled. It sets the usage limit of local storage during backfilling when creating an index. | | [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) | Newly added | Controls whether to enable the acceleration of `ADD INDEX` and `CREATE INDEX` DDL operations to improve the speed of backfilling when creating an index. | -| [`tidb_ddl_flashback_concurrency`](/system-variables.md#tidb_ddl_flashback_concurrency-new-in-v630) | Newly added | Controls the concurrency of `flashback cluster`. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | +| [`tidb_ddl_flashback_concurrency`](/system-variables.md#tidb_ddl_flashback_concurrency-new-in-v630) | Newly added | Controls the concurrency of `flashback cluster`. The feature controlled by this variable is not fully functional in TiDB v6.3.0. Do not change the default value. | | [`tidb_enable_clustered_index`](/system-variables.md#tidb_enable_clustered_index-new-in-v50) | Modified | The default value of this variable changes from `INT_ONLY` to `ON`. | | [`tidb_enable_exchange_partition`](/system-variables.md#tidb_enable_exchange_partition) | Deprecated | Controls whether to enable the [`exchange partitions with tables`](/partitioned-table.md#partition-management) feature. The default value is `ON`, that is, `exchange partitions with tables` is enabled by default. | -| [`tidb_enable_foreign_key`](/system-variables.md#tidb_enable_foreign_key-new-in-v630) | Newly added | Controls whether to enable the `FOREIGN KEY` feature. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | -| [`tidb_enable_general_plan_cache`](/system-variables.md#tidb_enable_general_plan_cache-new-in-v630) | Newly added | Controls whether to enable the General Plan Cache feature. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | +| [`tidb_enable_foreign_key`](/system-variables.md#tidb_enable_foreign_key-new-in-v630) | Newly added | Controls whether to enable the `FOREIGN KEY` feature. The feature controlled by this variable is not fully functional in TiDB v6.3.0. Do not change the default value. | +| [`tidb_enable_general_plan_cache`](/system-variables.md#tidb_enable_general_plan_cache-new-in-v630) | Newly added | Controls whether to enable the General Plan Cache feature. The feature controlled by this variable is not fully functional in TiDB v6.3.0. Do not change the default value. | | [`tidb_enable_metadata_lock`](/system-variables.md#tidb_enable_metadata_lock-new-in-v630)| Newly added | Specifies whether to enable the [Metadata lock](/metadata-lock.md) feature. | -| [`tidb_enable_null_aware_anti_join`](/system-variables.md#tidb_enable_null_aware_anti_join-new-in-v630) | Newly added | Controls whether TiDB applies Null Aware Hash Join when ANTI JOIN is generated by subqueries led by special set operators `NOT IN` and `!= ALL`. | +| [`tidb_enable_null_aware_anti_join`](/system-variables.md#tidb_enable_null_aware_anti_join-new-in-v630) | Newly added | Controls whether TiDB applies Null-Aware Hash Join when Anti Join is generated by subqueries led by special set operators `NOT IN` and `!= ALL`. | | [`tidb_enable_pseudo_for_outdated_stats`](/system-variables.md#tidb_enable_pseudo_for_outdated_stats-new-in-v530) | Modified | Controls the behavior of the optimizer on using statistics of a table when the statistics are outdated. The default value changes from `ON` to `OFF`, which means the optimizer still keeps using the statistics of the table even if the statistics of this table are outdated. | -| [`tidb_enable_rate_limit_action`](/system-variables.md#tidb_enable_rate_limit_action) | Modified | Controls whether to enable the dynamic memory control feature for the operator that reads data. When this variable is set to `ON`, the memory usage might not be under the control of [tidb_mem_quota_query](/system-variables.md#tidb_mem_quota_query). Therefore, the default value is changed from `ON` to `OFF`. | -| [`tidb_enable_tiflash_read_for_write_stmt`](/system-variables.md#tidb_enable_tiflash_read_for_write_stmt-new-in-v630) | Newly added | Controls whether read requests in SQL write statements are pushed down to TiFlash. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | +| [`tidb_enable_rate_limit_action`](/system-variables.md#tidb_enable_rate_limit_action) | Modified | Controls whether to enable the dynamic memory control feature for the operator that reads data. When this variable is set to `ON`, the memory usage might not be under the control of [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query). Therefore, the default value is changed from `ON` to `OFF`. | +| [`tidb_enable_tiflash_read_for_write_stmt`](/system-variables.md#tidb_enable_tiflash_read_for_write_stmt-new-in-v630) | Newly added | Controls whether read requests in SQL write statements are pushed down to TiFlash. The feature controlled by this variable is not fully functional in TiDB v6.3.0. Do not change the default value. | | [`tidb_enable_unsafe_substitute`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | Controls whether to replace expressions with generated columns in an unsafe way. | -| [`tidb_general_plan_cache_size`](/system-variables.md#tidb_general_plan_cache_size-new-in-v630) | Newly added | Controls the maximum number of execution plans that can be cached by General Plan Cache. The feature controlled by this variable is not fully functional in the current TiDB version. Do not change the default value. | +| [`tidb_general_plan_cache_size`](/system-variables.md#tidb_general_plan_cache_size-new-in-v630) | Newly added | Controls the maximum number of execution plans that can be cached by General Plan Cache. The feature controlled by this variable is not fully functional in TiDB v6.3.0. Do not change the default value. | | [`tidb_last_plan_replayer_token`](/system-variables.md#tidb_enable_unsafe_substitute-new-in-v630) | Newly added | Read-only and used to obtain the result of the last `PLAN REPLAYER DUMP` execution in the current session. | | [`tidb_opt_force_inline_cte`](/system-variables.md#tidb_opt_force_inline_cte-new-in-v630) | Newly added | Controls whether common table expressions (CTEs) in the entire session are inlined or not. The default value is `OFF`, which means that inlining CTE is not enforced by default. | | [`tidb_opt_three_stage_distinct_agg`](/system-variables.md#tidb_opt_three_stage_distinct_agg-new-in-v630) | Newly added | Specifies whether to rewrite a `COUNT(DISTINCT)` aggregation into a three-stage aggregation in MPP mode. The default value is `ON`. | | [`tidb_partition_prune_mode`](/system-variables.md#tidb_partition_prune_mode-new-in-v51) | Modified | Specifies whether to enable dynamic pruning. Since v6.3.0, the default value changes to `dynamic`. | | [`tidb_rc_read_check_ts`](/system-variables.md#tidb_rc_read_check_ts-new-in-v600) | Modified | Used for optimizing the timestamp acquisition, which is suitable for scenarios with read-committed isolation level where read-write conflicts are rare. This feature is oriented to specific service workloads and might cause performance regression in other scenarios. For this reason, since v6.3.0, the scope of this variable changes from `GLOBAL \| SESSION` to `INSTANCE`. That means you can enable this feature for specific TiDB instances. | -| [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) | Newly added | Used for optimizing the acquisition of timestamps and is suitable for scenarios with few point-write conflicts in read-committed isolation level of pessimistic transactions. Enabling this variable can avoid the latency and overhead brought by obtaining the global timestamps during the execution of point-write statements | +| [`tidb_rc_write_check_ts`](/system-variables.md#tidb_rc_write_check_ts-new-in-v630) | Newly added | Used for optimizing the acquisition of timestamps and is suitable for scenarios with few point-write conflicts in RC isolation level of pessimistic transactions. Enabling this variable can avoid the latency and overhead brought by obtaining the global timestamps during the execution of point-write statements | | [`tiflash_fastscan`](/system-variables.md#tiflash_fastscan-new-in-v630) | Newly added | Controls whether to enable FastScan. If [FastScan](/develop/dev-guide-use-fastscan.md) is enabled (set to `ON`), TiFlash provides more efficient query performance, but does not guarantee the accuracy of the query results or data consistency. | ### Configuration file parameters @@ -296,12 +296,8 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + Backup & Restore (BR) - - PITR now aggregates a batch of files, which would greatly reduce the number of backup files required [#13232](https://github.com/tikv/tikv/issues/13232) @[Leavrth](https://github.com/Leavrth) - - PITR now supports automatically configuring the number of TiFlash replicas based on the upstream cluster configuration after the restoration [#37208](https://github.com/pingcap/tidb/issues/37208) @[YuJuncen](https://github.com/YuJuncen) - - + TiDB Binlog - - - Fix a bug that Drainer cannot send requests correctly to Pump when compressor is set to gzip [#1152](https://github.com/pingcap/tidb-binlog/issues/1152) @[lichunzhu](https://github.com/lichunzhu) + - PITR can merge a batch of files, which greatly reduces the number of backup files required [#13232](https://github.com/tikv/tikv/issues/13232) @[Leavrth](https://github.com/Leavrth) + - PITR supports automatically configuring the number of TiFlash replicas based on the upstream cluster configuration after the restoration [#37208](https://github.com/pingcap/tidb/issues/37208) @[YuJuncen](https://github.com/YuJuncen) + TiCDC @@ -327,7 +323,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix the issue that the privilege check is skipped for `PREPARE` statements [#35784](https://github.com/pingcap/tidb/issues/35784) @[lcwangchao](https://github.com/lcwangchao) - Fix the issue that the system variable `tidb_enable_noop_variable` can be set to `WARN` [#36647](https://github.com/pingcap/tidb/issues/36647) @[lcwangchao](https://github.com/lcwangchao) - - Fix the issue that when an expression index is defined, the `ORDINAL_POSITION` column of the `INFORMAITON_SCHEMA`.`COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) + - Fix the issue that when an expression index is defined, the `ORDINAL_POSITION` column of the `INFORMAITON_SCHEMA.COLUMNS` table might be incorrect [#31200](https://github.com/pingcap/tidb/issues/31200) @[bb7133](https://github.com/bb7133) - Fix the issue that TiDB does not report an error when the timestamp is larger than `MAXINT32` [#31585](https://github.com/pingcap/tidb/issues/31585) @[bb7133](https://github.com/bb7133) - Fix the issue that TiDB server cannot be started when the enterprise plugin is used [#37319](https://github.com/pingcap/tidb/issues/37319) @[xhebox](https://github.com/xhebox) - Fix the incorrect output of `SHOW CREATE PLACEMENT POLICY` [#37526](https://github.com/pingcap/tidb/issues/37526) @[xhebox](https://github.com/xhebox) @@ -360,14 +356,14 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + TiKV - Fix the issue that PD does not reconnect to TiKV after the Region heartbeat is interrupted [#12934](https://github.com/tikv/tikv/issues/12934) @[bufferflies](https://github.com/bufferflies) - - Fix a bug that Regions might be overlapped if Raftstore is busy [#13160](https://github.com/tikv/tikv/issues/13160) @[5kbpers](https://github.com/5kbpers) + - Fix the issue that Regions might be overlapped if Raftstore is busy [#13160](https://github.com/tikv/tikv/issues/13160) @[5kbpers](https://github.com/5kbpers) - Fix the issue that the PD client might cause deadlocks [#13191](https://github.com/tikv/tikv/issues/13191) @[bufferflies](https://github.com/bufferflies) [#12933](https://github.com/tikv/tikv/issues/12933) @[BurtonQin](https://github.com/BurtonQin) - Fix the issue that TiKV might panic when encryption is disabled [#13081](https://github.com/tikv/tikv/issues/13081) @[jiayang-zheng](https://github.com/jiayang-zheng) - Fix the wrong expression of `Unified Read Pool CPU` in Dashboard [#13086](https://github.com/tikv/tikv/issues/13086) @[glorv](https://github.com/glorv) - Fix the issue that the TiKV service is unavailable for several minutes when a TiKV instance is in an isolated network environment [#12966](https://github.com/tikv/tikv/issues/12966) @[cosven](https://github.com/cosven) - Fix the issue that TiKV mistakenly reports a `PessimisticLockNotFound` error [#13425](https://github.com/tikv/tikv/issues/13425) @[sticnarf](https://github.com/sticnarf) - - Fix a bug that PITR might cause data loss in some situations [#13281](https://github.com/tikv/tikv/issues/13281) @[YuJuncen](https://github.com/YuJuncen) - - Fix a bug that causes checkpoint not advanced when there are some long pessimistic transactions [#13304](https://github.com/tikv/tikv/issues/13304) @[YuJuncen](https://github.com/YuJuncen) + - Fix the issue that PITR might cause data loss in some situations [#13281](https://github.com/tikv/tikv/issues/13281) @[YuJuncen](https://github.com/YuJuncen) + - Fix the issue that causes checkpoint not advanced when there are some long pessimistic transactions [#13304](https://github.com/tikv/tikv/issues/13304) @[YuJuncen](https://github.com/YuJuncen) - Fix the issue that TiKV does not distinguish the datetime type (`DATETIME`, `DATE`, `TIMESTAMP` and `TIME`) and `STRING` type in JSON [#13417](https://github.com/tikv/tikv/issues/13417) @[YangKeao](https://github.com/YangKeao) - Fix incompatibility with MySQL of comparison between JSON bool and other JSON value [#13386](https://github.com/tikv/tikv/issues/13386) [#37481](https://github.com/pingcap/tidb/issues/37481) @[YangKeao](https://github.com/YangKeao) @@ -375,39 +371,43 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix PD panics caused by the issue that gRPC handles errors inappropriately when `enable-forwarding` is enabled [#5373](https://github.com/tikv/pd/issues/5373) @[bufferflies](https://github.com/bufferflies) - Fix the issue that unhealthy Region might cause PD panic [#5491](https://github.com/tikv/pd/issues/5491) @[nolouch](https://github.com/nolouch) - - Fix the bug that the TiFlash learner replica might not be created [#5401](https://github.com/tikv/pd/issues/5401) @[HunDunDM](https://github.com/HunDunDM) + - Fix the issue that the TiFlash learner replica might not be created [#5401](https://github.com/tikv/pd/issues/5401) @[HunDunDM](https://github.com/HunDunDM) + TiFlash - - Fix a bug that the window function might cause TiFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) - - Fix a bug that wrong data input for `date(CAST(value AS DATETIME))` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) + - Fix the issue that a window function might cause TiFlash to crash when the query is canceled [#5814](https://github.com/pingcap/tiflash/issues/5814) @[SeaRise](https://github.com/SeaRise) + - Fix the issue that wrong data input for `CAST(value AS DATETIME)` causing high TiFlash sys CPU [#5097](https://github.com/pingcap/tiflash/issues/5097) @[xzhangxian1008](https://github.com/xzhangxian1008) - Fix the issue that the result of `CAST(Real/Decimal AS time)` is inconsistent with MySQL [#3779](https://github.com/pingcap/tiflash/issues/3779) @[mengxin9014](https://github.com/mengxin9014) - Fix the issue that some obsolete data in storage cannot be deleted [#5570](https://github.com/pingcap/tiflash/issues/5570) @[JaySon-Huang](https://github.com/JaySon-Huang) - - Fix a bug that page GC might block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) + - Fix the issue that page GC might block creating tables [#5697](https://github.com/pingcap/tiflash/issues/5697) @[JaySon-Huang](https://github.com/JaySon-Huang) - Fix the panic that occurs after creating the primary index with a column containing the `NULL` value [#5859](https://github.com/pingcap/tiflash/issues/5859) @[JaySon-Huang](https://github.com/JaySon-Huang) + Tools + Backup & Restore (BR) - - Fix a bug that might cause the information of the checkpoint being stale [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) - - Fix a bug that the regions are not balanced because the concurrency is set too large during the restoration [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) - - Fix a bug that might cause log backup checkpoint TS stuck when TiCDC exists in the cluster [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) - - Fix a bug that might lead to backup and restoration failure if some special character exists in the authorization key of external storage [#37469](https://github.com/pingcap/tidb/issues/37469) [@MoCuishle28](https://github.com/MoCuishle28) + - Fix the issue that might cause the information of the checkpoint being stale [#36423](https://github.com/pingcap/tidb/issues/36423) @[YuJuncen](https://github.com/YuJuncen) + - Fix the issue that the regions are not balanced because the concurrency is set too large during the restoration [#37549](https://github.com/pingcap/tidb/issues/37549) @[3pointer](https://github.com/3pointer) + - Fix the issue that might cause log backup checkpoint TS stuck when TiCDC exists in the cluster [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) + - Fix the issue that might lead to backup and restoration failure if special characters exist in the authorization key of external storage [#37469](https://github.com/pingcap/tidb/issues/37469) [@MoCuishle28](https://github.com/MoCuishle28) + + + TiDB Binlog + + - Fix the issue that Drainer cannot send requests correctly to Pump when compressor is set to gzip [#1152](https://github.com/pingcap/tidb-binlog/issues/1152) @[lichunzhu](https://github.com/lichunzhu) + TiCDC - - Fix a bug that TiCDC returns an inaccurate error for a wrong PD address with a grpc service [#6458](https://github.com/pingcap/tiflow/issues/6458) @[crelax](https://github.com/crelax) - - Fix a bug that the `cdc cause cli changefeed list` command does not return failed changefeeds [#6334](https://github.com/pingcap/tiflow/issues/6334) @[asddongmen](https://github.com/asddongmen) - - Fix a bug that TiCDC is unavailable when changefeed initialization fails [#6859](https://github.com/pingcap/tiflow/issues/6859) @[asddongmen](https://github.com/asddongmen) + - Fix the issue that TiCDC returns an inaccurate error for a wrong PD address with a grpc service [#6458](https://github.com/pingcap/tiflow/issues/6458) @[crelax](https://github.com/crelax) + - Fix the issue that the `cdc cause cli changefeed list` command does not return failed changefeeds [#6334](https://github.com/pingcap/tiflow/issues/6334) @[asddongmen](https://github.com/asddongmen) + - Fix the issue that TiCDC is unavailable when changefeed initialization fails [#6859](https://github.com/pingcap/tiflow/issues/6859) @[asddongmen](https://github.com/asddongmen) + TiDB Data Migration (DM) - Fix the issue that DM reports the `Specified key was too long` error [#5315](https://github.com/pingcap/tiflow/issues/5315) @[lance6716](https://github.com/lance6716) - Fix goroutine leak when relay meets an error [#6193](https://github.com/pingcap/tiflow/issues/6193) @[lance6716](https://github.com/lance6716) - - Fix the bug that when `collation_compatible` is set to `"strict"`, DM might generate SQL with duplicated collation [#6832](https://github.com/pingcap/tiflow/issues/6832) @[lance6716](https://github.com/lance6716) + - Fix the issue that when `collation_compatible` is set to `"strict"`, DM might generate SQL with duplicated collations [#6832](https://github.com/pingcap/tiflow/issues/6832) @[lance6716](https://github.com/lance6716) - Reduce the appearance of the warning message "found error when get timezone from binlog status_vars" in DM-worker log [#6628](https://github.com/pingcap/tiflow/issues/6628) @[lyzx2001](https://github.com/lyzx2001) - - Fix a bug that latin1 data might be corrupted during replication [#7028](https://github.com/pingcap/tiflow/issues/7028) @[lance6716](https://github.com/lance6716) + - Fix the issue that latin1 data might be corrupted during replication [#7028](https://github.com/pingcap/tiflow/issues/7028) @[lance6716](https://github.com/lance6716) + TiDB Lightning From a43237ac87b1a2cf730fec9c557f6d0dc4bb6f43 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Thu, 29 Sep 2022 16:23:47 +0800 Subject: [PATCH 099/104] move a binlog fix to bug fix --- releases/release-6.3.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index af6eaeb9a7697..7104648209254 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -391,16 +391,16 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix the issue that might cause log backup checkpoint TS stuck when TiCDC exists in the cluster [#37822](https://github.com/pingcap/tidb/issues/37822) @[YuJuncen](https://github.com/YuJuncen) - Fix the issue that might lead to backup and restoration failure if special characters exist in the authorization key of external storage [#37469](https://github.com/pingcap/tidb/issues/37469) [@MoCuishle28](https://github.com/MoCuishle28) - + TiDB Binlog - - - Fix the issue that Drainer cannot send requests correctly to Pump when compressor is set to gzip [#1152](https://github.com/pingcap/tidb-binlog/issues/1152) @[lichunzhu](https://github.com/lichunzhu) - + TiCDC - Fix the issue that TiCDC returns an inaccurate error for a wrong PD address with a grpc service [#6458](https://github.com/pingcap/tiflow/issues/6458) @[crelax](https://github.com/crelax) - Fix the issue that the `cdc cause cli changefeed list` command does not return failed changefeeds [#6334](https://github.com/pingcap/tiflow/issues/6334) @[asddongmen](https://github.com/asddongmen) - Fix the issue that TiCDC is unavailable when changefeed initialization fails [#6859](https://github.com/pingcap/tiflow/issues/6859) @[asddongmen](https://github.com/asddongmen) + + TiDB Binlog + + - Fix the issue that Drainer cannot send requests correctly to Pump when compressor is set to gzip [#1152](https://github.com/pingcap/tidb-binlog/issues/1152) @[lichunzhu](https://github.com/lichunzhu) + + TiDB Data Migration (DM) - Fix the issue that DM reports the `Specified key was too long` error [#5315](https://github.com/pingcap/tiflow/issues/5315) @[lance6716](https://github.com/lance6716) From bfd04b07190d2a530b6d5747940260db3ab45f33 Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Thu, 29 Sep 2022 17:11:12 +0800 Subject: [PATCH 100/104] address comment on br issue #13232 --- releases/release-6.3.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index 7104648209254..b236a5ed1a9b5 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -296,7 +296,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi + Backup & Restore (BR) - - PITR can merge a batch of files, which greatly reduces the number of backup files required [#13232](https://github.com/tikv/tikv/issues/13232) @[Leavrth](https://github.com/Leavrth) + - PITR can merge small files generated in log backup, which greatly reduces the number of backup files [#13232](https://github.com/tikv/tikv/issues/13232) @[Leavrth](https://github.com/Leavrth) - PITR supports automatically configuring the number of TiFlash replicas based on the upstream cluster configuration after the restoration [#37208](https://github.com/pingcap/tidb/issues/37208) @[YuJuncen](https://github.com/YuJuncen) + TiCDC From 684688343df34963c9a167e7d6e1276f19b25a6e Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Thu, 29 Sep 2022 18:34:08 +0800 Subject: [PATCH 101/104] Apply suggestions from code review Co-authored-by: Aolin Co-authored-by: Grace Cai --- releases/release-6.3.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index b236a5ed1a9b5..d18421b768413 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -335,11 +335,11 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix the issue that `FLASHBACK TABLE` does not work properly [#37386](https://github.com/pingcap/tidb/issues/37386) @[tiancaiamao](https://github.com/tiancaiamao) - Fix the issue of failing to handle `prepared` statement flags in the typical MySQL protocol [#36731](https://github.com/pingcap/tidb/issues/36731) @[dveeden](https://github.com/dveeden) - Fix the issue of incorrect TiDB status that might appear on startup in some extreme cases [#36791](https://github.com/pingcap/tidb/issues/36791) @[xhebox](https://github.com/xhebox) - - Fix the issue that `information_schema.variables_info` does not comply with security enhanced mode (SEM) [#37586](https://github.com/pingcap/tidb/issues/37586) @[CbcWestwolf](https://github.com/CbcWestwolf) + - Fix the issue that `INFORMATION_SCHEMA.VARIABLES_INFO` does not comply with security enhanced mode (SEM) [#37586](https://github.com/pingcap/tidb/issues/37586) @[CbcWestwolf](https://github.com/CbcWestwolf) - Fix the issue that casting string to string goes wrong in queries with `UNION` [#31678](https://github.com/pingcap/tidb/issues/31678) @[cbcwestwolf](https://github.com/cbcwestwolf) - Fix the wrong result that occurs when enabling dynamic mode in partitioned tables for TiFlash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) - Fix the issue that the cast and comparison between binary strings and JSON in TiDB are incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that `json_objectagg` and `json_arrayagg` in TiDB are not compatible with MySQL on binary values [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that `JSON_OBJECTAGG ` and `JSON_ARRAYAGG` in TiDB are not compatible with MySQL on binary values [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the comparison between JSON opaque values causes panic [#37315](https://github.com/pingcap/tidb/issues/37315) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the single precision float cannot be used in JSON aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the `UNION` operator might return unexpected empty result [#36903](https://github.com/pingcap/tidb/issues/36903) @[tiancaiamao](https://github.com/tiancaiamao) @@ -421,7 +421,7 @@ We would like to thank the following contributors from the TiDB community: - @[AnnieoftheStars](https://github.com/AnnieoftheStars) - @[AntiTopQuark](https://github.com/AntiTopQuark) - @[blacktear23](https://github.com/blacktear23) -- @[BurtonQin](https://github.com/BurtonQin): First-time contributor +- @[BurtonQin](https://github.com/BurtonQin) (First-time contributor) - @[crelax](https://github.com/crelax) - @[eltociear](https://github.com/eltociear) - @[fuzhe1989](https://github.com/fuzhe1989) From 49e886dc2711f873652b9b8c4b509672c693e3ee Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Thu, 29 Sep 2022 18:37:00 +0800 Subject: [PATCH 102/104] add 6.3 release to About TiDB in TOC --- TOC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TOC.md b/TOC.md index 4c8ca8680683b..480a41eac9f45 100644 --- a/TOC.md +++ b/TOC.md @@ -4,7 +4,7 @@ - [Docs Home](https://docs.pingcap.com/) - About TiDB - [TiDB Introduction](/overview.md) - - [TiDB 6.2 Release Notes](/releases/release-6.2.0.md) + - [TiDB 6.3 Release Notes](/releases/release-6.3.0.md) - [Basic Features](/basic-features.md) - [Experimental Features](/experimental-features.md) - [MySQL Compatibility](/mysql-compatibility.md) From 56460886ae629b5756facda11b1a6d6f819747bb Mon Sep 17 00:00:00 2001 From: shichun-0415 Date: Thu, 29 Sep 2022 20:00:44 +0800 Subject: [PATCH 103/104] address comments --- releases/release-6.3.0.md | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index d18421b768413..e879ca6397874 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -31,9 +31,7 @@ In v6.3.0-DMR, the key new features and improvements are as follows: TiDB supports [PARTITION BY RANGE COLUMNS (column_list)](/partitioned-table.md#range-columns-partitioning). `column_list` is no longer limited to a single column. The basic feature is the same as MySQL. -* EXCHANGE PARTITION becomes GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) - - [EXCHANGE PARTITION](/partitioned-table.md#partition-management) becomes GA after performance and stability improvements. +* [EXCHANGE PARTITION](/partitioned-table.md#partition-management) becomes GA [#35996](https://github.com/pingcap/tidb/issues/35996) @[ymkzpx](https://github.com/ymkzpx) * TiDB supports two more [window functions](/tiflash/tiflash-supported-pushdown-calculations.md) [#5579](https://github.com/pingcap/tiflash/issues/5579) @[SeaRise](https://github.com/SeaRise) @@ -138,16 +136,18 @@ In v6.3.0-DMR, the key new features and improvements are as follows: When [`dynamic pruning`](/partitioned-table.md#dynamic-pruning-mode) is enabled, the optimizer selects execution plans based on [GlobalStats](/statistics.md#collect-statistics-of-partitioned-tables-in-dynamic-pruning-mode). Before GlobalStats are fully collected, using pseudo statistics might cause performance regression. In v6.3.0, this issue is addressed by maintaining the `static` mode if you enable dynamic pruning before GlobalStats are collected. TiDB remains in the `static` mode until GlobalStats are collected. This ensures performance stability when you change the partition pruning settings. +### Ease of use + +* Improve compatibility of SQL-based data Placement Rules [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) + + TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature is not compatible with TiFlash due to conflicts in implementation mechanisms. TiDB v6.3.0 optimizes this feature, and [improves compatibility of SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). + ### MySQL compatibility * Improve MySQL 8.0 compatibility by adding support for four regular expression functions: `REGEXP_INSTR()`, `REGEXP_LIKE()`, `REGEXP_REPLACE()`, and `REGEXP_SUBSTR()` [#23881](https://github.com/pingcap/tidb/issues/23881) @[windtalker](https://github.com/windtalker) For more details about the compatibility with MySQL, see [Regular expression compatibility with MySQL](/functions-and-operators/string-functions.md#regular-expression-compatibility-with-mysql). -* Improve compatibility of SQL-based data Placement Rules [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) - - TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature is not compatible with TiFlash due to conflicts in implementation mechanisms. TiDB v6.3.0 optimizes this feature, and [improves compatibility of SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). - * The `CREATE USER` and `ALTER USER` statements support the `ACCOUNT LOCK/UNLOCK` option [#37051](https://github.com/pingcap/tidb/issues/37051) @[CbcWestwolf](https://github.com/CbcWestwolf) When you create a user using the [`CREATE USER`](/sql-statements/sql-statement-create-user.md) statement, you can specify whether the created user is locked using the `ACCOUNT LOCK/UNLOCK` option. A locked user cannot log in to the database. @@ -162,9 +162,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Backup and restore -* PITR supports GCS and Azure Blob Storage as backup storages @[joccau](https://github.com/joccau) +* PITR supports [GCS](/br/backup-storage-gcs.md) and [Azure Blob Storage](/br/backup-storage-azblob.md) as backup storages @[joccau](https://github.com/joccau) - PITR supports [GCS](/br/backup-storage-gcs.md) and [Azure Blob Storage](/br/backup-storage-azblob.md) as backup storages. If your TiDB cluster is deployed on GCP or Azure, you can use the PITR feature after upgrading your cluster to v6.3.0. + If your TiDB cluster is deployed on GCP or Azure, you can use the PITR feature after upgrading your cluster to v6.3.0. * BR supports AWS S3 Object Lock [#13442](https://github.com/tikv/tikv/issues/13442) @[3pointer](https://github.com/3pointer) @@ -172,13 +172,11 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Data migration -* TiDB Lightning supports importing Parquet files exported by Apache Hive into TiDB [#37536](https://github.com/pingcap/tidb/issues/37536) @[buchuitoudegou](https://github.com/buchuitoudegou) - - TiDB Lightning supports [importing Parquet files exported by Apache Hive into TiDB](/tidb-lightning/tidb-lightning-data-source.md#parquet), thereby achieving data migration from Hive to TiDB. +* TiDB Lightning supports [importing Parquet files exported by Apache Hive into TiDB](/tidb-lightning/tidb-lightning-data-source.md#parquet) [#37536](https://github.com/pingcap/tidb/issues/37536) @[buchuitoudegou](https://github.com/buchuitoudegou) -* DM adds a new configuration item `safe-mode-duration` in the task configuration file [#6224](https://github.com/pingcap/tiflow/issues/6224) @[okJiang](https://github.com/okJiang) +* DM adds a new configuration item `safe-mode-duration` [#6224](https://github.com/pingcap/tiflow/issues/6224) @[okJiang](https://github.com/okJiang) - DM adds a new configuration item `safe-mode-duration` in the [task configuration file](/dm/task-configuration-file-full.md). You can adjust the automatic safe mode duration after DM exits abnormally. The default value is 60 seconds. If `safe-mode-duration` is set to `"0s"`, an error is reported when DM tries to enter safe mode after an abnormal restart. + This configuration item is added to the [task configuration file](/dm/task-configuration-file-full.md). You can adjust the automatic safe mode duration after DM exits abnormally. The default value is 60 seconds. If `safe-mode-duration` is set to `"0s"`, an error is reported when DM tries to enter safe mode after an abnormal restart. ### TiDB data share subscription From a6b224217f8bdc1713883cad881f58b8885d160c Mon Sep 17 00:00:00 2001 From: shichun-0415 <89768198+shichun-0415@users.noreply.github.com> Date: Fri, 30 Sep 2022 11:22:16 +0800 Subject: [PATCH 104/104] Apply suggestions from code review --- releases/release-6.3.0.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/releases/release-6.3.0.md b/releases/release-6.3.0.md index e879ca6397874..0a57bbaf2d5f6 100644 --- a/releases/release-6.3.0.md +++ b/releases/release-6.3.0.md @@ -138,9 +138,9 @@ In v6.3.0-DMR, the key new features and improvements are as follows: ### Ease of use -* Improve compatibility of SQL-based data Placement Rules [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) +* Address the conflict between SQL-based data Placement Rules and TiFlash replicas [#37171](https://github.com/pingcap/tidb/issues/37171) @[lcwangchao](https://github.com/lcwangchao) - TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature is not compatible with TiFlash due to conflicts in implementation mechanisms. TiDB v6.3.0 optimizes this feature, and [improves compatibility of SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). + TiDB v6.0.0 provides SQL-based data Placement Rules. But this feature conflicts with TiFlash replicas due to implementation issues. TiDB v6.3.0 optimizes the implementation mechanisms, and [resolves the conflict between SQL-based data Placement Rules and TiFlash](/placement-rules-in-sql.md#known-limitations). ### MySQL compatibility @@ -337,7 +337,7 @@ Since v6.3.0, TiCDC no longer supports configuring Pulsar sink. [kop](https://gi - Fix the issue that casting string to string goes wrong in queries with `UNION` [#31678](https://github.com/pingcap/tidb/issues/31678) @[cbcwestwolf](https://github.com/cbcwestwolf) - Fix the wrong result that occurs when enabling dynamic mode in partitioned tables for TiFlash [#37254](https://github.com/pingcap/tidb/issues/37254) @[wshwsh12](https://github.com/wshwsh12) - Fix the issue that the cast and comparison between binary strings and JSON in TiDB are incompatible with MySQL [#31918](https://github.com/pingcap/tidb/issues/31918) [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - - Fix the issue that `JSON_OBJECTAGG ` and `JSON_ARRAYAGG` in TiDB are not compatible with MySQL on binary values [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) + - Fix the issue that `JSON_OBJECTAGG` and `JSON_ARRAYAGG` in TiDB are not compatible with MySQL on binary values [#25053](https://github.com/pingcap/tidb/issues/25053) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the comparison between JSON opaque values causes panic [#37315](https://github.com/pingcap/tidb/issues/37315) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the single precision float cannot be used in JSON aggregation funtions [#37287](https://github.com/pingcap/tidb/issues/37287) @[YangKeao](https://github.com/YangKeao) - Fix the issue that the `UNION` operator might return unexpected empty result [#36903](https://github.com/pingcap/tidb/issues/36903) @[tiancaiamao](https://github.com/tiancaiamao)