Skip to content
This repository has been archived by the owner on Feb 6, 2024. It is now read-only.

refactor: make create table idempotent #286

Merged
merged 13 commits into from
Jan 23, 2024

Conversation

ZuLiangWang
Copy link
Contributor

@ZuLiangWang ZuLiangWang commented Dec 5, 2023

Rationale

Store the intermediate data for table creation through procedures to support idempotent table creation.

Detailed Changes

  • Store the intermediate data for table creation through procedure.
  • Supports retrying from the upper layer to complete failed table creation procedures.

Test Plan

Pass all unit tests and integration tests.

@ShiKaiWi
Copy link
Member

@ZuLiangWang After a bit of reviewing, I find the table id is not persisted in the TableAssign procedure, and table creation may fail if a new table id is allocated in the second retry. And is it true?

@ShiKaiWi
Copy link
Member

@ZuLiangWang After a bit of reviewing, I find the table id is not persisted in the TableAssign procedure, and table creation may fail if a new table id is allocated in the second retry. And is it true?

After digging into the code not included by this PR, I find the table id will be fetched from the table meta data, so the second creation will use the same table id as before.

server/cluster/metadata/topology_manager.go Outdated Show resolved Hide resolved
server/cluster/metadata/topology_manager.go Outdated Show resolved Hide resolved
server/cluster/metadata/topology_manager.go Outdated Show resolved Hide resolved
server/coordinator/factory.go Outdated Show resolved Hide resolved
## Rationale
Currently the tests on the dev branch are not running properly.

## Detailed Changes
* Cherry pick some changes on main branch.
* Fix integration test.

## Test Plan
Pass CI.
@ZuLiangWang ZuLiangWang force-pushed the refactor_create_table_dev branch from 9c39c63 to 3426364 Compare January 10, 2024 08:44
@ZuLiangWang ZuLiangWang changed the base branch from dev to main January 10, 2024 08:44
@ZuLiangWang ZuLiangWang force-pushed the refactor_create_table_dev branch from 3426364 to df98b9e Compare January 10, 2024 08:46
@ZuLiangWang ZuLiangWang force-pushed the refactor_create_table_dev branch from df98b9e to 71792f2 Compare January 10, 2024 08:48
server/cluster/metadata/cluster_metadata.go Outdated Show resolved Hide resolved
server/cluster/metadata/cluster_metadata.go Outdated Show resolved Hide resolved
server/cluster/metadata/cluster_metadata.go Outdated Show resolved Hide resolved
server/cluster/metadata/topology_manager.go Outdated Show resolved Hide resolved
server/coordinator/persist_shard_picker.go Outdated Show resolved Hide resolved
server/etcdutil/util.go Outdated Show resolved Hide resolved
server/coordinator/persist_shard_picker.go Outdated Show resolved Hide resolved
server/coordinator/persist_shard_picker.go Outdated Show resolved Hide resolved
server/coordinator/persist_shard_picker.go Outdated Show resolved Hide resolved
@ShiKaiWi ShiKaiWi merged commit d880dd2 into apache:main Jan 23, 2024
2 of 3 checks passed
ShiKaiWi pushed a commit to apache/horaedb that referenced this pull request Jan 23, 2024
## Rationale
Refer to: apache/incubator-horaedb-meta#286,
since we implement idempotent table creation, the error message when
repeatedly creating a table has changed, and the results of the
integration test need to be modified.

## Detailed Changes
* Update `create_tables.result`

## Test Plan
No need.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants