Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids #27994

Merged
merged 3 commits into from
Dec 6, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
{
"title": "SHOW-ANALYZE-TABLETS",
"language": "en"
}
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

<version since="dev">

## SHOW-ANALYZE-TABLETS

</version>

### Name

SHOW ANALYZE TABLETS

### Description

Used to show tablets and information of their belonging table

grammar:

```sql
SHOW ANALYZE TABLETS tablet-ids;
```

illustrate:

1. tablet-ids:one or more tablet-ids, with comma separated
2. Columns of result keep same with result of `SHOW-DATA` for the same table

### Example

1. show information of four tablet-ids (actually, three tablet-ids. Result will be deduplicated)

```sql
SHOW ANALYZE TABLETS 27028,78880,78382,27028;
```

```
+---------------------+-----------+-----------+--------------+-----------+--------------+----------------+
| DbName | TableName | TableSize | PartitionNum | BucketNum | ReplicaCount | TabletIds |
+---------------------+-----------+-----------+--------------+-----------+--------------+----------------+
| default_cluster:db1 | kec | 613.000 B | 379 | 604 | 604 | [78880, 78382] |
| default_cluster:db1 | test | 1.874 KB | 1 | 1 | 1 | [27028] |
+---------------------+-----------+-----------+--------------+-----------+--------------+----------------+
```

### Keywords

SHOW, ANALYZE, TABLETS

### Best Practice

4 changes: 3 additions & 1 deletion docs/sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -867,7 +867,8 @@
"sql-manual/sql-reference/Database-Administration-Statements/RECOVER",
"sql-manual/sql-reference/Database-Administration-Statements/KILL",
"sql-manual/sql-reference/Database-Administration-Statements/ADMIN-REBALANCE-DISK",
"sql-manual/sql-reference/Database-Administration-Statements/ADMIN-CANCEL-REBALANCE-DISK"
"sql-manual/sql-reference/Database-Administration-Statements/ADMIN-CANCEL-REBALANCE-DISK",
"sql-manual/sql-reference/Database-Administration-Statements/UNSET-VARIABLE"
]
},
{
Expand Down Expand Up @@ -1024,6 +1025,7 @@
"type": "category",
"label": "Show",
"items": [
"sql-manual/sql-reference/Show-Statements/SHOW-ANALYZE-TABLETS",
"sql-manual/sql-reference/Show-Statements/SHOW-ALTER-TABLE-MATERIALIZED-VIEW",
"sql-manual/sql-reference/Show-Statements/SHOW-ALTER",
"sql-manual/sql-reference/Show-Statements/SHOW-BACKUP",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
{
"title": "SHOW-ANALYZE-TABLETS",
"language": "zh-CN"
}
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

<version since="dev">

## SHOW-ANALYZE-TABLETS

</version>

### Name

SHOW ANALYZE TABLETS

### Description

该语句用于展示指定Tablets归属的表的信息

语法:

```sql
SHOW ANALYZE TABLETS tablet-ids;
```

说明:

1. tablet-ids:指一到多个tablet-id构成的列表。如有多个,使用逗号分隔
2. 结果中 table 相关的信息和 `SHOW-DATA` 语句的口径一致

### Example

1. 展示3个tablet-id的相关信息(tablet-id可去重)

```sql
SHOW ANALYZE TABLETS 27028,78880,78382,27028;
```

```
+---------------------+-----------+-----------+--------------+-----------+--------------+----------------+
| DbName | TableName | TableSize | PartitionNum | BucketNum | ReplicaCount | TabletIds |
+---------------------+-----------+-----------+--------------+-----------+--------------+----------------+
| default_cluster:db1 | kec | 613.000 B | 379 | 604 | 604 | [78880, 78382] |
| default_cluster:db1 | test | 1.874 KB | 1 | 1 | 1 | [27028] |
+---------------------+-----------+-----------+--------------+-----------+--------------+----------------+
```

### Keywords

SHOW, ANALYZE, TABLETS

### Best Practice

4 changes: 4 additions & 0 deletions fe/fe-core/src/main/cup/sql_parser.cup
Original file line number Diff line number Diff line change
Expand Up @@ -4152,6 +4152,10 @@ show_param ::=
{:
RESULT = new ShowAnalyzeTaskStatus(jobId);
:}
| KW_ANALYZE KW_TABLETS integer_list:tabletIds
{:
RESULT = new ShowAnalyzeTabletsStmt(tabletIds);
:}
| KW_CATALOG KW_RECYCLE KW_BIN opt_wild_where
{:
RESULT = new ShowCatalogRecycleBinStmt(parser.where);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

package org.apache.doris.analysis;

import org.apache.doris.catalog.Column;
import org.apache.doris.catalog.ScalarType;
import org.apache.doris.common.UserException;
import org.apache.doris.qe.ShowResultSetMetaData;

import com.google.common.collect.ImmutableList;

import java.util.List;

/**
* ShowAnalyzeTabletsStmt is used to show statistics of tablets info.
* syntax:
* SHOW ANALYZE TABLETS tablet_ids
*/
public class ShowAnalyzeTabletsStmt extends ShowStmt {
private List<Long> tabletIds;

private static final ImmutableList<String> TITLE_NAMES = new ImmutableList.Builder<String>()
.add("DbName")
.add("TableName")
.add("TableSize")
.add("PartitionNum")
.add("BucketNum")
.add("ReplicaCount")
.add("TabletIds")
.build();

public ShowAnalyzeTabletsStmt(List<Long> tabletIds) {
this.tabletIds = tabletIds;
}

public List<Long> getTabletIds() {
return tabletIds;
}

@Override
public void analyze(Analyzer analyzer) throws UserException {
if (tabletIds == null || tabletIds.isEmpty()) {
throw new UserException("Please supply at least one tablet id");
}
}

@Override
public ShowResultSetMetaData getMetaData() {
ShowResultSetMetaData.Builder builder = ShowResultSetMetaData.builder();
for (String title : TITLE_NAMES) {
builder.addColumn(new Column(title, ScalarType.createVarchar(128)));
}
return builder.build();
}

@Override
public RedirectStatus getRedirectStatus() {
return RedirectStatus.FORWARD_NO_SYNC;
}

@Override
public String toSql() {
StringBuilder sb = new StringBuilder();
sb.append("SHOW ANALYZE TABLETS ");

for (long tabletId : tabletIds) {
sb.append(tabletId);
sb.append(", ");
}

String tmp = sb.toString();
return tmp.substring(tmp.length() - 1);
}

@Override
public String toString() {
return toSql();
}
}
61 changes: 61 additions & 0 deletions fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
import org.apache.doris.analysis.PartitionNames;
import org.apache.doris.analysis.ShowAlterStmt;
import org.apache.doris.analysis.ShowAnalyzeStmt;
import org.apache.doris.analysis.ShowAnalyzeTabletsStmt;
import org.apache.doris.analysis.ShowAnalyzeTaskStatus;
import org.apache.doris.analysis.ShowAuthorStmt;
import org.apache.doris.analysis.ShowBackendsStmt;
Expand Down Expand Up @@ -168,6 +169,7 @@
import org.apache.doris.common.proc.TrashProcNode;
import org.apache.doris.common.profile.ProfileTreeNode;
import org.apache.doris.common.profile.ProfileTreePrinter;
import org.apache.doris.common.util.DebugUtil;
import org.apache.doris.common.util.ListComparator;
import org.apache.doris.common.util.LogBuilder;
import org.apache.doris.common.util.LogKey;
Expand Down Expand Up @@ -237,6 +239,7 @@
import java.util.Collection;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
Expand Down Expand Up @@ -417,6 +420,8 @@ public ShowResultSet execute() throws AnalysisException {
handleShowCreateCatalog();
} else if (stmt instanceof ShowAnalyzeStmt) {
handleShowAnalyze();
} else if (stmt instanceof ShowAnalyzeTabletsStmt) {
handleShowAnalyzeTablet();
} else if (stmt instanceof AdminCopyTabletStmt) {
handleCopyTablet();
} else if (stmt instanceof ShowCatalogRecycleBinStmt) {
Expand Down Expand Up @@ -2704,6 +2709,62 @@ private void handleShowAnalyze() {
resultSet = new ShowResultSet(showStmt.getMetaData(), resultRows);
}

private void handleShowAnalyzeTablet() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to belongxxx?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

ShowAnalyzeTabletsStmt showStmt = (ShowAnalyzeTabletsStmt) stmt;
List<List<String>> rows = new ArrayList<>();

Env env = Env.getCurrentEnv();

TabletInvertedIndex invertedIndex = Env.getCurrentInvertedIndex();
Map<Long, HashSet<Long>> tableToTabletIdsMap = new HashMap<>();
for (long tabletId : showStmt.getTabletIds()) {
TabletMeta tabletMeta = invertedIndex.getTabletMeta(tabletId);
if (tabletMeta == null) {
continue;
}
Database db = env.getInternalCatalog().getDbNullable(tabletMeta.getDbId());
if (db == null) {
continue;
}
long tableId = tabletMeta.getTableId();
Table table = db.getTableNullable(tableId);
if (table == null) {
continue;
}

if (!tableToTabletIdsMap.containsKey(tableId)) {
tableToTabletIdsMap.put(tableId, new HashSet<>());
}
tableToTabletIdsMap.get(tableId).add(tabletId);
}

for (long tableId : tableToTabletIdsMap.keySet()) {
Table table = env.getInternalCatalog().getTableByTableId(tableId);
List<String> line = new ArrayList<>();
line.add(table.getDatabase().getFullName());
line.add(table.getName());

OlapTable olapTable = (OlapTable) table;
Pair<Double, String> tableSizePair = DebugUtil.getByteUint((long) olapTable.getDataSize());
String readableSize = DebugUtil.DECIMAL_FORMAT_SCALE_3.format(tableSizePair.first) + " "
+ tableSizePair.second;
line.add(readableSize);
line.add(new Long(olapTable.getPartitionNum()).toString());
int totalBucketNum = 0;
Set<String> partitionNamesSet = table.getPartitionNames();
for (String partitionName : partitionNamesSet) {
totalBucketNum += table.getPartition(partitionName).getDistributionInfo().getBucketNum();
}
line.add(new Long(totalBucketNum).toString());
line.add(new Long(olapTable.getReplicaCount()).toString());
line.add(tableToTabletIdsMap.get(tableId).toString());

rows.add(line);
}

resultSet = new ShowResultSet(showStmt.getMetaData(), rows);
}

private void handleCopyTablet() throws AnalysisException {
AdminCopyTabletStmt copyStmt = (AdminCopyTabletStmt) stmt;
long tabletId = copyStmt.getTabletId();
Expand Down