Skip to content

Commit

Permalink
[SPARK-16229][SQL] Drop Empty Table After CREATE TABLE AS SELECT fails
Browse files Browse the repository at this point in the history
#### What changes were proposed in this pull request?
In `CREATE TABLE AS SELECT`, if the `SELECT` query failed, the table should not exist. For example,

```SQL
CREATE TABLE tab
STORED AS TEXTFILE
SELECT 1 AS a, (SELECT a FROM (SELECT 1 AS a UNION ALL SELECT 2 AS a) t) AS b
```
The above query failed as expected but an empty table `t` is created.

This PR is to drop the created table when hitting any non-fatal exception.

#### How was this patch tested?
Added a test case to verify the behavior

Author: gatorsmile <gatorsmile@gmail.com>

Closes #13926 from gatorsmile/dropTableAfterException.
  • Loading branch information
gatorsmile authored and cloud-fan committed Jul 6, 2016
1 parent 909c6d8 commit 21eadd1
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@

package org.apache.spark.sql.hive.execution

import scala.util.control.NonFatal

import org.apache.spark.sql.{AnalysisException, Row, SparkSession}
import org.apache.spark.sql.catalyst.catalog.{CatalogColumn, CatalogTable}
import org.apache.spark.sql.catalyst.plans.logical.{InsertIntoTable, LogicalPlan}
Expand Down Expand Up @@ -87,8 +89,15 @@ case class CreateHiveTableAsSelectCommand(
throw new AnalysisException(s"$tableIdentifier already exists.")
}
} else {
sparkSession.sessionState.executePlan(InsertIntoTable(
metastoreRelation, Map(), query, overwrite = true, ifNotExists = false)).toRdd
try {
sparkSession.sessionState.executePlan(InsertIntoTable(
metastoreRelation, Map(), query, overwrite = true, ifNotExists = false)).toRdd
} catch {
case NonFatal(e) =>
// drop the created table.
sparkSession.sessionState.catalog.dropTable(tableIdentifier, ignoreIfNotExists = true)
throw e
}
}

Seq.empty[Row]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -554,6 +554,21 @@ class HiveDDLSuite
}
}

test("Create Cataloged Table As Select - Drop Table After Runtime Exception") {
withTable("tab") {
intercept[RuntimeException] {
sql(
"""
|CREATE TABLE tab
|STORED AS TEXTFILE
|SELECT 1 AS a, (SELECT a FROM (SELECT 1 AS a UNION ALL SELECT 2 AS a) t) AS b
""".stripMargin)
}
// After hitting runtime exception, we should drop the created table.
assert(!spark.sessionState.catalog.tableExists(TableIdentifier("tab")))
}
}

test("desc table for data source table") {
withTable("tab1") {
val tabName = "tab1"
Expand Down

0 comments on commit 21eadd1

Please sign in to comment.