Skip to content

Commit

Permalink
[SPARK-34152][SQL][FOLLOWUP] Do not uncache the temp view if it doesn…
Browse files Browse the repository at this point in the history
…'t exist

### What changes were proposed in this pull request?

This PR fixes a mistake in #31273. When CREATE OR REPLACE a temp view, we need to uncache the to-be-replaced existing temp view. However, we shouldn't uncache if there is no existing temp view.

This doesn't cause real issues because the uncache action is failure-safe. But it produces a lot of warning messages.

### Why are the changes needed?

Avoid unnecessary warning logs.

### Does this PR introduce _any_ user-facing change?

no

### How was this patch tested?

manually run tests and check the warning messages.

Closes #31650 from cloud-fan/warnning.

Authored-by: Wenchen Fan <wenchen@databricks.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
cloud-fan authored and dongjoon-hyun committed Feb 25, 2021
1 parent f7ac2d6 commit dffb01f
Showing 1 changed file with 10 additions and 8 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ case class CreateViewCommand(

if (viewType == LocalTempView) {
val aliasedPlan = aliasPlan(sparkSession, analyzedPlan)
if (replace && !isSamePlan(catalog.getRawTempView(name.table), aliasedPlan)) {
if (replace && needsToUncache(catalog.getRawTempView(name.table), aliasedPlan)) {
logInfo(s"Try to uncache ${name.quotedString} before replacing.")
checkCyclicViewReference(analyzedPlan, Seq(name), name)
CommandUtils.uncacheTableOrView(sparkSession, name.quotedString)
Expand All @@ -139,7 +139,7 @@ case class CreateViewCommand(
val db = sparkSession.sessionState.conf.getConf(StaticSQLConf.GLOBAL_TEMP_DATABASE)
val viewIdent = TableIdentifier(name.table, Option(db))
val aliasedPlan = aliasPlan(sparkSession, analyzedPlan)
if (replace && !isSamePlan(catalog.getRawGlobalTempView(name.table), aliasedPlan)) {
if (replace && needsToUncache(catalog.getRawGlobalTempView(name.table), aliasedPlan)) {
logInfo(s"Try to uncache ${viewIdent.quotedString} before replacing.")
checkCyclicViewReference(analyzedPlan, Seq(viewIdent), viewIdent)
CommandUtils.uncacheTableOrView(sparkSession, viewIdent.quotedString)
Expand Down Expand Up @@ -193,15 +193,17 @@ case class CreateViewCommand(
}

/**
* Checks if the temp view (the result of getTempViewRawPlan or getRawGlobalTempView) is storing
* the same plan as the given aliased plan.
* Checks if need to uncache the temp view being replaced.
*/
private def isSamePlan(
private def needsToUncache(
rawTempView: Option[LogicalPlan],
aliasedPlan: LogicalPlan): Boolean = rawTempView match {
case Some(TemporaryViewRelation(_, Some(p))) => p.sameResult(aliasedPlan)
case Some(p) => p.sameResult(aliasedPlan)
case _ => false
// The temp view doesn't exist, no need to uncache.
case None => false
// Do not need to uncache if the to-be-replaced temp view plan and the new plan are the
// same-result plans.
case Some(TemporaryViewRelation(_, Some(p))) => !p.sameResult(aliasedPlan)
case Some(p) => !p.sameResult(aliasedPlan)
}

/**
Expand Down

0 comments on commit dffb01f

Please sign in to comment.