Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-21831][TEST] Remove spark.sql.hive.convertMetastoreOrc config in HiveCompatibilitySuite #19043

Closed
wants to merge 2 commits into from

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Aug 24, 2017

What changes were proposed in this pull request?

SPARK-19025 removes SQLBuilder, so we don't need the following in HiveCompatibilitySuite.

// Ensures that the plans generation use metastore relation and not OrcRelation
// Was done because SqlBuilder does not work with plans having logical relation
TestHive.setConf(HiveUtils.CONVERT_METASTORE_ORC, false)

How was this patch tested?

Pass the existing Jenkins tests.

@@ -106,7 +106,7 @@ private[spark] object HiveUtils extends Logging {
.doc("When set to true, the built-in ORC reader and writer are used to process " +
"ORC tables created by using the HiveQL syntax, instead of Hive serde.")
.booleanConf
.createWithDefault(false)
.createWithDefault(true)
Copy link
Member Author

@dongjoon-hyun dongjoon-hyun Aug 24, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I set this to show HiveCompatibilitySuite pass with true, too.
After Jenkins passes, I will revert this line. This PR is a test-only fix intentially.

@dongjoon-hyun
Copy link
Member Author

dongjoon-hyun commented Aug 24, 2017

HiveCompatibilitySuite passed in Test build #81098, but HiveDDLSuite seems not to be tested with that option before. It should be handled in a different PR because it will be a bug.

@SparkQA
Copy link

SparkQA commented Aug 24, 2017

Test build #81098 has finished for PR 19043 at commit 2a72f7e.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Aug 24, 2017

Test build #81099 has finished for PR 19043 at commit 10fd9ab.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member Author

Hi, @gatorsmile .
Could you review this PR?

@gatorsmile
Copy link
Member

LGTM

@asfgit asfgit closed this in 522e1f8 Aug 26, 2017
@dongjoon-hyun
Copy link
Member Author

Thank you for reviewing and merging!

@dongjoon-hyun dongjoon-hyun deleted the SPARK-21831 branch August 26, 2017 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants