[SPARK-30412][SQL][TESTS] Eliminate warnings in Java tests regarding to deprecated Spark SQL API #27081
Conversation
| @Test | ||
| public void testTypedAggregationAverage() { | ||
| KeyValueGroupedDataset<String, Tuple2<String, Integer>> grouped = generateGroupedDataset(); | ||
| Dataset<Tuple2<String, Double>> agged = grouped.agg(typed.avg(v -> (double)(v._2() * 2))); | ||
| Dataset<Tuple2<String, Double>> agged = grouped.agg( | ||
| org.apache.spark.sql.expressions.javalang.typed.avg(v -> (double)(v._2() * 2))); |
There was a problem hiding this comment.
don't think it's a good idea to use full class name. you may use rename a class on import
import org.apache.spark.sql.expressions.javalang.{typed => javaTyped}
There was a problem hiding this comment.
The purpose of using full path to deprecated methods is to put any references to deprecated class under the annotation @SuppressWarnings("deprecation"). Even if we forget that your code doesn't compile in Java, it still refers to deprecated class.
| @@ -25,43 +25,50 @@ | |||
|
|
|||
| import org.apache.spark.sql.Dataset; | |||
| import org.apache.spark.sql.KeyValueGroupedDataset; | |||
| import org.apache.spark.sql.expressions.javalang.typed; | |||
There was a problem hiding this comment.
suggest to use rename a class on import, e.g. import org.apache.spark.sql.expressions.javalang.{typed => javaTyped}
instead of use full class name in rest of code
There was a problem hiding this comment.
It is Java not Scala. What do you propose doesn't work here.
|
Test build #116049 has finished for PR 27081 at commit
|
|
Merged to master. |
What changes were proposed in this pull request?
In the PR, I propose to add the
@SuppressWarnings("deprecation")annotation to Java tests for deprecated Spark SQL APIs.Why are the changes needed?
This eliminates the following warnings:
and highlights warnings about real problems.
Does this PR introduce any user-facing change?
No
How was this patch tested?
By existing test suites
Java8DatasetAggregatorSuite.java,JavaDataFrameSuite.javaandJavaDatasetAggregatorSuite.java.