[SPARK-25075] Fix some complilation error scala 2.13 part 1

[karolchmist]

What changes were proposed in this pull request?

Adapting the code to Scala 2.13

I didn't find a specific JIRA ticket for it, so I used the umbrella ticket as a reference.

Why are the changes needed?

I'm working on adapting REPL to Scala 2.13.x.

I opened a PR for it, but it was some time ago and the code changed since, with many conflicts in respect to the master branch.

So I think it's better to open first a number of smaller PRs that compile both in Scala 2.12 and 2.13. Once the modules that REPL depends on compile, I'll open a new PR that fixed the REPL itself.

Does this PR introduce any user-facing change?

No

How was this patch tested?

export SBT_MAVEN_PROFILES=scala-2.13
./build/sbt core/compile core/test:compile

Each module has its own commit.

It was compiled Scala 2.12 as well, without the SBT_MAVEN_PROFILES variable set.

[AmplabJenkins]

Can one of the admins verify this patch?

[dongjoon-hyun]

We already make core module to pass Scala 2.13 compilation three days ago. Is this required?

• 3ad4863

cc @srowen

[srowen]

Yeah I don't see that any of those changes are needed to compile?

[niebloomj]

Woohoo! Nice PR!

[karolchmist]

I have these errors below. Is there a CI job testing 2.13? I couldn't find any here nor here...

[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:656: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         if (opt.clOption != null) { childArgs += (opt.clOption, opt.value) }
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:688: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += (args.primaryResource, args.mainClass)
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:693: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         Option(args.driverMemory).foreach { m => childArgs += ("--memory", m) }
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:694: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         Option(args.driverCores).foreach { c => childArgs += ("--cores", c) }
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:696: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += (args.master, args.primaryResource, args.mainClass)
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:719: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += ("--primary-py-file", args.primaryResource)
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:720: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += ("--class", "org.apache.spark.deploy.PythonRunner")
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:723: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += ("--primary-r-file", mainFile)
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:724: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += ("--class", "org.apache.spark.deploy.RRunner")
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:727: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]           childArgs += ("--jar", args.primaryResource)
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:729: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += ("--class", args.mainClass)
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:732: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         args.childArgs.foreach { arg => childArgs += ("--arg", arg) }
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:741: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += (args.primaryResource, "")
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:747: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += (args.primaryResource, "")
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:749: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += (args.primaryResource, args.mainClass)
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:775: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]           childArgs += ("--arg", arg)
[error] [warn] 
[error] [warn] /home/karol/workspace/open-source/spark/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:780: multiarg infix syntax looks like a tuple and will be deprecated
[error] [warn]         childArgs += ("--proxy-user", args.proxyUser)
[error] [warn] 

[srowen]

There is no 2.13 CI/CD job yet - it doesn't compile, we know that. I'm submitting PRs to get much or all of it to compile, then tests will have to be evaluated.

I'm close to having everything up to REPL compiling which would allow you to at least work on the 2.13 REPL.

Those are warnings, not errors; I think your build is configured to raise them as errors. Let's only focus on errors for now as it's going to be big enough to maintain 2.12/2.13 compatibility without separate source trees already.

[karolchmist]

Ok, I won't do any more changes for now so we don't overlap, but please let me know if I can help. Closing this PR.

[dongjoon-hyun]

Thank you for closing, @karolchmist . Thanks, @srowen .