typechecker.Conexts not found when upgrading from 0.6.2 -> 0.7.0

[ngbinh]

Not sure if anyone like me but I get this error when upgrading refined from 0.6.2 to 0.7.0.

[error] Class scala.tools.nsc.typechecker.Contexts not found - continuing with a stub.
[error 01:39:37:AM] Class scala.tools.nsc.typechecker.Contexts not found - continuing with a stub.
[error] Class scala.tools.nsc.typechecker.Contexts not found - continuing with a stub.
[error 01:39:37:AM] Class scala.tools.nsc.typechecker.Contexts not found - continuing with a stub.
[error] two errors found

[fthomas]

Oh dear! This is probably caused by #251. Are you using the Eval predicate?

[fthomas]

I can reproduce this in one of my projects. The strange thing is that it only happens on the first compile after starting sbt. Compilation succeeds without errors or warnings on subsequent compiles. This is independent of the Scala version (tested with 2.11.8 and 2.12.1) - they only differ in the error message. I get

[error] Class scala.tools.nsc.Global not found - continuing with a stub.

on 2.11.8.

[ngbinh]

We are using a lot of Regex but no Eval I believe. It happens on scala 2.12.1 for us

[ngbinh]

...and it happens to us consistently

[fthomas]

I currently have no idea how to fix this other than reverting #251. @clhodapp, do you have any ideas?

[clhodapp]

Yikes! Let's try to gather a bit more detail about what everyone's dependency topologies look like and/or get an open source reproduction.

[fthomas]

Here is a reproduction: fthomas/typelevel-webapp#4

[clhodapp]

Currently, I am thinking that the problem is the use of eval in https://github.com/fthomas/refined/blob/master/modules/core/shared/src/main/scala/eu/timepit/refined/macros/RefineMacro.scala. I think that using Context#eval in a macro actually creates a compile-time dependency on scala-compiler.

Interestingly, refined's tests pass due to the line,

refined/build.sbt

Line 66 in c734e3e

"org.scala-lang" % "scala-compiler" % scalaVersion.value % Provided,

, in the build. I was actually a bit uneasy about that line before even though I know that e.g. shapeless has an equivalent line simply because I didn't know why it was needed to make the tests pass. Now I think I know why it needs to be there. And you can actually get the tests to exhibit the same issue that the downstream projects are seeing by removing that line from the build. This is an interesting case where being a test in the same project has slightly different semantics for classpath management from being a downstream project.

Now onto how to fix it (if I am right)... the very-short-term workaround would be to have every project that wants to leverage refineMV, refineMT, or refined.auto add "org.scala-lang" % "scala-compiler" % scalaVersion.value % Provided to its build.

For a longer-term fix, we are running up against the limitations of sbt and that fact that it tries to be maven-compatible. There is not a perfect way to express a compile-time dependency in sbt. The provided scope is not actually a compile-time dependency (even though a lot of Scala macro projects have treated as being such). Instead, it is a promise that you will handle getting some library onto the classpath at runtime (with the implication being that the build tool should get it onto the classpath at compile time for your project but not for any downstream ones).

Since we don't have access to a way to express proper compile-time dependencies at compile time, we really only have two choices:
a) Go back to a runtime dependency on the compiler. If we do this, at least it will officially be "safe" for in-the-know downstream runnable projects to take the compiler off of their runtime classpaths with some sbt-foo (previously, it was unsafe and could lead to classloader failures).
b) Stop using eval. I actually think that this is quite possible since refined only accepts literals into its macros. Based on a quick scan of the implementation, I really do think that this is possible, though it would make the macro code slightly longer. The benefit of this if I am right is that the dependency on scala-compiler from refined-core just becomes not-a-thing (not even provided).

@fthomas would you mind giving me a chance to try rewriting the macro to not use eval in the next couple days?

[clhodapp]

Oh, I do suppose that if you don't object we could also just add "Put "org.scala-lang" % "scala-compiler" % scalaVersion.value % Provided into your build.sbt" to the README and call it a day if you are OK with that.

[fthomas]

Thanks for the analysis, @clhodapp. It would be great if we could rewrite the macro to not use eval since that has already troubled me in the past (#3). Currently it is used to have the Validate and RefType type class instances available during macro expansion so that we can check if a literal conforms to some predicate. If there is a way to avoid eval there, I'm all ears!

[ngbinh]

Awesome. I will try the work around

[fthomas]

I've amended the README and put "org.scala-lang" % "scala-compiler" % scalaVersion.value % Provided, to the libraryDependencies required for using refined. I consider this a temporary workaround until:

1. we have rewritten the macros to not use eval or
2. added back scala-compiler as runtime dependency.

[ngbinh]

confirmed that adding scala-compiler solves the problem. I think we can close this issue now.