Update Scala.js to 1.8.0 #597

armanbilge · 2021-12-21T14:37:31Z

Towards #590.

tgodzik · 2021-12-21T14:43:15Z

I think you also need to update https://github.com/scalameta/mdoc/blob/main/project/plugins.sbt#L4

armanbilge · 2021-12-21T14:59:29Z

Hmm, anywhere else to change it?

tgodzik · 2021-12-21T15:14:41Z

Hmm, anywhere else to change it?

Looks like Scala 3 doesn't yet support it. Maybe updating to 3.1.1-RC2 would help? (change all 3.1.0 to 3.1.1-RC2 in the project.) and probably also here https://github.com/scalameta/mdoc/blob/791a78ad620f85f38a3cef6a5a605662678930b4/mdoc-sbt/src/sbt-test/sbt-mdoc/basic/project/plugins.sbt

armanbilge · 2021-12-21T16:30:47Z

Looks like Scala 3 doesn't yet support it.

That's right, now that Scala.js compiler plugin is built-in to Scala 3 it no longer has an independent release cycle. I don't think I see a PR upgrading Scala 3 to Scala.js 1.8 in https://github.com/lampepfl/dotty/pulls?q=is%3Apr+author%3Asjrd+.

So then, the real problem is that the scalajs-library_2.13-1.8.0.jar artifact is ending up on the classpath for Scala 3 mdoc plugin. Seems that should be held at 1.7.1.

tgodzik · 2021-12-21T16:40:10Z

Maybe we could just add:

def scala3js = "1.7.1"
....
      if2 = List(
        "org.scala-js" % "scalajs-compiler" % scalajs cross CrossVersion.full,
        "org.scala-js" %% "scalajs-linker" % scalajs
      ),
      if3 = List(
        "org.scala-js" %% "scalajs-linker" % scala3js cross CrossVersion.for3Use2_13
      )

in https://github.com/scalameta/mdoc/blob/main/build.sbt#L394 ?

armanbilge · 2021-12-21T16:40:39Z

We think the same! :)

armanbilge · 2021-12-21T16:47:26Z

Btw, I am also concerned this setting will depend on Scala 2/3. Right now it's 1.8.0, so we'll see what happens.

mdoc/build.sbt

Line 441 in 7ee8081

"SCALAJS_VERSION" -> scala2js,

The tricky part is, IIUC this property is usually derived automatically by the sbt-mdoc plugin. So I fear that we may have to introduce some custom logic in the plugin to not let this exceed the maximum supported Scala.js version when using with Scala 3.

armanbilge · 2021-12-21T17:09:23Z

Hmm, pinning Scala 3 to 1.7.1 still didn't fix it. Locally if I run unitJS/test I can replicate the IRVersionNotSupportedException failure. But, if I run show unitJS/managedClasspath I don't see any Scala.js 1.8 artifacts on the classpath.

Edit: also checked show unitJS/Test/managedClasspath

Edit 2: problem is in jsdocs project.

sbt:mdocRoot> show jsdocs/libraryDependencies
[info] * org.scala-lang:scala3-library_sjs1:3.1.0
[info] * org.scala-js:scalajs-library_2.13:1.8.0
[info] * org.scala-js:scalajs-test-bridge_2.13:1.8.0:test
[info] * org.scala-js:scalajs-dom:2.0.0

Edit 3: checked one of my Scala 3.1 / Scala.js 1.8 projects which works fine, and it has this:

sbt:http4s-dom> show tests/libraryDependencies
[info] * org.scala-lang:scala3-library_sjs1:3.1.0
[info] * org.scala-js:scalajs-library_2.13:1.8.0
[info] * org.scala-js:scalajs-test-bridge_2.13:1.8.0:test
[info] * org.scalameta:munit:0.7.29:test
[info] * org.typelevel:munit-cats-effect-3:1.0.7:test

This reverts commit 7ee8081.

armanbilge · 2021-12-21T17:39:47Z

@sjrd would you mind helping us understand what might be happening here? Thanks.

sjrd · 2021-12-21T17:43:31Z

build.sbt

+def scala2js = "1.8.0"
+def scala3js = "1.7.1"


There's no reason to make those different. You can always use 1.8.0, since Scala.js IR is backward binary compatible.

Right, except we seem to be running into forward-compatibility problem here only on Scala 3.

mdoc.internal.markdown.ModifierException: mdoc:js exception Caused by: org.scalajs.ir.IRVersionNotSupportedException: Failed to deserialize a file compiled with Scala.js 1.8 (supported up to: 1.7): /home/runner/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-js/scalajs-library_2.13/1.8.0/scalajs-library_2.13-1.8.0.jar:/scala/volatile.sjsir

That must be due to something in your infrastructure that I am not familiar with. This is because the linker being used is 1.7.x instead of being 1.8.0. But Scala 3 per se doesn't even get to choose that. It must be something else in your infrastructure that selects the 1.7.x linker, whereas it should always select the 1.8.0 linker.

Thanks. So @tgodzik it's back to my original question, where else do I need to change the version number 😆

No idea 😓 There seem to be no place where we define 1.7 linker so that must be brought in by the compiler maybe? Doesn't the Scala 3 compiler need to be able to read 1.8 ?

For locally released version everything seems to be correct:

cs resolve -t org.scalameta:mdoc-js_3:2.2.22+71-5ab2f66a+20211222-1821-SNAPSHOT | grep -B 12 -A 10 linker Result: └─ org.scalameta:mdoc-js_3:2.2.22+71-5ab2f66a+20211222-1821-SNAPSHOT ├─ org.scala-js:scalajs-linker_2.13:1.8.0 │ ├─ com.google.javascript:closure-compiler:v20211201 │ ├─ org.scala-js:scalajs-ir_2.13:1.8.0 │ │ └─ org.scala-lang:scala-library:2.13.6 -> 2.13.7 │ ├─ org.scala-js:scalajs-linker-interface_2.13:1.8.0 │ │ ├─ org.scala-js:scalajs-ir_2.13:1.8.0 │ │ │ └─ org.scala-lang:scala-library:2.13.6 -> 2.13.7 │ │ ├─ org.scala-js:scalajs-logging_2.13:1.1.1 │ │ │ └─ org.scala-lang:scala-library:2.13.2 -> 2.13.7 │ │ └─ org.scala-lang:scala-library:2.13.6 -> 2.13.7 │ ├─ org.scala-lang:scala-library:2.13.6 -> 2.13.7 │ └─ org.scala-lang.modules:scala-parallel-collections_2.13:0.2.0 │ └─ org.scala-lang:scala-library:2.13.0 -> 2.13.7 ├─ org.scala-lang:scala3-library_3:3.1.0 │ └─ org.scala-lang:scala-library:2.13.6 -> 2.13.7

@sjrd what would cause it to always work after you upgrade version such as https://github.com/lampepfl/dotty/pull/13734/files ?

IIUC (which I very well may not) the Scala 3 compiler uses the .class files of dependencies for compiling. Linking is what uses the .sjsir files and depends on the IR version (hence it is the linker throwing this error), but there is no Scala 3 linker—as described in this blog post:

The .sjsir format and specification is independent of Scala, its compiler or its standard library. In fact, there is nothing Scala-specific in the entire linker, other than a few optimizations targeted at Scala-like code. This is important because it means that the linker is independent of the version of Scala: we use the same linker irrespective of the version of Scala that was used to create the .sjsir files.

Classpath for the compiler seems to be:

/home/tgodzik/Documents/mdoc/tests/jsdocs/target/scala-3.1.0/classes:/home/tgodzik/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_sjs1_3/3.1.0/scala3-library_sjs1_3-3.1.0.jar:/home/tgodzik/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-js/scalajs-library_2.13/1.8.0/scalajs-library_2.13-1.8.0.jar:/home/tgodzik/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-js/scalajs-dom_sjs1_3/2.0.0/scalajs-dom_sjs1_3-2.0.0.jar:/home/tgodzik/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.6/scala-library-2.13.6.jar

And the compiler isn't able to read? Should the classpath contain the linker?

sjrd · 2021-12-21T17:45:40Z

So then, the real problem is that the scalajs-library_2.13-1.8.0.jar artifact is ending up on the classpath for Scala 3 mdoc plugin. Seems that should be held at 1.7.1.

That shouldn't be a problem. Users even outside of mdoc can use Scala 3 with Scala.js 1.8.0, and they would get version 1.8.0.

armanbilge · 2021-12-26T16:29:30Z

So @tgodzik raised a good question in #597 (comment) about why it would work after the corresponding SJS changes in the Scala 3 compiler.

What's interesting is that the update to SJS 1.7 went exactly the same way in #540. Things were mostly working, except Scala 3 was having IRVersionNotSupportedException that finally went away after 1cd61ab which updated to Scala 3.1.

armanbilge · 2022-01-05T06:51:33Z

I went down the rabbit hole with this again in http4s-dom. By explicitly adding a dependency to the SJS 1.8 linker to my docs project, like so:

lazy val docs =
  project
    .in(file("mdocs"))
    .enablePlugins(MdocPlugin)
    .settings(
      mdocJS := Some(jsdocs),
      libraryDependencies += "org.scala-js" %% "scalajs-linker" % "1.8.0" cross CrossVersion.for3Use2_13,
    ...

I was able to get this error:

[error] (run-main-1) java.lang.NoSuchMethodError: 'org.scalajs.ir.Types$PrimRef org.scalajs.ir.Types$PrimRef$.unapply(org.scalajs.ir.Types$PrimRef)'
[error] java.lang.NoSuchMethodError: 'org.scalajs.ir.Types$PrimRef org.scalajs.ir.Types$PrimRef$.unapply(org.scalajs.ir.Types$PrimRef)'
[error]         at dotty.tools.backend.sjs.JSEncoding$.toIRTypeInternal(JSEncoding.scala:302)
[error]         at dotty.tools.backend.sjs.JSEncoding$.toIRType(JSEncoding.scala:298)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genApplyMethodStatically(JSCodeGen.scala:3555)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genSuperCall(JSCodeGen.scala:2063)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genApply(JSCodeGen.scala:2016)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genStatOrExpr(JSCodeGen.scala:1705)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genStat(JSCodeGen.scala:1552)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genStatOrExpr$$anonfun$5(JSCodeGen.scala:1784)
[error]         at scala.collection.immutable.List.map(List.scala:246)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genStatOrExpr(JSCodeGen.scala:1784)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genStat(JSCodeGen.scala:1552)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genMethodWithCurrentLocalNameScope$$anonfun$1(JSCodeGen.scala:1464)
[error]         at dotty.tools.backend.sjs.JSCodeGen.withPerMethodBodyState$$anonfun$1(JSCodeGen.scala:114)
[error]         at dotty.tools.backend.sjs.ScopedVar$.withScopedVars(ScopedVar.scala:35)
[error]         at dotty.tools.backend.sjs.JSCodeGen.withPerMethodBodyState(JSCodeGen.scala:115)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genMethodWithCurrentLocalNameScope(JSCodeGen.scala:1482)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genMethod$$anonfun$1(JSCodeGen.scala:1383)
[error]         at dotty.tools.backend.sjs.ScopedVar$.withScopedVars(ScopedVar.scala:35)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genMethod(JSCodeGen.scala:1384)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genScalaClass$$anonfun$2(JSCodeGen.scala:370)
[error]         at scala.collection.immutable.List.foreach(List.scala:333)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genScalaClass(JSCodeGen.scala:375)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genCompilationUnit$$anonfun$8$$anonfun$1(JSCodeGen.scala:246)
[error]         at dotty.tools.backend.sjs.ScopedVar$.withScopedVars(ScopedVar.scala:35)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genCompilationUnit$$anonfun$3(JSCodeGen.scala:250)
[error]         at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error]         at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error]         at scala.collection.immutable.List.foreach(List.scala:333)
[error]         at dotty.tools.backend.sjs.JSCodeGen.genCompilationUnit(JSCodeGen.scala:252)
[error]         at dotty.tools.backend.sjs.JSCodeGen.run(JSCodeGen.scala:164)
[error]         at dotty.tools.backend.sjs.GenSJSIR.run(GenSJSIR.scala:15)
[error]         at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:308)
[error]         at scala.collection.immutable.List.map(List.scala:246)
[error]         at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:309)
[error]         at dotty.tools.dotc.Run.runPhases$4$$anonfun$4(Run.scala:261)
[error]         at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error]         at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error]         at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
[error]         at dotty.tools.dotc.Run.runPhases$5(Run.scala:272)
[error]         at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:280)
[error]         at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[error]         at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
[error]         at dotty.tools.dotc.Run.compileUnits(Run.scala:289)
[error]         at dotty.tools.dotc.Run.compileSources(Run.scala:222)
[error]         at mdoc.internal.markdown.MarkdownCompiler.$anonfun$2(MarkdownCompiler.scala:123)
[error]         at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[error]         at scala.util.Try$.apply(Try.scala:210)
[error]         at mdoc.internal.markdown.MarkdownCompiler.compileSources(MarkdownCompiler.scala:123)
[error]         at mdoc.modifiers.JsModifier.postProcess(JsModifier.scala:150)
[error]         at mdoc.modifiers.JsModifier.postProcess(JsModifier.scala:133)
[error]         at mdoc.internal.markdown.Processor.processDocument$$anonfun$4$$anonfun$1(Processor.scala:62)
[error]         at mdoc.internal.markdown.Processor.runModifier(Processor.scala:72)
[error]         at mdoc.internal.markdown.Processor.processDocument$$anonfun$3(Processor.scala:63)
[error]         at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error]         at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error]         at scala.collection.immutable.List.foreach(List.scala:333)
[error]         at mdoc.internal.markdown.Processor.processDocument(Processor.scala:64)
[error]         at mdoc.internal.markdown.Markdown$.toMarkdown(Markdown.scala:132)
[error]         at mdoc.internal.cli.MainOps.handleMarkdown(MainOps.scala:83)
[error]         at mdoc.internal.cli.MainOps.handleFile(MainOps.scala:111)
[error]         at mdoc.internal.cli.MainOps.$anonfun$1(MainOps.scala:157)
[error]         at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:169)
[error]         at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:165)
[error]         at scala.collection.immutable.List.foldLeft(List.scala:79)
[error]         at mdoc.internal.cli.MainOps.generateCompleteSite(MainOps.scala:159)
[error]         at mdoc.internal.cli.MainOps.run(MainOps.scala:178)
[error]         at mdoc.internal.cli.MainOps$.process(MainOps.scala:270)
[error]         at mdoc.Main$.process(Main.scala:26)
[error]         at mdoc.Main$.process(Main.scala:21)
[error]         at mdoc.Main$.main(Main.scala:16)
[error]         at mdoc.Main.main(Main.scala)
[error]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
[error]         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]         at java.base/java.lang.reflect.Method.invoke(Method.java:568)
[error] stack trace is suppressed; run last docs / Compile / bgRunMain for the full output

sjrd · 2022-01-05T07:06:00Z

Wait, does that mean you somehow have the linker and its dependencies on the classpath of the compiler? (The one used to run the compiler) That could cause weird issues. The compiler plugin contains a copy of the ir classes, and they could be incompatible with the ones that the linker is supposed to have, if it's not exactly the same version.

armanbilge · 2022-01-05T07:09:32Z

@sjrd you mean like this?

mdoc/build.sbt

Lines 394 to 411 in 979b8f4

    
           lazy val js = project 
        
             .in(file("mdoc-js")) 
        
             .settings( 
        
               sharedSettings, 
        
               moduleName := "mdoc-js", 
        
               Compile / unmanagedSourceDirectories ++= multiScalaDirectories("js").value, 
        
               libraryDependencies ++= crossSetting( 
        
                 scalaVersion.value, 
        
                 if2 = List( 
        
                   "org.scala-js" % "scalajs-compiler" % scalajs cross CrossVersion.full, 
        
                   "org.scala-js" %% "scalajs-linker" % scalajs 
        
                 ), 
        
                 if3 = List( 
        
                   "org.scala-js" %% "scalajs-linker" % scalajs cross CrossVersion.for3Use2_13 
        
                 ) 
        
               ) 
        
             ) 
        
             .dependsOn(mdoc)

sjrd · 2022-01-05T07:11:14Z

Hum yes, exactly. 😕

armanbilge · 2022-01-05T07:13:52Z

And the mdoc project dependsOned above of course depends on the Scala 3 compiler:

mdoc/build.sbt

Lines 217 to 219 in 979b8f4

    
           if3 = List( 
        
             "org.scala-lang" %% "scala3-compiler" % scalaVersion.value, 
        
             ("org.scalameta" %% "scalameta" % V.scalameta)

armanbilge · 2022-01-05T07:18:58Z

@sjrd does this mean that, for the purposes of mdoc.js, the Scala 3 version and Scala.js version will be tied to each other? Or is there some workaround for this by using multiple classloaders or similar techniques.

sjrd · 2022-01-05T07:20:40Z

Using class loaders would solve the issue. If you load the compiler in one class loader, and the linker in another one, they won't conflict.

keynmol · 2022-02-22T22:21:55Z

Superseded (:crossed_fingers: ) by #629

Update Scala.js to 1.8.0

62fd580

Bump version everywhere

cb3aab3

Different Scala.js for Scala 2 and 3

7ee8081

Revert "Different Scala.js for Scala 2 and 3"

5681080

This reverts commit 7ee8081.

sjrd reviewed Dec 21, 2021

View reviewed changes

fdietze mentioned this pull request Dec 23, 2021

Update sbt-scalajs, scalajs-compiler, ... to 1.8.0 outwatch/outwatch#551

Closed

armanbilge mentioned this pull request Dec 23, 2021

Update sbt-scalajs, scalajs-compiler, ... to 1.8.0 typelevel/cats#4082

Closed

This was referenced Jan 1, 2022

Update sbt-scalajs, scalajs-compiler, ... to 1.8.0 in series/3.x typelevel/cats-effect#2645

Merged

Pin Scala.js to 1.7.x typelevel/sbt-typelevel#40

Merged

Update to Scala.js 1.8.0 outwatch/outwatch#563

Merged

armanbilge mentioned this pull request Jan 5, 2022

mdoc.js is brittle to Scala.js updates #601

Closed

keynmol closed this Feb 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Scala.js to 1.8.0 #597

Update Scala.js to 1.8.0 #597

armanbilge commented Dec 21, 2021

tgodzik commented Dec 21, 2021

armanbilge commented Dec 21, 2021

tgodzik commented Dec 21, 2021

armanbilge commented Dec 21, 2021

tgodzik commented Dec 21, 2021

armanbilge commented Dec 21, 2021

armanbilge commented Dec 21, 2021

armanbilge commented Dec 21, 2021 •

edited

Loading

armanbilge commented Dec 21, 2021

sjrd Dec 21, 2021

armanbilge Dec 21, 2021

sjrd Dec 21, 2021

armanbilge Dec 21, 2021

tgodzik Dec 22, 2021

tgodzik Dec 22, 2021

tgodzik Dec 22, 2021

armanbilge Dec 22, 2021

tgodzik Dec 22, 2021

sjrd commented Dec 21, 2021

armanbilge commented Dec 26, 2021

armanbilge commented Jan 5, 2022

sjrd commented Jan 5, 2022

armanbilge commented Jan 5, 2022

sjrd commented Jan 5, 2022

armanbilge commented Jan 5, 2022

armanbilge commented Jan 5, 2022

sjrd commented Jan 5, 2022

keynmol commented Feb 22, 2022

Update Scala.js to 1.8.0 #597

Update Scala.js to 1.8.0 #597

Conversation

armanbilge commented Dec 21, 2021

tgodzik commented Dec 21, 2021

armanbilge commented Dec 21, 2021

tgodzik commented Dec 21, 2021

armanbilge commented Dec 21, 2021

tgodzik commented Dec 21, 2021

armanbilge commented Dec 21, 2021

armanbilge commented Dec 21, 2021

armanbilge commented Dec 21, 2021 • edited Loading

armanbilge commented Dec 21, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjrd commented Dec 21, 2021

armanbilge commented Dec 26, 2021

armanbilge commented Jan 5, 2022

sjrd commented Jan 5, 2022

armanbilge commented Jan 5, 2022

sjrd commented Jan 5, 2022

armanbilge commented Jan 5, 2022

armanbilge commented Jan 5, 2022

sjrd commented Jan 5, 2022

keynmol commented Feb 22, 2022

armanbilge commented Dec 21, 2021 •

edited

Loading