Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scala Native #3057

Merged
merged 78 commits into from
Sep 21, 2022
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
47bf28d
kernel.native
armanbilge Jun 18, 2022
5096ca3
kernel-testkit.native
armanbilge Jun 18, 2022
38c3a25
laws.native
armanbilge Jun 18, 2022
6f509f2
std.native
armanbilge Jun 18, 2022
d1afc95
core.native compiles
armanbilge Jun 18, 2022
689dcc7
example.native
armanbilge Jun 18, 2022
f588289
tests.native compiling
armanbilge Jun 18, 2022
ae44a3b
Add missing javalib
armanbilge Jun 18, 2022
c36ad50
Fix FiberMonitor
armanbilge Jun 18, 2022
9040aa7
Fix file name
armanbilge Jun 18, 2022
2336b7e
Use busy-wait scheduler for tests
armanbilge Jun 18, 2022
557da3c
fix busy wait scheduler
armanbilge Jun 18, 2022
8fe1977
All tests pass
armanbilge Jun 18, 2022
2b96471
Setup snapshotting
armanbilge Jun 18, 2022
50ff72d
Weird scaladoc issue on native
armanbilge Jun 18, 2022
904a40b
Fix Scala 3 compile
armanbilge Jun 18, 2022
2db9428
Merge branch 'typelevel:series/3.x' into experiment/native
armanbilge Jun 27, 2022
13999c2
Updates
armanbilge Jun 27, 2022
e063f5c
Try setting up CI
armanbilge Jun 27, 2022
a620bef
scalafmt+scalafix
armanbilge Jun 27, 2022
280a314
Add ciNative alias
armanbilge Jun 27, 2022
29df46b
Explicit types for implicits
armanbilge Jun 27, 2022
d0cd453
De-boot-leg
armanbilge Jun 27, 2022
2e2ae42
Fix MiMa
armanbilge Jun 27, 2022
1ff75b5
Fix workflow
armanbilge Jun 27, 2022
c1c5b17
Implement `SecureRandom` for Native
armanbilge Jun 28, 2022
bd6b6ea
Hopefully docs work now ...
armanbilge Jun 28, 2022
2ccf82e
Tweak Native `IOApp` exiting
armanbilge Jun 28, 2022
468d92b
Use `DetectPlatform` as needed
armanbilge Jun 28, 2022
162a3bf
Scala Native supports `os.version`
armanbilge Jun 28, 2022
57257f4
Just use global IORuntime for tests
armanbilge Jun 28, 2022
ccdb3e1
Fix compile
armanbilge Jun 28, 2022
ce7ce1d
Extract `PollingExecutorScheduler` abstraction
armanbilge Jun 28, 2022
3c1e3b2
Fix Native CI alias
armanbilge Jun 28, 2022
d113169
Privatize and finalize stuff
armanbilge Jun 28, 2022
259f2f1
Simplifying
armanbilge Jun 28, 2022
370d507
Use `j.u.ArrayDeque` instead of `ListBuffer`
armanbilge Jun 28, 2022
1815732
Use `j.u.PriorityQueue`
armanbilge Jun 28, 2022
495cfe2
First pass at better event loop
armanbilge Jun 28, 2022
76f0a6d
Fix stupid mistake
armanbilge Jun 28, 2022
7d4e846
Just directly execute 0-sleep tasks
armanbilge Jun 28, 2022
d710e54
Save some allocations
armanbilge Jun 28, 2022
439cb9d
Fix use of `cachedNow` for `sleep`ing
armanbilge Jun 28, 2022
affcada
Less contention on Native
armanbilge Jun 28, 2022
dfca087
Merge remote-tracking branch 'upstream/series/3.x' into experiment/na…
armanbilge Jun 28, 2022
49ea8ce
Re-order steps in event loop
armanbilge Jun 29, 2022
0ac7d09
Merge branch 'fix/syncstep-uncancelable-3.x' into experiment/native
armanbilge Jun 29, 2022
bb6a2f0
Merge branch 'fix/syncstep-uncancelable-3.x' into experiment/native
armanbilge Jun 29, 2022
d9d1589
Formatting
armanbilge Jun 29, 2022
f28b163
Merge branch 'fix/syncstep-uncancelable-3.x' into experiment/native
armanbilge Jun 29, 2022
5111712
Add support for microsecond time
armanbilge Jun 30, 2022
e624931
Fix binding
armanbilge Jun 30, 2022
05e6a4c
Add fallback for Windows
armanbilge Jun 30, 2022
7efbac0
Make `installGlobal` public
armanbilge Aug 24, 2022
693d193
Add explanation doc b/c, well, even I forgot (:
armanbilge Aug 25, 2022
eb55dd2
poll as the last step in the loop
armanbilge Aug 25, 2022
519e5ce
Cleanup canceled timers before polling
armanbilge Aug 26, 2022
d093f95
Use secure UUIDGen for Native
armanbilge Sep 1, 2022
54f2b02
Bump Scala Native version
armanbilge Sep 1, 2022
bbb5d0d
Surround blocking calls with cede
armanbilge Sep 4, 2022
8c8ae32
Update `nowMicros` with SN 0.4.7 APIs
armanbilge Sep 5, 2022
989ad9c
More cleanup for `nowMicros`
armanbilge Sep 6, 2022
2b3bc18
`guarantee(cede)` after `blocking` `thunk`
armanbilge Sep 6, 2022
ff197d1
Add failing test for `SecureRandom#nextInt`
armanbilge Sep 9, 2022
6ab05af
Correctly override `SecureRandom#next`
armanbilge Sep 9, 2022
82b2665
Gracefully handle negative sleeps
armanbilge Sep 11, 2022
6075d85
Merge remote-tracking branch 'upstream/series/3.x' into experiment/na…
armanbilge Sep 13, 2022
7d9b802
Add `pollEvery` parameter for fairness, default 64
armanbilge Sep 13, 2022
ea87873
`try`/`catch` like the WSTP
armanbilge Sep 13, 2022
9e27023
`ScheduledTask` -> `SleepTask`
armanbilge Sep 13, 2022
39cdd77
Refactor `sleep` cancelation
armanbilge Sep 13, 2022
1221506
Rip out `cachedNow` stuff
armanbilge Sep 14, 2022
a14be23
Revert "Make `installGlobal` public"
armanbilge Sep 14, 2022
0f3ea9c
Fix `suspend` impl
armanbilge Sep 14, 2022
e442f8f
Merge remote-tracking branch 'upstream/series/3.x' into experiment/na…
armanbilge Sep 19, 2022
989e648
Add native+macOS job
armanbilge Sep 19, 2022
dd56b2a
Add implementation note/limits for `SecureRandom`
armanbilge Sep 21, 2022
2a3c545
Document limitations for `IOApp` on Native
armanbilge Sep 21, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
os: [ubuntu-latest, windows-latest, macos-latest]
scala: [3.1.2, 2.12.16, 2.13.8]
java: [temurin@8, temurin@11, temurin@17, graalvm@11]
ci: [ciJVM, ciJS, ciFirefox, ciChrome]
ci: [ciJVM, ciNative, ciJS, ciFirefox, ciChrome]
exclude:
- scala: 3.1.2
java: temurin@11
Expand Down Expand Up @@ -53,6 +53,16 @@ jobs:
scala: 2.12.16
- ci: ciChrome
scala: 2.12.16
- ci: ciNative
java: temurin@11
- ci: ciNative
java: temurin@17
- ci: ciNative
java: graalvm@11
- os: windows-latest
ci: ciNative
- os: macos-latest
ci: ciNative
- ci: ciJS
java: temurin@11
- ci: ciJS
Expand Down Expand Up @@ -210,6 +220,11 @@ jobs:
shell: bash
run: example/test-js.sh ${{ matrix.scala }}

- name: Test Example Native App Using Binary
if: matrix.ci == 'ciNative' && matrix.os == 'ubuntu-latest'
shell: bash
run: example/test-native.sh ${{ matrix.scala }}

- name: Scalafix tests
if: matrix.scala == '2.13.8' && matrix.ci == 'ciJVM' && matrix.os == 'ubuntu-latest'
shell: bash
Expand Down
55 changes: 42 additions & 13 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,11 @@ ThisBuild / githubWorkflowBuild := Seq(
name = Some("Test Example JavaScript App Using Node"),
cond = Some(s"matrix.ci == 'ciJS' && matrix.os == '$PrimaryOS'")
),
WorkflowStep.Run(
List("example/test-native.sh ${{ matrix.scala }}"),
name = Some("Test Example Native App Using Binary"),
cond = Some(s"matrix.ci == 'ciNative' && matrix.os == '$PrimaryOS'")
),
WorkflowStep.Run(
List("cd scalafix", "sbt test"),
name = Some("Scalafix tests"),
Expand Down Expand Up @@ -198,7 +203,7 @@ ThisBuild / githubWorkflowBuildMatrixExclusions := {
ci <- jsCiVariants.tail
} yield MatrixExclude(Map("ci" -> ci, "scala" -> scala))

val jsJavaAndOSFilters = jsCiVariants.flatMap { ci =>
val nativeJsJavaAndOSFilters = (CI.Native.command :: jsCiVariants).flatMap { ci =>
val javaFilters =
(ThisBuild / githubWorkflowJavaVersions).value.filterNot(Set(ScalaJSJava)).map { java =>
MatrixExclude(Map("ci" -> ci, "java" -> java.render))
Expand All @@ -214,7 +219,7 @@ ThisBuild / githubWorkflowBuildMatrixExclusions := {
MatrixExclude(Map("os" -> Windows, "java" -> GraalVM.render))
)

scalaJavaFilters ++ windowsAndMacScalaFilters ++ jsScalaFilters ++ jsJavaAndOSFilters ++ flakyFilters
scalaJavaFilters ++ windowsAndMacScalaFilters ++ jsScalaFilters ++ nativeJsJavaAndOSFilters ++ flakyFilters
}

lazy val useJSEnv =
Expand Down Expand Up @@ -257,14 +262,15 @@ val CatsVersion = "2.8.0"
val Specs2Version = "4.16.0"
val ScalaCheckVersion = "1.16.0"
val DisciplineVersion = "1.4.0"
val CoopVersion = "1.1.1"
val CoopVersion = "1.2.0"

val MacrotaskExecutorVersion = "1.0.0"

tlReplaceCommandAlias("ci", CI.AllCIs.map(_.toString).mkString)
addCommandAlias("release", "tlRelease")

addCommandAlias(CI.JVM.command, CI.JVM.toString)
addCommandAlias(CI.Native.command, CI.Native.toString)
addCommandAlias(CI.JS.command, CI.JS.toString)
addCommandAlias(CI.Firefox.command, CI.Firefox.toString)
addCommandAlias(CI.Chrome.command, CI.Chrome.toString)
Expand All @@ -276,12 +282,27 @@ addCommandAlias(
val jsProjects: Seq[ProjectReference] =
Seq(kernel.js, kernelTestkit.js, laws.js, core.js, testkit.js, testsJS, std.js, example.js)

val nativeProjects: Seq[ProjectReference] =
Seq(
kernel.native,
kernelTestkit.native,
laws.native,
core.native,
testkit.native,
tests.native,
std.native,
example.native)

val undocumentedRefs =
jsProjects ++ Seq[ProjectReference](benchmarks, example.jvm, tests.jvm, tests.js)
jsProjects ++ nativeProjects ++ Seq[ProjectReference](
benchmarks,
example.jvm,
tests.jvm,
tests.js)

lazy val root = project
.in(file("."))
.aggregate(rootJVM, rootJS)
.aggregate(rootJVM, rootJS, rootNative)
.enablePlugins(NoPublishPlugin)
.enablePlugins(ScalaUnidocPlugin)
.settings(
Expand All @@ -306,11 +327,13 @@ lazy val rootJVM = project

lazy val rootJS = project.aggregate(jsProjects: _*).enablePlugins(NoPublishPlugin)

lazy val rootNative = project.aggregate(nativeProjects: _*).enablePlugins(NoPublishPlugin)

/**
* The core abstractions and syntax. This is the most general definition of Cats Effect, without
* any concrete implementations. This is the "batteries not included" dependency.
*/
lazy val kernel = crossProject(JSPlatform, JVMPlatform)
lazy val kernel = crossProject(JSPlatform, JVMPlatform, NativePlatform)
.in(file("kernel"))
.settings(
name := "cats-effect-kernel",
Expand All @@ -322,12 +345,15 @@ lazy val kernel = crossProject(JSPlatform, JVMPlatform)
.jsSettings(
libraryDependencies += "org.scala-js" %%% "scala-js-macrotask-executor" % MacrotaskExecutorVersion % Test
)
.nativeSettings(
libraryDependencies += "io.github.cquiroz" %%% "scala-java-time" % "2.4.0"
)
Comment on lines +366 to +368
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think Scala Native has the same artifact size concerns as Scala.js, so I reckon we should ship java.time support out-of-the-box.


/**
* Reference implementations (including a pure ConcurrentBracket), generic ScalaCheck
* generators, and useful tools for testing code written against Cats Effect.
*/
lazy val kernelTestkit = crossProject(JSPlatform, JVMPlatform)
lazy val kernelTestkit = crossProject(JSPlatform, JVMPlatform, NativePlatform)
.in(file("kernel-testkit"))
.dependsOn(kernel)
.settings(
Expand Down Expand Up @@ -357,7 +383,7 @@ lazy val kernelTestkit = crossProject(JSPlatform, JVMPlatform)
* dependency issues. As a consequence of this split, some things which are defined in
* kernelTestkit are *tested* in the Test scope of this project.
*/
lazy val laws = crossProject(JSPlatform, JVMPlatform)
lazy val laws = crossProject(JSPlatform, JVMPlatform, NativePlatform)
.in(file("laws"))
.dependsOn(kernel, kernelTestkit % Test)
.settings(
Expand All @@ -372,7 +398,7 @@ lazy val laws = crossProject(JSPlatform, JVMPlatform)
* contains some general datatypes built on top of IO which are useful in their own right, as
* well as some utilities (such as IOApp). This is the "batteries included" dependency.
*/
lazy val core = crossProject(JSPlatform, JVMPlatform)
lazy val core = crossProject(JSPlatform, JVMPlatform, NativePlatform)
.in(file("core"))
.dependsOn(kernel, std)
.settings(
Expand Down Expand Up @@ -706,7 +732,7 @@ lazy val core = crossProject(JSPlatform, JVMPlatform)
* Test support for the core project, providing various helpful instances like ScalaCheck
* generators for IO and SyncIO.
*/
lazy val testkit = crossProject(JSPlatform, JVMPlatform)
lazy val testkit = crossProject(JSPlatform, JVMPlatform, NativePlatform)
.in(file("testkit"))
.dependsOn(core, kernelTestkit)
.settings(
Expand All @@ -720,7 +746,7 @@ lazy val testkit = crossProject(JSPlatform, JVMPlatform)
/**
* Unit tests for the core project, utilizing the support provided by testkit.
*/
lazy val tests: CrossProject = crossProject(JSPlatform, JVMPlatform)
lazy val tests: CrossProject = crossProject(JSPlatform, JVMPlatform, NativePlatform)
.in(file("tests"))
.dependsOn(core, laws % Test, kernelTestkit % Test, testkit % Test)
.enablePlugins(BuildInfoPlugin, NoPublishPlugin)
Expand Down Expand Up @@ -768,7 +794,7 @@ lazy val testsJVM = tests
* implementations will require IO, and thus those tests will be located within the core
* project.
*/
lazy val std = crossProject(JSPlatform, JVMPlatform)
lazy val std = crossProject(JSPlatform, JVMPlatform, NativePlatform)
.in(file("std"))
.dependsOn(kernel)
.settings(
Expand Down Expand Up @@ -816,12 +842,15 @@ lazy val std = crossProject(JSPlatform, JVMPlatform)
ProblemFilters.exclude[MissingClassProblem]("cats.effect.std.JavaSecureRandom$")
)
)
.nativeSettings(
Compile / doc / sources := Seq.empty
)

/**
* A trivial pair of trivial example apps primarily used to show that IOApp works as a practical
* runtime on both target platforms.
*/
lazy val example = crossProject(JSPlatform, JVMPlatform)
lazy val example = crossProject(JSPlatform, JVMPlatform, NativePlatform)
.in(file("example"))
.dependsOn(core)
.enablePlugins(NoPublishPlugin)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,12 @@ private[effect] sealed abstract class WorkStealingThreadPool private ()
def reportFailure(cause: Throwable): Unit
private[effect] def reschedule(runnable: Runnable): Unit
private[effect] def canExecuteBlockingCode(): Boolean
private[unsafe] def liveFibers()
: (Set[Runnable], Map[WorkerThread, (Option[Runnable], Set[Runnable])], Set[Runnable])
}

private[unsafe] sealed abstract class WorkerThread private () extends Thread {
private[unsafe] def isOwnedBy(threadPool: WorkStealingThreadPool): Boolean
private[unsafe] def monitor(fiber: Runnable): WeakBag.Handle
private[unsafe] def index: Int
}
Original file line number Diff line number Diff line change
Expand Up @@ -218,14 +218,14 @@ private[effect] object FiberMonitor {
}
}

private[FiberMonitor] final val BagReferences
: ConcurrentLinkedQueue[WeakReference[WeakBag[Runnable]]] =
new ConcurrentLinkedQueue()

private[FiberMonitor] final val Bags: ThreadLocal[WeakBag[Runnable]] =
ThreadLocal.withInitial { () =>
val bag = new WeakBag[Runnable]()
BagReferences.offer(new WeakReference(bag))
bag
}

private[FiberMonitor] final val BagReferences
: ConcurrentLinkedQueue[WeakReference[WeakBag[Runnable]]] =
new ConcurrentLinkedQueue()
}
Loading