Skip to content

Commit

Permalink
Cleanup python example, tweaks docs in response to feedback, fix gith…
Browse files Browse the repository at this point in the history
…ub actions (#3925)

* Make pip/pex example multi-module
* Make bundling PEX actually work
* Clean up english and code in general
* Split up python and typescript examples into separate pages

Other miscellaneous doc improvements and fixes to the github actions
config
  • Loading branch information
lihaoyi authored Nov 9, 2024
1 parent 5e3b6bf commit b345524
Show file tree
Hide file tree
Showing 46 changed files with 616 additions and 638 deletions.
26 changes: 22 additions & 4 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,39 +59,57 @@ jobs:
# We also try to group tests together to manually balance out the runtimes of each jobs
- java-version: 17
millargs: "'{main,scalalib,testrunner,bsp,testkit}.__.testCached'"
install-android-sdk: false

- java-version: '11'
millargs: "'{scalajslib,scalanativelib,kotlinlib}.__.testCached'"
install-android-sdk: false

- java-version: 17
millargs: "contrib.__.testCached"
install-android-sdk: false

- java-version: 17
install-android-sdk: true
millargs: "'example.javalib.__.local.testCached'"
install-android-sdk: true

- java-version: 17
millargs: "'example.scalalib.__.local.testCached'"
install-android-sdk: false

- java-version: 17
install-android-sdk: true
millargs: "'example.kotlinlib.__.local.testCached'"
install-android-sdk: true

- java-version: '11'
millargs: "'example.thirdparty[{mockito,acyclic,commons-io}].local.testCached'"
install-android-sdk: false

- java-version: 17
millargs: "'example.thirdparty[{fansi,jimfs,netty,gatling}].local.testCached'"
install-android-sdk: false

- java-version: '17'
millargs: "'example.thirdparty[arrow].local.testCached'"
install-android-sdk: false

- java-version: '11'
millargs: "'example.{depth,extending}.__.local.testCached'"

install-android-sdk: false
# Most of these integration tests should not depend on which mode they
# are run in, so just run them in `local`
- java-version: '17'
millargs: "'integration.{failure,feature,ide}.__.local.testCached'"

install-android-sdk: false
# These invalidation tests need to be exercised in both execution modes
# to make sure they work with and without -i/--no-server being passed
- java-version: 17
millargs: "'integration.invalidation.__.fork.testCached'"
install-android-sdk: false

- java-version: 17
millargs: "'integration.invalidation.__.server.testCached'"
install-android-sdk: false

uses: ./.github/workflows/run-mill-action.yml
with:
Expand Down
3 changes: 2 additions & 1 deletion docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,8 @@
** xref:extending/running-jvm-code.adoc[]
** xref:extending/writing-plugins.adoc[]
** xref:extending/meta-build.adoc[]
** xref:extending/new-language.adoc[]
** xref:extending/example-typescript-support.adoc[]
** xref:extending/example-python-support.adoc[]
// This section focuses on diving into deeper, more advanced topics for Mill.
// These are things that most Mill developers would not encounter day to day,
// but people developing Mill plugins or working on particularly large or
Expand Down
32 changes: 32 additions & 0 deletions docs/modules/ROOT/pages/extending/example-python-support.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
= Example: Python Support

include::partial$gtag-config.adoc[]

This section demonstrates how to integrate `Python` support into `Mill`.
We will define a simple `PythonModule` trait that can resolve dependencies,
perform type checking on local code, and bundle an executable.

NOTE: This integration is for educational purposes only, showcasing common technique
used in building language toolchains, and is not intended for production use.

== Basic Python Build Pipeline

include::partial$example/extending/python/1-hello-python.adoc[]

== Re-usable PythonModule

include::partial$example/extending/python/2-python-modules.adoc[]

== PythonModule `moduleDeps`

include::partial$example/extending/python/3-python-module-deps.adoc[]

== PIP dependencies and bundling

include::partial$example/extending/python/4-python-libs-bundle.adoc[]



As mentioned, The `PythonModule` examples here demonstrate
how to add support for a new language toolchain in Mill.
A production-ready version would require more work to enhance features and performance.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
= Support for New Languages
= Example: Typescript Support

include::partial$gtag-config.adoc[]

Expand All @@ -11,21 +11,21 @@ The TypeScript integration here is not intended for production usage, but is
instead meant for illustration purposes of the techniques typically used in
implementing language toolchains.

== Basic TypeScript Pipeline
== Basic TypeScript Build Pipeline

include::partial$example/extending/newlang/1-hello-typescript.adoc[]
include::partial$example/extending/typescript/1-hello-typescript.adoc[]

== Re-usable TypeScriptModule

include::partial$example/extending/newlang/2-typescript-modules.adoc[]
include::partial$example/extending/typescript/2-typescript-modules.adoc[]

== TypeScriptModule `moduleDeps`

include::partial$example/extending/newlang/3-module-deps.adoc[]
include::partial$example/extending/typescript/3-module-deps.adoc[]

== NPM dependencies and bundling

include::partial$example/extending/newlang/4-npm-deps-bundle.adoc[]
include::partial$example/extending/typescript/4-npm-deps-bundle.adoc[]



Expand All @@ -39,25 +39,6 @@ to any language you need: whether it's TypeScript or some other language, most p
language toolchains have similar concepts of `compile`, `run`, `bundle`, etc.


== Python Support

This section demonstrates how to integrate `Python` support into `Mill`.
We will define a simple `PythonModule` trait that can resolve dependencies,
perform type checking on local code, and optimize the final bundle.

`Note`: This integration is for `educational purposes only`, showcasing common technique
used in building language toolchains, and is not intended for production use.

include::partial$example/extending/newlang/5-hello-python.adoc[]

include::partial$example/extending/newlang/6-python-modules.adoc[]

include::partial$example/extending/newlang/7-python-module-deps.adoc[]

include::partial$example/extending/newlang/8-python-libs-bundle.adoc[]



As mentioned, The `PythonModule` examples here demonstrate
As mentioned, The `PythonModule` examples here demonstrate
how to add support for a new language toolchain in Mill.
A production-ready version would require more work to enhance features and performance.
39 changes: 6 additions & 33 deletions docs/modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,33 +2,6 @@

include::partial$gtag-config.adoc[]

```graphviz
digraph G {
rankdir=LR
node [shape=box width=0 height=0 style=filled fillcolor=white]
bgcolor=transparent
newrank=true;
subgraph cluster_0 {
style=dashed
node [shape=box width=0 height=0 style=filled fillcolor=white]
label = "foo";

"foo.sources" -> "foo.compile" -> "foo.classPath" -> "foo.assembly"
"foo.resources" -> "foo.assembly"
"foo.classPath"
}
subgraph cluster_1 {
style=dashed
node [shape=box width=0 height=0 style=filled fillcolor=white]
label = "bar";

"foo.classPath" -> "bar.compile" [constraint=false]
"foo.classPath" -> "bar.classPath"
"bar.sources" -> "bar.compile" -> "bar.classPath" -> "bar.assembly"
"bar.resources" -> "bar.assembly"
}
}
```

Mill is a fast, scalable, multi-language build tool that supports Java, Scala,
and Kotlin:
Expand All @@ -39,8 +12,8 @@ or xref:comparisons/gradle.adoc[2-4x faster than Gradle]
* Mill's typed config language and immutable xref:depth/design-principles.adoc[task graph]
helps keep builds clean and understandable
* Mill scales well from small single-module projects
to xref:depth/large-builds.adoc[large monorepos] with hundreds of modules
* Mill is an alternative to https://bazel.build/[Bazel]
for xref:depth/large-builds.adoc[large monorepos] with hundreds of modules in multiple languages
Although the Java compiler is very fast and the Java language is easy to learn,
JVM build tools have a reputation for being sluggish and confusing. Mill tries to
Expand All @@ -51,10 +24,10 @@ Java platform's performance and usability:
xref:depth/evaluation-model.adoc#_caching_at_each_layer_of_the_evaluation_model[caches]
and xref:cli/flags.adoc#_jobs_j[parallelizes] build
tasks, keeping your workflows fast and responsive. Mill adds minimal overhead over
the logic necessary to build your project, while providing tools to let you identify
and resolve bottlenecks in your build
the logic necessary to build your project, and avoids the long configuration or initialization
times often necessary in other build tools like Gradle or SBT
* *Maintainability*: Mill goes beyond YAML and Bash, with config and custom logic written in
* *Maintainability*: Mill's config and custom logic is written in
xref:javalib/intro.adoc#_custom_build_logic[concise type-checked code],
and an immutable xref:depth/design-principles.adoc[module tree and task graph]. This
catches config issues early, and helps IDEs
Expand All @@ -64,7 +37,7 @@ understand your Mill build better than any other build system
* *Flexibility*: Mill's tasks and modules allow anything from adding
xref:fundamentals/tasks.adoc#primitive-tasks[simple build steps], up to
entire xref:extending/new-language.adoc[language toolchains].
entire xref:extending/example-python-support.adoc[language toolchains].
You can xref:extending/import-ivy-plugins.adoc[import any JVM library] in your build,
use Mill's rich ecosystem of xref:extending/thirdparty-plugins.adoc[Third-Party Mill Plugins],
or xref:extending/writing-plugins.adoc[write plugins] yourself and
Expand Down
9 changes: 4 additions & 5 deletions docs/modules/ROOT/pages/kotlinlib/intro.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
// what Mill is, hopefully have downloaded an example to try out, and be
// interested in learning more about the Mill build tool

= (Experimental) Kotlin with Mill
= Kotlin with Mill
:page-aliases: Kotlin_Intro_to_Mill.adoc

include::partial$gtag-config.adoc[]
Expand All @@ -22,10 +22,9 @@ include::partial$gtag-config.adoc[]

include::partial$Intro_Header.adoc[]

NOTE: *Kotlin support in Mill is experimental*! A lot of stuff works, which is
documented under this section, but a lot of stuff doesn't. In particular,
support for Android, KotlinJS and Kotlin-Multi-Platform is still in its infancy.
The API is not yet stable and may evolve. Try it out but please be aware of its
NOTE: Mill Kotlin support is currently focused on the JVM, with a lot of APIs
still under active development. It is expected to continue evolving over time
as Android and Multiplatform support is fleshed out. Try it out, but please be aware of its
limitations!

== Simple Kotlin Module
Expand Down
9 changes: 7 additions & 2 deletions docs/modules/ROOT/pages/why-mill.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ can trivially load it into their Chrome browser to analyze and figure out what.

Apart from the Mill Chrome Profile, Mill also provides the `./mill visualize` command, which
is useful to show the logical dependency graph between tasks. For example, we can use
`./mill visualize __.compile` (double `__` means wildcard) to
`./mill visualize __.compile` (double underscore means wildcard) to
show the dependency graph between the modules of the Netty build below:

image::comparisons/NettyCompileGraph.svg[]
Expand Down Expand Up @@ -319,7 +319,7 @@ Unlike most other build tools, Mill builds can be explored interactively in your
IDE. If you do not know what something does, it's documentation, definition, or usages is always
one click away in IntelliJ or VSCode. This isn't a new experience for Java developers, as it
is what you would be used to day-to-day in your application code! But Mill brings that same
polished experience to your build system - traditionally some that that has been opaque
polished experience to your build system - traditionally something that has been opaque
and hard to understand - and does so in a way that no other build tool does.


Expand Down Expand Up @@ -385,6 +385,11 @@ show its value via the Mill command line to force it to evaluate:
17
```

Note that as `lineCount` is a `Task`, we get automatic caching, invalidation, and
parallelization: these are things that every `Task` gets for free, without the task
author to do anything. And although we wrote the `lineCount` logic in the main
`build.mill` file for this example, if it grows complex enough to get messy it is
easy to move it to your own xref:extending/writing-plugins.adoc[custom plugins]

### Overriding Tasks

Expand Down
2 changes: 1 addition & 1 deletion docs/supplemental-ui/partials/nav-tree.hbs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ show all of them all the time
<ul class="nav-list">
{{#each navigation}}

<li class="nav-item{{#if (eq ./url @root.page.url)}} is-current-page{{/if}}{{#if (and (not ../../level) (gt @index 3))}} is-active{{/if}}" data-depth="{{or ../level 0}}">
<li class="nav-item{{#if (eq ./url @root.page.url)}} is-current-page{{/if}}{{#if (and (not ../../level) (gt @index 2))}} is-active{{/if}}" data-depth="{{or ../level 0}}">
{{#if ./content}}
{{#if ./items.length}}
<button class="nav-item-toggle"></button>
Expand Down
Loading

0 comments on commit b345524

Please sign in to comment.