Generating stardoc target fails in bzlmod on module_extension when using old Bazel 6 and/or legacy extractor

[luispadron]

Im trying to update a repository to support stardoc 0.6.2 so we can consume it via bzlmod, and im running into what seems like a bzlmod extensions issue:

Exception in thread "main" net.starlark.java.eval.Starlark$UncheckedEvalException: InvalidStarlarkValueException thrown during Starlark evaluation
        at <starlark>.module_extension(<builtin>:0)
        at <starlark>.<toplevel>(/private/var/tmp/_bazel_lpadron/7c5e30dd733c097d91026d4bca2f7997/execroot/_main/bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/docs/library_doc_stardoc.runfiles/rules_apple~2.3.0/apple/internal/local_provisioning_profiles.bzl:113)
Caused by: net.starlark.java.eval.Starlark$InvalidStarlarkValueException: invalid Starlark value: class java.lang.Object
        at net.starlark.java.eval.Starlark.fromJava(Starlark.java:214)
        at net.starlark.java.eval.MethodDescriptor.call(MethodDescriptor.java:216)
        at net.starlark.java.eval.BuiltinFunction.fastcall(BuiltinFunction.java:77)
        at net.starlark.java.eval.Starlark.fastcall(Starlark.java:695)
        at net.starlark.java.eval.Eval.evalCall(Eval.java:682)
        at net.starlark.java.eval.Eval.eval(Eval.java:497)
        at net.starlark.java.eval.Eval.execAssignment(Eval.java:109)
        at net.starlark.java.eval.Eval.exec(Eval.java:268)
        at net.starlark.java.eval.Eval.execStatements(Eval.java:82)
        at net.starlark.java.eval.Eval.execFunctionBody(Eval.java:66)
        at net.starlark.java.eval.StarlarkFunction.fastcall(StarlarkFunction.java:174)
        at net.starlark.java.eval.Starlark.fastcall(Starlark.java:695)
        at net.starlark.java.eval.Starlark.execFileProgram(Starlark.java:985)
        at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:476)
        at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:450)
        at com.google.devtools.build.skydoc.SkydocMain.recursiveEval(SkydocMain.java:450)
        at com.google.devtools.build.skydoc.SkydocMain.eval(SkydocMain.java:289)
        at com.google.devtools.build.skydoc.SkydocMain.main(SkydocMain.java:151)

This fails during a load of a file defined in a bzl_library that has a dep on a module extension. Looks similar to #123

[luispadron]

Defining bzl_library:

bzl_library(
    name = "library",
    srcs = ["library.bzl"],
    visibility = ["//visibility:public"],
    deps = [
        ":hmap",
        ":import_middleman",
        ":precompiled_apple_resource_bundle",
        ":utils.bzl",
        "//rules/framework:vfs_overlay",
        "//rules/library:resources",
        "//rules/library:xcconfig",
        "@bazel_skylib//lib:paths",
        "@bazel_skylib//lib:selects",
        "@bazel_skylib//lib:sets",
        "@bazel_skylib//rules:write_file",
        "@build_bazel_rules_apple//apple", # where the error seems to originate from
        "@build_bazel_rules_swift//swift",
    ],
)

Stardoc usage:

    stardoc(
        name = "library_doc",
        out = "library.gen.md",
        input = "//rules:library.bzl",
        tags = [
            "no-cache",
            "no-sandbox",  # https://github.com/bazelbuild/stardoc/issues/112
        ],
        deps = ["//rules:library"],
    )

This depends on this bzl_library defined here.

Snippet of the library.bzl file which imports apple.bzl where error seems to come from:

...

load("@build_bazel_rules_apple//apple:apple.bzl", "apple_dynamic_framework_import", "apple_static_framework_import")
load("@build_bazel_rules_apple//apple/internal/resource_rules:apple_intent_library.bzl", "apple_intent_library")
load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")

...

[fmeum]

@luispadron Which version of Bazel are you using? On recent rolling releases of Bazel 7, stardoc use starlark_doc_extract, which does support module_extension.

[brentleyjones]

This is pre-7. I'm running into the same thing trying to use stardoc 0.6.2 with Bazel 6.3.2.

[luispadron]

Correct, was testing myself with 6.3.2

[luispadron]

A work around for now for us was to remove the load of the apple.bzl file. We now load from the internal file vs. the umbrella file which means the module extension is no longer loaded and so error is "fixed"

[brentleyjones]

For rules_apple, we will probably just have to use a version of Bazel 7 to generate our docs 😕.

[tetromino]

This category of errors will be fixed once bazel 7 stable is out. But in the meantime - it sounds like we should update the bundled jar (from bazel 6.3.2) and make a new Stardoc release. Unfortunately, I'm on baby bonding leave at the moment and not able to spend >5 minutes on a computer, but @brandjon may be able to do it.

[tetromino]

Alas, regenerating the legacy extractor jar (commit cb533e6) did not fix this crash.

So if we want to fix this properly, we'd probably need to explicitly add module_extension and friends to skydoc/fakebuildapi in Bazel source, and then regenerate Stardoc's jar at that commit.

Or alternatively, simply require Bazel 7 (which is now LTS) and the new extractor when generating docs from any .bzl file that uses module_extension...

[tetromino]

Update: we are not planning to fix this; Stardoc 0.7.x will require Bazel 7 or newer, see #212.