Host release candidates in the BCR to address release candence mismatch

[tpudlik]

Description of the feature request:

The FR is for the BCR to allow uploading release candidate versions to the registry, which might be git repositories (rather than HTTP archives).

So really there are two related items:

1. Allow release candidate versions. This is mostly a policy decision. We should double-check that the SemVer syntax for pre-release versions is supported and correctly interpreted (i.e., 2.0.0-alpha is a higher version than 1.0.0 but a lower version than 2.0.0), though the documentation implies this is already the case.
2. Allow release candidate versions to be git repositories rather than HTTP archives.

It's possible that we might want special handling of release candidate versions (e.g., warn the user if version resolution bumps them from a stable version of a transitive dependency to a release candidate; skip release candidates when incrementing versions if the corresponding released version is also in the BCR, i.e. prefer going from 1.0.0 straight to 2.0.0 if it's available, rather than 2.0.0-alpha). To support this, release candidate versions might need special metadata.

@Wyverald

Which category does this issue belong to?

External Dependency

What underlying problem are you trying to solve with this feature?

Pigweed (a Bazel module) depends on the pico-sdk (another Bazel module), but has a higher release cadence. We effectively release Pigweed continuously, while the pico-sdk has historically been released only once every few months.

This leads to issues when a bug affecting Pigweed is discovered in the pico-sdk. Even once the fix is merged into the upstream pico-sdk, we can't rely on the fix in Pigweed until a release is cut. For local development, we can use a git_override, but there's no way for downstream users of Pigweed to inherit this when they specify a version of Pigweed that needs the fix as a bazel_dep. This means we need to go beyond the BCR for managing dependencies (e.g. through maintaining our own registry).

To continue expressing our dependencies exclusively via the BCR, we'd like to be able to say that we depend on the release candidate (even though the release has not been cut yet, and there's no HTTP archive). Note that this is different from patching the pico-sdk in the BCR. We don't want to patch it. We just want to make the BCR versions more granular than the GitHub releases.

Another dependency for which we'd like to do something similar is rules_cc. There's a release candidate for rules_cc (actually a HTTP archive!) that introduces a new API. But it's not officially released yet, and so not in the BCR.

The reality of software development is that different projects have wildly different release cadences, and so sometimes modules need to depend on yet-unreleased versions of other modules. It should be possible to express this through the BCR.

As Bazel is adopted by large, multi-build system projects, being able to decouple BCR versions from official releases in this way will become increasingly important. These projects will not always be releasing frequently enough for the ecosystem to rely exclusively on released versions, and their maintainers may be unwilling to cut releases just to make their (relatively few) Bazel users' dependency management tidier.

Which operating system are you running Bazel on?

Any

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

See https://issues.pigweed.dev/issues/360950550#comment3 and following for further context. Basically, this FR attempts to propose a long-term solution for managing unreleased dependencies that falls short of setting up separate registries.

[fmeum]

I think that if we want to add support for this, we should also consider the "single-module registry" idea that has been floated around before. Assuming that a Git repo contains a version of a Bazel module with other versions available in the BCR, we could conceivably allow deps such as:
bazel_dep(name = "github.com/org/repo", version = "v1.2.3-20191109021931-daa7c04131f5") (inspired by Go's pseudo-version).

Such a dep would not need to count as an override as its version is amenable to relaxed semver comparison (the format needs to be slightly more complex if the base tag itself uses relaxed semver or -).

The advantage is that projects could choose to depend on deps that aren't in the BCR yet at all. Hopefully the more cumbersome dep statement would still encourage users to add released versions to the BCR.

[meteorcloudy]

This is probably already technically possible. You can submit rc modules to BCR with git_repository in source.json, can you try to test if it works by sending a PR like this? Then we can evaluate if we should officially support this as a policy.

[armandomontanez]

Related: I started a discussion about how we should name these versions here: bazelbuild/bazel-central-registry#2888

[armandomontanez]

Seems to break pretty early on since there's no source archive URL to grab out:

bazelbuild/bazel-central-registry#3015

[armandomontanez]

bazelbuild/bazel-central-registry#3580 is the first BCR entry to use a git_repository source.

I originally wanted to use .bcr.20250113215903+4b6e6475 (time down to the second of the commit merge for more granularity) as the suffix but hit two issues:

1. Java crashes with large integers. Had to shorten the timestamp to be YYYYmmdd. I could have split it up into YYYYmmdd.HHMMSS, but for now I'm just going to hope there won't be a need to roll back to an earlier commit from the same day.
2. Everything after the plus (+4b6e6475) was ignored in a way where the expected version source string was *.bcr.20250113. I didn't want to debug this, so I switched to using a dash (-), which has the unfortunate side effect of allowing the commit hash to participate in version resolution.

[meteorcloudy]

I believe + can no longer appear in version due to #23127