Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider a WORKSPACE alias rule #3219

Closed
cgrushko opened this issue Jun 20, 2017 · 26 comments
Closed

Consider a WORKSPACE alias rule #3219

cgrushko opened this issue Jun 20, 2017 · 26 comments
Labels
P3 We're not considering working on this, but happy to review a PR. (No assignee) type: feature request

Comments

@cgrushko
Copy link
Contributor

Something like:

WORKSPACE:

http_archive(
    name = "com_google_protobuf",
    ...
)

alias(
    name = "com_google_protobuf_cc",
    actual = "com_google_protobuf",
)

Depending on com_google_protobuf_cc would be equivalent to com_google_protobuf.

The use-case is highlighted in
https://groups.google.com/forum/#!topic/bazel-discuss/859ybHQZnuI

@laszlocsomor
Copy link
Contributor

How about bind()?

@cgrushko
Copy link
Contributor Author

bind() doesn't do what I describe above.

@philwo philwo added P3 We're not considering working on this, but happy to review a PR. (No assignee) type: feature request labels Jun 21, 2017
@kchodorow
Copy link
Contributor

--override_repository=com_google_protobuf_cc=/path/to/com_google_protobuf can technically do this.

I still disagree with your premise. A repository should be ~a project. Also, if you're aliasing these, all of these repositories need to have the exact same targets, which is awkward.

IIUC, the concern is that @com_google_protobuf_java has one implementation and @com_google_protobuf_javalite has another and some projects will want to use them interchangably? That seems like it can be handled at several other levels.

@cgrushko
Copy link
Contributor Author

will want to use them interchangably?

They need to be used together in the same code base (e.g., Java code that's reused between server and Android).

@kchodorow
Copy link
Contributor

That sounds like it should be handled at the configuration layer.

@jayconrod
Copy link
Contributor

Without aliases, what is the the best practice for renaming a workspace? I would expect aliases to be the main tool for this, and I don't see how renaming would work without them.

Suppose I have a project that uses @io_bazel_rules_go, but at some point, that repository changes its name to @build_bazel_rules_go. I may have some dependencies that refer to it using the new name and some using the old name.

@gertvdijk
Copy link
Contributor

Without aliases, what is the the best practice for renaming a workspace?

I have the same question really. This stops me from making a transition when renaming WORKSPACE rule names in dependency (or consumer) projects. Would be great leave some aliases to cover a certain transition period.

Bumped into this when trying to build a JGit development tree in Gerrit. One of the two decided it was a good idea to rename all the rules to a new format. (mailing list discussion)

@dslomov
Copy link
Contributor

dslomov commented Mar 22, 2019

This is implemented with repository remapping

@dslomov dslomov closed this as completed Mar 22, 2019
@Globegitter
Copy link

@dslomov How is that solved with repository remapping? The link provided only shows an example with local_repository. How can I apply a remapping for the current workspace, i.e. myproject in the example?

@Globegitter
Copy link

Also there is still missing documentation of this new field here: https://docs.bazel.build/versions/master/be/workspace.html#local_repository

@nicolov
Copy link
Contributor

nicolov commented May 2, 2019

Repository remapping does remap labels in the BUILD files, but won't do anything to labels in .bzl files. This makes the feature useless when project have their own macros in .bzl files. As a reference, I've tried to make TensorFlow use @com_google_protobuf instead of @protobuf_archive to avoid double compilations.

@dslomov
Copy link
Contributor

dslomov commented May 2, 2019

Repository remapping does remap labels in the BUILD files, but won't do anything to labels in .bzl files.

It should also remap labels in .bzl files as well - do you have a repro where it does not work?

This makes the feature useless when project have their own macros in .bzl files. As a reference, I've tried to make TensorFlow use @com_google_protobuf instead of @protobuf_archive to avoid double compilations.

@dslomov
Copy link
Contributor

dslomov commented May 2, 2019

@dslomov How is that solved with repository remapping? The link provided only shows an example with local_repository. How can I apply a remapping for the current workspace, i.e. myproject in the example?

Every repository rule has a repo_mapping argument, not just local_repository.
What do you mean by "apply a remapping for the current workspace, i.e. myproject in the example?".

If a repository you depend on wants to refer to @some_repo but you want that to mean the main project, you can do the following:

workspace(name = "myproject")

http_archive(
       name = "xyz", ....,
       repo_mapping = { "@some_repo" : "@myproject" }
)

@nicolov
Copy link
Contributor

nicolov commented May 2, 2019

It should also remap labels in .bzl files as well - do you have a repro where it does not work?

I have remapped @protobuf_archive to @com_google_protobuf when adding tensorflow to my project's workspace. This worked, but then my build was crashing when trying to use a tensorflow skylark rule that still tried to add @protobuf_archive as a dep/

@Globegitter
Copy link

Globegitter commented May 8, 2019

@dslomov What I mean is, I want to be able to alias the external repository itself in my current workspace. So In the example you gave, I suppose I would want something like:

workspace(
    name = "myproject",
    repo_mapping  = { "@xyz": "@abc" },
)

http_archive(
       name = "xyz", ....,
       repo_mapping = { "@some_repo" : "@myproject" }
)

or as the issue creator suggested:

workspace(name = "myproject")

http_archive(
       name = "xyz", ....,
       repo_mapping = { "@some_repo" : "@myproject" }
)

alias(
    name = "abc",
    actual = "xyz",
)

As far as I can see from the documentation that is currently not possible, or am I wrong here?

My concrete use-case is we use yarn_install with an internal workspace name, but the name npm seems to become the standard for this workspace, so to make our internal migration easier I would like to be able to have two names for the same external repository that we can reference in our own BUILD files.

Hope that clarifies the problem.

@dslomov
Copy link
Contributor

dslomov commented May 8, 2019

Hmm, why don't you just call the xyz repository abc?

@dslomov
Copy link
Contributor

dslomov commented May 8, 2019

Ah I see, sorry... What forces you to use npm as a name?

@Globegitter
Copy link

Globegitter commented May 9, 2019

rules_nodejs started to use @npm as its default name for its dependencies. E.g. see: https://github.com/bazelbuild/rules_nodejs/blob/master/packages/jasmine/src/jasmine_node_test.bzl#L31 So it is just that we can be compatible with the default.

Also I have started to write a plugin for gazelle to auto generate BUILD files, and while it is probably possible to add an option to provide a custom name, I would like to just assume npm for now for simplicity.

Renaming things permanently is a bigger task due to internal reasons, even if it is possible in an automated manner. So being able to alias the name in our current workspace would be preferable. It also would make it simpler to change workspace names of some of our own private external dependencies with a transition period.

@Globegitter
Copy link

@dslomov any more thoughts on this?

@Globegitter
Copy link

@dslomov Another use-case that just came up when implementing toolchaings for rules_nodejs is that per default we provide a nodejs repository for each supported platform with the name nodejs_<os>_<arch> but for usability and backwards-compatibility reasons it would be nice to provide an alias of the nodejs_<host_os>_<host_arch> to just nodejs as users do interact occasionally directly with tool provided in that repository, e.g. the package manager but also sometimes even node directly to get access to the repl etc. In this case though we would want to generate the alias and provide a .bzl file with it.

@kayasoze
Copy link

I suggest this bug be reopened, as the use-case identified by @Globegitter, which I share the need for, remains unsolved.

@dslomov
Copy link
Contributor

dslomov commented Aug 20, 2019

If the usage is in your own project, why can't you just rename the references with a simple search and replace? It is more maintainable and less confusing than aliases.
If your other dependencies expect different names, use repo_mapping to remap.

@ccate
Copy link

ccate commented Aug 29, 2019

mature projects (ie the family of protobuf projects) have over time changed the names of expected repos, as well as their strategies for fetching dependencies. Working with older versions of protobuf is a nightmare. I believe the request for aliasing repo's is valid.

@chrXiTer
Copy link

https://github.com/bazelbuild/rules_python

2019-07-26: The canonical name of this repo has been changed from @io_bazel_rules_python to just @rules_python, in accordance with convention. Please update your WORKSPACE file and labels that reference this repo accordingly.

https://github.com/bazelbuild/rules_python/blob/master/python/pip.bzl

if "rules_python" not in native.existing_rules():
message = "=" * 79 + """\n
It appears that you are trying to import rules_python without using its
canonical name, "@rules_python". This does not work. Please change your
WORKSPACE file to import this repo with name = "rules_python" instead.
"""
if "io_bazel_rules_python" in native.existing_rules():
message += """\n
Note that the previous name of "@io_bazel_rules_python" is no longer used.
See bazelbuild/rules_python#203 for context.
"""
message += "=" * 79
fail(message)

MD TNND

Life is so hard

@TheButlah
Copy link

This looks like an important and relevant feature - why is this issue still closed?

@TheButlah
Copy link

TheButlah commented May 15, 2020

Followup: I agree with @dslomov's point that aliasing repos are not necessary within the main workspace, as you should in theory have full control over that code and be able to find and replace to a name of your choice. Repo mapping fits the rest of the use cases - allowing a translation between the main workspace and the external workspaces names.

However, I'm having the exact same issue as @nicolov. In my case I refer to the flatbuffers repo as @flatbuffers instead of @com_github_google_flatbuffers. In order to compile a flatbuffers schema, I load a macro from the flatbuffers workspace. The loading of the macro via @flatbuffers works fine, but the contents of the macro itself references @com_github_google_flatbuffers in its body. I expected that providing repo_mapping to the http_archive rule I used to add the flatbuffers workspace would have translated this, but it did not - when the macro expands in my main workspace, it references the longer name, and therefore I get a compilation error about a non-existing repository.

markww pushed a commit to google-research/raksha that referenced this issue Sep 27, 2022
Although Bazel gives you the option of naming your http_archive
repositories whatever you want, it apparently would really prefer if
everyone chose the same name everywhere. This becomes apparent where
fuzztest really wants the top level `WORKSPACE` to call absl
`com_google_absl`, and will not allow me to use the shorter name `absl`.
This is especially annoying because both `WORKSPACE`s describe `absl`
using the same URL and sha256 hash, so you'd hope it would identify them
as the same dependency? Discussions online do not give me great hope in
flexibility from Bazel on this point:
bazelbuild/bazel#3219

In any case, for now I am duplicating the dependency in two different
`http_archive`s. I will remove the shorter, more readable `absl` and
`re2` in the future once I have cut over to the new dependency.
markww pushed a commit to google-research/raksha that referenced this issue Sep 27, 2022
…741)

Although Bazel gives you the option of naming your http_archive
repositories whatever you want, it apparently would really prefer if
everyone chose the same name everywhere. This becomes apparent where
fuzztest really wants the top level `WORKSPACE` to call absl
`com_google_absl`, and will not allow me to use the shorter name `absl`.
This is especially annoying because both `WORKSPACE`s describe `absl`
using the same URL and sha256 hash, so you'd hope it would identify them
as the same dependency? Discussions online do not give me great hope in
flexibility from Bazel on this point:
bazelbuild/bazel#3219

In any case, for now I am duplicating the dependency in two different
`http_archive`s. I will remove the shorter, more readable `absl` and
`re2` in the future once I have cut over to the new dependency.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P3 We're not considering working on this, but happy to review a PR. (No assignee) type: feature request
Projects
None yet
Development

No branches or pull requests