Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for when there are so many file arguments it creates the Command To Long error #320

Merged
merged 7 commits into from
Jun 12, 2020

Conversation

opbro
Copy link
Contributor

@opbro opbro commented May 29, 2020

When there are to many file arguments to py_wheel from py_package on Windows 10 I was getting
this error related to wheelmaker.exe

command is longer than CreateProcessW's limit (32768 characters)

Added additional rules to get around this issue.

opbro and others added 2 commits May 28, 2020 20:49
Added new rule for putting package files path into a single file to a…
@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@opbro
Copy link
Contributor Author

opbro commented May 29, 2020

@googlebot I signed it!

@googlebot
Copy link

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@opbro opbro requested a review from pstradomski June 2, 2020 01:54
Copy link
Contributor

@pstradomski pstradomski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@pstradomski
Copy link
Contributor

andyscott, thundergolfer: For some reason github would let me merge this PR; I'm not sure why though, as I'm not listed in CODEOWNERS. As this looks like unintentional misconfiguration, I'm defering to your decision whether to merge or not. The change looks good to me.

@opbro
Copy link
Contributor Author

opbro commented Jun 3, 2020

@thundergolfer or @andyscott have any thoughts on this change?

@pstradomski pstradomski merged commit c82a8cc into bazelbuild:master Jun 12, 2020
thundergolfer pushed a commit to gmweaver/rules_python that referenced this pull request Jul 31, 2020
add zipextended.py to BUILD

fix import path

use pip internal unzip

use pip internal unzip tools

remove custom zipfile implementation

Add documentation of community / Bazel team ownership (bazelbuild#308)

This adds a more nuanced CODEOWNERS and explains its purpose in CONTRIBUTING.md.

Fixes bazelbuild#291.

point README readers to new 0.0.2 release (bazelbuild#302)

update version in version.bzl (bazelbuild#303)

Fix for when there are so many file arguments it creates the Command To Long error (bazelbuild#320)

Fix failing build on CI by specifying pip package version (bazelbuild#329)

`bazel build //...` was failing due to "googleapis-common-protos[grpc]"
pip package being unavailable.

It seems to be caused by latest googleapis-common-protos release.
Specify googleapis-common-protos in requirements.txt to be in the previous
version (1.51.0) to fix this.

Fixes bazelbuild#321.

"Skylark" is an outdated name of the language, please use "starlark" instead (bazelbuild#327)

Co-authored-by: Andy Scott <andyscott@users.noreply.github.com>

Support python interpreter target in pip_import. (bazelbuild#312)

* Support python interpreter target in pip_import.

This allows users to use a custom python interpreter that is built by
another repository rule instead of using a pre-built interpreter binary
that is checked-in.

This tangentially addresses bazelbuild#257 since a common setup is to use the
custom built interpreter in the python toolchain.

For example, see: https://github.com/kku1993/bazel-hermetic-python

* Actually use interpreter path.

Co-authored-by: Andy Scott <andyscott@users.noreply.github.com>

Address bazelbuild#289 (bazelbuild#328)

Co-authored-by: Andy Scott <andyscott@users.noreply.github.com>

Fix errors with incompatible_disallow_empty_glob (bazelbuild#315)

Allow py_library sources to be empty.
If --incompatible_disallow_empty_glob is set then generated py_library
targets will fail if there are no .py files.

Examples are pymssql==2.1.4 and cx-Oracle==7.2.3.

Set `allow_empty = True` for glob().

Bazel issue for incompatible_disallow_empty_glob:
bazelbuild/bazel#8195

Co-authored-by: Andy Scott <andyscott@users.noreply.github.com>

rebuild piptool and whltool

update filename, add links to pip function definitions

remove typing checks

chore: github setup improvements (bazelbuild#334)

Remove mention and usage of Bazel Federation (bazelbuild#339)

It's currently a stalled project so it's not useful for us to direct new users there in our README.
Separately it is harder to develop on rules_python since it is currently not self-contained.
For example it's hard to find or adjust the version of rules_pkg without looking/editing in the federation repo.
Tony says this is an okay change: bazelbuild/bazel-federation@63f9746#commitcomment-40577834

leaner implementation of pip unzip

remove unzip.py

temp test with tools

2nd test with tools

final test with tools

replace with master tools

merge original tools

feat(examples): move examples to a nested WORKSPACE (bazelbuild#337)

This lets users understand the example in isolation. They can copy/paste the example directory
and it works correctly.

This refactors the existing examples which are quite weak, only really demonstrating pip usage.
This makes room for examples demonstrating other features (like protocol buffers) or package
managers (like poetry).

In a later commit I'll add bazel-integration-testing so we get a test target that confirms
the examples build (including their WORKSPACE being self-contained)

warn against putting .par file changes in PR. (bazelbuild#342)
thundergolfer pushed a commit to gmweaver/rules_python that referenced this pull request Jul 31, 2020
add zipextended.py to BUILD

fix import path

use pip internal unzip

use pip internal unzip tools

remove custom zipfile implementation

Add documentation of community / Bazel team ownership (bazelbuild#308)

This adds a more nuanced CODEOWNERS and explains its purpose in CONTRIBUTING.md.

Fixes bazelbuild#291.

point README readers to new 0.0.2 release (bazelbuild#302)

update version in version.bzl (bazelbuild#303)

Fix for when there are so many file arguments it creates the Command To Long error (bazelbuild#320)

Fix failing build on CI by specifying pip package version (bazelbuild#329)

`bazel build //...` was failing due to "googleapis-common-protos[grpc]"
pip package being unavailable.

It seems to be caused by latest googleapis-common-protos release.
Specify googleapis-common-protos in requirements.txt to be in the previous
version (1.51.0) to fix this.

Fixes bazelbuild#321.

"Skylark" is an outdated name of the language, please use "starlark" instead (bazelbuild#327)

Co-authored-by: Andy Scott <andyscott@users.noreply.github.com>

Support python interpreter target in pip_import. (bazelbuild#312)

* Support python interpreter target in pip_import.

This allows users to use a custom python interpreter that is built by
another repository rule instead of using a pre-built interpreter binary
that is checked-in.

This tangentially addresses bazelbuild#257 since a common setup is to use the
custom built interpreter in the python toolchain.

For example, see: https://github.com/kku1993/bazel-hermetic-python

* Actually use interpreter path.

Co-authored-by: Andy Scott <andyscott@users.noreply.github.com>

Address bazelbuild#289 (bazelbuild#328)

Co-authored-by: Andy Scott <andyscott@users.noreply.github.com>

Fix errors with incompatible_disallow_empty_glob (bazelbuild#315)

Allow py_library sources to be empty.
If --incompatible_disallow_empty_glob is set then generated py_library
targets will fail if there are no .py files.

Examples are pymssql==2.1.4 and cx-Oracle==7.2.3.

Set `allow_empty = True` for glob().

Bazel issue for incompatible_disallow_empty_glob:
bazelbuild/bazel#8195

Co-authored-by: Andy Scott <andyscott@users.noreply.github.com>

rebuild piptool and whltool

update filename, add links to pip function definitions

remove typing checks

chore: github setup improvements (bazelbuild#334)

Remove mention and usage of Bazel Federation (bazelbuild#339)

It's currently a stalled project so it's not useful for us to direct new users there in our README.
Separately it is harder to develop on rules_python since it is currently not self-contained.
For example it's hard to find or adjust the version of rules_pkg without looking/editing in the federation repo.
Tony says this is an okay change: bazelbuild/bazel-federation@63f9746#commitcomment-40577834

leaner implementation of pip unzip

remove unzip.py

temp test with tools

2nd test with tools

final test with tools

replace with master tools

merge original tools

feat(examples): move examples to a nested WORKSPACE (bazelbuild#337)

This lets users understand the example in isolation. They can copy/paste the example directory
and it works correctly.

This refactors the existing examples which are quite weak, only really demonstrating pip usage.
This makes room for examples demonstrating other features (like protocol buffers) or package
managers (like poetry).

In a later commit I'll add bazel-integration-testing so we get a test target that confirms
the examples build (including their WORKSPACE being self-contained)

warn against putting .par file changes in PR. (bazelbuild#342)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants