Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ruby release targets #11468

Closed
wants to merge 42 commits into from

Conversation

deannagarcia
Copy link
Member

@deannagarcia deannagarcia commented Jan 5, 2023

Add bazel targets to create ruby release artifacts.

Should be run with:

bazel run ruby:release
bazel run ruby:jruby_release 

@deannagarcia deannagarcia requested review from a team as code owners January 5, 2023 17:23
@deannagarcia deannagarcia requested review from perezd and esorot and removed request for a team, perezd and esorot January 5, 2023 17:23
@@ -651,6 +651,18 @@
"sha256": "66fdef91e9739348df7a096aa384a5685f4e875584cce89386a7a47251c4d8e9",
"url": "https://repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar"
},
{
"coord": "org.jruby:jruby-complete:9.2.20.1",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does "JRuby support version" become something we have to have an established policy on now as well? cc @devjgm

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does "JRuby support version" become something we have to have an established policy on now as well? cc @devjgm

Most likely, yes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As luck would have it, we should probably move off of 9.2 real soon, bump 9.3.x to the latest patch release, and add 9.4.

https://twitter.com/jruby/status/1611063274459566106

Copy link
Contributor

@perezd perezd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JasonLunn JasonLunn self-requested a review January 5, 2023 18:54
@deannagarcia deannagarcia changed the title Ruby targets Add ruby release targets Jan 5, 2023
@mkruskal-google
Copy link
Member

bazel run ruby:release --define=ruby_platform=c
bazel run ruby:jruby_release --define=ruby_platform=java

These defines aren't necessary anymore, and don't actually do anything. The new pattern is to use target_compatible_with (e.g.

target_compatible_with = select({
) and just allow the toolchain resolution to pick the appropriate target

protobuf_deps.bzl Outdated Show resolved Hide resolved
ruby/BUILD.bazel Outdated Show resolved Hide resolved
ruby/BUILD.bazel Outdated Show resolved Hide resolved
@deannagarcia
Copy link
Member Author

bazel run ruby:release --define=ruby_platform=c
bazel run ruby:jruby_release --define=ruby_platform=java

These defines aren't necessary anymore, and don't actually do anything. The new pattern is to use target_compatible_with (e.g.

target_compatible_with = select({

) and just allow the toolchain resolution to pick the appropriate target

I implemented that but now when I test I get:

Dependency chain:
    //ruby:jruby_release (06aa77)   <-- target platform (@local_config_platform//:host) didn't satisfy constraint @platforms//:incompatible

How can I use this and test locally?

@deannagarcia
Copy link
Member Author

bazel run ruby:release --define=ruby_platform=c
bazel run ruby:jruby_release --define=ruby_platform=java

These defines aren't necessary anymore, and don't actually do anything. The new pattern is to use target_compatible_with (e.g.

target_compatible_with = select({

) and just allow the toolchain resolution to pick the appropriate target

Used this and deleted comments about using the other flags.

# --- end runfiles.bash initialization ---

# Make a temporary directory and move to it to do all packaging work
mkdir -p tmp
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're in the sandbox now do we need a tmp directory?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I needed to make a new directory to start with a clean directory since this ends up being run in bazel-bin's runfile directory where a copy of all of the dependency files are.

Copy link
Member

@mkruskal-google mkruskal-google Jan 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This gets run in bazel-bin/ruby/jruby_release.runfiles/com_google_protobuf/, which should only contain your dependencies. Why do we need to copy all of them into a fresh folder? This seems redundant, we could just run gem build from this directory


# Move all generated files to lib/google/protobuf
mkdir -p lib/google/protobuf
cp "$(rlocation com_google_protobuf/src/google/protobuf/any_pb.rb)" lib/google/protobuf
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use dirname to copy these with a wildcard

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went back and forth on this. rlocation doesn't support wildcards, but I think you are suggesting that I get the dirname and then use wildcards which could be possible.

I ended up manually listing them as more of a precaution. Everywhere else the files are wildcarded in which I think could make it very easy to lose a file. This will fail in that case, which I think ends up being a good check. On the other hand, a new file being added could cause this to fail silently. What do you think about that tradeoff?

Copy link
Member

@mkruskal-google mkruskal-google Jan 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we were missing a file I would expect the gem build/test to fail in our CI. This seems a bit redundant to me, and not worth the maintenance burden.

One alternative that might work nicely is to add an sh_test rule that actually runs the tests using the built gem. That way we can actually run our tests locally and not have to wait for the CI to be run over a PR. It would also simplify the ruby_install.yml file I think


# Move all generated files to lib/google/protobuf
mkdir -p lib/google/protobuf
cp "$(rlocation com_google_protobuf/src/google/protobuf/any_pb.rb)" lib/google/protobuf
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ln or mv would be slightly more efficient here

# rvm use ruby-3.0

# Make a temporary directory and move to it to do all packaging work
mkdir -p tmp
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comments here

ruby/build_jruby_release.sh Show resolved Hide resolved
$HOME/bin/bazel run ruby:jruby_release
gem install bazel-bin/ruby/jruby_release.runfiles/com_google_protobuf/tmp/google-protobuf-*
if: ${{ contains(matrix.ruby, 'jruby') }}
- name: Test installation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should add a sanity check to the beginning to make sure these tests fail before installation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know how to run in GHA expecting a failure? I can't find a good way of doing that (but can confirm that I have seen this fail when the gems were not correct).

Copy link
Member

@mkruskal-google mkruskal-google Jan 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

! bazel test <target> will invert the result

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants