[CI] Enable federated learning plugin in buildkite #8301
Conversation
| @@ -22,7 +22,7 @@ command_wrapper="tests/ci_build/ci_build.sh gpu_build_centos7 docker --build-arg | |||
| echo "--- Build libxgboost from the source" | |||
| $command_wrapper tests/ci_build/prune_libnccl.sh | |||
| $command_wrapper tests/ci_build/build_via_cmake.sh -DUSE_CUDA=ON -DUSE_NCCL=ON \ | |||
| -DUSE_OPENMP=ON -DHIDE_CXX_SYMBOLS=ON -DUSE_NCCL_LIB_PATH=ON \ | |||
| -DUSE_OPENMP=ON -DPLUGIN_FEDERATED=ON -DHIDE_CXX_SYMBOLS=ON -DUSE_NCCL_LIB_PATH=ON \ | |||
There was a problem hiding this comment.
Is gRPC a runtime dependency? (i.e. would users get error if they don't have gRPC installed on their machine?) If so, we shouldn't enable the federated learning plugin in the Python wheel on PyPI.
There was a problem hiding this comment.
We can built a separate artifact with gRPC for testing purposes
There was a problem hiding this comment.
It's a build time dependency since we are linking statically.
There was a problem hiding this comment.
It's kind of a cross-cutting concern though, we'd want it to work with cpu/gpu/mgpu etc.
| # Install gRPC for Federated Learning | ||
| git clone -b v1.49.1 https://github.com/grpc/grpc.git && \ | ||
| pushd grpc && \ | ||
| git submodule update --init && \ | ||
| cmake -S . -B build -GNinja && \ | ||
| cmake --build build --target install && \ | ||
| popd && \ | ||
| rm -fr grpc && \ |
There was a problem hiding this comment.
It could be better to use grpc-cpp package from Conda, as we won't have to build from the source
There was a problem hiding this comment.
Tried this, but looks like the libprotobuf conda package doesn't install the cmake modules: conda-forge/libprotobuf-feedstock#68
|
As discussed offline, @hcho3 is taking this over. Feel free to close it when you are ready. Thanks! |
No description provided.