Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] cudf submodule sync job failed #1306

Closed
jlowe opened this issue Jul 31, 2023 · 3 comments
Closed

[BUG] cudf submodule sync job failed #1306

jlowe opened this issue Jul 31, 2023 · 3 comments
Assignees
Labels
bug Something isn't working build

Comments

@jlowe
Copy link
Contributor

jlowe commented Jul 31, 2023

The regular cudf submodule sync job has been failing lately in the cudf build:

09:56:54  [INFO]      [exec] CMake Error at /home/jenkins/agent/workspace/spark-rapids-jni_submodule-sync-dev/thirdparty/cudf/cpp/build/arrow-targets.cmake:52 (set_target_properties):
09:56:54  [INFO]      [exec]   The link interface of target "cudf::arrow_static" contains:
09:56:54  [INFO]      [exec] 
09:56:54  [INFO]      [exec]     arrow::xsimd
09:56:54  [INFO]      [exec] 
09:56:54  [INFO]      [exec]   but the target was not found.  Possible reasons include:
09:56:54  [INFO]      [exec] 
09:56:54  [INFO]      [exec]     * There is a typo in the target name.
09:56:54  [INFO]      [exec]     * A find_package call is missing for an IMPORTED target.
09:56:54  [INFO]      [exec]     * An ALIAS target is missing.
09:56:54  [INFO]      [exec] 
09:56:54  [INFO]      [exec] Call Stack (most recent call first):
09:56:54  [INFO]      [exec]   /home/jenkins/agent/workspace/spark-rapids-jni_submodule-sync-dev/thirdparty/cudf/cpp/build/arrow-config.cmake:68 (include)
09:56:54  [INFO]      [exec]   /usr/local/cmake-3.26.4-linux-x86_64/share/cmake-3.26/Modules/CMakeFindDependencyMacro.cmake:76 (find_package)
09:56:54  [INFO]      [exec]   /home/jenkins/agent/workspace/spark-rapids-jni_submodule-sync-dev/thirdparty/cudf/cpp/build/cudf-dependencies.cmake:270 (find_dependency)
09:56:54  [INFO]      [exec]   /home/jenkins/agent/workspace/spark-rapids-jni_submodule-sync-dev/thirdparty/cudf/cpp/build/cudf-config.cmake:90 (include)
09:56:54  [INFO]      [exec]   /home/jenkins/agent/workspace/spark-rapids-jni_submodule-sync-dev/target/libcudfjni/_deps/rapids-cmake-src/rapids-cmake/find/package.cmake:125 (find_package)
09:56:54  [INFO]      [exec]   CMakeLists.txt:95 (rapids_find_package)
09:56:54  [INFO]      [exec] 
09:56:54  [INFO]      [exec] 
09:56:54  [INFO]      [exec] CMake Warning:
09:56:54  [INFO]      [exec]   Manually-specified variables were not used by the project:
09:56:54  [INFO]      [exec] 
09:56:54  [INFO]      [exec]     RMM_LOGGING_LEVEL
09:56:54  [INFO]      [exec] 
09:56:54  [INFO]      [exec] 
09:56:54  [INFO]      [exec] CMake Generate step failed.  Build files cannot be regenerated correctly.
09:56:54  [INFO] ------------------------------------------------------------------------
09:56:54  [INFO] BUILD FAILURE
09:56:54  [INFO] ------------------------------------------------------------------------
@gerashegalov
Copy link
Collaborator

gerashegalov commented Jul 31, 2023

when the CI build it was advancing thirdparty/cudf from 7746af421c88c59e2261d5edfcfe61b0fc7835fd
to 9d465f258ffdfe88e7b661cce8cae59741113ae0

To reproduce locally:

  1. checkout spark-rapids-jni at branch-23.10 HEAD
  2. update submodule init if necessary
  3. cd thirdparty/cudf
  4. git checkout -f 9d465f258ffdfe88e7b661cce8cae59741113ae0
  5. cd back to spark-rapids-jni root
  6. run ./build/build-in-docker -X clean -Dlibcudf.clean.skip=false install -DCPP_PARALLEL_LEVEL=6 -DGPU_ARCHS=NATIVE -DskipTests -Dsubmodule.check.skip=true

@gerashegalov
Copy link
Collaborator

rapidsai/cudf#13790 resolves the original build failure re arrow::xsimd

however there is another roadblock down the road:

[INFO]      [exec]   Could not find a package configuration file provided by "ArrowAcero" with
[INFO]      [exec]   any of the following names:
[INFO]      [exec] 
[INFO]      [exec]     ArrowAceroConfig.cmake
[INFO]      [exec]     arrowacero-config.cmake
[INFO]      [exec] -- Configuring incomplete, errors occurred!
[INFO]      [exec] CMake Error at /usr/local/cmake-3.26.4-linux-x86_64/share/cmake-3.26/Modules/CMakeFindDependencyMacro.cmake:76 (find_package):
[INFO]      [exec]   By not providing "FindArrowAcero.cmake" in CMAKE_MODULE_PATH this project
[INFO]      [exec]   has asked CMake to find a package configuration file provided by
[INFO]      [exec]   "ArrowAcero", but CMake did not find one.
[INFO]      [exec] 
[INFO]      [exec]   Could not find a package configuration file provided by "ArrowAcero" with
[INFO]      [exec]   any of the following names:
[INFO]      [exec] 
[INFO]      [exec]     ArrowAceroConfig.cmake
[INFO]      [exec]     arrowacero-config.cmake
[INFO]      [exec] 
[INFO]      [exec]   Add the installation prefix of "ArrowAcero" to CMAKE_PREFIX_PATH or set
[INFO]      [exec]   "ArrowAcero_DIR" to a directory containing one of the above files.  If
[INFO]      [exec]   "ArrowAcero" provides a separate development package or SDK, be sure it has
[INFO]      [exec]   been installed.
[INFO]      [exec] Call Stack (most recent call first):
[INFO]      [exec]   /home/gshegalov/gits/NVIDIA/spark-rapids-jni/target/libcudf-install/lib64/cmake/ArrowDataset/ArrowDatasetConfig.cmake:55 (find_dependency)
[INFO]      [exec]   /usr/local/cmake-3.26.4-linux-x86_64/share/cmake-3.26/Modules/CMakeFindDependencyMacro.cmake:76 (find_package)
[INFO]      [exec]   /home/gshegalov/gits/NVIDIA/spark-rapids-jni/target/libcudf-install/lib64/cmake/cudf/cudf-config.cmake:176 (find_dependency)
[INFO]      [exec]   /home/gshegalov/gits/NVIDIA/spark-rapids-jni/target/cmake-build/_deps/rapids-cmake-src/rapids-cmake/find/package.cmake:125 (find_package)
[INFO]      [exec]   CMakeLists.txt:112 (rapids_find_package)

rapids-bot bot pushed a commit to rapidsai/cudf that referenced this issue Aug 1, 2023
Arrow 12.0 uses the vendored CMake target `arrow::xsimd` instead of the global target name of `xsimd`. We need to use the new name so that libcudf can be used from the build directory by other projects.

Found by issue: NVIDIA/spark-rapids-jni#1306

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #13790
rapids-bot bot pushed a commit to rapidsai/cudf that referenced this issue Aug 2, 2023
Contiuation of #13790 as more changes are needed to support Arrow 12 builds from source ( both static and shared ). This fixes issues when building against Arrow with S3 enabled, and corrects missing `acero` targets.

NVIDIA/spark-rapids-jni#1306

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - David Wendt (https://github.com/davidwendt)
  - Gera Shegalov (https://github.com/gerashegalov)
  - Bradley Dice (https://github.com/bdice)

URL: #13794
@gerashegalov
Copy link
Collaborator

fixed by:
rapidsai/cudf#13790
rapidsai/cudf#13794

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working build
Projects
None yet
Development

No branches or pull requests

2 participants