Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Using system arrow does not trigger find thrift #6321

Closed
Yohahaha opened this issue Jul 3, 2024 · 5 comments
Closed

[VL] Using system arrow does not trigger find thrift #6321

Yohahaha opened this issue Jul 3, 2024 · 5 comments

Comments

@Yohahaha
Copy link
Contributor

Yohahaha commented Jul 3, 2024

Problem description

I build&install arrow first, then velox using system arrow, but seems the action of found thrift not executed.

System information

n/a

CMake log

cmake  -B \
	"_build/release" \
	-DTREAT_WARNINGS_AS_ERRORS=1 -DENABLE_ALL_WARNINGS=1 -DVELOX_BUILD_MINIMAL="OFF" -DVELOX_BUILD_TESTING="ON" -DCMAKE_BUILD_TYPE=Release \
	-GNinja -DMAX_LINK_JOBS=20 -DMAX_HIGH_MEM_JOBS=20 -DVELOX_FORCE_COLORED_OUTPUT=ON \
	-DVELOX_ENABLE_PARQUET=ON -DVELOX_ENABLE_HDFS=ON -DVELOX_BUILD_TESTING=OFF -DVELOX_BUILD_TEST_UTILS=OFF -DVELOX_ENABLE_DUCKDB=OFF -DVELOX_ENABLE_PARSE=OFF
-- Building using CMake version: 3.29.3
-- Build type: Release
-- Using ccache: /usr/bin/ccache
CMake Warning at CMakeLists.txt:237 (find_package):
  By not providing "Findlibhdfs3.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "libhdfs3",
  but CMake did not find one.

  Could not find a package configuration file provided by "libhdfs3" with any
  of the following names:

    libhdfs3Config.cmake
    libhdfs3-config.cmake

  Add the installation prefix of "libhdfs3" to CMAKE_PREFIX_PATH or set
  "libhdfs3_DIR" to a directory containing one of the above files.  If
  "libhdfs3" provides a separate development package or SDK, be sure it has
  been installed.


Setting CMAKE_CXX_FLAGS=-mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2
FINAL CMAKE_CXX_FLAGS= -mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2 -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wnon-virtual-dtor        -Wno-implicit-fallthrough          -Wno-class-memaccess          -Wno-comment          -Wno-int-in-bool-context          -Wno-redundant-move          -Wno-array-bounds          -Wno-maybe-uninitialized          -Wno-unused-result          -Wno-format-overflow          -Wno-strict-aliasing
-- Setting Boost source to AUTO
-- Found Boost: /usr/local/lib/cmake/Boost-1.84.0/BoostConfig.cmake (found suitable version "1.84.0", minimum required is "1.77.0") found components: atomic context date_time filesystem program_options regex system thread
-- Using SYSTEM Boost
-- Setting gflags source to AUTO
-- Using SYSTEM gflags
-- Setting fmt source to AUTO
-- Using SYSTEM fmt
-- Setting re2 source to AUTO
-- Found RE2 via pkg-config.
-- Using SYSTEM re2
-- Setting Protobuf source to AUTO
-- Using SYSTEM Protobuf
-- Setting simdjson source to BUNDLED
-- Building simdjson from source
-- Adding -Og to compile flag
-- Setting folly source to AUTO
-- Found Boost: /usr/local/lib/cmake/Boost-1.84.0/BoostConfig.cmake (found suitable version "1.84.0", minimum required is "1.51.0") found components: context filesystem program_options regex system thread
-- Found folly: /usr/local
-- Using SYSTEM folly
-- Setting xsimd source to AUTO
-- Using SYSTEM xsimd
-- Building stemmer from source
-- Setting Arrow source to AUTO
-- Using SYSTEM Arrow
CMake Deprecation Warning at velox/tpch/gen/dbgen/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Configuring done (0.1s)
-- Generating done (0.1s)
-- Build files have been written to: /root/oap-velox/_build/release
make[1]: 离开目录“/root/oap-velox”
make[1]: 进入目录“/root/oap-velox”
cmake --build _build/release -j 30
[69/602] Building CXX object velox/dwio/parquet/thrift/CMakeFiles/velox_dwio_parquet_thrift.dir/ParquetThriftTypes.cpp.o
FAILED: velox/dwio/parquet/thrift/CMakeFiles/velox_dwio_parquet_thrift.dir/ParquetThriftTypes.cpp.o
/usr/bin/ccache /usr/lib64/ccache/c++ -DFOLLY_HAVE_INT128_T=1 -DVELOX_DISABLE_GOOGLETEST -DVELOX_ENABLE_HDFS3 -DVELOX_ENABLE_PARQUET -I/root/oap-velox/. -I/root/oap-velox/velox/external/xxhash -isystem /root/oap-velox/velox -isystem /root/oap-velox/velox/external -mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2 -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wnon-virtual-dtor        -Wno-implicit-fallthrough          -Wno-class-memaccess          -Wno-comment          -Wno-int-in-bool-context          -Wno-redundant-move          -Wno-array-bounds          -Wno-maybe-uninitialized          -Wno-unused-result          -Wno-format-overflow          -Wno-strict-aliasing -Werror -O3 -DNDEBUG -std=gnu++17 -fPIC -fdiagnostics-color=always -MD -MT velox/dwio/parquet/thrift/CMakeFiles/velox_dwio_parquet_thrift.dir/ParquetThriftTypes.cpp.o -MF velox/dwio/parquet/thrift/CMakeFiles/velox_dwio_parquet_thrift.dir/ParquetThriftTypes.cpp.o.d -o velox/dwio/parquet/thrift/CMakeFiles/velox_dwio_parquet_thrift.dir/ParquetThriftTypes.cpp.o -c /root/oap-velox/velox/dwio/parquet/thrift/ParquetThriftTypes.cpp
In file included from /root/oap-velox/velox/dwio/parquet/thrift/ParquetThriftTypes.cpp:23:
/root/oap-velox/velox/dwio/parquet/thrift/ParquetThriftTypes.h:25:10: 致命错误:thrift/TApplicationException.h:没有那个文件或目录
   25 | #include <thrift/TApplicationException.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@Yohahaha
Copy link
Contributor Author

Yohahaha commented Jul 3, 2024

@PHILO-HE could you help take a look?

@PHILO-HE
Copy link
Contributor

PHILO-HE commented Jul 3, 2024

@Yohahaha, we have proposed a fix: facebookincubator/velox#10355.

@Yohahaha
Copy link
Contributor Author

Yohahaha commented Jul 3, 2024

@Yohahaha, we have proposed a fix: facebookincubator/velox#10355.

could you help explain how gluten fix this issue temporarily?

@PHILO-HE
Copy link
Contributor

PHILO-HE commented Jul 3, 2024

@Yohahaha, we have proposed a fix: facebookincubator/velox#10355.

could you help explain how gluten fix this issue temporarily?

@Yohahaha, to work around this issue, please "export Arrow_SOURCE=BUNDLED" in your env. to let velox always use bundled arrow.

@Yohahaha
Copy link
Contributor Author

Yohahaha commented Jul 3, 2024

@Yohahaha, we have proposed a fix: facebookincubator/velox#10355.

could you help explain how gluten fix this issue temporarily?

@Yohahaha, to work around this issue, please "export Arrow_SOURCE=BUNDLED" in your env. to let velox always use bundled arrow.

I see....

@Yohahaha Yohahaha closed this as completed Jul 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants