Support finding installed arrow libraries from system

[PHILO-HE]

Gluten project needs to pre-build arrow before building velox. And it's also possible that arrow
libs have already been installed, e.g., via vcpkg. Then, velox doesn't need to build arrow from
source again.

[netlify]

✅ Deploy Preview for meta-velox canceled.

Name	Link
🔨 Latest commit	fd84b5c
🔍 Latest deploy log	https://app.netlify.com/sites/meta-velox/deploys/667a38fe3091ca0007489699

[PHILO-HE]

@majetideepak, @assignUser, do you have any comment?

[majetideepak]

@assignUser can you please help review this? Thanks!

[assignUser]

Thanks for the contribution! I do agree that we should be able to use an existing arrow install but I think we should implement it a bit differently.

There is no real reason for arrow to be in thirdparty it's just a remnant and everything else was moved to resolve_dependency already so I think arrow should follow.

We can then use the usual find_package/build-if-not-found approach utilizing arrowConfig.cmake instead hard coding things like you did here.

There might be a few quirks to keep in mind when using arrow with fetch content, there is an example here: https://github.com/amoeba/arrow-cmake-fetchcontent/

It's probably best to utilize the subdir approach we use for folly and boost in CMake/resolve_dependency for arrow too.

[PHILO-HE]

Thanks for the contribution! I do agree that we should be able to use an existing arrow install but I think we should implement it a bit differently.

There is no real reason for arrow to be in thirdparty it's just a remnant and everything else was moved to resolve_dependency already so I think arrow should follow.

We can then use the usual find_package/build-if-not-found approach utilizing arrowConfig.cmake instead hard coding things like you did here.

There might be a few quirks to keep in mind when using arrow with fetch content, there is an example here: https://github.com/amoeba/arrow-cmake-fetchcontent/

It's probably best to utilize the subdir approach we use for folly and boost in CMake/resolve_dependency for arrow too.

@assignUser, thanks so much for your comment. Just updated the pr with resolve_dependency.

To find arrow from system, I tried to let arrowConfig.cmake be used. It was installed under system path. But target re-definition error for some third-party lib is reported. So currently I added FindArrow.cmake in velox for finding arrow lib.

I also tried using FetchContent_MakeAvailable to get and build arrow source code. But there are also some target conflict issues with the main project. So I keep using the existing ExternalProject_Add to isolate arrow build.

Please kindly let me know your suggestions. Thanks!

[PHILO-HE]

Hi @assignUser, could you help review again? Thanks!

[assignUser]

Thanks, this lookgs good! While I would like to re-work this to use FetchContent, we do share a number of dependencies and I can see that leading to trouble.

This finally aligns the existing bundled Arrow build with resolve_dependency and additionally allows for use of system Arrow. So we should merge this for now and we can try FetchContent again later in a follow up :)

[PHILO-HE]

Thanks, this lookgs good! While I would like to re-work this to use FetchContent, we do share a number of dependencies and I can see that leading to trouble.

This finally aligns the existing bundled Arrow build with resolve_dependency and additionally allows for use of system Arrow. So we should merge this for now and we can try FetchContent again later in a follow up :)

@assignUser, sounds good! Thanks so much for your review!

[facebook-github-bot]

@bikramSingh91 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@bikramSingh91 merged this pull request in 0d80228.

[conbench-facebook]

Conbench analyzed the 1 benchmark run on commit 0d802284.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.