VCPKG port for onnxruntime and modernize cmake builds #7150
Comments
|
Nice! Also, I found finally they added versioning support. https://devblogs.microsoft.com/cppblog/take-control-of-your-vcpkg-dependencies-with-versioning-support/ That was really a blocking issue for us. If without it, Let's say if we put onnxruntime there, but later on protobuf gets a version that breaks onnxruntime build, what should we do? We can't publish a new release immediately just because Google protobuf did so. This was my main concern.
The biggest problem is how to handling ONNX. ONNX Runtime is not using a release version of ONNX. Can vcpkg accept such an ONNX package?
Sorry I don't know how to do that. Welcome to contribute! |
|
Hi, so for now |
|
Not yet. Welcome to contribute! |
You are not required or expected to immediately publish a new version in this case. The vcpkg maintainers ensure that all packages stay compatible, so as part of updating protobuf, they will ensure that all dependent packages continue to work (and include patches for them to make them compatible, if needed). |
…LLY_DISCONNECTED=ON` (#15323) ### Description Rework some external targets to ease building with `-DFETCHCONTENT_FULLY_DISCONNECTED=ON` This will allow package managers to more easily provide an onnxruntime package by reducing the amount of patching needed downstream at each version. ### Motivation and Context Availability of onnxruntime in some C++ package managers #7150 conan-io/conan-center-index#16699 microsoft/vcpkg#20548 My initial intent is to get this in conan but the PR would most likely be useful (though not tested) to vcpkg as well (and maybe others). I tried to get only a first batch of not too specific patches (i.e. not specific to conan). The first commit reworks `flatbuffers` and just extends what @snnn did in #13991 The second commit reworks `pytorch_cpuinfo` The third commit reworks `google_nsync`
…LLY_DISCONNECTED=ON` (microsoft#15323) ### Description Rework some external targets to ease building with `-DFETCHCONTENT_FULLY_DISCONNECTED=ON` This will allow package managers to more easily provide an onnxruntime package by reducing the amount of patching needed downstream at each version. ### Motivation and Context Availability of onnxruntime in some C++ package managers microsoft#7150 conan-io/conan-center-index#16699 microsoft/vcpkg#20548 My initial intent is to get this in conan but the PR would most likely be useful (though not tested) to vcpkg as well (and maybe others). I tried to get only a first batch of not too specific patches (i.e. not specific to conan). The first commit reworks `flatbuffers` and just extends what @snnn did in microsoft#13991 The second commit reworks `pytorch_cpuinfo` The third commit reworks `google_nsync`
**Description**: Adds support for cmake find_package. **Motivation and Context** As mentioned in issue #7150 onnxruntime doesn't have support for CMake find_package, this PR adds that and also adds the CMake package version file. Now anyone can link onnxruntime like this: ```cmake find_package(onnxruntime) add_executable(test Source.cpp) target_link_libraries(test PRIVATE onnxruntime::onnxruntime) ``` this also simplifies #3124
|
I tested luncliff's registry luncliff/vcpkg-registry , but encountered a similar problem with conan: conan-io/conan-center-index#17380 (comment) , luncliff/vcpkg-registry/issues/136#issuecomment-1851206111 |
### Changes
1. CMake option `onnxruntime_USE_VCPKG`. It will be used in the vcpkg
port
* Unit test may fail because this option leads to a mixture of
unexpected external library versions.
Especially ONNX, Protobuf, and Flatbuffers version can be different
2. Overhaul of `onnxruntime_external_deps.cmake`
* Make `FetchContent_Declare` to try `find_package`.
See
https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html
* Relocated `FetchContent_Declare` and `FetchContent_MakeAvailable`(or
`onnxruntime_fetchcontent_makeavailable`) to closer lines.
It was too hard to navigate the entire file to search related
sections...
* Alias `IMPORTED` targets like build targets (e.g. `ONNX::onnx` -->
`onnx`)
```cmake
# The script uses `find_package` with the changes.
# In this case, use vcpkg to search dependencies
# See https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html
include(external/onnxruntime_external_deps.cmake)
```
3. Create CMakePresets.json and presets to [run vcpkg in manifest
mode](https://learn.microsoft.com/en-us/vcpkg/concepts/manifest-mode)
* Currently, it's NOT for training build
* Main triplets are `x64-windows` and `x64-osx`
```pwsh
Push-Location "cmake"
cmake --preset "x64-windows-vcpkg"
cmake --build --preset "x64-windows-vcpkg-debug"
Pop-Location
```
```bash
pushd "cmake"
cmake --preset "x64-osx-vcpkg"
cmake --build --preset "x64-osx-vcpkg-debug"
popd
```
4. Updated tools/ci_build/build.py
* `--use_vcpkg` option: it needs `CMAKE_TOOLCHAIN_FILE` with
[vcpkg.cmake toolchain
script](https://github.com/microsoft/vcpkg/blob/master/scripts/buildsystems/vcpkg.cmake)
* `--compile_no_warning_as_error` is recommended because library version
differences will cause unexpected compiler warnings
```bash
python ./tools/ci_build/build.py \
--compile_no_warning_as_error \
--use_vcpkg \
--cmake_extra_defines "CMAKE_TOOLCHAIN_FILE:FILEPATH=${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake" \
--cmake_extra_defines "VCPKG_TARGET_TRIPLET=..."
```
5. Created Job `Vcpkg` for Windows and macOS
* Show how to setup and use vcpkg.
Similar to the CMakePresets.json usage
### Motivation and Context
* Help #7150
* Help microsoft/vcpkg#36850
* luncliff/vcpkg-registry#212
* microsoft/vcpkg#39881
* luncliff/vcpkg-registry#215
* luncliff/vcpkg-registry#216
* luncliff/vcpkg-registry#227
*
https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html
*
https://github.com/microsoft/vcpkg/blob/master/scripts/buildsystems/vcpkg.cmake
### Future Works?
More feature coverage with the vcpkg supported libraries
* CUDA feature support
* Training feature support
For most build configs, it is done. A few of our build pipelines still need to use the submodules, for example, the webassembly pipeline needs to get emsdk from a submodule. I think it is not a big issue.
Now all deps are fetch by cmake's FetchContent. So by default it works.
Done. |
|
/Users/xwg/vcpkg/vcpkg/ports/onnxruntime: error: onnxruntime does not exist onnxruntime is deleted? |
|
It has not been checked in yet. |
Will it be submitted later? |
Provider a vcpkg port for onnxruntime for easier builds and linking into external projects.
There's an initial effort made that can be used as a starting template for this
microsoft/vcpkg#14903
This port succeeds for x64-windows-static-md but none of the other standard triplets.
Overall the onnxruntime cmakelist.txt needs cleaning to support such a port in an efficient manner
The text was updated successfully, but these errors were encountered: