SwiftPM fails to resolve system libraries that are present in the target SDK (--swift-sdk)
#7409
Open
1 task done
Assignees
Labels
Comments
MaxDesiatov
pushed a commit
that referenced
this issue
Apr 18, 2024
pkgconfig: Apply PKG_CONFIG_SYSROOTDIR when generating paths ### Motivation: SwiftPM's pkg-config implementation sets the `pc_sysrootdir` variable, but most `.pc` files do not use this variable directly. Instead, they rely on the pkg-config tool rewriting the generated paths to include the sysroot prefix when necessary. SwiftPM does not do this, so it does not generate the correct compiler flags to use libraries from a sysroot. This problem was reported in issue #7409 ### Modifications: There are two major pkg-config implementations which handle sysroot differently: * `pkg-config` (the original https://pkg-config.freedesktop.org implementation) prepends sysroot after variable expansion, when it creates the compiler flag lists * `pkgconf` (the newer http://pkgconf.org implementation) prepends sysroot to variables when they are defined, so sysroot is included when they are expanded `pkg-config`'s method skips single character compiler flags, such as `-I` and `-L`, and has special cases for longer options. It does not handle spaces between the flags and their values properly, and prepends sysroot multiple times in some cases, such as when the .pc file uses the `sysroot_dir` variable directly or has been rewritten to hard-code the sysroot prefix. `pkgconf`'s method handles spaces correctly, although it also makes extra checks to ensure that sysroot is not applied more than once. In 2024 `pkg-config` is the more popular option according to Homebrew installation statistics, but the major Linux distributions have generally switched to `pkgconf`. We will use `pkgconf`'s method here as it seems more robust than `pkg-config`'s, and `pkgconf`'s greater popularity on Linux means libraries developed there may depend on the specific way it handles `.pc` files. ### Result: SwiftPM will now apply the sysroot prefix to compiler flags, such as include (`-I`) and library (`-L`) search paths. This is a partial fix for #7409. The sysroot prefix is only applied when the `PKG_CONFIG_SYSROOT_DIR` environment variable is set. A future commit could apply an appropriate sysroot automatically when the `--experimental-swift-sdk` flag is used.
euanh
added a commit
to euanh/swift-package-manager
that referenced
this issue
Apr 19, 2024
SwiftPM's pkg-config implementation sets the `pc_sysrootdir` variable, but most `.pc` files do not use this variable directly. Instead, they rely on the pkg-config tool rewriting the generated paths to include the sysroot prefix when necessary. SwiftPM does not do this, so it does not generate the correct compiler flags to use libraries from a sysroot. This problem was reported in issue swiftlang#7409 There are two major pkg-config implementations which handle sysroot differently: * `pkg-config` (the original https://pkg-config.freedesktop.org implementation) prepends sysroot after variable expansion, when it creates the compiler flag lists * `pkgconf` (the newer http://pkgconf.org implementation) prepends sysroot to variables when they are defined, so sysroot is included when they are expanded `pkg-config`'s method skips single character compiler flags, such as `-I` and `-L`, and has special cases for longer options. It does not handle spaces between the flags and their values properly, and prepends sysroot multiple times in some cases, such as when the .pc file uses the `sysroot_dir` variable directly or has been rewritten to hard-code the sysroot prefix. `pkgconf`'s method handles spaces correctly, although it also makes extra checks to ensure that sysroot is not applied more than once. In 2024 `pkg-config` is the more popular option according to Homebrew installation statistics, but the major Linux distributions have generally switched to `pkgconf`. We will use `pkgconf`'s method here as it seems more robust than `pkg-config`'s, and `pkgconf`'s greater popularity on Linux means libraries developed there may depend on the specific way it handles `.pc` files. SwiftPM will now apply the sysroot prefix to compiler flags, such as include (`-I`) and library (`-L`) search paths. This is a partial fix for swiftlang#7409. The sysroot prefix is only applied when the `PKG_CONFIG_SYSROOT_DIR` environment variable is set. A future commit could apply an appropriate sysroot automatically when the `--experimental-swift-sdk` flag is used.
MaxDesiatov
pushed a commit
that referenced
this issue
Apr 19, 2024
…7472) Cherry pick of #7461 SwiftPM's pkg-config implementation sets the `pc_sysrootdir` variable, but most `.pc` files do not use this variable directly. Instead, they rely on the pkg-config tool rewriting the generated paths to include the sysroot prefix when necessary. SwiftPM does not do this, so it does not generate the correct compiler flags to use libraries from a sysroot. This problem was reported in issue #7409 There are two major pkg-config implementations which handle sysroot differently: * `pkg-config` (the original https://pkg-config.freedesktop.org implementation) prepends sysroot after variable expansion, when it creates the compiler flag lists * `pkgconf` (the newer http://pkgconf.org implementation) prepends sysroot to variables when they are defined, so sysroot is included when they are expanded `pkg-config`'s method skips single character compiler flags, such as `-I` and `-L`, and has special cases for longer options. It does not handle spaces between the flags and their values properly, and prepends sysroot multiple times in some cases, such as when the .pc file uses the `sysroot_dir` variable directly or has been rewritten to hard-code the sysroot prefix. `pkgconf`'s method handles spaces correctly, although it also makes extra checks to ensure that sysroot is not applied more than once. In 2024 `pkg-config` is the more popular option according to Homebrew installation statistics, but the major Linux distributions have generally switched to `pkgconf`. We will use `pkgconf`'s method here as it seems more robust than `pkg-config`'s, and `pkgconf`'s greater popularity on Linux means libraries developed there may depend on the specific way it handles `.pc` files. SwiftPM will now apply the sysroot prefix to compiler flags, such as include (`-I`) and library (`-L`) search paths. This is a partial fix for #7409. The sysroot prefix is only applied when the `PKG_CONFIG_SYSROOT_DIR` environment variable is set. A future commit could apply an appropriate sysroot automatically when the `--experimental-swift-sdk` flag is used. **Scope**: Limited to packages relying on system libraries that utilize pkg-config. **Risk**: Low, changes are isolated and the scope is limited to a small fraction of packages. **Testing**: Automated with new test cases. **Issues**: #7409 **Reviewer**: @MaxDesiatov.
euanh
added a commit
to euanh/swift-package-manager
that referenced
this issue
Apr 26, 2024
SwiftPM's pkg-config implementation sets the `pc_sysrootdir` variable, but most `.pc` files do not use this variable directly. Instead, they rely on the pkg-config tool rewriting the generated paths to include the sysroot prefix when necessary. SwiftPM does not do this, so it does not generate the correct compiler flags to use libraries from a sysroot. This problem was reported in issue swiftlang#7409 There are two major pkg-config implementations which handle sysroot differently: * `pkg-config` (the original https://pkg-config.freedesktop.org implementation) prepends sysroot after variable expansion, when it creates the compiler flag lists * `pkgconf` (the newer http://pkgconf.org implementation) prepends sysroot to variables when they are defined, so sysroot is included when they are expanded `pkg-config`'s method skips single character compiler flags, such as `-I` and `-L`, and has special cases for longer options. It does not handle spaces between the flags and their values properly, and prepends sysroot multiple times in some cases, such as when the .pc file uses the `sysroot_dir` variable directly or has been rewritten to hard-code the sysroot prefix. `pkgconf`'s method handles spaces correctly, although it also makes extra checks to ensure that sysroot is not applied more than once. In 2024 `pkg-config` is the more popular option according to Homebrew installation statistics, but the major Linux distributions have generally switched to `pkgconf`. We will use `pkgconf`'s method here as it seems more robust than `pkg-config`'s, and `pkgconf`'s greater popularity on Linux means libraries developed there may depend on the specific way it handles `.pc` files. SwiftPM will now apply the sysroot prefix to compiler flags, such as include (`-I`) and library (`-L`) search paths. This is a partial fix for swiftlang#7409. The sysroot prefix is only applied when the `PKG_CONFIG_SYSROOT_DIR` environment variable is set. A future commit could apply an appropriate sysroot automatically when the `--experimental-swift-sdk` flag is used.
furby-tm
pushed a commit
to wabiverse/swift-package-manager
that referenced
this issue
May 15, 2024
…ng#7461) pkgconfig: Apply PKG_CONFIG_SYSROOTDIR when generating paths ### Motivation: SwiftPM's pkg-config implementation sets the `pc_sysrootdir` variable, but most `.pc` files do not use this variable directly. Instead, they rely on the pkg-config tool rewriting the generated paths to include the sysroot prefix when necessary. SwiftPM does not do this, so it does not generate the correct compiler flags to use libraries from a sysroot. This problem was reported in issue swiftlang#7409 ### Modifications: There are two major pkg-config implementations which handle sysroot differently: * `pkg-config` (the original https://pkg-config.freedesktop.org implementation) prepends sysroot after variable expansion, when it creates the compiler flag lists * `pkgconf` (the newer http://pkgconf.org implementation) prepends sysroot to variables when they are defined, so sysroot is included when they are expanded `pkg-config`'s method skips single character compiler flags, such as `-I` and `-L`, and has special cases for longer options. It does not handle spaces between the flags and their values properly, and prepends sysroot multiple times in some cases, such as when the .pc file uses the `sysroot_dir` variable directly or has been rewritten to hard-code the sysroot prefix. `pkgconf`'s method handles spaces correctly, although it also makes extra checks to ensure that sysroot is not applied more than once. In 2024 `pkg-config` is the more popular option according to Homebrew installation statistics, but the major Linux distributions have generally switched to `pkgconf`. We will use `pkgconf`'s method here as it seems more robust than `pkg-config`'s, and `pkgconf`'s greater popularity on Linux means libraries developed there may depend on the specific way it handles `.pc` files. ### Result: SwiftPM will now apply the sysroot prefix to compiler flags, such as include (`-I`) and library (`-L`) search paths. This is a partial fix for swiftlang#7409. The sysroot prefix is only applied when the `PKG_CONFIG_SYSROOT_DIR` environment variable is set. A future commit could apply an appropriate sysroot automatically when the `--experimental-swift-sdk` flag is used.
furby-tm
pushed a commit
to wabiverse/swift-package-manager
that referenced
this issue
May 15, 2024
…ng#7461) pkgconfig: Apply PKG_CONFIG_SYSROOTDIR when generating paths ### Motivation: SwiftPM's pkg-config implementation sets the `pc_sysrootdir` variable, but most `.pc` files do not use this variable directly. Instead, they rely on the pkg-config tool rewriting the generated paths to include the sysroot prefix when necessary. SwiftPM does not do this, so it does not generate the correct compiler flags to use libraries from a sysroot. This problem was reported in issue swiftlang#7409 ### Modifications: There are two major pkg-config implementations which handle sysroot differently: * `pkg-config` (the original https://pkg-config.freedesktop.org implementation) prepends sysroot after variable expansion, when it creates the compiler flag lists * `pkgconf` (the newer http://pkgconf.org implementation) prepends sysroot to variables when they are defined, so sysroot is included when they are expanded `pkg-config`'s method skips single character compiler flags, such as `-I` and `-L`, and has special cases for longer options. It does not handle spaces between the flags and their values properly, and prepends sysroot multiple times in some cases, such as when the .pc file uses the `sysroot_dir` variable directly or has been rewritten to hard-code the sysroot prefix. `pkgconf`'s method handles spaces correctly, although it also makes extra checks to ensure that sysroot is not applied more than once. In 2024 `pkg-config` is the more popular option according to Homebrew installation statistics, but the major Linux distributions have generally switched to `pkgconf`. We will use `pkgconf`'s method here as it seems more robust than `pkg-config`'s, and `pkgconf`'s greater popularity on Linux means libraries developed there may depend on the specific way it handles `.pc` files. ### Result: SwiftPM will now apply the sysroot prefix to compiler flags, such as include (`-I`) and library (`-L`) search paths. This is a partial fix for swiftlang#7409. The sysroot prefix is only applied when the `PKG_CONFIG_SYSROOT_DIR` environment variable is set. A future commit could apply an appropriate sysroot automatically when the `--experimental-swift-sdk` flag is used.
|
The underlying pkg-config support in Swift Package Manager now handles |
--experimental-swift-sdk)--swift-sdk)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is it reproducible with SwiftPM command-line tools:
swift build,swift test,swift packageetc?Description
Currently, when cross-compiling packages with system dependencies, SwiftPM will parse
.pcfiles from the SDK but then take those paths as-is without considering the path of the SDK itself.For example,
glib-2.0on Ubuntu Jammy defines the following.pcfile:Building a SwiftPM package using a Swift SDK that contains this file will result in flags like
-I /usr/lib/x86_64-linux-gnu/glib-2.0. This path is not correct, the expected behaviour is to use a path inside the SDK root:-I $SDK_ROOT/usr/lib/x86_64-linux-gnu/glib-2.0One possible
solutionworkaround is to manually setPKG_CONFIG_SYSROOT_DIRenv variable to point into the SDK_ROOT. When set, this variable should be prepended to all resolved paths from thepcfile.Proper fix: the SDK sys root should be automatically prepended to the pkg-config search path when using a Swift SDK.
Expected behavior
Building Swift packages that rely on system libraries present in a Swift SDK should be possible. SwiftPM should pass flags that reference the files inside the SDK.
Actual behavior
SwiftPM passes flags that reference files in the host filesystem and not inside the SDK and thus the build fails or is incorrect.
Steps to reproduce
libglib2.0-devinstalled (see this issue on Swift SDK Generator: Can't get a custom Swift SDK working withglib-2.0swift-sdk-generator#89)Swift Package Manager version/commit hash
No response
Swift & OS version (output of
swift --version ; uname -a)swift-driver version: 1.90.11.1 Apple Swift version 5.10 (swiftlang-5.10.0.13 clang-1500.3.9.4)
Target: arm64-apple-macosx14.0
Darwin tobias-m1-pro.local 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:44 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6000 arm64
The text was updated successfully, but these errors were encountered: