[clang][Driver][Darwin] Optionally use xcselect to find macOS SDK #119670
Conversation
This is a scaled down version of https://reviews.llvm.org/D136315. The intent is largely the same as before[^1], but I've scaled down the scope to try to avoid the issues that the previous patch caused: - the changes are now opt-in based on enabling `CLANG_USE_XCSELECT` - this only works when targeting macOS on a macOS host (this is the only case supported by `libxcselect`[^2]) We also introduce an environment variable `CLANG_NO_XCSELECT` that disables this behaviour if Clang is configured with `CLANG_USE_XCSELECT=ON`. This is needed to avoid breaking tests. Another reason to leave this as opt-in for now is that there are some bugs in libxcselect that need fixing before it is safe to use by default for all users. This has been reported to Apple as FB16081077. [^1]: See also https://reviews.llvm.org/D109460 and llvm#45225. [^2]: https://developer.apple.com/documentation/xcselect?language=objc
|
@llvm/pr-subscribers-clang @llvm/pr-subscribers-clang-driver Author: Carlo Cabrera (carlocab) ChangesThis is a scaled down version of https://reviews.llvm.org/D136315. The intent is largely the same as before1, but I've scaled down the scope to try to avoid the issues that the previous patch caused:
We also introduce an environment variable Another reason to leave this as opt-in for now is that there are some bugs in libxcselect that need fixing before it is safe to use by default for all users. This has been reported to Apple as FB16081077. Full diff: https://github.com/llvm/llvm-project/pull/119670.diff 5 Files Affected:
diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 27e8095534a65c..3c7096a27c509b 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -214,6 +214,39 @@ if(GCC_INSTALL_PREFIX AND NOT USE_DEPRECATED_GCC_INSTALL_PREFIX)
"See https://github.com/llvm/llvm-project/pull/77537 for detail.")
endif()
+if(APPLE)
+ check_include_file(xcselect.h CLANG_HAVE_XCSELECT_H)
+ if(CLANG_HAVE_XCSELECT_H)
+ include(CheckSymbolExists)
+ list(APPEND CMAKE_REQUIRED_LIBRARIES xcselect)
+ check_symbol_exists(xcselect_host_sdk_path xcselect.h CLANG_HAVE_XCSELECT_HOST_SDK_PATH)
+ list(REMOVE_ITEM CMAKE_REQUIRED_LIBRARIES xcselect)
+ endif()
+endif()
+
+cmake_dependent_option(CLANG_USE_XCSELECT "Use libxcselect to find the macOS SDK." OFF
+ "APPLE;CLANG_HAVE_XCSELECT_HOST_SDK_PATH" OFF)
+
+if(DEFAULT_SYSROOT AND CLANG_USE_XCSELECT)
+ message(FATAL_ERROR "Setting DEFAULT_SYSROOT is incompatible with CLANG_USE_XCSELECT.")
+endif()
+
+if(CLANG_USE_XCSELECT)
+ set(XCSELECT_VALID_POLICIES LATEST MATCHING_ONLY MATCHING_PREFERRED)
+ set(CLANG_XCSELECT_HOST_SDK_POLICY "LATEST" CACHE STRING
+ "Policy to use for xcselect. One of: ${XCSELECT_VALID_POLICIES}")
+ set_property(CACHE CLANG_XCSELECT_HOST_SDK_POLICY PROPERTY STRINGS ${XCSELECT_VALID_POLICIES})
+ string(TOUPPER ${CLANG_XCSELECT_HOST_SDK_POLICY} CLANG_XCSELECT_HOST_SDK_POLICY)
+ list(JOIN XCSELECT_VALID_POLICIES "|" XCSELECT_POLICY_REGEX)
+ if(NOT CLANG_XCSELECT_HOST_SDK_POLICY MATCHES "^XCSELECT_HOST_SDK_POLICY_(${XCSELECT_POLICY_REGEX})$")
+ if(NOT CLANG_XCSELECT_HOST_SDK_POLICY IN_LIST XCSELECT_VALID_POLICIES)
+ message(FATAL_ERROR
+ "CLANG_XCSELECT_HOST_SDK_POLICY (${CLANG_XCSELECT_HOST_SDK_POLICY}) must be one of: ${XCSELECT_VALID_POLICIES}")
+ endif()
+ set(CLANG_XCSELECT_HOST_SDK_POLICY "XCSELECT_HOST_SDK_POLICY_${CLANG_XCSELECT_HOST_SDK_POLICY}")
+ endif()
+endif()
+
set(ENABLE_LINKER_BUILD_ID OFF CACHE BOOL "pass --build-id to ld")
set(ENABLE_X86_RELAX_RELOCATIONS ON CACHE BOOL
diff --git a/clang/include/clang/Config/config.h.cmake b/clang/include/clang/Config/config.h.cmake
index 27ed69e21562bf..9ae0c6a4b9e11d 100644
--- a/clang/include/clang/Config/config.h.cmake
+++ b/clang/include/clang/Config/config.h.cmake
@@ -86,4 +86,10 @@
/* Whether CIR is built into Clang */
#cmakedefine01 CLANG_ENABLE_CIR
+/* Whether to use xcselect to find the macOS SDK */
+#cmakedefine CLANG_USE_XCSELECT
+
+/* Policy to use for xcselect */
+#cmakedefine CLANG_XCSELECT_HOST_SDK_POLICY ${CLANG_XCSELECT_HOST_SDK_POLICY}
+
#endif
diff --git a/clang/lib/Driver/CMakeLists.txt b/clang/lib/Driver/CMakeLists.txt
index 4fd10bf671512f..299de2ef30470c 100644
--- a/clang/lib/Driver/CMakeLists.txt
+++ b/clang/lib/Driver/CMakeLists.txt
@@ -14,6 +14,10 @@ if(WIN32)
set(system_libs version)
endif()
+if(CLANG_USE_XCSELECT)
+ set(system_libs xcselect)
+endif()
+
add_clang_library(clangDriver
Action.cpp
Compilation.cpp
diff --git a/clang/lib/Driver/ToolChains/Darwin.cpp b/clang/lib/Driver/ToolChains/Darwin.cpp
index 87380869f6fdab..8d3e0130a5a432 100644
--- a/clang/lib/Driver/ToolChains/Darwin.cpp
+++ b/clang/lib/Driver/ToolChains/Darwin.cpp
@@ -29,6 +29,10 @@
#include "llvm/TargetParser/Triple.h"
#include <cstdlib> // ::getenv
+#ifdef CLANG_USE_XCSELECT
+#include <xcselect.h> // ::xcselect_host_sdk_path
+#endif
+
using namespace clang::driver;
using namespace clang::driver::tools;
using namespace clang::driver::toolchains;
@@ -2257,17 +2261,26 @@ void Darwin::AddDeploymentTarget(DerivedArgList &Args) const {
// Warn if the path does not exist.
if (!getVFS().exists(A->getValue()))
getDriver().Diag(clang::diag::warn_missing_sysroot) << A->getValue();
- } else {
- if (char *env = ::getenv("SDKROOT")) {
- // We only use this value as the default if it is an absolute path,
- // exists, and it is not the root path.
- if (llvm::sys::path::is_absolute(env) && getVFS().exists(env) &&
- StringRef(env) != "/") {
- Args.append(Args.MakeSeparateArg(
- nullptr, Opts.getOption(options::OPT_isysroot), env));
- }
+ } else if (const char *env = ::getenv("SDKROOT"); env && *env) {
+ // We only use this value as the default if it is an absolute path,
+ // exists, and it is not the root path.
+ if (llvm::sys::path::is_absolute(env) && getVFS().exists(env) &&
+ StringRef(env) != "/") {
+ Args.append(Args.MakeSeparateArg(
+ nullptr, Opts.getOption(options::OPT_isysroot), env));
}
}
+#ifdef CLANG_USE_XCSELECT
+ else if (const char *env = ::getenv("CLANG_NO_XCSELECT");
+ getTriple().isMacOSX() && (!env || !*env)) {
+ if (char *p;
+ !::xcselect_host_sdk_path(CLANG_XCSELECT_HOST_SDK_POLICY, &p)) {
+ Args.append(Args.MakeSeparateArg(
+ nullptr, Opts.getOption(options::OPT_isysroot), p));
+ ::free(p);
+ }
+ }
+#endif
// Read the SDKSettings.json file for more information, like the SDK version
// that we can pass down to the compiler.
diff --git a/clang/test/lit.cfg.py b/clang/test/lit.cfg.py
index 7e7934d5fe0f5f..d05ad1ec6c5fa6 100644
--- a/clang/test/lit.cfg.py
+++ b/clang/test/lit.cfg.py
@@ -366,3 +366,7 @@ def calculate_arch_features(arch_string):
# possibly be present in system and user configuration files, so disable
# default configs for the test runs.
config.environment["CLANG_NO_DEFAULT_CONFIG"] = "1"
+
+# Configuring clang with CLANG_USE_XCSELECT=ON breaks some tests, so disable
+# its behaviour while running tests.
+config.environment["CLANG_NO_XCSELECT"] = "1"
Footnotes |
It might also be nicer to set this only for the tests that break instead of globally. |
All done, thanks. I don't quite understand why the failing tests are caused to fail when |
I think the issue is that the implicit |
Yea, this seems right. Adding appropriate |
This is a good guess, it seems to help: diff --git a/clang/test/Driver/arc.c b/clang/test/Driver/arc.c
index 7f6ac81aad17..70cd66899dff 100644
--- a/clang/test/Driver/arc.c
+++ b/clang/test/Driver/arc.c
@@ -1,11 +1,8 @@
-// XFAIL: xcselect
-// FIXME: There's no reason why this should fail with CLANG_USE_XCSELECT.
-
-// RUN: not %clang -ObjC -target i386-apple-darwin10 -stdlib=libstdc++ -m32 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck %s
-// RUN: not %clang -x objective-c -target i386-apple-darwin10 -stdlib=libstdc++ -m32 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck %s
-// RUN: not %clang -x objective-c++ -target i386-apple-darwin10 -stdlib=libstdc++ -m32 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck %s
-// RUN: not %clang -x c -target i386-apple-darwin10 -stdlib=libstdc++ -m32 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck -check-prefix NOTOBJC %s
-// RUN: not %clang -x c++ -target i386-apple-darwin10 -stdlib=libstdc++ -m32 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck -check-prefix NOTOBJC %s
+// RUN: not %clang -ObjC -target i386-apple-macosx10.6 -stdlib=libstdc++ -m32 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck %s
+// RUN: not %clang -x objective-c -target i386-apple-macosx10.6 -stdlib=libstdc++ -m32 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck %s
+// RUN: not %clang -x objective-c++ -target i386-apple-macosx10.6 -stdlib=libstdc++ -m32 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck %s
+// RUN: not %clang -x c -target i386-apple-macosx10.6 -stdlib=libstdc++ -m32 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck -check-prefix NOTOBJC %s
+// RUN: not %clang -x c++ -target i386-apple-macosx10.6 -stdlib=libstdc++ -m32 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck -check-prefix NOTOBJC %s
// RUN: not %clang -x objective-c -target x86_64-apple-darwin11 -mmacos-version-min=10.5 -fobjc-arc %s -fsyntax-only 2>&1 | FileCheck -check-prefix NOTSUPPORTED %s
// Just to test clang is working.This change makes
This is an option, but not the ideal one since the default target triple still uses the Is there any reason why default builds don't just use the macOS spelling, at least for modern versions of macOS? |
I don't think anyone has looked into changing the host guess script. I am ok to switch to macOS spelling.
The downside here is that when investigating running each command, it might be confusing that the error is not reproducing unless an env is set. Alternative is just |
|
After excluding the I've left it as
|
|
For reference, the failure from dropping Failure Log |
Using them for `-darwin*` triples makes many tests fail.
c51df70 to
8c0ea7f
Compare
That looks like if that no SDK is provided, the SDK version is going to match deployment target. Let's break the NOSDK test into a separate file and mark that file as There is a corner case I guess that |
|
Thanks, done. |
| @@ -43,8 +43,8 @@ | |||
|
|
|||
| // RUN: %clang -target x86_64-apple-macos10.13 -mlinker-version=520 \ | |||
| // RUN: -### %t.o 2>&1 \ | |||
| // RUN: | FileCheck --check-prefix=NOSDK %s | |||
| // RUN: | FileCheck --check-prefix=INFERRED-SDK %s | |||
There was a problem hiding this comment.
I think you want to delete those two. The test will fail if no CLANG_USE_XCSELECT is used.
There was a problem hiding this comment.
I think they'll pass since I've just relaxed the text to match against. But I'll double check locally for both enabling and disabling CLANG_USE_XCSELECT and delete them if it fails.
There was a problem hiding this comment.
I see. The value of the test is to check the behavior of matching the version when SDK is not provided. I don't think current test provide any value.
There was a problem hiding this comment.
I think it does provide some value, particularly for builds with CLANG_USE_XCSELECT=ON (since there is a stricter test that was separated out for builds where it is disabled).
That is, we check that the first two arguments to -platform_version, "macos" and "10.13.0", are correctly inferred from -target, and that the last argument looks like a version scheme, even when no -isysroot flag is provided.
That still seems valuable to me, even if it's not the same as the original test.
However, if you're still not convinced, I can remove it.
This is a scaled down version of https://reviews.llvm.org/D136315.
The intent is largely the same as before1, but I've scaled down the scope to try to avoid the issues that the previous patch caused:
CLANG_USE_XCSELECTlibxcselect2)We also introduce an environment variable
CLANG_NO_XCSELECTthat disables this behaviour if Clang is configured withCLANG_USE_XCSELECT=ON. This is needed to avoid breaking tests.Another reason to leave this as opt-in for now is that there are some bugs in libxcselect that need fixing before it is safe to use by default for all users. This has been reported to Apple as FB16081077.
Footnotes
See also https://reviews.llvm.org/D109460 and regression: locally-built Clang can't find C++ stdlib headers on Mac #45225. ↩
https://developer.apple.com/documentation/xcselect?language=objc ↩