Compiler-provided inttypes.h is not recognized as belonging to any module

[llvmbot]

Bugzilla Link	51307
Version	trunk
OS	MacOS X
Reporter	LLVM Bugzilla Contributor
CC	@DougGregor,@zygoloid,@vsapsai

Extended Description

Attempting to compile a module containing a header that imports inttypes.h with -fmodules-strict-decluse enabled results in an error despite providing Clang with the Darwin module that provides inttypes.h:

While building module 'test':
In file included from <module-includes>:1:
./test.h:1:10: error: module test does not depend on a module exporting 'inttypes.h'
#include <inttypes.h>
         ^
1 error generated.

Clang finds inttypes.h within the compiler-provided headers, but does not consider the header to be built-in. The header is not present in the module map alongside the compiler-provided headers so there is no other module to depend on to access the header when using -fmodules-strict-decluse.

Most other compiler-provided headers are present in either the module map1 or as built-in headers that are treated as belonging to the module that provides the header they're wrapping2. inttypes.h and a few other headers are absent from both. inttypes.h appears like it should be treated as a built-in header. It's not clear how the other headers should be handled.

Platform-agnostic headers that appear to be missing from both the module map and built-in header list:

inttypes.h
builtins.h
stdnoreturn.h
varargs.h
vadefs.h

There are a number of platform-specific headers that are also absent from the module map.

Steps to reproduce:

Requires macOS with Xcode 12.x installed. Tested with both Clang from Xcode 12.5.1 (Apple clang version 12.0.5 (clang-1205.0.22.11)) and from main (bdf4c7b).

Precompile _Builtin_stddef_max_align_t

clang -cc1 -fsyntax-only -std=gnu11
-resource-dir $(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/*
-isysroot $(xcrun --show-sdk-path)
-internal-isystem $(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang//include
-internal-externc-isystem $(xcrun --show-sdk-path)/usr/include
-fgnuc-version=4.2.1 -fmodules
-emit-module
-fmodule-name=_Builtin_stddef_max_align_t
$(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang//include/module.modulemap
-o _Builtin_stddef_max_align_t.pcm

Precompile Darwin

clang -cc1 -fsyntax-only -std=gnu11
-resource-dir $(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/*
-isysroot $(xcrun --show-sdk-path)
-internal-isystem $(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang//include
-internal-externc-isystem $(xcrun --show-sdk-path)/usr/include
-fgnuc-version=4.2.1 -fmodules
-emit-module
-fmodule-map-file=$(echo $(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang//include/module.modulemap)
-fmodule-file=_Builtin_stddef_max_align_t=_Builtin_stddef_max_align_t.pcm
-fmodule-name=Darwin
$(xcrun --show-sdk-path)/usr/include/module.modulemap
-o Darwin.pcm

Attempt to precompile the module whose header imports inttypes.h

echo "#include <inttypes.h>" > test.h
echo -e "module test {\n use Darwin\n header "test.h"\n export *\n}" > test.modulemap

clang -cc1 -fsyntax-only -std=gnu11
-resource-dir $(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/*
-isysroot $(xcrun --show-sdk-path)
-internal-isystem $(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang//include
-internal-externc-isystem $(xcrun --show-sdk-path)/usr/include
-fgnuc-version=4.2.1 -fmodules
-emit-module
-fmodules-strict-decluse
-fmodule-map-file=$(echo $(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang//include/module.modulemap)
-fmodule-file=_Builtin_stddef_max_align_t=_Builtin_stddef_max_align_t.pcm
-fmodule-map-file=$(xcrun --show-sdk-path)/usr/include/module.modulemap
-fmodule-file=Darwin=Darwin.pcm
-fmodule-name=test
test.modulemap -o test.pcm

[llvmbot]

https://reviews.llvm.org/D107296 addresses the inttypes.h case.

I'm less sure what to do about the other compiler-provided headers that are also absent from the module map and not treated as built-in headers.

[vsapsai]

Possible fix with a test case
Here is the test case I've mentioned in the code review. Does the test case look reasonable? If yes, we can move on to other test failures it triggers.

[llvmbot]

The test case looks fine to me, though, as you note, the supporting files do cause failures in other test cases.

[vsapsai]

Possible fix for resource directory injection
My understanding is those errors are caused by how clang always prefers builtin headers from resource directory, even if it conflicts with the provided header search paths. Another instance of this problem can be seen running the command

echo '#include <mach/std_types.h>\nuint64_t test(void);' | clang -fsyntax-only -fmodules -fmodules-cache-path=/tmp/modules.noindex -isystem $(xcrun --sdk macosx --show-sdk-path)/usr/include -x objective-c -

The error is

:2:1: error: declaration of 'uint64_t' must be imported from module 'Darwin.POSIX._types._uint64_t' before it is required

and typedef unsigned long long uint64_t; isn't visible because the path SDK/usr/include is provided explicitly, so it comes before the resource directory. But during module building clang treats[1, 2] resource directory as coming before SDK/usr/include. This inconsistency breaks visibility tracking mechanism.

I have experimented with fixing it (patch attached) but don't have a test case yet and due to

If possible, don't stat the header until we need to. [...]
need to check the fix doesn't introduce unnecessary stat calls.

The most helpful path forward for me is if you help to create a test case for the resource directory injection. And then we can knock down the entire domino chain of fixes. But if you are not interested in this task, that is totally OK and understandable.

[1]

llvm-project/clang/lib/Lex/ModuleMap.cpp

Lines 308 to 311 in 77b435a

	auto Role = headerKindToRole(Header.Kind);
	Module::Header H = {Header.FileName, std::string(Path.str()), *File};
	addHeader(Mod, H, Role);
	return true;

[2]

llvm-project/clang/lib/Lex/ModuleMap.cpp

Lines 1158 to 1164 in 77b435a

	// If we have both a builtin and system version of the file, the
	// builtin version may want to inject macros into the system header, so
	// force the system header to be treated as a textual header in this
	// case.
	Header.Kind = headerRoleToKind(ModuleMap::ModuleHeaderRole(
	headerKindToRole(Header.Kind) | ModuleMap::TextualHeader));
	Header.HasBuiltinHeader = true;

[llvmbot]

I'm sorry, but due to other obligations I'm not able to spend time on the related issue your test uncovered.