[bug] compiler.libcxx for dependencies with prebuilt packages

[smessmer]

Source issue: cryfs/cryfs#336

I'm using conan_cmake_run to call conan from cmake and pass in the argument SETTINGS compiler.libcxx=libstdc++11. This calls conan install with -s compiler.libcxx=libstdc++11 before running the rest of the CMakeLists.txt because my application can only work with libstdc++11.

I noticed that the build works successfully when specifying -DCMAKE_BUILD_TYPE=RelWithDebInfo but runs into gcc ABI linking errors to one of the dependencies (spdlog) when specifying -DCMAKE_BUILD_TYPE=Release.

After some debugging, I noticed that -DCMAKE_BUILD_TYPE=Release uses a prebuilt package of spdlog from conan servers while -DCMAKE_BUILD_TYPE=RelWithDebInfo builds the dependency locally, possibly because there is no prebuilt package for RelWithDebInfo. Is it possible that it somehow downloads a package that was built with libstdc++ instead of libstdc++11 even though I set the global setting to libstdc++11?

If that analysis is correct and the compiler.libcxx setting is not considered when searching for prebuilt binaries, then that should probably be fixed by adding it to the metadata for prebuilt packages and only downloading and using a prebuilt package if the compiler.libcxx setting matches.

Environment Details (include every applicable attribute)

• Operating System+version: CentOS 7
• Compiler+version: devtoolset-7 (gcc 7)
• Conan version: 1.26.1
• Python version: 3.6

Steps to reproduce (Include if Applicable)

1. Start fresh CentOS 7 docker container and install requirements

$ sudo docker run -ti centos:centos7 /bin/bash
[root@a0098fc49d53]# yum install git wget make centos-release-scl python3 libcurl-devel fuse-devel
[root@a0098fc49d53]# yum install devtoolset-7
[root@a0098fc49d53]# scl enable devtoolset-7 bash
[root@a0098fc49d53]# pip3 install conan
[root@a0098fc49d53]# cd ~
[root@a0098fc49d53]# wget https://github.com/Kitware/CMake/releases/download/v3.14.6/cmake-3.14.6-Linux-x86_64.sh
[root@a0098fc49d53]# chmod +x cmake-3.14.6-Linux-x86_64.sh
[root@a0098fc49d53]# ./cmake-3.14.6-Linux-x86_64.sh
[root@a0098fc49d53]# export PATH=$PATH:/root/bin

2. Inside the docker container, build CryFS with -DCMAKE_BUILD_TYPE=Release

[root@a0098fc49d53]# git clone https://github.com/cryfs/cryfs
[root@a0098fc49d53]# git checkout f179b8c4a433a4ebf893fe43d82fd3bd2c4fe513
[root@a0098fc49d53]# cd cryfs
[root@a0098fc49d53]# mkdir build
[root@a0098fc49d53]# cd build
[root@a0098fc49d53]# cmake .. -DCMAKE_BUILD_TYPE=Release
[root@a0098fc49d53]# make

3. Notice that it fails.
4. Try the same thing with -DCMAKE_BUILD_TYPE=RelWithDebInfo and notice that it succeeds.

Logs (Executed commands with output) (Include/Attach if Applicable)

Run with -DCMAKE_BUILD_TYPE=Release fails: https://pastebin.com/wMuQAEdL
Run with -DCMAKE_BUILD_TYPE=RelWithDebInfo succeeds: https://pastebin.com/979iSsxp

[smessmer]

I confirmed that adding the BUILD all setting to conan_cmake_run (which then calls conan install --build instead of conan install --build missing) also fixes the problem, so the issue is definitely that it gets the wrong prebuilt package from the servers.

[memsharded]

I am having a look and even if in the command line appears 2 times -s compiler.libcxx=libstdc++ -s compiler.libcxx=libstdc++11, the actual profile in both cases is:

Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=7

So it doesn't seem that the root cause is the libcxx setting, but it is possible that the ConanCenter pre-built packages are incompatible with CentOS7 (glibc issues). ConanCenter is doing some changes to the docker images (https://blog.conan.io/2020/06/17/Conan-Docker-Images.html) to improve this situation, but it is likely that packages have not still been rebuilt with these images yet.

Please @uilianries @SSE4 could you have a look?

[smessmer]

I compared the compiled dependency library (libspdlog.a) using `nm -C` and it is the gcc ABI issue. In the working build (i.e. when built locally), it has the new GCC ABI `std::__cxx11::string` and when downloading it as a package from Conan servers, it has the old GCC ABI `std::string`.

…

On June 29, 2020 01:48:14 James ***@***.***> wrote: I am having a look and even if in the command line appears 2 times -s compiler.libcxx=libstdc++ -s compiler.libcxx=libstdc++11, the actual profile in both cases is: Configuration: [settings] arch=x86_64 arch_build=x86_64 build_type=Release compiler=gcc compiler.libcxx=libstdc++11 compiler.version=7 So it doesn't seem that the root cause is the libcxx setting, but it is possible that the ConanCenter pre-built packages are incompatible with CentOS7 (glibc issues). ConanCenter is doing some changes to the docker images (https://blog.conan.io/2020/06/17/Conan-Docker-Images.html) to improve this situation, but it is likely that packages have not still been rebuilt with these images yet. Please @uilianries @SSE4 could you have a look? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[memsharded]

Then this could be a bug in the spdlog recipe in ConanCenter, that is not following correctly the libcxx value, when it is not the system default or something, it is worth checking that possibility too.

[SSE4]

from the log, it downloads the right package c0fb34b00d4c04a33793ba290326808d0fd5de33:

271     Package_ID: c0fb34b00d4c04a33793ba290326808d0fd5de33
272         [options]
273             fPIC: True
274             header_only: False
275             no_exceptions: False
276             shared: False
277             wchar_filenames: False
278             wchar_support: False
279         [settings]
280             arch: x86_64
281             build_type: Release
282             compiler: gcc
283             compiler.libcxx: libstdc++11
284             compiler.version: 7
285             os: Linux
286         [requires]
287             fmt/6.0.0:66c5327ebdcecae0a01a863939964495fa019a06
288         Outdated from recipe: True

checking the package itself, it seems to be really built with libstdc++11:

conan@700acaae5e1d:~$ nm -a /home/conan/.conan/data/spdlog/1.4.2/_/_/package/c0fb34b00d4c04a33793ba290326808d0fd5de33/lib/libspdlog.a | grep err_handler_ | c++filt
0000000000003350 T spdlog::logger::err_handler_(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
0000000000000000 b spdlog::logger::err_handler_(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::err_counter
0000000000000008 b spdlog::logger::err_handler_(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::last_report_time
0000000000000020 b spdlog::logger::err_handler_(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::mutex
                 U spdlog::logger::err_handler_(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)

that seems like the application itself (cryfs-stats) isn't been built with right ABI.

going to check in docker and provide more details.

[SSE4]

okay, digging more into the details, it seems like _GLIBCXX_USE_CXX11_ABI has no effect on devtoolset-7 (!!!):
https://stackoverflow.com/questions/49393888/how-can-i-use-the-new-c-11-abi-with-devtoolset-7-on-centos-rhel
https://stackoverflow.com/questions/47951367/d-glibcxx-use-cxx11-abi-1-ineffective-for-devtoolset-7-on-centos-7
https://bugzilla.redhat.com/show_bug.cgi?id=1546704
quote:

Define _GLIBCXX_USE_CXX11_ABI gets ignored by gcc in devtoolset-7.
As a result the old CoW-String Implementation is used. 

We've tried hard, but it is not possible to support this, neither on RHEL6 nor on RHEL7, which is why it is forcefully disabled.  It will work in RHEL8 (and be the default there as well).

thus, compiler.libcxx=libstdc++11 is ignored on devtoolset-7. the setting basically has no effect in this case, and building spdlog or any other library with such setting still produces binaries with old ABI.

[SSE4]

given the above information, I doubt there is anything could be done on conan side. this appears to be a limitation of devtoolset-7 itself, and I suppose we can't do anything with broken toolchains silently ignoring certain flags. in such cases settings and package ids will be unreliable :(

the best user(s) can do, is installing with something like -s spdlog:compiler.libcxx=libstdc++, or just use libstdc++ everywhere and be happy.

[memsharded]

Great investigation, thanks very much @SSE4 !

Then I need to understand @smessmer feedback:

I compared the compiled dependency library (libspdlog.a) using nm -C and
it is the gcc ABI issue. In the working build (i.e. when built locally), it
has the new GCC ABI std::__cxx11::string and when downloading it as a
package from Conan servers, it has the old GCC ABI std::string.

Shouldn't it be the opposite? The locally built is using libstdc++, while the one retrieved from ConanCenter is with libstdc++11?

[SSE4]

yes, it's the opposite

[smessmer]

oh wow, thanks for the detailed analysis. You're absolutely right, it's the opposite, I didn't notice that.

Could conan detect if a toolchain ignores _GLIBCXX_USE_CXX11_ABI? It could for example compile a small compiler_test.cpp file similar to CheckCXXSourceCompiles, and report an error if the generated symbols mismatch the expectation? A better error message would go a long way here.

[SSE4]

technically, this is possible, conan could detect broken toolchains and report warnings about them
but I am not sure if it's ideologically conan's responsibility to check for bugs in toolchains
@memsharded WDYT?

[memsharded]

Your PR #5740 has been merged, for CONAN_V2_MODE.
I think now it is very complicated, and almost impossible to do it without breaking. But sounds like for Conan 2.0, it would be good to continue your work and check the libcxx real possible values and defaults with an actual compilation, to get as close to 100% as possible the real value of the libcxx of the current toolchain.

[SSE4]

okay, seems like there is a still room for improvement, we can detect an actual libcxx value in the new detector.
I guess there were already some issues asking for that, so it's worth to take a look.

[memsharded]

I have implemented in #7776 a real detector of libcxx, which should fix this in the future. At the moment is only CONAN_V2_MODE, because it would be breaking.

It will only be applied for auto-detection of profile. Users manually provided profiles will not be verified, it is the responsibility of the users to correctly provide their configuration for their toolchain, as it is very challenging to do a robust verification of profiles, as they might depend on many factors, can require environment settings, build_requires and other stuff to make sense.