Feature: improve compiler detection

[SSE4]

Changelog: Feature: improve compiler detection for CONAN_V2_MODE.
Docs: conan-io/docs#1789
closes: #4322

things to test:

• Mac apple clang
• Mac apple clang (gcc wrapper)
• Mac authentic clang (brew install llvm)
• Mac authentic gcc (brew install gcc)
• Mac LLVM-GCC (from old Xcode)
• Linux gcc
• Linux clang
• Windows Visual Studio
• Windows gcc (MinGW)
• Windows gcc (MSYS)
• Windows gcc (Cygwin)
• Windows clang (clang-cl)
• Windows clang (MSYS)
• Windows QCC (QNX)
• Windows icc (Intel)
• Linux icc (Intel)
• Mac icc (Intel)
• ccache (compiler wrapper)
• distcc (compiler wrapper)

what else?

• Refer to the issue that supports this Pull Request.
• If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
• I've read the Contributing guide.
• I've followed the PEP8 style guides for Python code.
• I've opened another PR in the Conan docs repo to the develop branch, documenting this one.

^{Note: By default this PR will skip the slower tests and will use a limited set of python versions. Check here how to increase the testing level by writing some tags in the current PR body text.}

[SSE4]

ready for the review, I have tested all configurations mentioned above

[lasote]

I'm a bit surprised that using the g++ for fixing the detection requires a new class of more than 200 lines and several changes among 6 files.
As we always say, if this is a refactor or an improvements it should be done separately.
Cannot it be just fixed without implementing new things?

[sergeyklay]

Any news on this?

[lasote]

@sergeyklay for me this PR is not valid for closing the issue. @SSE4 if you want to move this forward open a PR with a simpler fix and later we can discuss if we want to introduce other improvements and why.

[SSE4]

rebased, review comments addressed

[memsharded]

This PR increases our test suite time by 50% in Linux, around 20% in others, depending on the run.

I am sorry, but this is not possible. Relying on building such a small app for auto-detecting compilers should probably not be the way.

Detecting the compiler versions was not a big issue so far (for the current model). We need to think if we want to follow this approach. In case we wanted, then probably:

• Use it only for some extreme cases, when it is absolutely necessary
• Make sure there is a fast and easy alternative for testing

So we need time to discuss this.

[SSE4]

interesting, I'll take a look at why does it slow down so much. tests I've added are mocked and shouldn't cause actual compiler commands to be run. so I am puzzled right now.

[memsharded]

It is interesting, but I saw some differences in the CI previous job, I have relaunched in https://conan-ci.jfrog.info/blue/organizations/jenkins/ConanTestSuite/detail/PR-5740/5/pipeline/180, and still see consistently slower builds than other PR. Could be a pure CI issue, sure, but I am curious to understand why the big difference (compare for example with recent PR build: https://conan-ci.jfrog.info/blue/organizations/jenkins/ConanTestSuite/detail/PR-7284/4/pipeline/179)

[SSE4]

@memsharded build you have referenced is definitely slower for Linux disabled revisions jobs - it takes 9m instead of usual 5-8m. I'll continue to investigate why does it take slower, maybe there is still a room for improvement.

[jgsogo]

Please, if time is a concern for the test suite, let's fix it in the test suite, not in the codebase. We cannot add legacy or complexity to the codebase because of the tests. Caches are not easy to manage, I'm afraid the environment could change and the detection should run again to detect a different compiler. Can it be an issue?

If we are trying to stabilize the conan_api and document it, then we need to think about Conan being a library, the environment will change and cache won't be cleaned

If we agree with the implemented functionality (I think it is a huge improvement), then we can figure out how to minimize this extra time in the tests...

[SSE4]

@jgsogo rebased, added CONAN_V2_MODE

[jgsogo]

With the v2_mode there can be no risk if want to add this feature... but we need to think how can we test this feature if this becomes the default in conan v2

[SSE4]

Q: why parsing gcc --version / gcc --dumpversion is weak?
A: there are several reasons:

• current regexp for gcc --version or gcc --dumpversion is "([0-9]+(\.[0-9])?)" is `` (

  conan/conans/client/conf/detect.py

  Line 28 in 02ccf9e

  installed_version = re.search("([0-9]+(\.[0-9])?)", out).group()

  ). it will obviously match 10.0 correctly, but not 10.11 (if it will be ever released). we also had troubles in the past (see Fix: parse GCC 10 version from command line #6551)
• GCC might be built with option --with-gcc-major-version-only
  (see issue Failure to detect gcc when gcc is compiled with --with-gcc-major-version-only #1127). some distros (e.g. Fedora are already using that). our regex may fail to match in this case.
• GCC might be built with option --with-pkgversion. this is something typical for custom toolchains (mobile, embedded, etc.). if output contains something like Android (5220042 based on r346389c) clang version 8.0.7 (it's real version string), then detection may result in an incorrect version number.
• output might be localized (e.g. to Russian, Chinese, etc.), which could cause some troubles to be correctly decoded by python in certain systems.

[memsharded]

This could easily be a namedtuple, but it is ok, no need to change it now

[jgsogo]

We had some concerns about testing, any idea/plan about how to test this new detection logic if it becomes the 2.0 default?

[danimtb]

I guess this is something to consider for the Conan CI itself, like having different images with environments to test compilers, build helpers, tools...