Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix PCRE2_DEBUG for multi-configuration builds #542

Merged
merged 1 commit into from
Nov 11, 2024

Conversation

NWilson
Copy link
Member

@NWilson NWilson commented Oct 30, 2024

On Windows (MSVC) and Mac (Xcode), one config.h file is shared by both Debug and Release builds.

So, I think we should instead pass -DPCRE2_DEBUG on the commandline instead of including it in the config file, and set it to true only for Debug builds.

This ensures that CMake users on Windows & Mac will get their assertions checked automatically.

Copy link
Contributor

@carenas carenas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically the same is true of the "Ninja Multi-Config" generator that is available in more platforms and will be a good option if added to the CI (ex: changing the Perl job in dev.yml)

This will also require further changes to CMakeFiles.txt so it can generate a working pcre2_test.sh to be able to complete ctest.

@NWilson
Copy link
Member Author

NWilson commented Nov 1, 2024

Oh yes, the CTest integration has been done a little unusually.

On Linux I usually run the pcre2 tests with "ninja test" or "make test" (from the Cmake build directory - I never use autoconf if I can avoid it). On windows I use "Cmake --build . --target RUN_TESTS". So those variations work at least.

I can test with the multi config ninja and make sure it works.

CMakeLists.txt Show resolved Hide resolved
@NWilson
Copy link
Member Author

NWilson commented Nov 6, 2024

I've added support for "Ninja Multi-Config" and tested on Linux & Windows. The Windows support was already working (due to the support in pcre2_test.bat) but I had to tweak pcre2_test.sh a little bit. Not too hard, all works nicely.

CMakeLists.txt Show resolved Hide resolved
@carenas
Copy link
Contributor

carenas commented Nov 7, 2024

Mac (Xcode)

Tested with Xcode 15.2 (which I never used before) but doesn't seem to be building anything in the IDE (although it says it succeeds)

it builds fine in the command line (even using the Xcode generator), but using ctest failed to find the binaries for both pcre2test and pcre2grep unless I use ctest -C Debug . while in the build directory.

@NWilson
Copy link
Member Author

NWilson commented Nov 7, 2024

Interesting. I don't have a Mac or Xcode to test with.

On Windows, I can run tests with either cmake --build . --target test (when using ninja Multi-config), or cmake --build . --target RUN_TESTS (when using native Visual Studio cmake backend); or alternatively using ctest -C Debug. However just ctest on its own doesn't work.

@NWilson NWilson force-pushed the user/niwilson/cmake-debug branch from ad33685 to 9488837 Compare November 8, 2024 11:07
@NWilson
Copy link
Member Author

NWilson commented Nov 8, 2024

Updated commit description:


Fix PCRE2_DEBUG for multi-configuration builds

  • Add support for Ninja Multi-Config generator on all platforms
  • Ensure one build job is using CMake 3.15
  • Provide a build type (Debug or RelWithDebInfo) on all CMake builds
  • Ensure PCRE2_DEBUG=ON is set for: dev.yml builds which use the
    RelWithDebInfo configuration

@@ -77,7 +77,7 @@ jobs:
submodules: true

- name: Configure
run: cmake -DPCRE2_SUPPORT_JIT=ON -DPCRE2_BUILD_PCRE2_16=ON -DPCRE2_BUILD_PCRE2_32=ON -DCMAKE_OSX_ARCHITECTURES='arm64;x86_64' -DCMAKE_C_FLAGS='-Wall -Wextra' -DCMAKE_COMPILE_WARNING_AS_ERROR=ON -B build
run: cmake -DPCRE2_SUPPORT_JIT=ON -DPCRE2_BUILD_PCRE2_16=ON -DPCRE2_BUILD_PCRE2_32=ON -DCMAKE_OSX_ARCHITECTURES='arm64;x86_64' -DCMAKE_C_FLAGS='-Wall -Wextra' -DCMAKE_COMPILE_WARNING_AS_ERROR=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo -B build
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think all the jobs in build.yml should be using RelWithDebInfo (for the jobs using CMake of course). And, we want PCRE2_DEBUG to be left to its default, which will disable the assertions.

This mirrors the configuration which we expect customers and downstream packages to be using.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thinking was that we use the default of the generator used since they are meant to match what a developer runs localy, but you probably have a lot more experience with cmake as a developer than any of us.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never run CMake without specifying a CMAKE_BUILD_TYPE. Some generators (like the Makefiles backend) just do weird things, and generate configuration that nobody really wants.

I guess we could have one example of that in the dev.yml... but it would only apply on Linux anyway, and we're expecting most Linux users to be using autoconf instead. So I'd just simplify things, and only test on actual proper CMake configurations (Debug, Release, RelWithDebInfo).

Copy link
Contributor

@carenas carenas Nov 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize I misread the context.

My comment was about jobs in dev.yml; for build.yml we want to mirror what end users would do for production, so while I think you are correct RelWithDebInfo is valid Release might be better (which will also test -O3 vs -O2 with gcc/clang)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I've personally always considered "RelWithDebInfo" to be the best for actual production releases.

To start with, no-one should be shipping binaries without having the debug info stored somewhere. Linux distributions always build with debug info, and ship the ELF+DWARF binaries in special "-debug" variant of the package, then run strip to publish the ordinary packages. That way, when users report a crash, you can symbolise it. Commercial vendors do the same thing (on all the companies and projects I've worked on) - build with symbols, store them, and ship the stripped binaries.

I also (personally) dislike O3 because it does too much inlining with GCC. Binary size for a C++ app can double, with no noticeable performance increase. For apps that are bigger than 10-20MB binary size, -Os is often the fastest in fact, because -O3 does so much loop unrolling it forces the code out of the L1 cache.

But yes, I totally acknowledge that people ship with -O2, -O3, or -Os.

Here's what I'll do: change build.yml to use Release (-O3), and in build.yml let's make sure we run the unit tests for each of -O2, -O3, and -Os.

.github/workflows/dev.yml Outdated Show resolved Hide resolved
Copy link
Member Author

@NWilson NWilson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'm happy with this and ready for it to be merged, as long as Carlo's happy.

CMakeLists.txt Outdated
MESSAGE(STATUS " Install prefix .................... : ${CMAKE_INSTALL_PREFIX}")
MESSAGE(STATUS " C compiler ........................ : ${CMAKE_C_COMPILER}")
if (CMAKE_CONFIGURATION_TYPES)
FOREACH (config IN LISTS CMAKE_CONFIGURATION_TYPES)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: can we use lowercase foreach?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, done.

But someone really needs to just run an autoformatter over the file, it's totally inconsistent at the moment. I can do that in a follow-up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the inconsistency is by design, see #115

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well 23 instances of "if" lowercase and 124 instances of "IF" can't be by design. The formatting should at least be consistent. It's not important though, given it has absolutely no functional impact.

Making it more idiomatic CMake would be nice in future, there are some good ideas in that thread.

Copy link
Contributor

@carenas carenas Nov 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I meant "by design" ironically, since that file has been hacked without any for way too long IMHO.

Indeed, I have to admit I was a little surprised no one mentioned taking over maintainer-ship of at least this file as part of our previous call for things that were needed to integrate with Excell

CMakeLists.txt Show resolved Hide resolved
RunTest Outdated Show resolved Hide resolved
@NWilson NWilson force-pushed the user/niwilson/cmake-debug branch from 9488837 to c849d82 Compare November 9, 2024 09:12
@zherczeg
Copy link
Collaborator

Is there anything more to do here, or is the patch ready?

@NWilson
Copy link
Member Author

NWilson commented Nov 11, 2024

Carlo has asked for a change to the flags we test with (-O3 rather than -O2 in build.yml). I'll do that quickly, then it'll be ready.

@NWilson NWilson force-pushed the user/niwilson/cmake-debug branch 4 times, most recently from 397be09 to bff90f5 Compare November 11, 2024 10:31
* Add support for Ninja Multi-Config generator on all platforms
* Ensure one build job is using CMake 3.15
* Provide a build type (Debug or RelWithDebInfo) on all CMake builds
* Ensure PCRE2_DEBUG=ON is set for: dev.yml builds which use the
  Release configuration
@NWilson NWilson force-pushed the user/niwilson/cmake-debug branch from 9139a36 to 69c6712 Compare November 11, 2024 11:17
@NWilson
Copy link
Member Author

NWilson commented Nov 11, 2024

OK, ready to merge.

I have encountered two issues while fiddling for a while with different compile options:

  • The GCC build doesn't pass with -O0 -Werror.
  • The Clang ASAN build doesn't pass with -O0 either. There's a stack overflow in the recursive JIT function scan_prefix, which uses a little more stack space in -O0 mode, and the recursion overflows the stack. The failing pattern is /X?(R||){3335}/ in testinput1.

These are pre-existing issues, so I've left them. We can address in a future PR.

Copy link
Collaborator

@zherczeg zherczeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@carenas
Copy link
Contributor

carenas commented Nov 11, 2024

LGTM

meant to hit the merge button, but failed somehow?

@zherczeg
Copy link
Collaborator

zherczeg commented Nov 11, 2024

LGTM is looks good to me. Comes from my WebKit times.

@zherczeg zherczeg merged commit a678783 into PCRE2Project:master Nov 11, 2024
17 checks passed
@NWilson NWilson deleted the user/niwilson/cmake-debug branch December 17, 2024 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants