Add coverage job

[NWilson]

Fixes #555

[zherczeg]

This looks useful.

pcre2_jit_compile.c.html#L102: 58.9k #define MACHINE_STACK_SIZE 32768

Interesting that a const has coverage. Some macros has coverage, others don't, so probably not all code is fully "covered" by the tool :)

[NWilson]

I'd guess that constants which are built into the binary will be counted as "code" and generate coverage.

Macros which are only used by the preprocessor won't generate coverage. They have to actually write out some text which is seen as C code by the compiler.

I'd be interested in trying out the "Coveralls" or "CodeCov" services, which look like good ways to track coverage over time, coverage diff in a PR, and so on.

[carenas]

it might be better to move this job into their own "tools.yml" in te long run, though

[NWilson]

it might be better to move this job into their own "tools.yml" in te long run, though

I feel like code-scanning tools (like CodeQL, or Clang Static Analyzer) should be in their own jobs.

However, code coverage should be part of the Release "Build" pipeline (not the "Dev" pipeline) because it's actually a fixed, reportable artifact of the release process: "we built the code; we ran the tests; the tests covered 85% of the code; now we release the binaries".

[carenas]

I think you are seeing too much on those names, I created the files as ways to automte the work that Philip does on each release, and build.yml was the first; with a simple objective to make sure the code actually builds in the few pltforms we could easily use through github actions. I was specially motivated to make sure macOS would work because neither Philip or Zoltan have access to it and the build was broken by a contributor merged patch and almost released once.

dev.yml came later after a lot more contributor provided patches were found to introduce problems that I thought we could have prevented with feedback at PR time. For those, the ideas was to cover as much of the code (hence why DEBUG is on) so we don't have blind spots, and probably a comparative code coverage job would be also sensible there.

Eitherway, we are a very small team, none of us doing this as "work", so our perspectives will obviously differ, and indeed some people assume that our jobs are a serious CI/CD when they are meant only as additional automation on top of developer directed tests, but feel free to tweak them for your needs as well.