Code coverage can be confusing

[cijothomas]

Originally from here:
#2915 (comment)

Copying the content:

The current code coverage result is not stable (each run will have slightly different results) and is not considered as a blocker for the PR.
I guess the codecov issue could either be related to the tool itself (e.g. some bug, although I think it's less likely to be the case) or the how the test cases were designed (e.g. some test cases require RPC interactions or some other operations that are sensitive to timing/race).

Given this is not the first time folks asked about this, I'm thinking - maybe for short term we should change the codecov CI so it only shows the coverage percentage but always return Success? In this way it still serves the purpose of giving us a rough estimation of the test coverage, but won't introduce confusion to the contributors? https://github.com/orgs/open-telemetry/teams/dotnet-approvers https://github.com/orgs/open-telemetry/teams/dotnet-maintainers what do you think?

[cijothomas]

Discussed on 2/22/22 SIG meeting, and we agree to do the below suggestion:

maybe for short term we should change the codecov CI so it only shows the coverage percentage but always return Success?

Any volunteers to help achieve this?

[cijothomas]

@eddynaka checking if you have some cycles to help here :)