Estimate code coverage for the current frontend codebase

[jiji14]

Estimate code coverage for the current frontend codebase

We've made significant progress in setting up unit tests for e-emission-phone and are actively writing test cases as we overhaul Angular services. To gauge the effectiveness of our test suite, we aim to estimate the code coverage.

You can find test codes here

Related Issue

Estimate code coverage for the current server codebase

I've researched 3 well-known code coverage tools that work with jest and GitHub actions.

jest-coverage

Jest-coverage is a tool that extends Jest to provide code coverage reports. To generate code coverage reports using Jest, we can run tests with coverage using the --coverage flag.

[ Report Sample ]

• This screenshot is our test case.

1. Function coverage: Determines if each function has been called.
2. Statement coverage: Checks if each statement in the program has been executed.
3. Branch coverage: Evaluates whether each branch of control structures, like if and case statements, has been executed.
4. Line coverage: Verifies if each executable line in the source file has been executed.

[ Pros ]

1. Integration with Jest. (No need for complex setup)

[ Cons]

1. Limited to Jest, limiting integration with other testing frameworks.
2. Reports are somewhat simple compared to other options

[ Resources ]
jest-coverage
stackoverflow - jest-coverage issue

Codecov

Codecov is an all-in-one code coverage reporting solution that focuses on integration and encourages healthy pull requests. It delivers coverage metrics directly into the modern workflow, particularly in pull requests.

[ Report Sample ]

1. hit: indicates that the source code was executed by the test suite.
2. miss: indicates that the source code was not executed by the test suite.
3. partial: indicates that the source code was not fully executed by the test suite; there are remaining branches that were not executed.

[ Pros ]

1. Forever Free for Open Source 😸
2. Provides detailed and customizable reports

[ Cons]

1. Initial setup can be somewhat complex and may require additional time.

[ Resources ]
codecov.io

Istanbul

Istanbul instruments your ES5 and ES2015+ JavaScript code with line counters, allowing you to track code coverage based on unit tests.

[ Pros ]

1. Offers more options for report formats and configuration compared to jest-coverage.

[ Cons]

1. For TypeScript, transpilation to JavaScript is required, introducing an additional step.

[ Report Sample ]

[ Resources ]
Gudie for instanbul

Jest-coverage VS Codecov VS Istanbul

• I did not consider JS Coverage as an option because it lacks support for CI/CD integration, and its last update is outdated.

[ Resources ]
code-coverage-js-in-2023

I am inclined towards Codecov because it is free for open-source projects and offers various features with user-friendly and readable reports.

[shankari]

@jiji14 before we use any third party (hosted) tools, we have to get approval from NREL IT.
GitHub is pre-approved, which is why we are able to use GitHub and GH Actions.
I agree that codecov seems cool, but we should get started with the process to request approval immediately
It may be that it is also pre-approved (best case scenario!)

If not, Jest seems pretty good, but does it cover end-to-end testing? I am not sure that code coverage tools will do well with end-to-end testing anyway since they work at the code lines level.

[jiji14]

@shankari
It does not cover end-to-end testing because code coverage tools check the source code line by line, as you mentioned. End-to-end testing is a UI-level automation, not code-level. However, there are other tools available for checking end-to-end testing coverage (although I haven't delved deeply into this).

I also agree that jest-coverage is a good alternative if we can't get approval with codecov.

[shankari]

@jiji14 sounds good. What additional feedback would you like from me?
I think that the next step is to contact IT for a "solution exception" for codecov and update the issue with what they find.

[jiji14]

@shankari
Nothing for now. I will contact IT and will update how it goes!

[jiji14]

@shankari

I need to submit a Security Impact Analysis. I am wondering who the System owner will be and how many members will have access to the account.

[shankari]

@jiji14 I am 99% sure you don't need to submit an SIA. The SIA is for systems that we want to run internally, which is why they are asking for a system owner. We want to use an external service, which would be a technology exception. Please cc me on the discussion with IT if needed and I can clarify if there are any questions.

[jiji14]

@shankari

I received approval from the IT team. Should I reach out to Codecov directly to request access for the open-source project? Additionally, I think you should sign up with your GitHub handle to set up Codecov in the e-mission-phone project. After I signed up with my GitHub handle, I could only see the repositories that I owned. You can find the setup documentation here.

[shankari]

@jiji14 can you add me to the ticket related to the this, so I can have the approval for the record?
I will sign up for codecov over the weekend 😄

[shankari]

@jiji14 thanks for adding me to the ticket. Signing up for codecov now...

[shankari]

Signed up!

[jiji14]

@shankari
Can you please add the repository token to the e-mission-phone repository?

Here are the steps:

1. Sign in to the Codecov website.

2. Click on the e-mission-phone repository in dashboard.

3. Copy the token.

4. Go to the repository settings and paste the token in Settings > Security and Variables > Actions.
   

After you add the token, I will submit Codecov Github action PR.
Thank you!

[shankari]

Added the token