rm gcovr dep implement json html report (#1099)

* rm gcovr dep implement json html report * spaces protection for gcov executable * fix html report * update docs html report remove gcovr
espressif · Jan 9, 2024 · 4723524 · 4723524
1 parent e7dbfc2
commit 4723524
Show file tree

Hide file tree

Showing 21 changed files with 595 additions and 272 deletions.
diff --git a/.github/actions/idf/entrypoint.sh b/.github/actions/idf/entrypoint.sh
@@ -8,7 +8,6 @@ export OLD_PATH=$PATH
 cd /github/workspace
 
 pip install --upgrade pip
-pip install --constraint espidf.constraints.txt  -r requirements.txt
 pip install --constraint espidf.constraints.txt -r esp_debug_adapter/requirements.txt
 
 export GIT_VERSION=$( echo "$a" | echo $(git --version) | sed -nre 's/^[^0-9]*(([0-9]+\.)*[0-9]+).*/\1/p')

diff --git a/.github/actions/idf/ui-entrypoint.sh b/.github/actions/idf/ui-entrypoint.sh
@@ -8,7 +8,6 @@ export OLD_PATH=$PATH
 cd /github/workspace
 
 pip install --upgrade pip
-pip install --constraint espidf.constraints.txt -r requirements.txt
 pip install --constraint espidf.constraints.txt -r esp_debug_adapter/requirements.txt
 
 export GIT_VERSION=$( echo "$a" | echo $(git --version) | sed -nre 's/^[^0-9]*(([0-9]+\.)*[0-9]+).*/\1/p')

diff --git a/docs/COVERAGE.md b/docs/COVERAGE.md
@@ -1,24 +1,22 @@
-# Using **gcov** and **gcovr** to Provide Code Coverage
+# Using **gcov** to Provide Code Coverage
 
-Source code coverage is data indicating the count and frequency of every program execution path that has been taken within a program’s runtime. [Gcov](https://en.wikipedia.org/wiki/Gcov) is a GCC tool that, when used in concert with the compiler, can generate log files indicating the execution count of each line of a source code. The [Gcovr](https://gcovr.com/) tool is utility for managing Gcov and generating summarized code coverage results.
+Source code coverage is data indicating the count and frequency of every program execution path that has been taken within a program’s runtime. [Gcov](https://en.wikipedia.org/wiki/Gcov) is a GCC tool that, when used in concert with the compiler, can generate log files indicating the execution count of each line of a source code.
 
 Please read [GCOV Code coverage](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/app_trace.html#gcov-source-code-coverage) to learn more about code coverage with gcov in ESP-IDF projects.
 
 ## Requirements
 
 Your ESP-IDF project should be configured to generate gcda/gcno coverage files using gcov as shown in [GCOV Code Coverage](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/app_trace.html#gcov-source-code-coverage). Please take a look at the [ESP-IDF gcov Example](https://github.com/espressif/esp-idf/tree/master/examples/system/gcov) as example project.
 
-This extension requires `gcovr` to generate JSON and HTML reports from generated files. This is installed as part of this extension [SETUP](./SETUP.md).
-
-Make sure you had properly configure the toolchain in `idf.customExtraPaths` or in your environment variable PATH since the gcov executable used is `{TOOLCHAIN_PREFIX}-gcov` (replacing TOOLCHAIN_PREFIX for your IDF_TARGET toolchain prefix) and `gcovr` exists in the same directory as your `${idf.pythonBinPath}` path.
+Make sure you had properly configure the toolchain in `idf.customExtraPaths` or in your environment variable PATH since the gcov executable used is `{TOOLCHAIN_PREFIX}-gcov` (replacing TOOLCHAIN_PREFIX for your IDF_TARGET toolchain prefix).
 
 The **ESP-IDF: Configure Project SDKConfig for Coverage** command can set required values in your project SDKConfig to enable Code Coverage.
 
 ## Editor Coverage
 
-For the text editor highlighting, the **ESP-IDF: Add Editor Coverage** command execute a child process with `gcovr -r . --gcov-executable {TOOLCHAIN_PREFIX}-gcov --json`. You can remove the coverage highlight with **ESP-IDF: Remove Editor Coverage**.
+For the text editor highlighting, the **ESP-IDF: Add Editor Coverage** command execute a child process with `{TOOLCHAIN_PREFIX}-gcov` to parse all gcda generated files and generate a JSON report. You can remove the coverage highlight with **ESP-IDF: Remove Editor Coverage**.
 
-> **NOTE:** This assumes you had configure your extension with Xtensa toolchain in `idf.customExtraPaths` and installed the `gcovr` from Debug Adapter's python requirements.
+> **NOTE:** This assumes you had configure your extension with Xtensa or RISC-V toolchain in `idf.customExtraPaths`.
 
 For the text editor, we use the json object generated by the previous command to highlight each line if it is covered or if it is not. We don't highlight noncode lines.
 
@@ -30,8 +28,6 @@ You can customize highlight color using the extension settings. Visual Studio co
 
 ## HTML report
 
-The **ESP-IDF: Get HTML Coverage Report for Project** execute a child process with command `gcovr -r . --gcov-executable {TOOLCHAIN_PREFIX}-gcov --html` for the HTML report.
-
-> **NOTE:** This assumes you had configure your extension with Xtensa toolchain in `idf.customExtraPaths` and installed the `gcovr` from Debug adapter's python requirements.
+The **ESP-IDF: Get HTML Coverage Report for Project** execute a child process with `{TOOLCHAIN_PREFIX}-gcov` to parse all gcda generated files for the HTML report.
 
-With the generated HTML from `gcovr`, a Webview Panel is launched to show the gcov html report.
+> **NOTE:** This assumes you had configure your extension with Xtensa toolchain in `idf.customExtraPaths`.
diff --git a/docs/SETUP.md b/docs/SETUP.md
@@ -46,7 +46,7 @@ Setup wizard provides 3 choices:
 - **Use Existing Setup**: This option will show previous setup used in the extension and existing setup if:
   1. `esp-idf.json` is found in the current `idf.toolsPath` (MacOS/Linux users) or `idf.toolsPathWin` (Windows users). This file is generated when you install ESP-IDF with the [IDF Windows Installer](https://github.com/espressif/idf-installer) or using [IDF-ENV](https://github.com/espressif/idf-env) or this extension.
 
-> **NOTE:** When running any of these choices, the setup wizard will install ESP-IDF Python packages, this extension (`EXTENSION_PATH`/requirements.txt and ESP-IDF debug adapter (`EXTENSION_PATH`/esp_debug_adapter/requirements.txt) python packages where `EXTENSION_PATH` is located in:
+> **NOTE:** When running any of these choices, the setup wizard will install ESP-IDF Python packages and ESP-IDF debug adapter (`EXTENSION_PATH`/esp_debug_adapter/requirements.txt) python packages where `EXTENSION_PATH` is located in:
 
 - Windows: `%USERPROFILE%\.vscode\extensions\espressif.esp-idf-extension-VERSION`
 - Linux & MacOSX: `$HOME/.vscode/extensions/espressif.esp-idf-extension-VERSION`
@@ -118,10 +118,6 @@ where:
 
 Make sure to install the extension and extension debug adapter Python requirements by running the following commands in your terminal:
 
-```
-PYTHON_INTERPRETER -m pip install --upgrade -r EXTENSION_PATH/requirements.txt
-```
-
 ```
 PYTHON_INTERPRETER -m pip install --upgrade -r EXTENSION_PATH/esp_debug_adapter/requirements.txt
 ```

diff --git a/docs/tutorial/code_coverage.md b/docs/tutorial/code_coverage.md
@@ -2,8 +2,6 @@
 
 ## Requirements
 
-This extension's code coverage feature uses the `gcovr` python package which is included in the extension's requirements.txt and installed during [Install tutorial](./install.md) as part of this extension ESP-IDF Debug Adapter Python Requirements when following the **ESP-IDF: Configure ESP-IDF Extension** command.
-
 Your ESP-IDF project should be configured to generate `gcda/gcno` coverage files using `gcov`. Please read [GCOV Code Coverage](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/app_trace.html#gcov-source-code-coverage) to learn more about code coverage with GCOV in ESP-IDF projects.
 
 Please take a look at [Coverage](../COVERAGE.md) for more information about code coverage in this extension.
@@ -79,6 +77,6 @@ Please review [Settings](../SETTINGS.md) to see how to modify these configuratio
 
 ## Troubleshooting
 
-Make sure you had properly configure the required toolchain in `idf.customExtraPaths` or in your environment variable PATH since the GCOV executable used is `{TOOLCHAIN_PREFIX}-gcov` (replacing `TOOLCHAIN_PREFIX` for your `IDF_TARGET` toolchain prefix) and `gcovr` exists in the same directory as your `${idf.pythonBinPath}` path.
+Make sure you had properly configure the required toolchain in `idf.customExtraPaths` or in your environment variable PATH since the GCOV executable used is `{TOOLCHAIN_PREFIX}-gcov` (replacing `TOOLCHAIN_PREFIX` for your `IDF_TARGET` toolchain prefix).
 
-An easy way is to verify this is to execute **ESP-IDF: Open ESP-IDF Terminal** and type `{TOOLCHAIN_PREFIX}-gcov --version` and `gcovr --version`.
+An easy way is to verify this is to execute **ESP-IDF: Open ESP-IDF Terminal** and type `{TOOLCHAIN_PREFIX}-gcov --version`.
diff --git a/docs/tutorial/install.md b/docs/tutorial/install.md
@@ -66,7 +66,7 @@
 
 > **NOTE**: Visual Studio Code has many places where to set configuration settings. This extension uses the `idf.saveScope` configuration setting to determine where to save settings, Global (User Settings), Workspace and WorkspaceFolder. Please review [vscode settings precedence](https://code.visualstudio.com/docs/getstarted/settings#_settings-precedence).
 
-> **NOTE:** the setup wizard will install ESP-IDF Python packages, this extension (`EXTENSION_PATH`/requirements.txt) and ESP-IDF debug adapter (`EXTENSION_PATH`/esp_debug_adapter/requirements.txt) python packages. Make sure that if using an existing python virtual environment that installing these packages doesn't affect your virtual environment. The `EXTENSION_PATH` is:
+> **NOTE:** the setup wizard will install ESP-IDF Python packages and ESP-IDF debug adapter (`EXTENSION_PATH`/esp_debug_adapter/requirements.txt) python packages. Make sure that if using an existing python virtual environment that installing these packages doesn't affect your virtual environment. The `EXTENSION_PATH` is:
 
 - Windows: `%USERPROFILE%\.vscode\extensions\espressif.esp-idf-extension-VERSION`
 - Linux & MacOSX: `$HOME/.vscode/extensions/espressif.esp-idf-extension-VERSION`

diff --git a/media/tutorials/coverage/html_report.png b/media/tutorials/coverage/html_report.png
diff --git a/requirements.txt b/requirements.txt
diff --git a/src/coverage/coverageService.ts b/src/coverage/coverageService.ts
@@ -12,19 +12,18 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-import { sep, join } from "path";
+import { basename, sep, join } from "path";
 import * as vscode from "vscode";
 import {
-  appendIdfAndToolsToPath,
-  dirExistPromise,
   extensionContext,
   getToolchainToolName,
   getWebViewFavicon,
-  spawn,
 } from "../utils";
 import * as idfConf from "../idfConfiguration";
 import { OutputChannel } from "../logger/outputChannel";
 import { Logger } from "../logger/logger";
+import { getGcovData } from "./gcdaPaths";
+import { createGcovHtmlReport, createGcovReportObj } from "./gcovHtmlReport";
 
 const COVERED_FACTOR = 0.9;
 const PARTIAL_FACTOR = 0.75;
@@ -48,96 +47,6 @@ export function getGcovExecutable(idfTarget: string) {
   return getToolchainToolName(idfTarget, "gcov");
 }
 
-export async function getGcovFilterPaths(workspacePath: vscode.Uri) {
-  const espAdfPath =
-    idfConf.readParameter("idf.espAdfPath", workspacePath) ||
-    process.env.ADF_PATH;
-  const espIdfPath =
-    idfConf.readParameter("idf.espIdfPath", workspacePath) ||
-    process.env.IDF_PATH;
-  const espMdfPath =
-    idfConf.readParameter("idf.espMdfPath", workspacePath) ||
-    process.env.MDF_PATH;
-
-  const pathsToFilter: string[] = [];
-
-  const idfExists = await dirExistPromise(espIdfPath);
-  if (idfExists) {
-    pathsToFilter.push(
-      "--filter",
-      join(espIdfPath, "components").replace(/\\/g, "/")
-    );
-  }
-  const adfExists = await dirExistPromise(espAdfPath);
-  if (adfExists) {
-    pathsToFilter.push(
-      "--filter",
-      join(espAdfPath, "components").replace(/\\/g, "/")
-    );
-  }
-  const mdfExists = await dirExistPromise(espMdfPath);
-  if (mdfExists) {
-    pathsToFilter.push(
-      "--filter",
-      join(espMdfPath, "components").replace(/\\/g, "/")
-    );
-  }
-  return pathsToFilter;
-}
-
-export async function buildJson(dirPath: vscode.Uri) {
-  const componentsDir = await getGcovFilterPaths(dirPath);
-  const idfTarget =
-    idfConf.readParameter("idf.adapterTargetName", dirPath) || "esp32";
-  const gcovTool = getGcovExecutable(idfTarget);
-
-  const result = await _runCmd(
-    "gcovr",
-    [
-      "--filter",
-      ".*",
-      ...componentsDir,
-      "--gcov-executable",
-      gcovTool,
-      "--json",
-    ],
-    dirPath.fsPath.replace(/\\/g, "/")
-  );
-  return JSON.parse(result);
-}
-
-export async function buildHtml(dirPath: vscode.Uri) {
-  const componentsDir = await getGcovFilterPaths(dirPath);
-  const idfTarget =
-    idfConf.readParameter("idf.adapterTargetName", dirPath) || "esp32";
-  const gcovTool = getGcovExecutable(idfTarget);
-  const result = await _runCmd(
-    "gcovr",
-    [
-      "--filter",
-      ".",
-      ...componentsDir,
-      "--gcov-executable",
-      gcovTool,
-      "--html",
-    ],
-    dirPath.fsPath
-  );
-  return result;
-}
-
-function _runCmd(cmd: string, args: string[], dirPath: string) {
-  const modifiedEnv = appendIdfAndToolsToPath(vscode.Uri.file(dirPath));
-  return spawn(cmd, args, { env: modifiedEnv, cwd: dirPath })
-    .then((resultBuffer) => resultBuffer.toString())
-    .catch((e) => {
-      const msg = e.error ? e.error.message : e;
-      Logger.error("Error on gcov cmd.\n" + msg, e);
-      OutputChannel.appendLine("Error building gcov cmd.\n" + msg);
-      return "";
-    });
-}
-
 export async function generateCoverageForEditors(
   dirPath: vscode.Uri,
   editors: readonly vscode.TextEditor[],
@@ -168,67 +77,73 @@ export async function generateCoverageForEditors(
         }
       }
 
-      for (const gcovFile of gcovJsonObj.files) {
-        const gcovFilePath = gcovFile.file as string;
-        if (
-          gcovObjFilePath.toLowerCase().indexOf(gcovFilePath.toLowerCase()) !==
-            -1 ||
-          editor.document.fileName
-            .replace(/\\/g, "/")
-            .toLowerCase()
-            .indexOf(gcovFilePath.toLowerCase()) !== -1
-        ) {
-          const coveredEditor: textEditorWithCoverage = {
-            allLines: [],
-            countPerLines: [],
-            coveredLines: [],
-            editor,
-            partialLines: [],
-            uncoveredLines: [],
-          };
-
-          for (let i = 0; i < editor.document.lineCount; i++) {
-            coveredEditor.allLines.push(editor.document.lineAt(i).range);
-          }
-
-          const maxLineCount =
-            gcovFile.lines.reduce((prev, curr) => {
-              return prev < curr.count ? curr.count : prev;
-            }, 0) || MIN_LINE_COUNT;
-
-          const coveredLineCount =
-            Math.round(maxLineCount * COVERED_FACTOR) || MIN_LINE_COUNT;
-          const partialLineCount =
-            Math.floor(maxLineCount * PARTIAL_FACTOR) || MIN_LINE_COUNT;
-
-          for (let covLine of gcovFile.lines) {
-            if (covLine && !covLine["gcovr/noncode"]) {
-              const lineRange = editor.document.lineAt(covLine.line_number - 1)
-                .range;
-
-              const rangeToDelIndex = coveredEditor.allLines.findIndex((r) => {
-                return r.isEqual(lineRange);
-              });
-              coveredEditor.allLines.splice(rangeToDelIndex, 1);
+      for (const gcovData of gcovJsonObj) {
+        for (const gcovFile of gcovData.files) {
+          const gcovFilePath = basename(gcovFile.file) as string;
+          if (
+            gcovObjFilePath
+              .toLowerCase()
+              .indexOf(gcovFilePath.toLowerCase()) !== -1 ||
+            editor.document.fileName
+              .replace(/\\/g, "/")
+              .toLowerCase()
+              .indexOf(gcovFilePath.toLowerCase()) !== -1
+          ) {
+            const coveredEditor: textEditorWithCoverage = {
+              allLines: [],
+              countPerLines: [],
+              coveredLines: [],
+              editor,
+              partialLines: [],
+              uncoveredLines: [],
+            };
+
+            for (let i = 0; i < editor.document.lineCount; i++) {
+              coveredEditor.allLines.push(editor.document.lineAt(i).range);
+            }
 
-              if (covLine.count >= coveredLineCount) {
-                coveredEditor.coveredLines.push(lineRange);
-              } else if (
-                partialLineCount <= covLine.count &&
-                covLine.count < coveredLineCount
-              ) {
-                coveredEditor.partialLines.push(lineRange);
-              } else {
-                coveredEditor.uncoveredLines.push(lineRange);
+            const maxLineCount =
+              gcovFile.lines.reduce((prev, curr) => {
+                return prev < curr.count ? curr.count : prev;
+              }, 0) || MIN_LINE_COUNT;
+
+            const coveredLineCount =
+              Math.round(maxLineCount * COVERED_FACTOR) || MIN_LINE_COUNT;
+            const partialLineCount =
+              Math.floor(maxLineCount * PARTIAL_FACTOR) || MIN_LINE_COUNT;
+
+            for (let covLine of gcovFile.lines) {
+              if (covLine && !covLine["gcovr/noncode"]) {
+                const lineRange = editor.document.lineAt(
+                  covLine.line_number - 1
+                ).range;
+
+                const rangeToDelIndex = coveredEditor.allLines.findIndex(
+                  (r) => {
+                    return r.isEqual(lineRange);
+                  }
+                );
+                coveredEditor.allLines.splice(rangeToDelIndex, 1);
+
+                if (covLine.count >= coveredLineCount) {
+                  coveredEditor.coveredLines.push(lineRange);
+                } else if (
+                  partialLineCount <= covLine.count &&
+                  covLine.count < coveredLineCount
+                ) {
+                  coveredEditor.partialLines.push(lineRange);
+                } else {
+                  coveredEditor.uncoveredLines.push(lineRange);
+                }
+                coveredEditor.countPerLines.push({
+                  range: lineRange,
+                  count: covLine.count,
+                });
               }
-              coveredEditor.countPerLines.push({
-                range: lineRange,
-                count: covLine.count,
-              });
             }
+            coveredEditors.push(coveredEditor);
+            break;
           }
-          coveredEditors.push(coveredEditor);
-          break;
         }
       }
     }
@@ -250,10 +165,11 @@ export async function previewReport(dirPath: vscode.Uri) {
       gcovHtmlPanel.reveal(column);
       return;
     }
-    const reportHtml = await buildHtml(dirPath);
+    const gcovObj = await getGcovData(dirPath);
+    const reportHtml = await createGcovHtmlReport(gcovObj);
     gcovHtmlPanel = vscode.window.createWebviewPanel(
       "gcoveragePreview",
-      "Coverage report",
+      "GCC Code Coverage Report",
       column
     );
     gcovHtmlPanel.iconPath = getWebViewFavicon(extensionContext.extensionPath);