Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency usage CLI #198920

Merged
merged 15 commits into from
Nov 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -2015,6 +2015,15 @@ module.exports = {
'@kbn/imports/no_group_crossing_imports': 'warn',
},
},
{
files: ['packages/kbn-dependency-usage/**/*.{ts,tsx}'],
rules: {
// disabling it since package is a CLI tool
'no-console': 'off',
// disabling it since package is marked as module and it requires extension for files written
'@kbn/imports/uniform_imports': 'off',
},
},
],
};

Expand Down
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,7 @@ packages/kbn-data-service @elastic/kibana-visualizations @elastic/kibana-data-di
packages/kbn-data-stream-adapter @elastic/security-threat-hunting
packages/kbn-data-view-utils @elastic/kibana-data-discovery
packages/kbn-datemath @elastic/kibana-data-discovery
packages/kbn-dependency-usage @elastic/kibana-security
packages/kbn-dev-cli-errors @elastic/kibana-operations
packages/kbn-dev-cli-runner @elastic/kibana-operations
packages/kbn-dev-proc-runner @elastic/kibana-operations
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ target
*.iml
*.log
types.eslint.config.js
types.eslint.config.cjs
__tmp__

# Ignore example plugin builds
Expand Down Expand Up @@ -159,3 +160,4 @@ x-pack/test/security_solution_playwright/playwright/.cache/
x-pack/test/security_solution_playwright/.auth/
x-pack/test/security_solution_playwright/.env
.codeql
.dependency-graph-log.json
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -1429,6 +1429,7 @@
"@kbn/core-ui-settings-server-mocks": "link:packages/core/ui-settings/core-ui-settings-server-mocks",
"@kbn/core-usage-data-server-mocks": "link:packages/core/usage-data/core-usage-data-server-mocks",
"@kbn/cypress-config": "link:packages/kbn-cypress-config",
"@kbn/dependency-usage": "link:packages/kbn-dependency-usage",
"@kbn/dev-cli-errors": "link:packages/kbn-dev-cli-errors",
"@kbn/dev-cli-runner": "link:packages/kbn-dev-cli-runner",
"@kbn/dev-proc-runner": "link:packages/kbn-dev-proc-runner",
Expand Down Expand Up @@ -1708,6 +1709,7 @@
"cypress-recurse": "^1.35.2",
"date-fns": "^2.29.3",
"dependency-check": "^4.1.0",
"dependency-cruiser": "^16.4.2",
"ejs": "^3.1.10",
"enzyme": "^3.11.0",
"enzyme-to-json": "^3.6.2",
Expand Down
153 changes: 153 additions & 0 deletions packages/kbn-dependency-usage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@

# @kbn/dependency-usage
elena-shostak marked this conversation as resolved.
Show resolved Hide resolved

A CLI tool for analyzing dependencies across packages and plugins. This tool provides commands to check dependency usage, aggregate it, debug dependency graphs, and more.

---

## Table of Contents
1. [Show all packages/plugins using a dependency](#show-all-packagesplugins-using-a-dependency)
2. [Show dependencies grouped by code owner](#show-dependencies-grouped-by-code-owner)
3. [List all dependencies for a package or directory](#list-all-dependencies-for-source-directory)
4. [Group by code owner with adjustable collapse depth](#group-by-code-owner-with-adjustable-collapse-depth)
5. [Show dependencies matching a pattern](#show-dependencies-matching-a-pattern)
6. [Verbose flag to debug dependency graph issues](#verbose-flag-to-debug-dependency-graph-issues)

---


### 1. Show all packages/plugins using a specific dependency

Use this command to list all packages or plugins within a directory that use a specified dependency.

```sh
bash scripts/dependency_usage.sh -d <dependency> -p <path_to_directory>
```
or
```sh
bash scripts/dependency_usage.sh --dependency-name <dependency> --paths <path_to_directory>
```

**Example**:
```sh
bash scripts/dependency_usage.sh -d rxjs -p x-pack/plugins/security_solution
```

- `-d rxjs`: Specifies the dependency to look for (`rxjs`).
- `-p x-pack/plugins/security_solution`: Sets the directory to search within (`x-pack/plugins/security_solution`).

---

### 2. Show dependencies grouped by code owner

Group the dependencies used within a directory by code owner.

```sh
bash scripts/dependency_usage.sh -p <path_to_directory> -g owner
```
or
```sh
bash scripts/dependency_usage.sh --paths <path_to_directory> --group-by owner
```

**Example**:
```sh
bash scripts/dependency_usage.sh -p x-pack/plugins -g owner
```

- `-p x-pack/plugins`: Sets the directory to scan for plugins using this dependency.
- `-g owner`: Groups results by code owner.
- **Output**: Lists all dependencies for `x-pack/plugins`, organized by code owner.

---

### 3. List all dependencies for source directory

To display all dependencies used within a specific directory.

```sh
bash scripts/dependency_usage.sh -p <path_to_directory>
```
or
```sh
bash scripts/dependency_usage.sh --paths <path_to_directory>
```

**Example**:
```sh
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution
```

- `-p x-pack/plugins/security_solution`: Specifies the package or directory for which to list all dependencies.
- **Output**: Lists all dependencies for `x-pack/plugins/security_solution`.

---

### 4. Group by code owner with adjustable collapse depth

When a package or plugin has multiple subteams, use the `--collapse-depth` option to control how granular the grouping by code owner should be.

#### Detailed Subteam Grouping
Shows all subteams within `security_solution`.

```sh
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 4
```

#### Collapsed Grouping
Groups the results under a higher-level owner (e.g., `security_solution` as a single group).

```sh
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -g owner --collapse-depth 1
```

**Explanation**:
- `-p x-pack/plugins/security_solution`: Specifies the directory to scan.
- `-g owner`: Groups results by code owner.
- `--collapse-depth`: Defines the depth for grouping, where higher numbers show more granular subteams.
- **Output**: Lists dependencies grouped by code owner at different levels of depth based on the `--collapse-depth` value.

---

### 5. Show dependencies matching a pattern

Search for dependencies that match a specific pattern (such as `react-*`) within a package and output the results to a specified file.

```sh
bash scripts/dependency_usage.sh -p <path_to_directory> -d '<pattern>' -o <output_file>
```

**Example**:
```sh
bash scripts/dependency_usage.sh -d 'react-*' -p x-pack/plugins/security_solution -o ./tmp/results.json
```

- `-p x-pack/plugins/security_solution`: Specifies the directory or package to search within.
- `-d 'react-*'`: Searches for dependencies that match the pattern `react-*`.
- `-o ./tmp/results.json`: Outputs the results to a specified file (`results.json` in the `./tmp` directory).
- **Output**: Saves a list of all dependencies matching `react-*` in `x-pack/plugins/security_solution` to `./tmp/results.json`.

---

### 6. Verbose flag to debug dependency graph issues

Enable verbose mode to log additional details for debugging dependency graphs. This includes generating a non-aggregated dependency graph in `.dependency-graph-log.json`.
elena-shostak marked this conversation as resolved.
Show resolved Hide resolved

```sh
bash scripts/dependency_usage.sh -p <path_to_directory> -o <output_file> -v
```

**Example**:
```sh
bash scripts/dependency_usage.sh -p x-pack/plugins/security_solution -o ./tmp/results.json
```
- `-p x-pack/plugins/security_solution`: Specifies the target directory or package to analyze.
- `-o ./tmp/results.json`: Saves the output to the `results.json` file in the `./tmp` directory.
- `-v`: Enables verbose mode.

**Output**: Saves a list of all dependencies in `x-pack/plugins/security_solution` to `./tmp/results.json`. Additionally, it logs a detailed, non aggregated dependency graph to `.dependency-graph-log.json` for debugging purposes.

---

For further information on additional flags and options, refer to the script's help command.

15 changes: 15 additions & 0 deletions packages/kbn-dependency-usage/jest.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the "Elastic License
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
* Public License v 1"; you may not use this file except in compliance with, at
* your election, the "Elastic License 2.0", the "GNU Affero General Public
* License v3.0 only", or the "Server Side Public License, v 1".
*/

/* eslint-disable no-restricted-syntax */
export default {
preset: '@kbn/test',
rootDir: '../..',
roots: ['<rootDir>/packages/kbn-dependency-usage'],
};
6 changes: 6 additions & 0 deletions packages/kbn-dependency-usage/kibana.jsonc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"devOnly": true,
"type": "shared-common",
"id": "@kbn/dependency-usage",
"owner": "@elastic/kibana-security"
elena-shostak marked this conversation as resolved.
Show resolved Hide resolved
}
11 changes: 11 additions & 0 deletions packages/kbn-dependency-usage/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"extends": "../../tsconfig.base.json",
"name": "@kbn/dependency-usage",
"private": true,
"version": "1.0.0",
"license": "Elastic License 2.0 OR AGPL-3.0-only OR SSPL-1.0",
"type": "module",
Copy link
Contributor Author

@elena-shostak elena-shostak Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

dependency-cruiser is a esm only package, so I need to adjust tsconfig to "module": "esnext" and have a "type": "module" for it in package.json, otherwise it was failing with

No "exports" main defined in /node_modules/dependency-cruiser/package.json

Making import with await didn't help and still produced the same error

const { cruise } = await import('dependency-cruiser');

"exports": {
"./src/*": "./src/*"
}
}
97 changes: 97 additions & 0 deletions packages/kbn-dependency-usage/src/cli.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the "Elastic License
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
* Public License v 1"; you may not use this file except in compliance with, at
* your election, the "Elastic License 2.0", the "GNU Affero General Public
* License v3.0 only", or the "Server Side Public License, v 1".
*/

import { identifyDependencyUsageWithCruiser } from './dependency_graph/providers/cruiser.ts';
import { configureYargs } from './cli';

jest.mock('chalk', () => ({
green: jest.fn((str) => str),
yellow: jest.fn((str) => str),
cyan: jest.fn((str) => str),
magenta: jest.fn((str) => str),
blue: jest.fn((str) => str),
bold: { magenta: jest.fn((str) => str), blue: jest.fn((str) => str) },
}));

jest.mock('./dependency_graph/providers/cruiser', () => ({
identifyDependencyUsageWithCruiser: jest.fn(),
}));

jest.mock('./cli', () => ({
...jest.requireActual('./cli'),
runCLI: jest.fn(),
}));

describe('dependency-usage CLI', () => {
const parser = configureYargs()
.fail((message: string) => {
throw new Error(message);
})
.exitProcess(false);

beforeEach(() => {
jest.spyOn(console, 'log').mockImplementation(() => {});
});

afterEach(() => {
jest.resetAllMocks();
});

it('should handle verbose option', () => {
const argv = parser.parse(['--paths', './plugins', '--verbose']);
expect(argv.verbose).toBe(true);

expect(identifyDependencyUsageWithCruiser).toHaveBeenCalledWith(
expect.any(Array),
undefined,
expect.objectContaining({ isVerbose: true })
);
});

it('should group results by specified group-by option', () => {
const argv = parser.parse(['--paths', './src', '--group-by', 'owner']);
expect(argv['group-by']).toBe('owner');

expect(identifyDependencyUsageWithCruiser).toHaveBeenCalledWith(
expect.any(Array),
undefined,
expect.objectContaining({ groupBy: 'owner' })
);
});

it('should use default values when optional arguments are not provided', () => {
const argv = parser.parse([]);
expect(argv.paths).toEqual(['.']);
expect(argv['dependency-name']).toBeUndefined();
expect(argv['collapse-depth']).toBe(1);
expect(argv.verbose).toBe(false);
});

it('should throw an error if summary is used without dependency-name', () => {
expect(() => {
parser.parse(['--summary', '--paths', './src']);
}).toThrow('Summary option can only be used when a dependency name is provided');
});

it('should validate collapse-depth as a positive integer', () => {
expect(() => {
parser.parse(['--paths', './src', '--collapse-depth', '0']);
}).toThrow('Collapse depth must be a positive integer');
});

it('should output results to specified output path', () => {
const argv = parser.parse(['--paths', './src', '--output-path', './output.json']);
expect(argv['output-path']).toBe('./output.json');
});

it('should print results to console if no output path is specified', () => {
const argv = parser.parse(['--paths', './src']);
expect(argv['output-path']).toBeUndefined();
});
});
Loading