Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output a warning on mulitple paths being tested in the same file. #452

Merged
merged 3 commits into from
Aug 1, 2024

Conversation

dblock
Copy link
Member

@dblock dblock commented Jul 29, 2024

Description

Adds support for story warnings, outputs a warning when multiple paths are being tested.

$ npm run test:spec--insecure -- --tests tests/indices/

> opensearch_api_tools@1.0.0 test:spec--insecure
> ts-node tools/src/tester/test.ts --opensearch-insecure --tests tests/indices/

OpenSearch 2.15.0

PASSED  cache.yaml (/Users/dblock/source/opensearch-project/opensearch-api-specification/dblock-opensearch-api-specification/tests/indices/cache.yaml)
PASSED  clone.yaml (/Users/dblock/source/opensearch-project/opensearch-api-specification/dblock-opensearch-api-specification/tests/indices/clone.yaml)
WARNING Multiple paths detected, please group similar tests together and move paths not being tested to the prologue.
  /{index}/_settings
  /{index}/_clone/{target}

This lets us catch the instances where we maybe be testing multiple/unrelated paths or where some should be in prologue. It helps make tests more contained, and warnings can be suppressed when absolutely necessary.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dblock dblock force-pushed the warn-multiple-paths branch from 297edcc to 2b6a10c Compare July 29, 2024 20:25
Copy link
Contributor

github-actions bot commented Jul 29, 2024

Spec Test Coverage Analysis

Total Tested
524 116 (22.14 %)

Copy link
Contributor

github-actions bot commented Jul 29, 2024

Changes Analysis

Commit SHA: df9e41a
Comparing To SHA: 659a3ce

API Changes

Summary

NO CHANGES

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/10205196222/artifacts/1766570560

API Coverage

Before After Δ
Covered (%) 500 (48.97 %) 500 (48.97 %) 0 (0 %)
Uncovered (%) 521 (51.03 %) 521 (51.03 %) 0 (0 %)
Unknown 24 24 0

@dblock dblock marked this pull request as ready for review July 29, 2024 21:47
json_schemas/test_story.schema.yaml Show resolved Hide resolved
tools/src/tester/StoryEvaluator.ts Outdated Show resolved Hide resolved
@dblock dblock force-pushed the warn-multiple-paths branch from b41c0a1 to 41c11c7 Compare July 30, 2024 21:27
@dblock dblock requested a review from nhtruong July 30, 2024 21:27
nhtruong
nhtruong previously approved these changes Jul 31, 2024
@dblock
Copy link
Member Author

dblock commented Jul 31, 2024

@nhtruong rebased

@dblock dblock force-pushed the warn-multiple-paths branch from 3c22d39 to f77f03d Compare July 31, 2024 20:00
dblock added 3 commits August 1, 2024 16:20
Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: dblock <dblock@amazon.com>
@dblock dblock force-pushed the warn-multiple-paths branch from f77f03d to df9e41a Compare August 1, 2024 20:20
@Xtansia Xtansia merged commit b67104c into opensearch-project:main Aug 1, 2024
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants