Skip to content

Commit

Permalink
Merge pull request #6 from lyubick/issue_#4
Browse files Browse the repository at this point in the history
ISSUE #4: JSON validation enabled.
  • Loading branch information
lyubick authored Jun 19, 2023
2 parents 774ff88 + 6d2a83f commit 46019fc
Show file tree
Hide file tree
Showing 13 changed files with 162 additions and 45 deletions.
30 changes: 17 additions & 13 deletions .github/workflows/self-test.yaml
Original file line number Diff line number Diff line change
@@ -1,25 +1,29 @@
---
name: Self Test

on:
'on':
pull_request:
branches: [ main ]
branches: ['main']

jobs:
runValidation:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Check if directory is not scanned, due to recursive is False (should PASS)
id: validation
uses: lyubick/action-YAML-schema-validator@test
- name: Run PyTest(s)
run: |
pip install -r requirements.txt
pip install -r requirements-test.txt
pytest test/ -rxXs
- name: Check YAML files inside directory
uses: ./
continue-on-error: true
with:
json-schema-file: test/json_schema.json
yaml-file-dir: test/
- name: Check if directory is scanned, due to recursive is True (should FAIL)
id: invalidation
json-schema-file: test/schema/json_schema.json
yaml-file-dir: test/YAMLs/
- name: Check JSON files inside directory
uses: ./
continue-on-error: true
uses: lyubick/action-YAML-schema-validator@test
with:
json-schema-file: test/json_schema.json
yaml-file-dir: test/
recursive: true
json-schema-file: test/schema/json_schema.json
yaml-file-dir: test/JSONs/
41 changes: 29 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# action-YAML-schema-validator
# YAML/JSON File(s) Validation against JSON Schema

## Overview
This action can validate YAML files against provided JSON Schema. In case if YAML file is in full compliance
with provided JSON Schema action will exit gracefully, crash (unhandled exception) will occur otherwise.
This action can validate YAML or JSON files against provided JSON Schema. In case if provided file is in full compliance
with provided JSON Schema, action will exit gracefully, crash (unhandled exception) will occur otherwise.

The JSON Schema format used, should be in compliance with https://json-schema.org/.

Expand All @@ -14,20 +14,20 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Check my YAML file with my JSON schema
- name: Check my YAML/JSON file with my JSON schema
id: validation
uses: lyubick/action-YAML-schema-validator@v1
uses: lyubick/action-YAML-schema-validator@v2
with:
json-schema-file: path/to/my/cool/schema.json
yaml-file-dir: path/to/my/cool/yaml/file.yaml
yaml-json-file-dir: path/to/my/cool/yaml/file.yaml
recursive: false
```
One should provide two parameters:
- `json-schema-file`, points to legit JSON Schema file
- `yaml-file-dir`, is a comma separated list that contains
- Single YAML files
- Directories that will be parsed for `.yaml` and `.yml` files
- `recursive`, True/False depending on if recursive scan for YAML files in directory required
- `yaml-json-file-dir`, is a comma separated list that contains
- Single YAML or JSON files
- Directories that will be parsed for `.yaml` or `.yml` or `.json` files
- `recursive`, True/False depending on if recursive scan for YAML or JSON files in directory required

## Results
### Success
Expand Down Expand Up @@ -56,15 +56,32 @@ Traceback ...
}
}
```
### YAML Valid file (compliant with Schema)

### Valid file (compliant with Schema)
#### YAML
```yaml
field1: Value1
```
#### JSON
```json
{
"field1": "Value1"
}
```

### YAML Invalid file (not compliant with Schema)
### Invalid file (not compliant with Schema)
#### YAML
```yaml
field2: Value2
```
#### JSON
```json
{
"field2": "Value1"
}
```

Validating this file will cause error (Exception) thus failing an action
```text
Failed validating 'additionalProperties' in schema:
Expand Down
Empty file added __init__.py
Empty file.
16 changes: 11 additions & 5 deletions action.yaml
Original file line number Diff line number Diff line change
@@ -1,17 +1,23 @@
name: 'YAML file validation against JSON Schema'
description: 'Validates YAML file schema against provided JSON Schema file'
---
name: 'YAML/JSON validation against JSON Schema'
description: |
Validates YAML or JSON file(s) against provided JSON Schema file
branding:
icon: check-circle
color: green
inputs:
json-schema-file:
description: 'JSON Schema file to validate against'
required: true
yaml-file-dir:
description: 'Comma separated list of YAML files and/or directories that require validation'
yaml-json-file-dir:
description: |
Comma separated list of YAML or JSON files and/or directories that require
validation.
required: true
recursive:
description: 'True/False, provide True if recursive scan for YAML files in directory required'
description: |
True/False, provide True if recursive scan for YAML or JSON files
in directory required.
required: false
default: 'false'
runs:
Expand Down
3 changes: 3 additions & 0 deletions requirements-test.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
jsonschema==4.17.3
pyyaml==6.0
pytest==7.3.2
3 changes: 3 additions & 0 deletions test/JSONs/yaml_invalid.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"field2": "Value2"
}
3 changes: 3 additions & 0 deletions test/JSONs/yaml_valid.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"field1": "Value1"
}
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
---
field2: Value2
1 change: 1 addition & 0 deletions test/yamls/yaml_valid.yaml → test/YAMLs/yaml_valid.yaml
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
---
field1: Value1
Empty file added test/__init__.py
Empty file.
File renamed without changes.
63 changes: 63 additions & 0 deletions test/test_sanity.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
from pathlib import Path

import validator
import jsonschema


class Test:
abs_path = Path(__file__).parent

def test1_load_schema(self):
schema = validator.load_schema(f'{self.abs_path}/schema/json_schema.json')
assert schema['title'] == 'TestConfig'
assert schema['description'] == 'Test Basic YAML File'

def test2_get_files(self):
files = validator.get_yaml_json_files_list(f'{self.abs_path}/YAMLs', is_recursive=True)
assert len(files) == 2

def test3_validate_valid_file(self):
schema = validator.load_schema(f'{self.abs_path}/schema/json_schema.json')
files = validator.get_yaml_json_files_list(f'{self.abs_path}/YAMLs/yaml_valid.yaml', False)
assert validator.validate_files(files, schema)

def test4_validate_invalid_file(self):
schema = validator.load_schema(f'{self.abs_path}/schema/json_schema.json')
files = validator.get_yaml_json_files_list(f'{self.abs_path}/YAMLs/yaml_invalid.yaml', False)
try:
validator.validate_files(files, schema)
assert False
except jsonschema.exceptions.ValidationError as exc:
assert exc.instance == {'field2': 'Value2'}

def test5_validate_valid_file_json(self):
schema = validator.load_schema(f'{self.abs_path}/schema/json_schema.json')
files = validator.get_yaml_json_files_list(f'{self.abs_path}/JSONs/yaml_valid.json', False)
assert validator.validate_files(files, schema)

def test6_validate_invalid_file_json(self):
schema = validator.load_schema(f'{self.abs_path}/schema/json_schema.json')
files = validator.get_yaml_json_files_list(f'{self.abs_path}/JSONs/yaml_invalid.json', False)
try:
validator.validate_files(files, schema)
assert False
except jsonschema.exceptions.ValidationError as exc:
assert exc.instance == {'field2': 'Value2'}

def test7_validate_folder_yaml(self):
schema = validator.load_schema(f'{self.abs_path}/schema/json_schema.json')
files = validator.get_yaml_json_files_list(f'{self.abs_path}/YAMLs/', False)
try:
validator.validate_files(files, schema)
assert False
except jsonschema.exceptions.ValidationError as exc:
assert exc.instance == {'field2': 'Value2'}

def test8_validate_folder_json(self):
schema = validator.load_schema(f'{self.abs_path}/schema/json_schema.json')
files = validator.get_yaml_json_files_list(f'{self.abs_path}/JSONs/', False)
try:
validator.validate_files(files, schema)
assert False
except jsonschema.exceptions.ValidationError as exc:
assert exc.instance == {'field2': 'Value2'}
46 changes: 31 additions & 15 deletions validator.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,49 +7,65 @@
from jsonschema import validate
from jsonschema.exceptions import ValidationError

if __name__ == '__main__':
args = sys.argv[1:]

schema_file = args[0]

if os.path.exists(schema_file):
if os.path.isfile(schema_file):
with open(args[0], 'r') as stream:
schema = json.loads("".join(stream.readlines()))
def load_schema(schema_file_path: str) -> json:
if os.path.exists(schema_file_path):
if os.path.isfile(schema_file_path):
with open(schema_file_path, 'r') as stream:
return json.loads("".join(stream.readlines()))
else:
raise f'Provided JSON Schema is not a file! Please provide legit JSON Schema file.'
else:
raise f'Provided JSON Schema file does not exist!'

yaml_input = list(args[1].split(','))

recursive = args[2].lower() == 'true'
def get_yaml_json_files_list(files_paths_list: str, is_recursive: bool) -> list[str]:
yaml_input = list(files_paths_list.split(','))

yaml_files = []
for yaml_object in yaml_input:
if os.path.isdir(yaml_object):
yaml_files.extend(
list(
filter(
lambda f: str(f).endswith('.yaml') or str(f).endswith('.yml'),
pathlib.Path(yaml_object).glob('**/*' if recursive else '*')
map(
lambda f: str(f), # Convert all paths to string, instead of Posix Path
filter(
lambda f: str(f).endswith('.yaml') or str(f).endswith('.yml') or str(f).endswith('.json'),
pathlib.Path(yaml_object).glob('**/*' if is_recursive else '*')
)
)
)
)
elif os.path.isfile(yaml_object):
yaml_files.append(yaml_object)

return yaml_files


def validate_files(yaml_files: list, json_schema: json):
for yaml_file in yaml_files:
if os.path.exists(yaml_file):
if os.path.isfile(yaml_file):
with open(yaml_file, 'r') as stream:
yaml_json = json.loads(json.dumps(yaml.safe_load(stream)))
if not yaml_file.endswith('.json'):
yaml_json = json.loads(json.dumps(yaml.safe_load(stream)))
else:
yaml_json = json.load(stream)
try:
validate(instance=yaml_json, schema=schema)
validate(instance=yaml_json, schema=json_schema)
except ValidationError as exc:
print(f'File `{yaml_file}` failed validation with >>>`{exc}`<<<', file=sys.stderr)
raise exc
else:
raise f'Provided YAML file is not a file! Please provide legit YAML file.'
else:
raise f'Provided YAML file does not exist!'
return True


if __name__ == '__main__':
args = sys.argv[1:]

schema = load_schema(args[0])
files = get_yaml_json_files_list(args[1], args[2].lower() == 'true')
validate_files(files, schema)

0 comments on commit 46019fc

Please sign in to comment.