Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement half of the CLI #8

Merged
merged 3 commits into from
Aug 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ jobs:
- name: GolangCI Lint
uses: golangci/golangci-lint-action@v6
with:
# TODO: Update this version.
version: v1.60.2
args: --out-format=colored-line-number
test:
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ running Terraform.
### Parsing Terraform Configuration Files
Parsing Terraform files is done using the [HCL package](https://github.com/hashicorp/hcl). Initially, the plan was to use an existing application such as [terraform-docs](https://github.com/terraform-docs/terraform-docs/) to preform the parsing step, but some of the fields of the `variable` block weren't implemented, such as validation rules.

TerraSchema parses each terraform configuration file as a HCL (HashiCorp Configuration Language) file and picks out any blocks which match the definition of an input variable in Terraform. A typical `variable` block looks like this:
TerraSchema parses each Terraform configuration file as a HCL (HashiCorp Configuration Language) file and picks out any blocks which match the definition of an input variable in Terraform. A typical `variable` block looks like this:

```hcl
variable "age" {
Expand Down
104 changes: 77 additions & 27 deletions cmd/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,34 +24,30 @@ var (
disallowAdditionalProperties bool
overwrite bool
allowEmpty bool
requireAll bool
outputStdOut bool
output string
input string

errReturned error
)

// rootCmd is the base command for terraschema
var rootCmd = &cobra.Command{
Use: "terraschema",
Short: "Generate JSON schema from HCL Variable Blocks in a Terraform/OpenTofu module",
Long: `TODO: Long description`,
Run: func(cmd *cobra.Command, args []string) {
path, err := filepath.Abs(input) // absolute path
if err != nil {
fmt.Printf("could not get absolute path: %v\n", err)
os.Exit(1)
}
output, err := jsonschema.CreateSchema(path, false)
if err != nil {
fmt.Printf("error creating schema: %v\n", err)
os.Exit(1)
}
jsonOutput, err := json.MarshalIndent(output, "", " ")
if err != nil {
fmt.Printf("error marshalling schema: %v\n", err)
Use: "terraschema",
Example: "terraschema -i /path/to/module -o /path/to/schema.json",
Short: "Generate JSON schema from HCL Variable Blocks in a Terraform/OpenTofu module",
Long: "TerraSchema is a CLI tool which scans Terraform configuration ('.tf') " +
"files, parses a list of variables along with their type and validation rules, and converts " +
"them to a schema which complies with JSON Schema Draft-07.\nThe default behaviour is to scan " +
"the current directory and output a schema file called 'schema.json' in the same location. " +
"\nFor more information see https://github.com/HewlettPackard/terraschema.",
Run: runCommand,
PostRun: func(cmd *cobra.Command, args []string) {
if errReturned != nil {
fmt.Printf("error: %v\n", errReturned)
os.Exit(1)
}

fmt.Println(string(jsonOutput))
},
}

Expand All @@ -61,18 +57,72 @@ func Execute() error {

func init() {
// TODO: implement
rootCmd.Flags().BoolVar(&overwrite, "overwrite", false, "overwrite existing schema file")
rootCmd.Flags().BoolVar(&overwrite, "overwrite", false, "allow overwriting an existing file")
// TODO: implement
rootCmd.Flags().BoolVar(&outputStdOut, "stdout", false,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arguably outputting to stdout by default is the UNIX way. But ultimately a judgement call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather not here because you'll want a file output in all cases, and it complicates error logging a lot. (Splitting stderr and stdout doesnt feel like the Go way to me personally)

"output schema content to stdout instead of a file and disable error output",
"output JSON Schema content to stdout instead of a file and disable error output",
)
// TODO: implement

rootCmd.Flags().BoolVar(&disallowAdditionalProperties, "disallow-additional-properties", false,
"set additionalProperties to false in the generated schema and in nested objects",
"set additionalProperties to false in the JSON Schema and in nested objects",
)

rootCmd.Flags().BoolVar(&allowEmpty, "allow-empty", false,
"allow an empty JSON Schema if no variables are found",
)

rootCmd.Flags().BoolVar(&requireAll, "require-all", false,
"set all variables to be 'required' in the JSON Schema, even if a default value is specified",
)

rootCmd.Flags().StringVarP(&input, "input", "i", ".",
"input folder containing a Terraform module",
)

// TODO: implement
rootCmd.Flags().BoolVar(&allowEmpty, "allow-empty", false, "allow empty schema if no variables are found, otherwise error")
rootCmd.Flags().StringVarP(&input, "input", "i", ".", "input folder containing .tf files")
// TODO: implement
rootCmd.Flags().StringVarP(&output, "output", "o", "schema.json", "output file path for schema")
rootCmd.Flags().StringVarP(&output, "output", "o", "schema.json",
"output path for the JSON Schema file",
)
}

func runCommand(cmd *cobra.Command, args []string) {
path, err := filepath.Abs(input) // absolute path
if err != nil {
errReturned = fmt.Errorf("could not get absolute path for %q: %w", input, err)

return
}

folder, err := os.Stat(path)
if err != nil {
errReturned = fmt.Errorf("could not access directory %q: %w", path, err)

return
}

if !folder.IsDir() {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw this works if input is a sym link to a directory (due to using Stat).

errReturned = fmt.Errorf("input %q is not a directory", path)

return
}

output, err := jsonschema.CreateSchema(path, jsonschema.CreateSchemaOptions{
RequireAll: requireAll,
AllowAdditionalProperties: !disallowAdditionalProperties,
AllowEmpty: allowEmpty,
})
if err != nil {
errReturned = fmt.Errorf("error creating schema: %w", err)

return
}

jsonOutput, err := json.MarshalIndent(output, "", " ")
if err != nil {
errReturned = fmt.Errorf("error marshalling schema: %w", err)

return
}

fmt.Println(string(jsonOutput))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to stdout (good default behaviour), so "-o" flag not implemented? (Or did I miss something?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented in the PR after this

}
51 changes: 33 additions & 18 deletions pkg/jsonschema/json-schema.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,37 +10,52 @@ import (
"github.com/HewlettPackard/terraschema/pkg/reader"
)

func CreateSchema(path string, strict bool) (map[string]any, error) {
type CreateSchemaOptions struct {
RequireAll bool
AllowAdditionalProperties bool
AllowEmpty bool
}

func CreateSchema(path string, options CreateSchemaOptions) (map[string]any, error) {
schemaOut := make(map[string]any)

varMap, err := reader.GetVarMap(path)
// GetVarMaps returns an error if no .tf files are found in the directory. We
// ignore this error for now.
if err != nil && !errors.Is(err, reader.ErrFilesNotFound) {
return schemaOut, fmt.Errorf("error reading tf files at %s: %w", path, err)
if err != nil {
if errors.Is(err, reader.ErrFilesNotFound) {
if options.AllowEmpty {
fmt.Printf("Info: no tf files were found in %q, creating empty schema\n", path)

return schemaOut, nil
}
} else {
return schemaOut, fmt.Errorf("error reading tf files at %q: %w", path, err)
}
}

if len(varMap) == 0 {
return schemaOut, nil
if options.AllowEmpty {
return schemaOut, nil
} else {
return schemaOut, errors.New("no variables found in tf files")
}
}

schemaOut["$schema"] = "http://json-schema.org/draft-07/schema#"

if strict {
schemaOut["additionalProperties"] = false
} else {
schemaOut["additionalProperties"] = true
}
schemaOut["additionalProperties"] = options.AllowAdditionalProperties

properties := make(map[string]any)
requiredArray := []any{}
for name, variable := range varMap {
if variable.Required {
if variable.Required && !options.RequireAll {
requiredArray = append(requiredArray, name)
}
if options.RequireAll {
requiredArray = append(requiredArray, name)
}
node, err := createNode(name, variable, strict)
node, err := createNode(name, variable, options)
if err != nil {
return schemaOut, fmt.Errorf("error creating node for %s: %w", name, err)
return schemaOut, fmt.Errorf("error creating node for %q: %w", name, err)
}

properties[name] = node
Expand All @@ -54,16 +69,16 @@ func CreateSchema(path string, strict bool) (map[string]any, error) {
return schemaOut, nil
}

func createNode(name string, v model.TranslatedVariable, strict bool) (map[string]any, error) {
func createNode(name string, v model.TranslatedVariable, options CreateSchemaOptions) (map[string]any, error) {
tc, err := reader.GetTypeConstraint(v.Variable.Type)
if err != nil {
return nil, fmt.Errorf("getting type constraint for %s: %w", name, err)
return nil, fmt.Errorf("getting type constraint for %q: %w", name, err)
}

nullableIsTrue := v.Variable.Nullable != nil && *v.Variable.Nullable
node, err := getNodeFromType(name, tc, nullableIsTrue, strict)
node, err := getNodeFromType(name, tc, nullableIsTrue, options)
if err != nil {
return nil, fmt.Errorf("%s: %w", name, err)
return nil, fmt.Errorf("%q: %w", name, err)
}

if v.Variable.Default != nil {
Expand Down
10 changes: 7 additions & 3 deletions pkg/jsonschema/json-schema_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,14 @@ func TestCreateSchema(t *testing.T) {
expected, err := os.ReadFile(filepath.Join(schemaPath, name, "schema.json"))
require.NoError(t, err)

result, err := CreateSchema(filepath.Join(tfPath, name), false)
result, err := CreateSchema(filepath.Join(tfPath, name), CreateSchemaOptions{
RequireAll: false,
AllowAdditionalProperties: true,
AllowEmpty: true,
})
require.NoError(t, err)

var expectedMap map[string]interface{}
var expectedMap map[string]any
err = json.Unmarshal(expected, &expectedMap)
require.NoError(t, err)

Expand Down Expand Up @@ -283,7 +287,7 @@ func TestSampleInput(t *testing.T) {

input, err := os.ReadFile(tc.filePath)
require.NoError(t, err)
var m interface{}
var m any

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious - this change is just a style thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

err = json.Unmarshal(input, &m)
require.NoError(t, err)

Expand Down
Loading