Terraform Syntax Testing and Target Syntax Generation

[jpogran]

• update to latest terraform grammer from extension
• testing infrastructure
• build target syntaxes

[radeksimko]

A few questions/suggestions:

I'm all for supporting more formats, but I think that we should also be aware of why we support any particular format. Do you mind adding a short list of all formats (cson, JSON, YAML, PLIST) and at least one example to each to a Readme? - Just to make sure it's a conscious decision to commit to supporting a format - i.e. we all know why we're doing this and will have something to refer to in the future.

Feel free to lookup ENG-030 - I did some research on this and left some notes with links there.

I hate to say this (after you have done all the work) 😅 but seeing it now all in action I would argue that the tests written in the caret comments syntax in https://github.com/PanAeon/vscode-tmgrammar-test would be a little easier to maintain, as one doesn't have to deal with index numbering and the test failures also seem easier to read/fix as you can directly see the piece of config being tested in the failure. Also it is written in TypeScript and uses the same textmate vscode library under the hood. That said I may not have the same context as you do as I have never written such tests before.

[radeksimko]

Suggested change

	const tokens = result.tokens;
	return tokens;
	return result.tokens;

[dbanck]

Thank you for all the work of putting this together!

I like the structure of the jest tests, but I'm also worried that having to fiddle with many indexes might hurt our DX in the future. I wondered if having a clever test helper would make things easier, but I'm unsure if something like this would be flexible enough for all scenarios.

expect(tokens.slice(0, 11)).toMatchScopes([
  'entity.name.type.terraform',

  'punctuation.definition.string.begin.terraform',
  'string.quoted.double.terraform',
  'punctuation.definition.string.end.terraform',

  'punctuation.definition.string.begin.terraform',
  'string.quoted.double.terraform',
  'punctuation.definition.string.end.terraform',
  
  'punctuation.section.block.begin.terraform',
]);

This doesn't account for tokens' length or exact position (yet).

[dbanck]

Another possible way is to make use of Jest snapshot tests. For example, like this:

expect(tokens.slice(0, 11)).toMatchInlineSnapshot(`
Array [
  Object {
    "endIndex": 8,
    "scopes": Array [
      "source.terraform",
      "meta.block.terraform",
      "entity.name.type.terraform",
    ],
    "startIndex": 0,
  },
  Object {
    "endIndex": 9,
    "scopes": Array [
      "source.terraform",
      "meta.block.terraform",
    ],
    "startIndex": 8,
  },
  Object {
    "endIndex": 10,
    "scopes": Array [
      "source.terraform",
      "meta.block.terraform",
      "string.quoted.double.terraform",
      "punctuation.definition.string.begin.terraform",
    ],
    "startIndex": 9,
  },
  ...
]
`);

This avoids typing object structures by hand and will verify correctness once generated but isn't easy to read.

[dbanck]

Like you mentioned in Slack, having vscode-tmgrammer-test generate snapshot tests seems like the best of both worlds. You avoid tedious writing and still get a more readable result than the Jest tests.

The snapshots seem a bit bloated, though. There are a lot of comment.terraform entries. We should make sure that the source file for the snapshot test only contains Terraform code and not any grammar test comments.

[jpogran]

Closing in favor of #3