Terraform validate reporting "Module not installed" unnecessarily

[wagnerone]

Issue Type: Bug

This is causing lots of distracting red alarm visuals in Vscode in the past few days.

The "main.tf" in the error below is a module calling another module, so it will never have an init executed in "itself".

Its init is one level up in the parent module that calls this module, which then calls another module.

[{
"resource": ".../main.tf",
"owner": "generated_diagnostic_collection_name#0",
"severity": 8,
"message": "Module not installed: This module is not yet installed. Run "terraform init" to install all modules required by this configuration.",
"source": "terraform validate",
"startLineNumber": 1,
"startColumn": 1,
"endLineNumber": 1,
"endColumn": 13
}]

Extension version: 2.21.0
VS Code version: Code - Insiders 1.66.0-insider (c5dece632b4b1ea98db872e1edcfc92994503274, 2022-03-24T15:07:58.609Z)
OS version: Darwin arm64 21.4.0
Restricted Mode: No

System Info

Item	Value
CPUs	Apple M1 Pro (10 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on oop_rasterization: enabled opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled
Load (avg)	6, 5, 5
Memory (System)	32.00GB (0.20GB free)
Process Argv	--crash-reporter-id 212a1510-41cd-4257-9265-09b65bc1f6a6
Screen Reader	no
VM	0%

A/B Experiments

vsliv695:30137379
vsins829:30139715
vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyl392:30422396
pythontb:30258533
pythonptprofiler:30281269
vshan820:30294714
pythondataviewer:30285072
vscod805cf:30301675
pythonvspyt200:30323110
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
pythonvs932:30404738
wslgetstarted:30449409
vsclayoutctrt:30405800
pythonvspyt640:30438691
vscscmwlcmc:30428972
vscgsvidc:30433759
pynewfile477:30451556

[staranto]

I've got the exact same problem as @wagnerone. I believe it started happening only in the last week or so.

Environment Information

Terraform Information

Version: 1.1.4
Platform: linux_amd64
Outdated: true

Visual Studio Code

Name	Version
Operating System	Linux x64 5.13.0-37-generic
VSCode	1.65.2

Visual Studio Code Extensions

Visual Studio Code Extensions(Click to Expand)

Extension	Author	Version
aws-toolkit-vscode	amazonwebservices	1.36.0
Bookmarks	alefragnani	13.2.4
explorer-exclude-vscode-extension	RedVanWorkshop	1.2.0
git-graph	mhutchie	1.30.0
githistory	donjayamanne	0.6.19
iam-legend	SebastianBille	1.0.7
jinjahtml	samuelcolvin	0.17.0
local-history	xyz	1.8.1
markdown-all-in-one	yzhang	3.4.0
RunOnSave	emeraldwalk	0.2.0
shellcheck	timonwong	0.18.10
terraform	hashicorp	2.21.0
todo-tree	Gruntfuggly	0.0.215
vscode-eslint	dbaeumer	2.2.2
vscode-yaml	redhat	1.5.1

[staranto]

error.log

Occurs with terraform-ls 0.25.2 and 0.26.0.

Only occurs after I content change/save any .tf. If I start vscode and simply look at .tf's, I don't get the error.

[wagnerone]

I concur with everything @staranto added.

[jpogran]

Hi @wagnerone and @staranto, thanks for the detailed reports. From your description and from looking at the error log you have terraform-ls.experimentalFeatures.validateOnSave enabled which means the extension runs terraform validate whenever VS Code triggers a file save. We recently fixed a terraform-ls#799 in 0.26.0 which prevented successful execution of terraform validate in some situations, so the good news is we now work correctly in those situations. The bad news is it appears terraform validate can't find your module declarations from the directory it's being run in.

As a short-term workaround you can set terraform-ls.experimentalFeatures.validateOnSave to false to prevent this from happening.

Can you both print a tree of the file structure you're describing so I can ensure I understand correctly how your project is structured? It can be simplified to only include the relevant paths. I have the paths shown in @staranto's log, but I want to be sure I understand correctly.

[staranto]

@jprogan - Thanks for the response.

1. Using terraform-ls 0.26.0, setting terraform-ls.experimentalFeatures.validateOnSave = false does, indeed, hide there error.
2. My project stricture is this (note, in my error log above I did a poor job of masking out some personal info in the folder names - in that log, replace all the /my/src/ with /my/src/iac/) --

my/src/iac        <- .tf in here has modules "test" { source = "./test" }
my/src/iac/_modules/ec2
my/src/iac/test   <- .tf that shows the error are in here.

For example my/src/iac/test/bastion.tf has --

module "ec2" {
    source = "../_modules/ec2"  # this line has error highlight
    ...
}

IOW.... root in iac/ includes module in iac/test, which includes module in iac/_modules/ec2

[radeksimko]

To add some more context here:

The extension (through language server) doesn't do anything more than call out to terraform validate under the hood and so all the behaviours of that command would affect the validation behaviours.

terraform validate is very strict in assuming you successfully ran terraform init prior to validating in that directory you want to validate, i.e. all providers and modules are installed.

There is a valid argument to be made about supporting some kind of "partial" validation of the part of configuration that "is validatable", but terraform validate isn't flexible enough to provide that today. Adding that functionality would require wider design discussion as there are use cases (often outside of editor) where the current strict behaviour still makes more sense.

This is one of the main reasons we make validation on save opt-in.

We could try to run validate in the "upstream" module which has the providers etc. installed, but that would only work if there is a 1-1 relationship, i.e. if you install the same module only once, i.e. there's only one choice of the context/directory in which to validate it.

This breaks pretty quickly with >1 installations. See for example https://github.com/hashicorp/terraform-ls/tree/main/internal/langserver/handlers/testdata/main-module-multienv/env where dev, prod and staging would all be inited, and therefore there would be 3 different contexts/ways in which we could validate the underlying application or database modules https://github.com/hashicorp/terraform-ls/tree/main/internal/langserver/handlers/testdata/main-module-multienv/modules

Which one should we pick? I think they can all be relevant, but you probably expect to mainly see diagnostics which are relevant to the installation point, not to the module's internal behaviour. e.g. you probably want to see mismatched module inputs/outputs at the installation point and missing required resource "..." attribute diagnostics only within the module. We could possibly do some filtering as part of hashicorp/terraform-ls#375 but the challenge still remains that we'd need to run validate at all installation points, i.e. if the same module is installed 5 times, we'd run validate 5 times every time you save the downstream module. I am not entirely convinced that is a scalable model but I'm open to other thoughts here.

We do plan to re-implement some parts of validate as part of #720

Does #720 sound like it would provide the validation you expect?

[radeksimko]

I'm going to close this optimistically in favour of #720 as I don't see anything else actionable here.

terraform validate as a CLI command has the limitations discussed and there's unfortunately not much we can do about these. #720 proposes a validation which does not involve that command as such would be more suitable for running "on save".

The only downside is that it may not be as detailed validation since we would only do the validation based on static provider schema, so e.g. it won't run custom ValidateFunc which typically provides additional dynamic validation (see e.g. AWS provider). In order to do such detailed validation we'd need access to the provider, i.e. it would need to be installed, i.e. terraform init would need to run beforehand - which takes us back to the originally reported error message (which in this context is relevant).

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.