Add StateShow

[micahkemp]

Included in this PR is the addition of the StateShow method.

This method returns the stdout of the executed terraform command, or an error if encountered.

[hashicorp-cla]

All committers have signed the CLA.

[radeksimko]

Hi @micahkemp
Thank you for the PR.

We tend to keep terraform-exec focused on automation use cases (rather than aiming for 100% coverage of all Terraform CLI commands).

With that in mind could you share your use case for this command in the context of automation?

Keep in mind that this command (unlike most others we integrated) does not provide machine-readable (JSON) output and AFAIK therefore doesn't provide any compatibility guarantees for the output format.

If we have a use case though we could propose enhancement of Terraform CLI so that it does offer JSON output - this would then make the integration much easier here.

[micahkemp]

This PR (as well as #206) was indeed crafted in the name of automation.

I'm working on enabling terraform import functionality in a CI/CD pipeline to minimize temptation to run any terraform commands outside of the pipeline.

In support of that, my pipeline aims to:

• Perform a terraform import (already in terraform-exec) in a graceful manner, such as skipping already imported resources but resulting in a non-zero exit code. This would be enabled by Add StateList #206, as it returns a list of existing resources which can be compared against.
• Perform a terraform show of the previously imported resources to make it easy to grab the current state in HCL. This could also be used to enable the graceful import, though by failure versus comparison.

There's certainly a lot more to it (dynamically created workspaces for feature branches, checking if the defined imports are still valid before merging the next PR, etc.), but these two PRs add vital functionality which will make my automated CI/CD much simpler when written in Go than in Shell.

As for potential enhancement requests to have the CLI output JSON that could help, I'm not sure how much help that would be. terraform show I specifically want to use because it spits out HCL, not JSON. terraform list could potentially be replaced by inspecting the terraform-json.State object, but it seems that would be much more involved when it comes to addresses within child modules.

[radeksimko]

Thank you for the added context - this is useful!

I think we are all aiming for a long-term goal where Terraform is capable of seamlessly importing a resource without additional human interaction 😄 Unfortunately this isn't how Terraform works today - yet - as you already noticed.

terraform show I specifically want to use because it spits out HCL, not JSON

I'm afraid that is a false assumption. It only happens to look like HCL and in many/most cases it is valid HCL, but it's not a design goal and it is certainly not guaranteed that command will always output valid HCL.

You can also review how the output is assembled here which also shows a number of annotations such as (tainted) etc. The main goal for that output is mainly human-readability more than anything else and breaking changes should be expected for these reasons.

Upcoming v1.1.0 release will bring a new add command which does promise generation of valid HCL:
hashicorp/terraform#28874

I assume that would address your use case, wouldn't it? I'd be happy to review a PR for the add command.

---

It's fair to say that users with lots of resources (probably your case?) will want to employ some automation around this process. I would however apply some caution because the resulting config (even the one generated by add) may not necessarily be 100% what you need (the linked PR mentions some reasons why) and the result should still be reviewed by a human anyway otherwise you risk persistent diffs during plans and other unwanted behaviours.

In other words I understand why you want to automate this, but I'm not sure I fully understand why you wish to do it in a CI pipeline (as opposed to a standalone script human runs). What would the output or next step from that pipeline be?

[micahkemp]

I'm afraid that is a false assumption. It only happens to look like HCL and in many/most cases it is valid HCL, but it's not a design goal and it is certainly not guaranteed that command will always output valid HCL.

This is acceptable to us. We aren't looking for the automation to take the result of terraform state show and shove it into a .tf file. Instead the engineer would take the output of terraform state show from the CI/CD pipeline, place it in the .tf file, and clean it up as needed. At a minimum IDs get removed, and in some cases ID references need to be substituted (I've done that in another project via some manually-run scripts).

Upcoming v1.1.0 release will bring a new add command which does promise generation of valid HCL:
hashicorp/terraform#28874

This looks very promising! I will spend some more time looking into this, but it looks like a replacement for state show certainly.

In other words I understand why you want to automate this, but I'm not sure I fully understand why you wish to do it in a CI pipeline (as opposed to a standalone script human runs). What would the output or next step from that pipeline be?

This is likely a longer topic than can truly be expressed here, but the CI/CD pipeline for import is largely intended to prevent engineers from configuring the remote state (and other) credentials that would permit them to run terraform commands locally. When multiple engineers are working on code and configuration it becomes more important for everything to go through review and automation to be deployed, and avoid as much manual running as possible.

Specifically it would permit the use of existing automation to assist the building of .tf content and validation that said content is a net-zero change from what is already deployed (outside of terraform). The steps generally look like:

Feature branch:

• engineer - creates stub .tf content for resources to import
• CICD - create temporary workspace
• CICD perform import
• CICD - state show to help engineer fill out .tf content
• CICD - force rm temporary workspace
• engineer - clean up output from state show and place in .tf, commit, push
• CICD - repeats steps above, but additionally terraform plan to show what's not quite right in the .tf content to get to a net-zero definition
• engineer - after getting to a clean plan, submit PR

Default branch:

• CICD - perform import
• CICD - terraform plan should be empty, no further actions to take

Our existing process is manual, and dependent on being run by just one or two terraform experts, who are (hopefully) less likely to make a mistake to the default workspace or deployed infrastructure. But we'd like to avoid this bottleneck, while also not promoting every engineer to run terraform commands locally, due to the risk of state not matching committed/merged configurations, or having orphaned remote resources from working on local state or manually created workspaces.

[radeksimko]

Thank you for the added detailed description of your CI/CD use case - this is very insightful.

I appreciate that the lack of compatibility guarantees is acceptable for you but as maintainers we also need to keep in mind this library is used by folks who have certain expectations which may not necessarily match with yours.

I'm just in the process of planning work on another project which will also require programmatic access to state and so I will likely have a use case for this either way. If we do add it though I'd really prefer consuming machine-readable (JSON) output, rather than the human-readable one.

I will try to propose JSON output flag for the two subcommands (state list and state show) and come back to this PR.

Thank you for your patience and understanding.

[micahkemp]

For this PR, I think it can be closed and potentially replaced by one for terraform add. I may try to find time to submit that one as well.

I believe terraform add is intended to result in valid output (not JSON), and thus would make sense to be able to automate around it.

I'll close this PR assuming you agree, or you can feel free to close it.

[radeksimko]

I would probably prefer to keep it open for a little bit longer if you don't mind.

As mentioned I have a potential use case for state show in the language server. I just haven't gotten through the full design phase of that piece of work yet and haven't explored all avenues - including terraform console - to be able to confirm this. Once I do I can decide whether state show is the best option there and whether we just need to add JSON flag and modify this PR, or if we conclude that console is better suited for the job then we can just close this PR.

I should have some time to work on that in coming weeks hopefully. 🤞🏻

[radeksimko]

After discussing this in more detail with the Core team and also outlining some use potential cases of our own in hashicorp/vscode-terraform#734 we came to conclusion that the discussed import use case along with the one linked can be addressed with existing functionality.

terraform state show doesn't do much more than filtering of JSON structures and that whole JSON file is already available through terraform show / Show().

Given a previously init-ed and apply-ed config.

package main

import (
	"context"
	"fmt"
	"log"
	"os"
	"os/exec"

	"github.com/hashicorp/terraform-exec/tfexec"
	tfjson "github.com/hashicorp/terraform-json"
)

func main() {
	workDir, err := os.Getwd()
	if err != nil {
		log.Fatal(err)
	}

	execPath, err := exec.LookPath("terraform")
	if err != nil {
		log.Fatal(err)
	}

	tf, err := tfexec.NewTerraform(workDir, execPath)
	if err != nil {
		log.Fatal(err)
	}

	ctx := context.Background()
	state, err := tf.Show(ctx)
	if err != nil {
		log.Fatal(err)
	}

	for _, resource := range state.Values.RootModule.Resources {
		if resource.Address == "random_pet.adam" {
			switch resource.Mode {
			case tfjson.ManagedResourceMode:
				fmt.Printf("resource ")
			case tfjson.DataResourceMode:
				fmt.Printf("data ")
			}
			fmt.Printf("%q ", resource.Type)
			fmt.Printf("%q {\n", resource.Name)

			for key, value := range resource.AttributeValues {
				if value != nil {
					fmt.Printf("  %s = %q\n", key, value)
				}
			}

			fmt.Printf("}\n")
		}
	}
}

$ go run main.go
resource "random_pet" "adam" {
  id = "quality-firefly"
  length = "2"
  separator = "-"
}

which produces equivalent output to

$ terraform state show random_pet.adam
# random_pet.adam:
resource "random_pet" "adam" {
    id        = "quality-firefly"
    length    = 2
    separator = "-"
}

Thank you for your patience and understanding.

I hope that helps.