how to disable terraform's output when running test

[owulveryck]

Hello and thank you for you tool.

I am using it to write some tests against my infrastructure.
I would like to inhibit the output of terraform while running go test and use the output of the functions to display what happened only if something fails:

output, err := terraform.InitAndPlanE(&testing.T{}, terraformOptions)
if err != nil {
    t.Log(output)
    t.Fail()
}

If the test succeeds, a go test will only return
PASS

Is there a way to easily achieve that?

[owulveryck]

I managed to do it by tweaking the logger package.

The patch is:

--- a/modules/logger/logger.go
+++ b/modules/logger/logger.go
@@ -11,6 +11,13 @@ import (
        "time"
 )

+var output io.Writer = os.Stdout
+
+// Use a new writer to send the logs to
+func Use(w io.Writer) {
+       output = w
+}
+
 // Logf logs the given format and arguments, formatted using fmt.Sprintf, to stdout, along with a timestamp and information
 // about what test and file is doing the logging. This is an alternative to t.Logf that logs to stdout immediately,
 // rather than buffering all log output and only displaying it at the very end of the test. This is useful because:
@@ -29,14 +36,14 @@ import (
 // Note that there is a proposal to improve t.Logf (https://github.com/golang/go/issues/24929), but until that's
 // implemented, this method is our best bet.
 func Logf(t *testing.T, format string, args ...interface{}) {
-       DoLog(t, 2, os.Stdout, fmt.Sprintf(format, args...))
+       DoLog(t, 2, output, fmt.Sprintf(format, args...))
 }

 // Log logs the given arguments to stdout, along with a timestamp and information about what test and file is doing the
 // logging. This is an alternative to t.Logf that logs to stdout immediately, rather than buffering all log output and
 // only displaying it at the very end of the test. See the Logf method for more info.
 func Log(t *testing.T, args ...interface{}) {
-       DoLog(t, 2, os.Stdout, args...)
+       DoLog(t, 2, output, args...)
 }

 // DoLog logs the given arguments to the given writer, along with a timestamp and information about what test and file is

In my tests, I simply do a

logger.Use(ioutil.Discard)

if I do not want the logs.

WDYT? Do you want me to submit a PR?

[brikis98]

Neat workaround, but I think that would suppress logging for everything in Terratest globally, which doesn't offer the granularity we need. Perhaps add some sort of logging level to terraform.Options, packer.Options, and other similar Options structs and pass it through?

[owulveryck]

What kind of granularity do you usually need?
Every test can use and set the Use function as needed.
On top of that, the API of the module/terraform is very flexible and catch the output anyway.
What do I miss? (I am new to terratest)

[brikis98]

Every test can use and set the Use function as needed.

Yes, but tests run in parallel, and this is a global setting, so tests will end up overwriting each other's settings.

[eschwartz]

Any update on this?

This is causing some pain in our automated test runs. We have tests that run OutputAll() against modules with a large number of outputs. When we run them in a CI system (Azure Devops), the CI tool truncates the test logs, and will not show us the actual test failure logs.

[tomaszsek]

Neat workaround, but I think that would suppress logging for everything in Terratest globally, which doesn't offer the granularity we need. Perhaps add some sort of logging level to terraform.Options, packer.Options, and other similar Options structs and pass it through?

That's is what I'm looking for. It's the best solution for me.

[mattolenik]

I think the logger package should be utilizing Go's built in log package, and the various Options structs should allow the passing in of a custom Go logger. This would let you specify whatever io.Writer you want, be it stdout, stderr, or something else entirely, like GinkgoWriter when using the Ginkgo test framework. It would be granular down the specific packages you're using, each option struct can have the same or different logger instance.

I also think packages like http-helper also need some way to adjust logging output. I'm using HTTPDoWithRetryE to wait for a service to be ready, and when it does, http-helper dumps the entire HTML document to stdout, which is a bit absurd (edit: in my case I realized I could fix this by using the HEAD method, but even then there's quite a bit of log noise). For that package I think it would expose some kind of HTTPHelper struct which could be initialized with a custom logger (and any other values, like testing.T), and the existing functions would be attached to that struct as methods. Compatibility could be maintained by extracting the existing functions into private functions that require an instance of said HTTPHelper struct, then have the existing public functions call into those with a default instance.

[tommyknows]

Also, Go's Logf function now does streaming during the test:
https://go-review.googlesource.com/c/go/+/127120

That's landed in Go 1.14, can confirm that this works.

Would be great to support some kind of io.Writer and the Logf function at least.

I'd propose - and if people agree, would start to implement - it with the following interface:

type Logger interface {
        Logf(t testing.TestingT, format string, args ...interface{})
}

Complete, "minimal" usage example:

package logger

import (
        "fmt"
        "io"
        "os"
        "testing"
        "time"

        tftesting "github.com/gruntwork-io/terratest/modules/testing"
)

// bareLogger uses *testing.T to log, by doing a type assertion.
type bareLogger struct{}

func (b *bareLogger) Logf(t tftesting.TestingT, format string, args ...interface{}) {
        if tt, ok := t.(*testing.T); ok {
                tt.Logf(format, args...)
        }
}

func TestLogger(t *testing.T) {
        RegisterLogger(&bareLogger{})
        RegisterLogger(&customLogger{os.Stdout})

        DoLog(t, "%s world", "hello")
        time.Sleep(time.Second * 3)
        DoLog(t, "%s world", "goodbye")

}

type customLogger struct {
        io.Writer
}

func (c *customLogger) Logf(_ tftesting.TestingT, format string, args ...interface{}) {
        _, _ = c.Write([]byte(fmt.Sprintf("CustomLogger: "+format+"\n", args...)))
}

type Logger interface {
        Logf(t tftesting.TestingT, format string, args ...interface{})
}

func DoLog(t tftesting.TestingT, format string, args ...interface{}) {
        for _, l := range loggers {
                l.Logf(t, format, args...)
        }
}

var loggers []Logger

func RegisterLogger(l Logger) {
        loggers = append(loggers, l)
}

I'd add a field to terraform.Options (Loggers []logger.Logger) which then get used to print the command output.

[blame19]

It would be good to have the same behavior for the k8s module.

[retpolanne]

@blame19 sent a patch adding this behavior to k8s and kubectl.