Add hook to allow roundtripping values

[quad]

We have request to a server that include a unique ID that needs to be included in the response:

Request:

{ "type": "request", "id": "abc123", "message": "hello!" }

Response:

{ "type": "response", "id": "abc123", "message": "nice to meet you!" }

The request is generated with code like the following:

request := Request{Type: "Request", Id: id.Random().String(), Message: "hello!"}

Right now, the first interaction will be recorded with the first random ID. All future replays will include the original id.

• Neither the AfterCaptureHook nor BeforeSaveHook trigger on replays, so they can't patch up the response with the correct id
• The BeforeResponseReplayHook does trigger on replays, but the i.Request contains the first recorded id value, so it can't patch up the response with the correct id

What is the correct way to go about fixing up an interaction in this way?

[dnaeon]

Hey @quad ,

Is it possible to provide a working code example which we can look at?

Thanks!

[quad]

Here's a minimal example, using the Request-Id header as the roundtripped value. This test will pass on its first run and then fail on subsequent runs.

package main

import (
	"net/http"
	"net/http/httptest"
	"net/url"
	"testing"

	"github.com/google/uuid"
	"gopkg.in/dnaeon/go-vcr.v3/cassette"
	"gopkg.in/dnaeon/go-vcr.v3/recorder"
)

func httptestMatcher(r *http.Request, i cassette.Request) bool {
	// Ignore the port in the HTTP request URL
	r_url := *r.URL
	r_url.Host = r_url.Hostname()

	// Ignore the port in the cassette request URL
	i_url, err := url.Parse(i.URL)
	if err != nil {
		return false
	}
	i_url.Host = i_url.Hostname()

	return r.Method == i.Method && r_url.String() == i_url.String()
}

func testHandler(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Request-Id", r.Header.Get("Request-Id"))
	w.WriteHeader(http.StatusOK)
}

func TestVcr(t *testing.T) {
	server := httptest.NewServer(http.HandlerFunc(testHandler))
	defer server.Close()

	rec, err := recorder.New("fixtures/" + t.Name())
	if err != nil {
		t.Fatal(err)
	}
	defer rec.Stop()
	rec.SetMatcher(httptestMatcher)

	req, err := http.NewRequest(http.MethodGet, server.URL, nil)
	if err != nil {
		t.Fatal(err)
	}

	req_id := uuid.NewString()
	req.Header.Add("Request-Id", req_id)

	resp, err := rec.GetDefaultClient().Do(req)
	if err != nil {
		t.Fatal(err)
	}
	defer resp.Body.Close()

	resp_id := resp.Header.Get("Request-Id")
	if resp_id != req_id {
		t.Errorf("Expected Request-Id %s, got %s", req_id, resp_id)
	}
}

[dnaeon]

Hey @quad ,

I'll try to look into this one soon. Thanks for providing some sample code to test out.

[dnaeon]

Hey @quad ,

I think the issue you have is because during each test execution you are creating a new UUID and then try to compare it against the one which is already in the recorded interaction.

This code here would always generate a new UUID, which will be different than the recorded one.

	req_id := uuid.NewString()
	req.Header.Add("Request-Id", req_id)

A possible solution would be to use a recorder.BeforeResponseReplayHook, which will mutate the HTTP response from the recorded interaction.

This is what you need to add to your recorder.

	// BeforeResponseHook which will return the ID we create above
	hook := func(i *cassette.Interaction) error {
		i.Response.Headers.Set("Request-Id", req_id)
		return nil
	}
	rec.AddHook(hook, recorder.BeforeResponseReplayHook)

The full code is here as well.

package main

import (
	"net/http"
	"net/http/httptest"
	"net/url"
	"testing"

	"github.com/google/uuid"
	"gopkg.in/dnaeon/go-vcr.v3/cassette"
	"gopkg.in/dnaeon/go-vcr.v3/recorder"
)

func httptestMatcher(r *http.Request, i cassette.Request) bool {
	// Ignore the port in the HTTP request URL
	r_url := *r.URL
	r_url.Host = r_url.Hostname()

	// Ignore the port in the cassette request URL
	i_url, err := url.Parse(i.URL)
	if err != nil {
		return false
	}
	i_url.Host = i_url.Hostname()

	return r.Method == i.Method && r_url.String() == i_url.String()
}

func testHandler(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Request-Id", r.Header.Get("Request-Id"))
	w.WriteHeader(http.StatusOK)
}

func TestVcr(t *testing.T) {
	server := httptest.NewServer(http.HandlerFunc(testHandler))
	defer server.Close()

	rec, err := recorder.New("fixtures/" + t.Name())
	if err != nil {
		t.Fatal(err)
	}
	defer rec.Stop()
	rec.SetMatcher(httptestMatcher)

	req, err := http.NewRequest(http.MethodGet, server.URL, nil)
	if err != nil {
		t.Fatal(err)
	}

	req_id := uuid.NewString()
	req.Header.Add("Request-Id", req_id)

	// BeforeResponseHook which will return the ID we create above
	hook := func(i *cassette.Interaction) error {
		i.Response.Headers.Set("Request-Id", req_id)
		return nil
	}
	rec.AddHook(hook, recorder.BeforeResponseReplayHook)

	resp, err := rec.GetDefaultClient().Do(req)
	if err != nil {
		t.Fatal(err)
	}
	defer resp.Body.Close()

	resp_id := resp.Header.Get("Request-Id")
	if resp_id != req_id {
		t.Errorf("Expected Request-Id %s, got %s", req_id, resp_id)
	}
}

[quad]

Hey @quad ,

I think the issue you have is because during each test execution you are creating a new UUID and then try to compare it against the one which is already in the recorded interaction.

💯 I used per-request unique UUID as a motivating example; but, yes, the recorded response needs to be mutated as a function of unique per-request data.

[…]
A possible solution would be to use a recorder.BeforeResponseReplayHook, which will mutate the HTTP response from the recorded interaction.
[…]

That's a neat solution! I hadn't considered capturing req_id via hook's closure as opposed to passed in via the hook's arguments.

I don't immediately see how this pattern could work across a cassette that recorded multiple requests; but, I'll play with it more.

Thank you! 🙇

[dnaeon]

One thing that comes to mind is to have a hook, where you keep mapping between API paths (e.g. /api/v1/foo) and the respective UUIDs you expect at these paths. Then within the hook simply test whether the API path matches and inject the corresponding UUID in the response.

Example hook (haven't tested it, but it should be good enough to illustrate the idea).

	hook := func(i *cassette.Interaction) error {
		req, err := i.GetHTTPRequest()
		if err != nil {
			return err
		}

		// Mapping between relative URLs and the expected UUIDs
		mappings := map[string]string{
			"/api/v1/foo": "uuid-1",
			"/api/v1/bar": "uuid-2",
		}

		for path, id := range mappings {
			if req.URL.Path == path {
				i.Response.Headers.Set("Request-Id", id)
			}
		}

		return nil
	}

[quad]

One thing that comes to mind is to have a hook, where you keep mapping between API paths (e.g. /api/v1/foo) and the respective UUIDs you expect at these paths. Then within the hook simply test whether the API path matches and inject the corresponding UUID in the response.

The problem is that the UUIDs are randomly generated; the request ID can be thought of as an idempotency key. In a real example, the UUID would be generated per-request by the client library under test.

At this point, what I'd love is a way to access the actual request in BeforeResponseReplayHook. Unfortunately, it only seems to be able to access the recorded request.