Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: reverse proxy #336

Merged
merged 16 commits into from
May 27, 2024
Merged

feat: reverse proxy #336

merged 16 commits into from
May 27, 2024

Conversation

mojtaba-esk
Copy link
Contributor

@mojtaba-esk mojtaba-esk commented May 15, 2024

This is a draft version as there is an issue which needs help from other team members.

To run it, use the following command

go test -v ./e2e/basic/ --run TestReverseProxy -timeout 60m

Closes #315

@mojtaba-esk mojtaba-esk requested review from smuu and tty47 May 15, 2024 10:15
@mojtaba-esk mojtaba-esk self-assigned this May 15, 2024
@mojtaba-esk mojtaba-esk requested a review from a team May 15, 2024 10:17
@mojtaba-esk mojtaba-esk changed the title Mojtaba/feat reverse proxy feat: reverse proxy May 15, 2024
@smuu
Copy link
Member

smuu commented May 15, 2024

Could you please provide the commands of how you testet it locally?

@smuu
Copy link
Member

smuu commented May 16, 2024

I started reviewing and to understand it better I have two questions:

  1. Does this solution require a reverse proxy (a LoadBalancer) per knuu instance we want to access?
  2. Does this solution need to have something pre-installed in the k8s cluster?

@mojtaba-esk
Copy link
Contributor Author

Could you please provide the commands of how you testet it locally?

yes it can be run via this command:

go test -v ./e2e/basic/ --run TestReverseProxy -timeout 60m

it creates a dummy instance with bittwister enabled and tries to call some APIs of bittwister to validate the proxy function.

@MSevey MSevey added the knuu label May 16, 2024
@mojtaba-esk mojtaba-esk marked this pull request as ready for review May 17, 2024 11:17
Copy link
Member

@smuu smuu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is 🤯
Thank you for all the work!
I added some questions and thoughts in this first round of review.

pkg/k8s/k8s_endpoint.go Outdated Show resolved Hide resolved
pkg/k8s/k8s_pod.go Outdated Show resolved Hide resolved
pkg/knuu/knuu.go Outdated Show resolved Hide resolved
pkg/knuu/knuu.go Outdated Show resolved Hide resolved
pkg/traefik/traefik.go Outdated Show resolved Hide resolved
pkg/traefik/traefik.go Show resolved Hide resolved
pkg/traefik/traefik.go Show resolved Hide resolved
proxy_sa.sh Outdated Show resolved Hide resolved
pkg/traefik/traefik.go Show resolved Hide resolved
pkg/k8s/k8s_service.go Outdated Show resolved Hide resolved
@mojtaba-esk mojtaba-esk requested a review from smuu May 21, 2024 13:50
@MSevey
Copy link
Member

MSevey commented May 21, 2024

This is a draft version as there is an issue which needs help from other team members.

To run it, use the following command

go test -v ./e2e/basic/ --run TestReverseProxy -timeout 60m

Closes #315

Since the issue doesn't have a lot of context, to help with alignment on async reviews, can we add some more detail to either the issue or the PR description?

Things I'd like to know are:

  • What is the current problem that this solves?
  • What is the scope of this PR as it relates to the full solution? Is this everything or are there planned next steps?
  • What are the current known limitations or issues?
  • Are there any assumptions that this PR is making?
  • Does this change come with any caveats?

// Check if the BitTwister service is set
out, err := main.BitTwister.Client().AllServicesStatus()
require.NoError(t, err, "Error getting all services status")
assert.NotEmpty(t, out, "No services found")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible it would be good to check for some explicit state other than not empty as this would not protect us from a bug in the code that returns an invalid non empty response.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could also use an nginx server instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delicate point, updated

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the resolution here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of just checking if it is empty or not, now we check for existence of a particular field which can yield only if the request goes through successfully.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[follow up] we should enable golangci-lint for this repo and enable the export comment linter.

Why?

Exports should have comments so when other packages import them, it is abundantly clear what they are importing.

The additional benefit of having comments is that it forces line separation for things like this. So when 1 new error is added, we don't have an unnecessarily long diff that is unhelpful in understanding the actual change.

Copilot can add comments to all of these, so it isn't a big lift to add comments and new lines here in a separate PR.

Could be done now and merge to clean up the diff here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah using a linter is a great idea in ci.
About the comments on Exported funcs, I blieve not every function needs it. It is attempted to have clear names for funcs and their arguments so it will be intuitive for users to use it. Otherwise yes we need comments to clarify stuff or even show an example to the users.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can have the discussion around which linters to enable, but we should make a follow up to track getting the linters on the code base.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an issue is created for it

pkg/k8s/k8s.go Show resolved Hide resolved
pkg/k8s/k8s_role.go Outdated Show resolved Hide resolved
pkg/k8s/k8s_rolebinding.go Outdated Show resolved Hide resolved
@mojtaba-esk mojtaba-esk requested review from smuu and MSevey May 22, 2024 17:24
@smuu
Copy link
Member

smuu commented May 23, 2024

In the prototype for big block tests, I added the following function to Instance:

func (i *Instance) AddHost(port int) (err error, host string) {
	prefix := fmt.Sprintf("%s-%d", i.k8sName, port)
	if err := traefikClient.AddHost(context.TODO(), i.k8sName, prefix, port); err != nil {
		return ErrAddingHost.WithParams(i.k8sName).Wrap(err), ""
	}
	host, err = traefikClient.URL(context.TODO(), prefix)
	if err != nil {
		return ErrGettingHost.WithParams(i.k8sName).Wrap(err), ""
	}
	return nil, host
}

It would be great to have a function with that functionality. Feel free to rename and modify the function at will.

smuu
smuu previously approved these changes May 23, 2024
tty47
tty47 previously approved these changes May 23, 2024
Copy link
Contributor

@tty47 tty47 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Co-authored-by: Jose Ramon Mañes <32740567+jrmanes@users.noreply.github.com>
@mojtaba-esk mojtaba-esk requested review from smuu and tty47 May 23, 2024 15:24
Makefile Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be a follow up but imo it would be better to DRY up this commands and just use the pkg and run variables.

ie:

pkgs := $(shell go list ./...)
run := .
count := 1

test: 
        KNUU_TIMEOUT=120m go test -v $(pkgs) -run $(run) -count=$(count) -timeout 120m
.PHONY: test

then you can just have the one makefile command and easily target whatever by setting the pkg, run, or count flags.

Targeting a directory

make test pkgs=./e2e/basic

Targeting a Test in a directory

make test pkgs=./e2e/basic run=TestJustThisTest

Run a test in a loop to debug

make test pkgs=./e2e/basic run=TestJustThisTest10Times count=10

This model has more flexibility and DRY's up the Makefile significantly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a good idea. specially when we add unit tests this make file can grow really fast. This way we can have a cleaner code. Let's address it in a separate PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated and tested 👌

pkg/knuu/errors.go Outdated Show resolved Hide resolved
Copy link
Member

@MSevey MSevey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think my previous review was fully resolved.

Please don't resolve a conversations unless the action was fully implemented with no deviation.

Responding to a comment and then dismissing the review by resolving the discussion makes it harder for re-reviews. As it isn't clear if comments were fully addressed so then reviewers have to go through and open up all the discussions to check. Or the same comments will just be left again.

Signed-off-by: Jose Ramon Mañes <jose@celestia.org>
tty47
tty47 previously approved these changes May 27, 2024
@mojtaba-esk mojtaba-esk disabled auto-merge May 27, 2024 13:30
@mojtaba-esk mojtaba-esk merged commit 931b66d into main May 27, 2024
3 of 4 checks passed
@mojtaba-esk mojtaba-esk deleted the mojtaba/feat-reverse-proxy branch May 27, 2024 13:30
@mojtaba-esk mojtaba-esk restored the mojtaba/feat-reverse-proxy branch June 13, 2024 08:05
@mojtaba-esk mojtaba-esk deleted the mojtaba/feat-reverse-proxy branch June 13, 2024 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

feat: reverse proxy
4 participants