Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retina multicloud #1267

Open
SRodi opened this issue Jan 24, 2025 · 1 comment
Open

Retina multicloud #1267

SRodi opened this issue Jan 24, 2025 · 1 comment
Assignees

Comments

@SRodi
Copy link
Member

SRodi commented Jan 24, 2025

Issue Description

Provide a streamlined approach deploy retina on multiple Kubernetes distributions. This will also allow to test Retina on multiple platforms.

Goal

Create an IaC sub-project to deploy multi-cloud clusters

  • GKE
  • EKS
  • Kind
  • ...

Definition of Done

A flow exist to create Kubernetes clusters on multiple cloud providers and deploy retina on it

@nddq
Copy link
Contributor

nddq commented Jan 24, 2025

related issue: #451
related PR: #667

github-merge-queue bot pushed a commit that referenced this issue Jan 28, 2025
# Description

Initial work on multi-cloud

* create modules for aks, gke, kind and retina
* use gcloud, azure, kind and helm providers
* create examples on how to use each module
* create automated tests in Go Terratest for each module example (only
initial work, since we require creds for public clouds)
* create integration test retina-kind (this can also be used as an
example for AKS, GKE and eventually EKS)
* create a Makefile for this subproject
* create GH workflow with automated tests


![diagram](https://github.com/user-attachments/assets/38ded8ec-0c31-4e5d-9bd0-8729115cf5b3)

## Related Issue

#1267 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed

All modules were tested and work as expected.

![Screenshot 2025-01-24 at 12 18
55](https://github.com/user-attachments/assets/2d222a13-2b12-4a11-a001-1b03a254e152)

See automated tests here
https://github.com/microsoft/retina/actions/runs/12993893617/job/36237118572


![image](https://github.com/user-attachments/assets/edafa98b-1d5d-44b0-a393-1bf7b7b70bda)



## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
github-merge-queue bot pushed a commit that referenced this issue Jan 30, 2025
# Description

* add fetchSensitiveOutputs function to override default logger to
prevent sensitive outputs from being logged
* add checkRetinaPodsRunning function to check if Retina pods are
running
* add make target for formatting both OpenTofu and go code
* modify make test target to run test ONLY for TestRetinaKindIntegration
till we get cloud creds
* refactor tests to use new func

## Related Issue

#1267 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed


![image](https://github.com/user-attachments/assets/fd711fa8-4064-4c57-87cc-2a7acc088f6f)

## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
github-merge-queue bot pushed a commit that referenced this issue Jan 31, 2025
# Description

* add module for Prometheus deployment
* add example and test for Prometheus module
* add tests for each testUtils.go functions
* refactor

## Related Issue

#1267 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed

Please add any relevant screenshots or GIFs to showcase the changes
made.

## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
github-merge-queue bot pushed a commit that referenced this issue Feb 4, 2025
# Description

* add integration test for Retina on GKE
* add package utils and refactor test directory

## Related Issue

#1267 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed


![image](https://github.com/user-attachments/assets/d80045e7-8342-47f0-89a4-950f4d3b9a4d)

## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
github-merge-queue bot pushed a commit that referenced this issue Feb 5, 2025
# Description

* create helm-release module
* delete retina and prometheus modules
* refactor integration tests as needed

## Related Issue

#1267 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed

![Screenshot 2025-02-04
093852](https://github.com/user-attachments/assets/d8c806cc-307a-4956-bdea-c12caf33315c)
![Screenshot 2025-02-04
114057](https://github.com/user-attachments/assets/fb46b254-0623-4fa8-9a96-f5abf67c56c0)

## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
github-merge-queue bot pushed a commit that referenced this issue Feb 11, 2025
# Description

Create a multi cloud multi cluster deployment where each cluster has a
deployment of Prometheus and Retina. Each cluster exposes Prometheus as
a load balancer. Both load balancers are connected to a single instance
of Grafana to visualize retina network observability metrics.

* Add module for Kubernetes load balancer service used by AKS and GKE
Prometheus instances
* Automate the data source config via Grafana module
* Add module for Azure Network Security Group
* Add module for Google Cloud Firewall
* Update retina-gke and retina-aks live stacks


![grafana-mc](https://github.com/user-attachments/assets/b24138cb-9b03-4d46-8231-ebba530ce486)

## Related Issue

#1267 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed


![image](https://github.com/user-attachments/assets/fc9ec2b5-9ca5-4a41-bff4-bb97c23bd67d)


![image](https://github.com/user-attachments/assets/2f4779cc-8677-4bc0-9a65-faebcddb0c94)

## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
github-merge-queue bot pushed a commit that referenced this issue Feb 27, 2025
# Description

* Create EKS module
* Create EKS example
* Create EKS unit and integration test with retina
* Create live/retina-eks to demo multi-cloud
* Update docs
* Update diagrams
* Update Makefile for this sub-project test/multicloud

## Related Issue

#1267 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/Contributing/overview).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed

Grafana Hubble DNS dashboard for EKS cluster

![Screenshot_26-2-2025_141028_srodi grafana
net](https://github.com/user-attachments/assets/d5e43699-83f9-429f-b7df-127a6e238859)

EKS cluster showing AWS nodes and retina logs

![Screenshot 2025-02-26
131742](https://github.com/user-attachments/assets/2bb9ec2c-7b13-40af-b10e-607e02467ffa)

## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
github-merge-queue bot pushed a commit that referenced this issue Feb 28, 2025
# Description

* Create examples/integration/retina-aks
* Create go integration test retina-aks

## Related Issue

#1267 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/Contributing/overview).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed

Logs tail to show successful test run


![image](https://github.com/user-attachments/assets/49003fcd-a4c7-4d26-a21e-2e72c14fb839)

## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
BeegiiK pushed a commit to BeegiiK/retina that referenced this issue Feb 28, 2025
# Description

* Create EKS module
* Create EKS example
* Create EKS unit and integration test with retina
* Create live/retina-eks to demo multi-cloud
* Update docs
* Update diagrams
* Update Makefile for this sub-project test/multicloud

## Related Issue

microsoft#1267 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/Contributing/overview).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed

Grafana Hubble DNS dashboard for EKS cluster

![Screenshot_26-2-2025_141028_srodi grafana
net](https://github.com/user-attachments/assets/d5e43699-83f9-429f-b7df-127a6e238859)

EKS cluster showing AWS nodes and retina logs

![Screenshot 2025-02-26
131742](https://github.com/user-attachments/assets/2bb9ec2c-7b13-40af-b10e-607e02467ffa)

## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
BeegiiK pushed a commit to BeegiiK/retina that referenced this issue Feb 28, 2025
…ft#1391)

# Description

* Create examples/integration/retina-aks
* Create go integration test retina-aks

## Related Issue

microsoft#1267 

## Checklist

- [x] I have read the [contributing
documentation](https://retina.sh/docs/Contributing/overview).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.

## Screenshots (if applicable) or Testing Completed

Logs tail to show successful test run


![image](https://github.com/user-attachments/assets/49003fcd-a4c7-4d26-a21e-2e72c14fb839)

## Additional Notes

Add any additional notes or context about the pull request here.

---

Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more
information on how to contribute to this project.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

2 participants