Skip to content

Commit

Permalink
Documented apps & charts (#66)
Browse files Browse the repository at this point in the history
* Update with grpcserver

* Add docs
  • Loading branch information
utr1903 authored Mar 18, 2024
1 parent bfaf205 commit f7b9b20
Show file tree
Hide file tree
Showing 10 changed files with 68 additions and 5 deletions.
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,17 @@ The repo replicates a commonly preferred organizational DevOps structure:

It can be stated that the tasks of individual teams are sort of loosely decoupled from each other. In this repo, every programming language stands for a different `devteam` where the central maintanence team is referred as the `opsteam` and responsible for the cluster components and the common resources which the `devteams` use (Kafka, MySQL). The `opsteam` uses the `ops` namespace and the `devteams` use their `<language>` namespace (e.g. `golang`) for their applications.

The current business logic is simple. Every team has 3 components: `simulator`, `kafkaconsumer` and `httpserver`. As it can be seen the diagram below, the `simulator` publishes a message (from its own namespace) to the `kafka` cluster (in the `ops` namespace). This stands for a `CREATE` request which is meant to store a random data in the `mysql` database (in the `ops` namespace). This is done via the `kafkaconsumer` which consumes the message from the `kafka` cluster and stores the data in the `mysql` database. Meanwhile, the `simulator` also performs `GET` and `DELETE` requests to the `httpserver` which then first goes to the `redis` cache and then queries against the `mysql` database.
The current business logic is simple. Every team has 4 components: `simulator`, `kafkaconsumer` `httpserver` and `grpcserver`. As it can be seen the diagram below, the `simulator` publishes a message (from its own namespace) to the `kafka` cluster (in the `ops` namespace). This stands for a `CREATE` request which is meant to store a random data in the `redis` cache and the `mysql` database (in the `ops` namespace). This is done via the `kafkaconsumer` which consumes the message from the `kafka` cluster and stores the data in the `mysql` database. Meanwhile, the `simulator` also performs `GET` and `DELETE` requests to the `httpserver` which then first goes to the `redis` cache and then queries against the `mysql` database and also the `grpcserver` which then goes to the `redis`.

![Application Architecture](./media/application_architecture.png)

To see each workflow in detail, refer to following docs:

- [kafkaconsumer](/infra/helm/kafkaconsumer/README.md)
- [grpcserver](/infra/helm/grpcserver/README.md)
- [httpserver](/infra/helm/httpserver/README.md)
- [latencymanager](/infra/helm/latencymanager/README.md)

That being said, the telemetry data which is generated from the applications within the `<language>` namespaces are sent to the `otelcollectors` in the `ops` namespace which will send the data to your observability backend. This is depicted in the diagram below.

![Telemetry Architecture](./media/telemetry_architecture.png)
Expand Down Expand Up @@ -80,13 +87,15 @@ infra/

- (commons library)
- httpserver
- grpcserver
- kafkaconsumer
- simulator

`<charts>` stands for:

- [cert-manager](/infra/helm/cert-manager)
- [httpserver](/infra/helm/httpserver/)
- [grpcserver](/infra/helm/grpcserver/)
- [kafka](/infra/helm/kafka/)
- [kafkaconsumer](/infra/helm/kafkaconsumer/)
- [latencymanager](/infra/helm/latencymanager/)
Expand Down
13 changes: 12 additions & 1 deletion infra/helm/grpcserver/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,14 @@
# gRPC server

To be implemented...
The `grpcserver` is running within the `<language>` namespace. It uses

- Redis cache

within the `ops` namespace.

The corresponding `GET` and `DELETE` producure works as follows:

1. `simulator` calls `grpcserver`
2. `httpserver` gets/deletes from/in `redis`

![workflow](/media/grpcserver_workflow.png)
16 changes: 15 additions & 1 deletion infra/helm/httpserver/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
# HTTP server

To be implemented...
The `httpserver` is running within the `<language>` namespace. It uses

- Redis cache
- MySQL database

within the `ops` namespace.

The corresponding `GET` and `DELETE` producure works as follows:

1. `simulator` calls `httpserver`
2. `httpserver` gets the boolean `increase.latency` flag from `redis`
- _periodic intentional error controlled by `latencymanager`_
3. `httpserver` gets/deletes from/in `mysql`

![workflow](/media/httpserver_workflow.png)
16 changes: 15 additions & 1 deletion infra/helm/kafkaconsumer/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
# Kafka consumer

To be implemented...
The `kafkaconsumer` is running within the `<language>` namespace. It uses

- Kafka broker
- Redis cache
- MySQL database

within the `ops` namespace.

The corresponding `CREATE` producure works as follows:
1. `simulator` publishes a message to `kafka`
2. `kafkaconsumer` consumes the message
3. `kafkaconsumer` stores a value in `redis`
4. `kafkaconsumer` stores a value in `mysql`

![workflow](/media/kafkaconsumer_workflow.png)
17 changes: 16 additions & 1 deletion infra/helm/latencymanager/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,18 @@
# Latency manger

To be implemented...
The `latencymanager` is running within the `<language>` namespace. It uses

- Redis cache

within the `ops` namespace.

The corresponding `increase` and `decrease` producure works as follows:

1. `cronjob` triggers `latencymanager`
- _either `increase` or `decrease`_
2. `latencymanager` sets the boolean `increase.latency` flag in `redis`
- _periodic intentional error to manipulate `httpserver` latency_

![workflow](/media/latencymanager_workflow.png)

Moreover, it sends a deployment marker event to New Relic whenever the latency change has been made.
Binary file modified media/application_architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/grpcserver_workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/httpserver_workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/kafkaconsumer_workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/latencymanager_workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit f7b9b20

Please sign in to comment.