Regis Cluster Example

ContinUSE · Aug 20, 2015 · 166469e · 166469e
1 parent ddac76a
commit 166469e
Show file tree

Hide file tree

Showing 16 changed files with 1,722 additions and 0 deletions.
diff --git a/examples/redis-cluster/README.md b/examples/redis-cluster/README.md
@@ -0,0 +1,81 @@
+# Redis Cluster
+This example provides the ability to automatically process such as initial redis cluster configuration, node expansion and node failover operation on Kubernetes environment.
+
+## Reliable, Scalable Redis Cluster on Kubernetes
+The followings are multi-node Redis Cluster Usage. This example that I tried to apply on a trial basis.
+
+### Redis Cluster Manager Pods
+Create redis cluster manager as follows:
+```
+$ kubectl create -f redis-cluster-manager.yaml
+replicationcontrollers/redis-cluster-manager
+```
+
+Log view for status check
+```
+$ kubectl get pods
+NAME                          READY     STATUS    RESTARTS   AGE
+redis-cluster-manager-8eo9n   1/1       Running   0          1m
+$ kubectl logs -f redis-cluster-manager-8eo9n
+Getting information for redis cluster...........
+.........
+........
+........
+The redis cluster need at least 6 nodes.
+  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
+                                 Dload  Upload   Total   Spent    Left  Speed
+```
+
+### Replicated redis cluster workers
+Create redis cluster worker as follows:
+```
+$ kubectl create -f redis-cluster-worker.yaml
+replicationcontrollers/redis-cluster-worker
+```
+You can view status check for redis-cluster-manager above "kubectl logs...." command.
+
+### Service creation for redis-cluster-worker pods
+```
+$ kubectl create -f redis-cluster-worker-service.yaml
+You have exposed your service on an external port on all nodes in your
+cluster.  If you want to expose this service to the external internet, you may
+need to set up firewall rules for the service port(s) (tcp:32100) to serve traffic.
+
+See http://releases.k8s.io/HEAD/docs/services-firewalls.md for more details.
+services/redis-cluster-worker
+```
+And redis-cli command Usage:
+The host IP address is one of node of kube, and port is 32100. (This port number defined in redis-cluster-worker-service.yaml)
+```
+$ sudo docker run --rm -it redis redis-cli -c -h 192.168.10.71 -p 32100
+192.168.10.71:32100> cluster info
+cluster_state:ok
+cluster_slots_assigned:16384
+cluster_slots_ok:16384
+cluster_slots_pfail:0
+cluster_slots_fail:0
+cluster_known_nodes:6
+cluster_size:3
+cluster_current_epoch:6
+cluster_my_epoch:3
+cluster_stats_messages_sent:2121
+cluster_stats_messages_received:2121
+192.168.10.71:32100>
+```
+
+### Node Add (1 master / 1 slave)
+```
+$ kubectl scale rc redis-cluster-worker --replicas=8
+scaled
+```
+See log view for ALL procedure such as node add and data reshard.
+
+# Node Failure Test
+kube-02 fail test
+```
+vagrant halt kube-02
+```
+It takes about five minutes for the automatic recovery. Checking the log view of redis-cluster-manager.
+
+### Issue
+The system is still unstable after recovery of failure.
diff --git a/examples/redis-cluster/backup/README.md b/examples/redis-cluster/backup/README.md
@@ -0,0 +1,45 @@
+# Redis Cluster
+This example provides the ability to automatically process such as initial redis cluster configuration, cluster expansion and node failover etc. on Kubernetes environment.
+
+## Reliable, Scalable Redis Cluster on Kubernetes
+The followings are multi-node MPICH cluster configuration on Kubernetes. It deploys a master with replicated workers.
+
+### Master Pods
+Create this master as follows:
+```
+kubectl create -f mpich-master.yaml
+```
+
+### Replicated mpich servers
+Create this worker as follows:
+```
+kubectl create -f mpich-controller.yaml
+```
+
+### Scale our replicated pods
+we will add more replicas for worker:
+```
+kubectl scale rc mpich-worker --replicas=3
+```
+
+### mpich master service
+Create the service by running: Using NodePort as 32000 tcp port
+```
+kubectl create -f mpich-service.yaml
+```
+
+### Connect to master & Basic test
+```
+$ ssh root@192.168.10.71 -p 32000 (Any one of kube cluster nodes)
+root@192.168.10.71's password: (root apsswrod as 'root')
+Last login: Thu Aug  6 04:11:46 2015 from mpich-master
+root@mpich-master:~# mpirun -f host_file -n 3 hostname
+mpich-master
+mpich-worker-1p1ko
+mpich-worker-r4j6z
+root@mpich-master:~#
+```
+
+### Conclusion
+Now We have a reliable, scalable MPICH cluster installation. By scaling the replication controller for mpich worker instances, we can increase or decrease the number of mpich-worker instances and auto update for host file on master. If you want to test for sample MPI program, visit to http://mpitutorial.com/tutorials/mpi-hello-world/. (MPICH package installed /tmp/mpich)
+
diff --git a/examples/redis-cluster/backup/redis-cluster-manager.yaml b/examples/redis-cluster/backup/redis-cluster-manager.yaml
@@ -0,0 +1,39 @@
+apiVersion: v1
+kind: ReplicationController
+metadata:
+  name: redis-cluster-manager
+  # namespace: redis-cluster-system
+spec:
+  replicas: 1
+  selector:
+    name: redis-cluster-manager
+  template:
+    metadata:
+      labels:
+        name: redis-cluster-manager
+        role: cluster-cluster-manager
+    spec:
+      containers:
+       - name: redis-cluster-manager
+         image: continuse/redis:v3
+         env:
+           - name: ETCDCTL_PEERS
+             value: "192.168.10.11:4001,192.168.10.12:4001,192.168.10.13:4001"
+           - name: KUBE_NAMESPACE
+             value: "default"
+           - name: KUBE_LABEL
+             value: "redis-cluster-worker"
+           - name: REPLICAS
+             value: "1"
+         ports:
+           - containerPort: 6379
+         resources:
+           limits:
+             cpu: "0.1"
+         volumeMounts:
+           - mountPath: /share
+             name: share
+      volumes:
+        - name: share
+          hostPath:
+            path : /continuse/mpi
diff --git a/examples/redis-cluster/backup/redis-cluster-manager.yaml.bak b/examples/redis-cluster/backup/redis-cluster-manager.yaml.bak
@@ -0,0 +1,34 @@
+apiVersion: v1
+kind: ReplicationController
+metadata:
+  name: redis-cluster-manager
+  # namespace: redis-cluster-system
+  labels:
+    name: redis-manager
+    role: cluster-manager
+spec:
+  replicas: 1
+  containers:
+    - name: redis-cluster-manager
+      image: continuse/redis:v3
+      env:
+        - name: ETCDCTL_PEERS
+          value: "192.168.10.11:4001,192.168.10.12:4001,192.168.10.13:4001"
+        - name: KUBE_NAMESPACE
+          value: "default"
+        - name: KUBE_LABEL
+          value: "redis-cluster-worker"
+        - name: REPLICAS
+          value: "1"
+      ports:
+        - containerPort: 6379
+      resources:
+        limits:
+          cpu: "0.1"
+      volumeMounts:
+        - mountPath: /share
+          name: share
+  volumes:
+    - name: share
+      hostPath:
+         path : /continuse/mpi
diff --git a/examples/redis-cluster/backup/redis-cluster-worker-service.yaml b/examples/redis-cluster/backup/redis-cluster-worker-service.yaml
@@ -0,0 +1,16 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: redis-cluster-worker
+  # namespace: redis-cluster-system
+  labels:
+    name: redis-worker
+spec:
+  type: NodePort
+  ports:
+    # the port that this service should serve on
+  - port: 6379
+    targetPort: 6379
+    nodePort: 32100
+  selector:
+    name: redis-cluster-worker
diff --git a/examples/redis-cluster/backup/redis-cluster-worker.yaml b/examples/redis-cluster/backup/redis-cluster-worker.yaml
@@ -0,0 +1,33 @@
+apiVersion: v1
+kind: ReplicationController
+metadata:
+  name: redis-cluster-worker
+  # namespace: mpich-system
+spec:
+  replicas: 6
+  selector:
+    name: redis-cluster-worker
+  template:
+    metadata:
+      labels:
+        name: redis-cluster-worker
+    spec:
+      containers:
+      - name: redis-cluster-worker
+        image: continuse/redis:v3
+        env:
+          - name: WORKER
+            value: "true"
+        ports:
+        - containerPort: 6379
+        resources:
+          limits:
+            cpu: "0.1"
+        volumeMounts:
+        - mountPath: /share
+          name: share
+      volumes:
+        - name: share
+          hostPath: 
+             path: /continuse/mpi
+
diff --git a/examples/redis-cluster/images/Dockerfile b/examples/redis-cluster/images/Dockerfile
@@ -0,0 +1,30 @@
+FROM       ubuntu:14.04
+MAINTAINER Jaewoo Lee <continuse@icloud.com>
+
+RUN apt-get update
+
+RUN apt-get install -y gcc make ruby expect telnet curl wget python-pip
+
+RUN wget http://download.redis.io/releases/redis-3.0.3.tar.gz
+RUN tar xvfz redis-3.0.3.tar.gz && cd redis-3.0.3 && make && make install
+
+# ETCD for python
+RUN cd /tmp && wget https://github.com/jplana/python-etcd/archive/0.4.1.tar.gz && \
+    tar xvfz 0.4.1.tar.gz && cd python-etcd-0.4.1 && pip install .
+
+# Python for Redis Cluster
+RUN wget https://github.com/ContinUSE/redis-py-cluster/archive/1.0.0.tar.gz \
+	&& tar xvfz 1.0.0.tar.gz \
+	&& cd redis-py-cluster-1.0.0 \
+	&& pip install .
+
+RUN gem install redis
+
+COPY redis.conf /redis.conf
+COPY auto_config_redis_cluster.py /auto_config_redis_cluster.py
+COPY entrypoint.sh /entrypoint.sh
+
+EXPOSE 6379
+
+#CMD    ["/usr/local/bin/redis-server", "/redis.conf"]
+CMD    ["/entrypoint.sh"]
diff --git a/examples/redis-cluster/images/README.md b/examples/redis-cluster/images/README.md
@@ -0,0 +1,81 @@
+# Redis Cluster
+This example provides the ability to automatically process such as initial redis cluster configuration, node expansion and node failover operation on Kubernetes environment.
+
+## Reliable, Scalable Redis Cluster on Kubernetes
+The followings are multi-node Redis Cluster Usage. This example that I tried to apply on a trial basis.
+
+### Redis Cluster Manager Pods
+Create redis cluster manager as follows:
+```
+$ kubectl create -f redis-cluster-manager.yaml
+replicationcontrollers/redis-cluster-manager
+```
+
+Log view for status check
+```
+$ kubectl get pods
+NAME                          READY     STATUS    RESTARTS   AGE
+redis-cluster-manager-8eo9n   1/1       Running   0          1m
+$ kubectl logs -f redis-cluster-manager-8eo9n
+Getting information for redis cluster...........
+.........
+........
+........
+The redis cluster need at least 6 nodes.
+  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
+                                 Dload  Upload   Total   Spent    Left  Speed
+```
+
+### Replicated redis cluster workers
+Create redis cluster worker as follows:
+```
+$ kubectl create -f redis-cluster-worker.yaml
+replicationcontrollers/redis-cluster-worker
+```
+You can view status check for redis-cluster-manager above "kubectl logs...." command.
+
+### Service creation for redis-cluster-worker pods
+```
+$ kubectl create -f redis-cluster-worker-service.yaml
+You have exposed your service on an external port on all nodes in your
+cluster.  If you want to expose this service to the external internet, you may
+need to set up firewall rules for the service port(s) (tcp:32100) to serve traffic.
+
+See http://releases.k8s.io/HEAD/docs/services-firewalls.md for more details.
+services/redis-cluster-worker
+```
+And redis-cli command Usage:
+The host IP address is one of node of kube, and port is 32100. (This port number defined in redis-cluster-worker-service.yaml)
+```
+$ sudo docker run --rm -it redis redis-cli -c -h 192.168.10.71 -p 32100
+192.168.10.71:32100> cluster info
+cluster_state:ok
+cluster_slots_assigned:16384
+cluster_slots_ok:16384
+cluster_slots_pfail:0
+cluster_slots_fail:0
+cluster_known_nodes:6
+cluster_size:3
+cluster_current_epoch:6
+cluster_my_epoch:3
+cluster_stats_messages_sent:2121
+cluster_stats_messages_received:2121
+192.168.10.71:32100>
+```
+
+### Node Add (1 master / 1 slave)
+```
+$ kubectl scale rc redis-cluster-worker --replicas=8
+scaled
+```
+See log view for ALL procedure such as node add and data reshard.
+
+# Node Failure Test
+kube-02 fail test
+```
+vagrant halt kube-02
+```
+It takes about five minutes for the automatic recovery. Checking the log view of redis-cluster-manager.
+
+### Issue
+The system is still unstable after recovery of failure.